FAQ: Can I use this as a drop in replacement for pytorch? #12

trholding · 2022-10-14T19:21:42Z

trholding
Oct 14, 2022

Awesome work. I have question. Can this be used as a drop in replacement for pytorch?

My expectation is that it would work nice on systems that don't have cuda, but where opencl is supported (AMD,ARM).

I also hope that it will be able change lines like these:

checkpoint = load_checkpoint(pth, torch.device("cuda")) or
checkpoint = load_checkpoint(pth, torch.device("cpu")) to

checkpoint = load_checkpoint(pth, torch.device("opencl"))

Please let me know, would love to build and try.

Answered by artyom-beilis

Oct 14, 2022

Yes.

However it is early version and lot of the operator/functions aren't implemented yet.

View full answer

artyom-beilis · 2022-10-14T19:43:13Z

artyom-beilis
Oct 14, 2022
Maintainer

Yes.

However it is early version and lot of the operator/functions aren't implemented yet.

6 replies

trholding Oct 17, 2022
Author

Building pytorch requires this particular version of setup tools:

I encountered this error:

python setup.py build --cmake-only
Building wheel torch-1.11.0a0+giteb74af1
Traceback (most recent call last):
  File "/home/skunkwork/experiments/pytorch/setup.py", line 309, in <module>
    cmake = CMake()
  File "/home/skunkwork/experiments/pytorch/tools/setup_helpers/cmake.py", line 102, in __init__
    self._cmake_command = CMake._get_cmake_command()
  File "/home/skunkwork/experiments/pytorch/tools/setup_helpers/cmake.py", line 126, in _get_cmake_command
    elif cmake is not None and CMake._get_version(cmake) >= distutils.version.LooseVersion("3.10.0"):
  File "/home/skunkwork/experiments/pytorch/tools/setup_helpers/cmake.py", line 137, in _get_version
    return distutils.version.LooseVersion(line.strip().split(' ')[2])
AttributeError: module 'setuptools._distutils' has no attribute 'version'

Solution:

pip install setuptools==59.5.0

trholding Oct 18, 2022
Author

For those who get an error during python setup.py install

.../third_party/gloo/gloo/transport/tcp/device.cc:151:39: error: aggregate ‘std::array<char, 64> hostname’ has incomplete type and cannot be defined
  151 |       std::array<char, HOST_NAME_MAX> hostname;
      |                                       ^~~~~~~~
[4/3397] Building CXX object third_party/gloo/gloo/CMakeFiles/gloo.dir/transport/tcp/pair.cc.o
ninja: build stopped: subcommand failed.

See facebookincubator/gloo#332 and apply this fix: Add #include <array> to line 15 in third_party/gloo/gloo/transport/tcp/device.cc

#include "gloo/transport/tcp/device.h"

#include <ifaddrs.h>
#include <netdb.h>
#include <netinet/in.h>
#include <string.h>
#include <array>

trholding Oct 23, 2022
Author

More fixes that need to be applied with newer GCC versions:

oneapi-src/oneDNN#1023 (comment)

file: primitive_cache.hpp line: after last #include

+ #include <thread>

google/breakpad@605c51e

file: exception_handler.cc line: 141

- static const unsigned kSigStackSize = std::max(16384, SIGSTKSZ);
+ const unsigned kSigStackSize = std::max<unsigned>(16384, SIGSTKSZ);

trholding Oct 23, 2022
Author

pytorch/pytorch#65824 (comment)

file: ir_emitter.cpp line: 1678
-              << " elements, which were unified to " << candidate->repr_str();
+              << " elements, which were unified to " << (*unified_value_type)->repr_str();

trholding Oct 24, 2022
Author

Note: must be built without tests enabled to avoid build failure.

artyom-beilis · 2022-10-23T18:27:41Z

artyom-beilis
Oct 23, 2022
Maintainer

I need to start working on integration with torch 1.12 ASAP - it should allow using backend without changes in the torch code itself.

4 replies

trholding Oct 23, 2022
Author

That sounds good!

trholding Oct 24, 2022
Author

When are you planning to start, would be willing to try

trholding Oct 24, 2022
Author

@artyom-beilis I have some questions, may seem dumb but I am new to this and I am interested in accelerating inference on AMD APUs, ARM CPUs with GPU and low end devices. I also don't like that the whole AI world is dependent on NVIDIA CUDA and its closed drivers - that's not open access to accelerated computing!

I was able to compile your version of pytorch and install after lots of trial and error. Does the following config look good? I am asking because of this: -- USE_OPENCL : OFF

Also heard that vulkan is less overhead than opencl and the next big thing, apparently there is support as I see from this flag: -- USE_VULKAN : OFF do you use VULKAN acceleration too?

I also disabled:

--   USE_CUDA              : 0
--   USE_ROCM              : 0

As I don't have need for it in the AMD APU system. clinfo shows no problems, radeontop also works, vulkan benchmarks also work, confirmed that OpenCL examples and kernels actually run on GPU. I don't think or remember that ROCM was used on the APU system to get opencl to work. Hope disabling the above was fine. I'll continue building pytorch_dlprim and dlprimitives and also test some pytorch inference code and get back to you with results. I'll also get back on your dlprimitives accelerated caffe after I figure out how to convert pytorch models. May take a day or two to complete.

Please have a look at this config summary to see if there are any problems:

-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.24.2
--   CMake command         : /usr/bin/cmake
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler id       : GNU
--   C++ compiler version  : 12.2.0
--   Using ccache if found : ON
--   Found ccache          : /usr/bin/ccache
--   CXX flags             :  -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow
--   Build type            : Release
--   Compile definitions   : ONNX_ML=1;ONNXIFI_ENABLE_EXT=1;ONNX_NAMESPACE=onnx_torch;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1;USE_EXTERNAL_MZCRC;MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS
--   CMAKE_PREFIX_PATH     : /usr/lib/python3.10/site-packages
--   CMAKE_INSTALL_PREFIX  : /home/skunkwork/experiments/pytorch/torch
--   USE_GOLD_LINKER       : OFF
-- 
--   TORCH_VERSION         : 1.11.0
--   CAFFE2_VERSION        : 1.11.0
--   BUILD_CAFFE2          : ON
--   BUILD_CAFFE2_OPS      : ON
--   BUILD_CAFFE2_MOBILE   : OFF
--   BUILD_STATIC_RUNTIME_BENCHMARK: OFF
--   BUILD_TENSOREXPR_BENCHMARK: OFF
--   BUILD_NVFUSER_BENCHMARK: OFF
--   BUILD_BINARY          : OFF
--   BUILD_CUSTOM_PROTOBUF : ON
--     Link local protobuf : ON
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : True
--     Python version      : 3.10.8
--     Python executable   : /usr/bin/python
--     Pythonlibs version  : 3.10.8
--     Python library      : /usr/lib/libpython3.10.so.1.0
--     Python includes     : /usr/include/python3.10
--     Python site-packages: lib/python3.10/site-packages
--   BUILD_SHARED_LIBS     : ON
--   CAFFE2_USE_MSVC_STATIC_RUNTIME     : OFF
--   BUILD_TEST            : False
--   BUILD_JNI             : OFF
--   BUILD_MOBILE_AUTOGRAD : OFF
--   BUILD_LITE_INTERPRETER: OFF
--   INTERN_BUILD_MOBILE   : 
--   USE_BLAS              : 1
--     BLAS                : generic
--   USE_LAPACK            : 1
--     LAPACK              : generic
--   USE_ASAN              : OFF
--   USE_CPP_CODE_COVERAGE : OFF
--   USE_CUDA              : 0
--   USE_ROCM              : 0
--   USE_EIGEN_FOR_BLAS    : ON
--   USE_FBGEMM            : ON
--     USE_FAKELOWP          : OFF
--   USE_KINETO            : ON
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : OFF
--   USE_GLOG              : OFF
--   USE_LEVELDB           : OFF
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : OFF
--   USE_METAL             : OFF
--   USE_PYTORCH_METAL     : OFF
--   USE_PYTORCH_METAL_EXPORT     : OFF
--   USE_FFTW              : ON
--   USE_MKL               : OFF
--   USE_MKLDNN            : ON
--   USE_MKLDNN_ACL        : OFF
--   USE_MKLDNN_CBLAS      : OFF
--   USE_NCCL              : OFF
--   USE_NNPACK            : ON
--   USE_NUMPY             : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCL            : OFF
--   USE_OPENCV            : OFF
--   USE_OPENMP            : ON
--   USE_TBB               : OFF
--   USE_VULKAN            : OFF
--   USE_PROF              : OFF
--   USE_QNNPACK           : ON
--   USE_PYTORCH_QNNPACK   : ON
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF
--   USE_DISTRIBUTED       : ON
--     USE_MPI               : ON
--     USE_GLOO              : ON
--     USE_GLOO_WITH_OPENSSL : OFF
--     USE_TENSORPIPE        : ON
--   USE_DEPLOY           : OFF
--   USE_BREAKPAD         : ON
--   Public Dependencies  : Threads::Threads;caffe2::mkldnn
--   Private Dependencies : pthreadpool;cpuinfo;qnnpack;pytorch_qnnpack;nnpack;XNNPACK;fbgemm;/usr/lib/libnuma.so;fp16;/usr/lib/libmpi_cxx.so;/usr/lib/libmpi.so;gloo;tensorpipe;aten_op_header_gen;foxi_loader;rt;fmt::fmt-header-only;kineto;gcc_s;gcc;dl
--   USE_COREML_DELEGATE     : OFF
-- Configuring done
-- Generating done

trholding Oct 24, 2022
Author

Also another question:

There is this awesome project https://github.com/ggerganov/whisper.cpp which is a port of OpenAI's Whisper Speech to Text to C++ and CPU, really works well. It uses the authors own C++ tensor library https://github.com/ggerganov/ggml which is in the line of Fabrice Bellard's LibNC https://bellard.org/libnc/ (unfortunately LibNC is closed source) . I have asked the author of ggml if OpenCL acceleration could be supported, but it seems that it is a great deal of work to support OpenCL from scratch. What I am thinking is that using dlprimitives in ggml could enable OpenCL support in a easier way, but I really do not know if that is the case. Can you look at https://github.com/ggerganov/ggml to see if this could be feasible if you have the time / interest? It could be another great use case for both projects involved. I'l be glad to support in testing and QA and some code fixes.

artyom-beilis · 2022-10-24T20:21:56Z

artyom-beilis
Oct 24, 2022
Maintainer

USE_OPENCL : OFF

Yes this is right. Built in OpenCL does not work it was more like placeholder.

Now once I integrate with 1.12 that provides support of private devices you'll be able just to use 'prinvatedeice1' or something like that need to test - with vanilla pytorch. It would simplify things

0 replies

artyom-beilis · 2022-10-24T20:23:25Z

artyom-beilis
Oct 24, 2022
Maintainer

I am interested in accelerating inference on AMD APUs, ARM CPUs with GPU and low end devices

DLPrim provides initial basic ONNX support. It tested on set of standard vision models (still limited but works) so I thing it would be much better since ONNX allows to disconnect inference from training.

1 reply

trholding Oct 25, 2022
Author

I've tested your tools, still new to neural networks and stuff. But pretty impressive so far even on older hardware and non NVIDIA. Yes ONNX seems to be a good standard and supported by many vendors - I think your choice was spot on.

artyom-beilis · 2022-10-24T20:26:29Z

artyom-beilis
Oct 24, 2022
Maintainer

There is this awesome project https://github.com/ggerganov/whisper.cpp which is a port of OpenAI's Whisper Speech to Text to C++ and CPU, really works well.

Not familiar with it.

What I am thinking is that using dlprimitives in ggml could enable OpenCL support in a easier way, but I really do not know if that is the case.

Give it a shut.

Currently my general directions are:

Improve pytorch support for training
Improve ONNX support in dlprimitives for inference

3 replies

trholding Oct 25, 2022
Author

Ack, I'll give it a try.

pytorch support is way more important. I wonder why the original authors of pytorch refuse to support opencl and vulkan which are more open and accessible! It's as if NVIDIA has a hand in it.

artyom-beilis Oct 25, 2022
Maintainer

pytorch refuse to support opencl a

Actually I'm getting lots of help from pytorch developers - without their advice I couldn't get here. The support of out-of-tree backend is fantastic thing and it is improved in 1.12 so currently doing out-of-tree backend is the way to go.

However today the bread and butter of the industry is nVidia - so it is hard to break it. Since finally all who work on pytorch are getting their salaries from a company that needs working product the CUDA/nVidia support is much more critical.

Also note that if you are AMD Linux user you probably may give ROCm - also it is not clear if older GPUs are supported at all.

artyom-beilis Oct 25, 2022
Maintainer

On the other hand ONNX OpenCL support that goes to end user may be actually much easier to break in - since OpenCL drivers are installed everywhere and it does not matter what kind of GPU it runs. And this is actually HUGE headache for the industry - having unified inference.

But of course pytorch support is as important as ONNX.

artyom-beilis · 2022-10-24T20:28:45Z

artyom-beilis
Oct 24, 2022
Maintainer

I'll also get back on your dlprimitives accelerated caffe after I figure out how to convert pytorch models.

Unfortunately CAFFE is dead. I like it a lot especially for its simple and easy to use code base but it isn't for up-to date standards especially in memory management. However it still provides complete Caffe + OpenCL implementation that works quite nicely.

2 replies

trholding Oct 25, 2022
Author

I also like Caffe cos it seemed to be a more compact and straight forward way to make use of models in C/C++ applications.

"This website is being deprecated - Caffe2 is now a part of PyTorch.
While the APIs will continue to work, we encourage you to use the PyTorch APIs."

I do not know what the above means, but I'm sure with that OpenCL support is gone.

artyom-beilis Oct 25, 2022
Maintainer

I also like Caffe cos it seemed to be a more compact and straight forward way to make use of models in C/C++ applications.

Unfortunately the development is stopped even some critical patches like support of CUDA11/CUDNN8 I prepare for Caffe were not merged/ignored - nobody maintains it. Also of course if Caffe gives enough features it is very nice and I like it.

Another big problem of Caffe is memory usage - using pytorch or ONNX I can do inference of Resnet50 and 4GB card using huge batch like 64, in caffe it is impossible due to memory model deficiency.

So I still use Caffe but all future developments are now pytorch.

artyom-beilis · 2022-10-26T21:41:02Z

artyom-beilis
Oct 26, 2022
Maintainer

I started working with pytorch nighly and you don't need to build pytorch any more, see true_out_of_tree_support branch

It is early and some nets fail, but this is the direction.

26 replies

artyom-beilis Oct 29, 2022
Maintainer

After installing MKL also I am getting the same error. Still figuring out the fix this linux distro.

Are you installing from pip? Are you installing CPU version? I tested both 1.13 (with privateuseone device) and 1.14 and I don't see any dependency on mkl? What is your environment? Try to find mkl in CMakeCache.txt to see where it comes from.

trholding Oct 30, 2022
Author

I installed torch from pip
MKL was already there which was also from pip and I think it was auto installed
I also installed https://archlinux.org/packages/community/x86_64/intel-oneapi-mkl/ after I got the build error
Yes I use the CPU version Ref: https://pytorch.org/get-started/locally/

pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu

I got version 1.14
System: Arch Linux 5.19.10-arch1-1 #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux
Env looks fine. Not pasting here.

CMake Cache output:

cat CMakeCache.txt | grep -i mkl
//The directory containing a CMake configuration file for MKLDNN.
MKLDNN_DIR:PATH=MKLDNN_DIR-NOTFOUND
//Choose MKL_ARCH options are: ia32;intel64
MKL_ARCH:STRING=intel64
//The directory containing a CMake configuration file for MKL.
MKL_DIR:PATH=/home/skunkwork/.local/lib/cmake/mkl
MKL_H:PATH=/home/skunkwork/.local/include
//Choose MKL_INTERFACE_FULL options are: intel_ilp64;intel_lp64
MKL_INTERFACE_FULL:STRING=intel_ilp64
//Choose MKL_LINK options are: static;dynamic;sdl
MKL_LINK:STRING=dynamic
//Choose MKL_MPI options are: intelmpi;openmpi;mpich;mpich2
MKL_MPI:STRING=intelmpi
//Choose MKL_THREADING options are: sequential;intel_thread;tbb_thread;gnu_thread
MKL_THREADING:STRING=intel_thread
mkl_core_file:FILEPATH=mkl_core_file-NOTFOUND
mkl_intel_ilp64_file:FILEPATH=mkl_intel_ilp64_file-NOTFOUND
mkl_intel_thread_file:FILEPATH=mkl_intel_thread_file-NOTFOUND

I think this is induced by the CMake from torch (site-packages/torch/share/cmake/Torch), I'll try an upgraded nightly tonight and see if this goes away.

cat ~/.local/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake | grep -i mkl
    append_torchlib_if_found(dnnl mkldnn)

trholding Oct 30, 2022
Author

Okay I am stuck. It seems it has to do with torch.

I found a similar issue here: microsoft/onnxruntime#9170

The last thing I tried was:

$ export MKL_ROOT=/opt/intel/mkl
$ export MKLROOT=/opt/intel/mkl
$ cmake -DUSE_INTERNAL_BLAS=ON -$ DCMAKE_PREFIX_PATH=/home/skunkwork/.local/lib/python3.10/site-packages/torch/share/cmake/Torch ..

The cmake output was different, but the final make / build failed with same error.

-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
CMake Warning (dev) at /home/skunkwork/.local/lib/python3.10/site-packages/torch/share/cmake/Caffe2/public/mkl.cmake:1 (find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  Environment variable MKL_ROOT is set to:

    /opt/intel/mkl

  For compatibility, CMake is ignoring the variable.
Call Stack (most recent call first):
  /home/skunkwork/.local/lib/python3.10/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:113 (include)
  /home/skunkwork/.local/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:4 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- MKL_ARCH: None, set to ` intel64` by default
-- MKL_LINK: None, set to ` dynamic` by default
-- MKL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_THREADING: None, set to ` intel_thread` by default
-- MKL_MPI: None, set to ` intelmpi` by default
CMake Warning at /home/skunkwork/.local/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /home/skunkwork/.local/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
  CMakeLists.txt:4 (find_package)

To fix this:

Environment variable MKL_ROOT is set to:
/opt/intel/mkl
For compatibility, CMake is ignoring the variable.

I added cmake_policy(SET CMP0074 NEW) to CMakeLists.txt

And the result is the same.

I think I've to figure out a manual way to link.

artyom-beilis Oct 30, 2022
Maintainer

It looks like it finds your local installation of mkl and asks to use it.

The the TorchConfig there is a line

if(ON)
   append_torchlib_if_found(dnnl mkldnn)
endif()

Try to change it to OFF or to

append_torchlib_if_found(dnnl)

Without mkldnn - since I don't use mkl in the code so no need to link with it,

trholding Oct 30, 2022
Author

Thank you so much. Was away. Will do that.

artyom-beilis · 2022-10-29T19:47:38Z

artyom-beilis
Oct 29, 2022
Maintainer

Now today or yesterday pytorch 1.13 was released and it is the version you can use (and 1.14 nighty as well)

The difference that in 1.13 you must refer to opencl device by name as privateuseone while 1.14 introduced the torch.utils.rename_privateuse1_backend function and you can give a better name like ocl - don't use opencl as it is reserverd

2 replies

trholding Oct 30, 2022
Author

Thanks for the build help. It worked. I am so happy!!!

Here are the results:

$ python mnist.py --device ocl:0
Accessing device #0:STONEY (stoney, LLVM 14.0.6, DRM 3.48, 6.0.5-arch1-1) on Clover
Train Epoch: 1 [0/60000 (0%)]   Loss: 2.326378
Train Epoch: 1 [640/60000 (1%)] Loss: 1.373418
Train Epoch: 1 [1280/60000 (2%)]        Loss: 0.674242
Train Epoch: 1 [1920/60000 (3%)]        Loss: 0.342724
Train Epoch: 1 [2560/60000 (4%)]        Loss: 0.282552
Train Epoch: 1 [3200/60000 (5%)]        Loss: 0.321669
Train Epoch: 1 [3840/60000 (6%)]        Loss: 0.117674
Train Epoch: 1 [4480/60000 (7%)]        Loss: 0.175405
Train Epoch: 1 [5120/60000 (9%)]        Loss: 0.294237
Train Epoch: 1 [5760/60000 (10%)]       Loss: 0.177331
Train Epoch: 1 [6400/60000 (11%)]       Loss: 0.147656
Train Epoch: 1 [7040/60000 (12%)]       Loss: 0.247967
Train Epoch: 1 [7680/60000 (13%)]       Loss: 0.097331
Train Epoch: 1 [8320/60000 (14%)]       Loss: 0.171668
Train Epoch: 1 [8960/60000 (15%)]       Loss: 0.100159
Train Epoch: 1 [9600/60000 (16%)]       Loss: 0.179712
Train Epoch: 1 [10240/60000 (17%)]      Loss: 0.098589
Train Epoch: 1 [10880/60000 (18%)]      Loss: 0.092788
Train Epoch: 1 [11520/60000 (19%)]      Loss: 0.056874
Train Epoch: 1 [12160/60000 (20%)]      Loss: 0.083909
Train Epoch: 1 [12800/60000 (21%)]      Loss: 0.139126
Train Epoch: 1 [13440/60000 (22%)]      Loss: 0.126386
Train Epoch: 1 [14080/60000 (23%)]      Loss: 0.037617
Train Epoch: 1 [14720/60000 (25%)]      Loss: 0.160326
Train Epoch: 1 [15360/60000 (26%)]      Loss: 0.078383
Train Epoch: 1 [16000/60000 (27%)]      Loss: 0.043570
Train Epoch: 1 [16640/60000 (28%)]      Loss: 0.145933
Train Epoch: 1 [17280/60000 (29%)]      Loss: 0.050387
Train Epoch: 1 [17920/60000 (30%)]      Loss: 0.054103
Train Epoch: 1 [18560/60000 (31%)]      Loss: 0.021652
Train Epoch: 1 [19200/60000 (32%)]      Loss: 0.035223
Train Epoch: 1 [19840/60000 (33%)]      Loss: 0.013079
Train Epoch: 1 [20480/60000 (34%)]      Loss: 0.027577
Train Epoch: 1 [21120/60000 (35%)]      Loss: 0.028688
Train Epoch: 1 [21760/60000 (36%)]      Loss: 0.132288
Train Epoch: 1 [22400/60000 (37%)]      Loss: 0.190936
Train Epoch: 1 [23040/60000 (38%)]      Loss: 0.075352
Train Epoch: 1 [23680/60000 (39%)]      Loss: 0.127283
Train Epoch: 1 [24320/60000 (41%)]      Loss: 0.102158
Train Epoch: 1 [24960/60000 (42%)]      Loss: 0.010638
Train Epoch: 1 [25600/60000 (43%)]      Loss: 0.030593
Train Epoch: 1 [26240/60000 (44%)]      Loss: 0.065338
Train Epoch: 1 [26880/60000 (45%)]      Loss: 0.105478
Train Epoch: 1 [27520/60000 (46%)]      Loss: 0.030327
Train Epoch: 1 [28160/60000 (47%)]      Loss: 0.042820
Train Epoch: 1 [28800/60000 (48%)]      Loss: 0.023040
Train Epoch: 1 [29440/60000 (49%)]      Loss: 0.063582
Train Epoch: 1 [30080/60000 (50%)]      Loss: 0.082858
Train Epoch: 1 [30720/60000 (51%)]      Loss: 0.076143
Train Epoch: 1 [31360/60000 (52%)]      Loss: 0.088471
Train Epoch: 1 [32000/60000 (53%)]      Loss: 0.103620
Train Epoch: 1 [32640/60000 (54%)]      Loss: 0.016941
Train Epoch: 1 [33280/60000 (55%)]      Loss: 0.050557
Train Epoch: 1 [33920/60000 (57%)]      Loss: 0.071508
Train Epoch: 1 [34560/60000 (58%)]      Loss: 0.093433
Train Epoch: 1 [35200/60000 (59%)]      Loss: 0.132329
Train Epoch: 1 [35840/60000 (60%)]      Loss: 0.073337
Train Epoch: 1 [36480/60000 (61%)]      Loss: 0.007668
Train Epoch: 1 [37120/60000 (62%)]      Loss: 0.111842
Train Epoch: 1 [37760/60000 (63%)]      Loss: 0.036615
Train Epoch: 1 [38400/60000 (64%)]      Loss: 0.011528
Train Epoch: 1 [39040/60000 (65%)]      Loss: 0.062670
Train Epoch: 1 [39680/60000 (66%)]      Loss: 0.047413
Train Epoch: 1 [40320/60000 (67%)]      Loss: 0.121324
Train Epoch: 1 [40960/60000 (68%)]      Loss: 0.099681
Train Epoch: 1 [41600/60000 (69%)]      Loss: 0.064086
Train Epoch: 1 [42240/60000 (70%)]      Loss: 0.054643
Train Epoch: 1 [42880/60000 (71%)]      Loss: 0.031469
Train Epoch: 1 [43520/60000 (72%)]      Loss: 0.005572
Train Epoch: 1 [44160/60000 (74%)]      Loss: 0.051205
Train Epoch: 1 [44800/60000 (75%)]      Loss: 0.124627
Train Epoch: 1 [45440/60000 (76%)]      Loss: 0.008519
Train Epoch: 1 [46080/60000 (77%)]      Loss: 0.040237
Train Epoch: 1 [46720/60000 (78%)]      Loss: 0.077708
Train Epoch: 1 [47360/60000 (79%)]      Loss: 0.009306
Train Epoch: 1 [48000/60000 (80%)]      Loss: 0.250737
Train Epoch: 1 [48640/60000 (81%)]      Loss: 0.091779
Train Epoch: 1 [49280/60000 (82%)]      Loss: 0.014412
Train Epoch: 1 [49920/60000 (83%)]      Loss: 0.032865
Train Epoch: 1 [50560/60000 (84%)]      Loss: 0.019158
Train Epoch: 1 [51200/60000 (85%)]      Loss: 0.042707
Train Epoch: 1 [51840/60000 (86%)]      Loss: 0.010261
Train Epoch: 1 [52480/60000 (87%)]      Loss: 0.087193
Train Epoch: 1 [53120/60000 (88%)]      Loss: 0.041311
Train Epoch: 1 [53760/60000 (90%)]      Loss: 0.080190
Train Epoch: 1 [54400/60000 (91%)]      Loss: 0.015097
Train Epoch: 1 [55040/60000 (92%)]      Loss: 0.006451
Train Epoch: 1 [55680/60000 (93%)]      Loss: 0.019437
Train Epoch: 1 [56320/60000 (94%)]      Loss: 0.163386
Train Epoch: 1 [56960/60000 (95%)]      Loss: 0.034762
Train Epoch: 1 [57600/60000 (96%)]      Loss: 0.120978
Train Epoch: 1 [58240/60000 (97%)]      Loss: 0.019372
Train Epoch: 1 [58880/60000 (98%)]      Loss: 0.010469
Train Epoch: 1 [59520/60000 (99%)]      Loss: 0.051015
Epoch in  66.0s

Test set: Average loss: 0.0508, Accuracy: 9805/10000 (98%)

Train Epoch: 2 [0/60000 (0%)]   Loss: 0.060654
Train Epoch: 2 [640/60000 (1%)] Loss: 0.041666
Train Epoch: 2 [1280/60000 (2%)]        Loss: 0.035993
Train Epoch: 2 [1920/60000 (3%)]        Loss: 0.051420
Train Epoch: 2 [2560/60000 (4%)]        Loss: 0.096240
Train Epoch: 2 [3200/60000 (5%)]        Loss: 0.055887
Train Epoch: 2 [3840/60000 (6%)]        Loss: 0.025478
Train Epoch: 2 [4480/60000 (7%)]        Loss: 0.081983
Train Epoch: 2 [5120/60000 (9%)]        Loss: 0.010014
Train Epoch: 2 [5760/60000 (10%)]       Loss: 0.013485
Train Epoch: 2 [6400/60000 (11%)]       Loss: 0.032643
Train Epoch: 2 [7040/60000 (12%)]       Loss: 0.047428
Train Epoch: 2 [7680/60000 (13%)]       Loss: 0.026647
Train Epoch: 2 [8320/60000 (14%)]       Loss: 0.008917
Train Epoch: 2 [8960/60000 (15%)]       Loss: 0.008614
Train Epoch: 2 [9600/60000 (16%)]       Loss: 0.008884
Train Epoch: 2 [10240/60000 (17%)]      Loss: 0.035276
Train Epoch: 2 [10880/60000 (18%)]      Loss: 0.123259
Train Epoch: 2 [11520/60000 (19%)]      Loss: 0.015595
Train Epoch: 2 [12160/60000 (20%)]      Loss: 0.050691
Train Epoch: 2 [12800/60000 (21%)]      Loss: 0.047752
Train Epoch: 2 [13440/60000 (22%)]      Loss: 0.008601
Train Epoch: 2 [14080/60000 (23%)]      Loss: 0.004042
Train Epoch: 2 [14720/60000 (25%)]      Loss: 0.004957
Train Epoch: 2 [15360/60000 (26%)]      Loss: 0.039756
Train Epoch: 2 [16000/60000 (27%)]      Loss: 0.091472
Train Epoch: 2 [16640/60000 (28%)]      Loss: 0.064199
Train Epoch: 2 [17280/60000 (29%)]      Loss: 0.008498
Train Epoch: 2 [17920/60000 (30%)]      Loss: 0.050529
Train Epoch: 2 [18560/60000 (31%)]      Loss: 0.003090
Train Epoch: 2 [19200/60000 (32%)]      Loss: 0.010912
Train Epoch: 2 [19840/60000 (33%)]      Loss: 0.004400
Train Epoch: 2 [20480/60000 (34%)]      Loss: 0.007175
Train Epoch: 2 [21120/60000 (35%)]      Loss: 0.004632
Train Epoch: 2 [21760/60000 (36%)]      Loss: 0.008872
Train Epoch: 2 [22400/60000 (37%)]      Loss: 0.137297
Train Epoch: 2 [23040/60000 (38%)]      Loss: 0.005859
Train Epoch: 2 [23680/60000 (39%)]      Loss: 0.022960
Train Epoch: 2 [24320/60000 (41%)]      Loss: 0.010931
Train Epoch: 2 [24960/60000 (42%)]      Loss: 0.023478
Train Epoch: 2 [25600/60000 (43%)]      Loss: 0.069565
Train Epoch: 2 [26240/60000 (44%)]      Loss: 0.020921
Train Epoch: 2 [26880/60000 (45%)]      Loss: 0.085850
Train Epoch: 2 [27520/60000 (46%)]      Loss: 0.009113
Train Epoch: 2 [28160/60000 (47%)]      Loss: 0.001644
Train Epoch: 2 [28800/60000 (48%)]      Loss: 0.006020
Train Epoch: 2 [29440/60000 (49%)]      Loss: 0.010898
Train Epoch: 2 [30080/60000 (50%)]      Loss: 0.000443
Train Epoch: 2 [30720/60000 (51%)]      Loss: 0.203181
Train Epoch: 2 [31360/60000 (52%)]      Loss: 0.058979
Train Epoch: 2 [32000/60000 (53%)]      Loss: 0.021109
Train Epoch: 2 [32640/60000 (54%)]      Loss: 0.036988
Train Epoch: 2 [33280/60000 (55%)]      Loss: 0.022490
Train Epoch: 2 [33920/60000 (57%)]      Loss: 0.032740
Train Epoch: 2 [34560/60000 (58%)]      Loss: 0.005683
Train Epoch: 2 [35200/60000 (59%)]      Loss: 0.033092
Train Epoch: 2 [35840/60000 (60%)]      Loss: 0.022096
Train Epoch: 2 [36480/60000 (61%)]      Loss: 0.167969
Train Epoch: 2 [37120/60000 (62%)]      Loss: 0.025452
Train Epoch: 2 [37760/60000 (63%)]      Loss: 0.034627
Train Epoch: 2 [38400/60000 (64%)]      Loss: 0.009330
Train Epoch: 2 [39040/60000 (65%)]      Loss: 0.003391
Train Epoch: 2 [39680/60000 (66%)]      Loss: 0.001685
Train Epoch: 2 [40320/60000 (67%)]      Loss: 0.005397
Train Epoch: 2 [40960/60000 (68%)]      Loss: 0.013949
Train Epoch: 2 [41600/60000 (69%)]      Loss: 0.109206
Train Epoch: 2 [42240/60000 (70%)]      Loss: 0.000649
Train Epoch: 2 [42880/60000 (71%)]      Loss: 0.001867
Train Epoch: 2 [43520/60000 (72%)]      Loss: 0.014802
Train Epoch: 2 [44160/60000 (74%)]      Loss: 0.014424
Train Epoch: 2 [44800/60000 (75%)]      Loss: 0.070203
Train Epoch: 2 [45440/60000 (76%)]      Loss: 0.022417
Train Epoch: 2 [46080/60000 (77%)]      Loss: 0.005601
Train Epoch: 2 [46720/60000 (78%)]      Loss: 0.006165
Train Epoch: 2 [47360/60000 (79%)]      Loss: 0.109785
Train Epoch: 2 [48000/60000 (80%)]      Loss: 0.024693
Train Epoch: 2 [48640/60000 (81%)]      Loss: 0.051214
Train Epoch: 2 [49280/60000 (82%)]      Loss: 0.053793
Train Epoch: 2 [49920/60000 (83%)]      Loss: 0.031350
Train Epoch: 2 [50560/60000 (84%)]      Loss: 0.044368
Train Epoch: 2 [51200/60000 (85%)]      Loss: 0.114331
Train Epoch: 2 [51840/60000 (86%)]      Loss: 0.012510
Train Epoch: 2 [52480/60000 (87%)]      Loss: 0.010729
Train Epoch: 2 [53120/60000 (88%)]      Loss: 0.006735
Train Epoch: 2 [53760/60000 (90%)]      Loss: 0.003996
Train Epoch: 2 [54400/60000 (91%)]      Loss: 0.027359
Train Epoch: 2 [55040/60000 (92%)]      Loss: 0.008420
Train Epoch: 2 [55680/60000 (93%)]      Loss: 0.026319
Train Epoch: 2 [56320/60000 (94%)]      Loss: 0.021779
Train Epoch: 2 [56960/60000 (95%)]      Loss: 0.002282
Train Epoch: 2 [57600/60000 (96%)]      Loss: 0.039314
Train Epoch: 2 [58240/60000 (97%)]      Loss: 0.015478
Train Epoch: 2 [58880/60000 (98%)]      Loss: 0.024258
Train Epoch: 2 [59520/60000 (99%)]      Loss: 0.112566
Epoch in  64.5s

Test set: Average loss: 0.0354, Accuracy: 9892/10000 (99%)

Train Epoch: 3 [0/60000 (0%)]   Loss: 0.020422
Train Epoch: 3 [640/60000 (1%)] Loss: 0.005346
Train Epoch: 3 [1280/60000 (2%)]        Loss: 0.072615
Train Epoch: 3 [1920/60000 (3%)]        Loss: 0.061316
Train Epoch: 3 [2560/60000 (4%)]        Loss: 0.023169
Train Epoch: 3 [3200/60000 (5%)]        Loss: 0.001526
Train Epoch: 3 [3840/60000 (6%)]        Loss: 0.008637
Train Epoch: 3 [4480/60000 (7%)]        Loss: 0.009571
Train Epoch: 3 [5120/60000 (9%)]        Loss: 0.010601
Train Epoch: 3 [5760/60000 (10%)]       Loss: 0.067344
Train Epoch: 3 [6400/60000 (11%)]       Loss: 0.003945
Train Epoch: 3 [7040/60000 (12%)]       Loss: 0.005526
Train Epoch: 3 [7680/60000 (13%)]       Loss: 0.011736
Train Epoch: 3 [8320/60000 (14%)]       Loss: 0.050748
Train Epoch: 3 [8960/60000 (15%)]       Loss: 0.009006
Train Epoch: 3 [9600/60000 (16%)]       Loss: 0.006108
Train Epoch: 3 [10240/60000 (17%)]      Loss: 0.003086
Train Epoch: 3 [10880/60000 (18%)]      Loss: 0.032249
Train Epoch: 3 [11520/60000 (19%)]      Loss: 0.002977
Train Epoch: 3 [12160/60000 (20%)]      Loss: 0.006912
Train Epoch: 3 [12800/60000 (21%)]      Loss: 0.132079
Train Epoch: 3 [13440/60000 (22%)]      Loss: 0.002056
Train Epoch: 3 [14080/60000 (23%)]      Loss: 0.011832
Train Epoch: 3 [14720/60000 (25%)]      Loss: 0.009438
Train Epoch: 3 [15360/60000 (26%)]      Loss: 0.023943
Train Epoch: 3 [16000/60000 (27%)]      Loss: 0.003752
Train Epoch: 3 [16640/60000 (28%)]      Loss: 0.003679
Train Epoch: 3 [17280/60000 (29%)]      Loss: 0.008968
Train Epoch: 3 [17920/60000 (30%)]      Loss: 0.011817
Train Epoch: 3 [18560/60000 (31%)]      Loss: 0.035943
Train Epoch: 3 [19200/60000 (32%)]      Loss: 0.045714
Train Epoch: 3 [19840/60000 (33%)]      Loss: 0.017717
Train Epoch: 3 [20480/60000 (34%)]      Loss: 0.001321
Train Epoch: 3 [21120/60000 (35%)]      Loss: 0.072993
Train Epoch: 3 [21760/60000 (36%)]      Loss: 0.004564
Train Epoch: 3 [22400/60000 (37%)]      Loss: 0.002843
Train Epoch: 3 [23040/60000 (38%)]      Loss: 0.007740
Train Epoch: 3 [23680/60000 (39%)]      Loss: 0.008501
Train Epoch: 3 [24320/60000 (41%)]      Loss: 0.002646
Train Epoch: 3 [24960/60000 (42%)]      Loss: 0.024915
Train Epoch: 3 [25600/60000 (43%)]      Loss: 0.005951
Train Epoch: 3 [26240/60000 (44%)]      Loss: 0.046029
Train Epoch: 3 [26880/60000 (45%)]      Loss: 0.000814
Train Epoch: 3 [27520/60000 (46%)]      Loss: 0.009131
Train Epoch: 3 [28160/60000 (47%)]      Loss: 0.052000
Train Epoch: 3 [28800/60000 (48%)]      Loss: 0.016432
Train Epoch: 3 [29440/60000 (49%)]      Loss: 0.011815
Train Epoch: 3 [30080/60000 (50%)]      Loss: 0.002107
Train Epoch: 3 [30720/60000 (51%)]      Loss: 0.003562
Train Epoch: 3 [31360/60000 (52%)]      Loss: 0.012214
Train Epoch: 3 [32000/60000 (53%)]      Loss: 0.101903
Train Epoch: 3 [32640/60000 (54%)]      Loss: 0.001532
Train Epoch: 3 [33280/60000 (55%)]      Loss: 0.062838
Train Epoch: 3 [33920/60000 (57%)]      Loss: 0.010577
Train Epoch: 3 [34560/60000 (58%)]      Loss: 0.025925
Train Epoch: 3 [35200/60000 (59%)]      Loss: 0.004507
Train Epoch: 3 [35840/60000 (60%)]      Loss: 0.004899
Train Epoch: 3 [36480/60000 (61%)]      Loss: 0.004373
Train Epoch: 3 [37120/60000 (62%)]      Loss: 0.008094
Train Epoch: 3 [37760/60000 (63%)]      Loss: 0.083419
Train Epoch: 3 [38400/60000 (64%)]      Loss: 0.061206
Train Epoch: 3 [39040/60000 (65%)]      Loss: 0.031591
Train Epoch: 3 [39680/60000 (66%)]      Loss: 0.004541
Train Epoch: 3 [40320/60000 (67%)]      Loss: 0.000478
Train Epoch: 3 [40960/60000 (68%)]      Loss: 0.004004
Train Epoch: 3 [41600/60000 (69%)]      Loss: 0.001429
Train Epoch: 3 [42240/60000 (70%)]      Loss: 0.007718
Train Epoch: 3 [42880/60000 (71%)]      Loss: 0.009427
Train Epoch: 3 [43520/60000 (72%)]      Loss: 0.037873
Train Epoch: 3 [44160/60000 (74%)]      Loss: 0.007009
Train Epoch: 3 [44800/60000 (75%)]      Loss: 0.001999
Train Epoch: 3 [45440/60000 (76%)]      Loss: 0.025647
Train Epoch: 3 [46080/60000 (77%)]      Loss: 0.004423
Train Epoch: 3 [46720/60000 (78%)]      Loss: 0.040767
Train Epoch: 3 [47360/60000 (79%)]      Loss: 0.005674
Train Epoch: 3 [48000/60000 (80%)]      Loss: 0.003527
Train Epoch: 3 [48640/60000 (81%)]      Loss: 0.005910
Train Epoch: 3 [49280/60000 (82%)]      Loss: 0.018807
Train Epoch: 3 [49920/60000 (83%)]      Loss: 0.004369
Train Epoch: 3 [50560/60000 (84%)]      Loss: 0.007872
Train Epoch: 3 [51200/60000 (85%)]      Loss: 0.106953
Train Epoch: 3 [51840/60000 (86%)]      Loss: 0.029376
Train Epoch: 3 [52480/60000 (87%)]      Loss: 0.000561
Train Epoch: 3 [53120/60000 (88%)]      Loss: 0.011492
Train Epoch: 3 [53760/60000 (90%)]      Loss: 0.006430
Train Epoch: 3 [54400/60000 (91%)]      Loss: 0.002241
Train Epoch: 3 [55040/60000 (92%)]      Loss: 0.005440
Train Epoch: 3 [55680/60000 (93%)]      Loss: 0.015889
Train Epoch: 3 [56320/60000 (94%)]      Loss: 0.020121
Train Epoch: 3 [56960/60000 (95%)]      Loss: 0.017298
Train Epoch: 3 [57600/60000 (96%)]      Loss: 0.001261
Train Epoch: 3 [58240/60000 (97%)]      Loss: 0.006338
Train Epoch: 3 [58880/60000 (98%)]      Loss: 0.073202
Train Epoch: 3 [59520/60000 (99%)]      Loss: 0.002699
Epoch in  54.3s

Test set: Average loss: 0.0327, Accuracy: 9901/10000 (99%)

Train Epoch: 4 [0/60000 (0%)]   Loss: 0.136767
Train Epoch: 4 [640/60000 (1%)] Loss: 0.000869
Train Epoch: 4 [1280/60000 (2%)]        Loss: 0.013660
Train Epoch: 4 [1920/60000 (3%)]        Loss: 0.003311
Train Epoch: 4 [2560/60000 (4%)]        Loss: 0.033691
Train Epoch: 4 [3200/60000 (5%)]        Loss: 0.002912
Train Epoch: 4 [3840/60000 (6%)]        Loss: 0.001323
Train Epoch: 4 [4480/60000 (7%)]        Loss: 0.004146
Train Epoch: 4 [5120/60000 (9%)]        Loss: 0.007213
Train Epoch: 4 [5760/60000 (10%)]       Loss: 0.002190
Train Epoch: 4 [6400/60000 (11%)]       Loss: 0.010412
Train Epoch: 4 [7040/60000 (12%)]       Loss: 0.007548
Train Epoch: 4 [7680/60000 (13%)]       Loss: 0.010867
Train Epoch: 4 [8320/60000 (14%)]       Loss: 0.003366
Train Epoch: 4 [8960/60000 (15%)]       Loss: 0.002380
Train Epoch: 4 [9600/60000 (16%)]       Loss: 0.001422
Train Epoch: 4 [10240/60000 (17%)]      Loss: 0.000642
Train Epoch: 4 [10880/60000 (18%)]      Loss: 0.003760
Train Epoch: 4 [11520/60000 (19%)]      Loss: 0.066069
Train Epoch: 4 [12160/60000 (20%)]      Loss: 0.024731
Train Epoch: 4 [12800/60000 (21%)]      Loss: 0.003921
Train Epoch: 4 [13440/60000 (22%)]      Loss: 0.000390
Train Epoch: 4 [14080/60000 (23%)]      Loss: 0.009083
Train Epoch: 4 [14720/60000 (25%)]      Loss: 0.051092
Train Epoch: 4 [15360/60000 (26%)]      Loss: 0.001966
Train Epoch: 4 [16000/60000 (27%)]      Loss: 0.006489
Train Epoch: 4 [16640/60000 (28%)]      Loss: 0.000282
Train Epoch: 4 [17280/60000 (29%)]      Loss: 0.005493
Train Epoch: 4 [17920/60000 (30%)]      Loss: 0.007122
Train Epoch: 4 [18560/60000 (31%)]      Loss: 0.097360
Train Epoch: 4 [19200/60000 (32%)]      Loss: 0.000341
Train Epoch: 4 [19840/60000 (33%)]      Loss: 0.010498
Train Epoch: 4 [20480/60000 (34%)]      Loss: 0.002287
Train Epoch: 4 [21120/60000 (35%)]      Loss: 0.001108
Train Epoch: 4 [21760/60000 (36%)]      Loss: 0.018645
Train Epoch: 4 [22400/60000 (37%)]      Loss: 0.030110
Train Epoch: 4 [23040/60000 (38%)]      Loss: 0.001106
Train Epoch: 4 [23680/60000 (39%)]      Loss: 0.001947
Train Epoch: 4 [24320/60000 (41%)]      Loss: 0.029748
Train Epoch: 4 [24960/60000 (42%)]      Loss: 0.007403
Train Epoch: 4 [25600/60000 (43%)]      Loss: 0.005610
Train Epoch: 4 [26240/60000 (44%)]      Loss: 0.000991
Train Epoch: 4 [26880/60000 (45%)]      Loss: 0.006730
Train Epoch: 4 [27520/60000 (46%)]      Loss: 0.000933
Train Epoch: 4 [28160/60000 (47%)]      Loss: 0.000725
Train Epoch: 4 [28800/60000 (48%)]      Loss: 0.028049
Train Epoch: 4 [29440/60000 (49%)]      Loss: 0.000959
Train Epoch: 4 [30080/60000 (50%)]      Loss: 0.014685
Train Epoch: 4 [30720/60000 (51%)]      Loss: 0.047288
Train Epoch: 4 [31360/60000 (52%)]      Loss: 0.002935
Train Epoch: 4 [32000/60000 (53%)]      Loss: 0.004192
Train Epoch: 4 [32640/60000 (54%)]      Loss: 0.007309
Train Epoch: 4 [33280/60000 (55%)]      Loss: 0.026882
Train Epoch: 4 [33920/60000 (57%)]      Loss: 0.010468
Train Epoch: 4 [34560/60000 (58%)]      Loss: 0.005188
Train Epoch: 4 [35200/60000 (59%)]      Loss: 0.002399
Train Epoch: 4 [35840/60000 (60%)]      Loss: 0.006627
Train Epoch: 4 [36480/60000 (61%)]      Loss: 0.071250
Train Epoch: 4 [37120/60000 (62%)]      Loss: 0.002847
Train Epoch: 4 [37760/60000 (63%)]      Loss: 0.000365
Train Epoch: 4 [38400/60000 (64%)]      Loss: 0.021992
Train Epoch: 4 [39040/60000 (65%)]      Loss: 0.019298
Train Epoch: 4 [39680/60000 (66%)]      Loss: 0.001479
Train Epoch: 4 [40320/60000 (67%)]      Loss: 0.001049
Train Epoch: 4 [40960/60000 (68%)]      Loss: 0.000635
Train Epoch: 4 [41600/60000 (69%)]      Loss: 0.010856
Train Epoch: 4 [42240/60000 (70%)]      Loss: 0.000646
Train Epoch: 4 [42880/60000 (71%)]      Loss: 0.003728
Train Epoch: 4 [43520/60000 (72%)]      Loss: 0.040958
Train Epoch: 4 [44160/60000 (74%)]      Loss: 0.003352
Train Epoch: 4 [44800/60000 (75%)]      Loss: 0.003019
Train Epoch: 4 [45440/60000 (76%)]      Loss: 0.006677
Train Epoch: 4 [46080/60000 (77%)]      Loss: 0.000525
Train Epoch: 4 [46720/60000 (78%)]      Loss: 0.000407
Train Epoch: 4 [47360/60000 (79%)]      Loss: 0.020695
Train Epoch: 4 [48000/60000 (80%)]      Loss: 0.008021
Train Epoch: 4 [48640/60000 (81%)]      Loss: 0.003343
Train Epoch: 4 [49280/60000 (82%)]      Loss: 0.003259
Train Epoch: 4 [49920/60000 (83%)]      Loss: 0.003762
Train Epoch: 4 [50560/60000 (84%)]      Loss: 0.002606
Train Epoch: 4 [51200/60000 (85%)]      Loss: 0.034897
Train Epoch: 4 [51840/60000 (86%)]      Loss: 0.000584
Train Epoch: 4 [52480/60000 (87%)]      Loss: 0.005119
Train Epoch: 4 [53120/60000 (88%)]      Loss: 0.001065
Train Epoch: 4 [53760/60000 (90%)]      Loss: 0.016994
Train Epoch: 4 [54400/60000 (91%)]      Loss: 0.049590
Train Epoch: 4 [55040/60000 (92%)]      Loss: 0.017113
Train Epoch: 4 [55680/60000 (93%)]      Loss: 0.006830
Train Epoch: 4 [56320/60000 (94%)]      Loss: 0.030506
Train Epoch: 4 [56960/60000 (95%)]      Loss: 0.000915
Train Epoch: 4 [57600/60000 (96%)]      Loss: 0.009812
Train Epoch: 4 [58240/60000 (97%)]      Loss: 0.004997
Train Epoch: 4 [58880/60000 (98%)]      Loss: 0.005363
Train Epoch: 4 [59520/60000 (99%)]      Loss: 0.012845
Epoch in  54.4s

Test set: Average loss: 0.0295, Accuracy: 9906/10000 (99%)

Train Epoch: 5 [0/60000 (0%)]   Loss: 0.000971
Train Epoch: 5 [640/60000 (1%)] Loss: 0.032364
Train Epoch: 5 [1280/60000 (2%)]        Loss: 0.016328
Train Epoch: 5 [1920/60000 (3%)]        Loss: 0.012576
Train Epoch: 5 [2560/60000 (4%)]        Loss: 0.007325
Train Epoch: 5 [3200/60000 (5%)]        Loss: 0.010492
Train Epoch: 5 [3840/60000 (6%)]        Loss: 0.019273
Train Epoch: 5 [4480/60000 (7%)]        Loss: 0.017304
Train Epoch: 5 [5120/60000 (9%)]        Loss: 0.000329
Train Epoch: 5 [5760/60000 (10%)]       Loss: 0.000880
Train Epoch: 5 [6400/60000 (11%)]       Loss: 0.000691
Train Epoch: 5 [7040/60000 (12%)]       Loss: 0.009014
Train Epoch: 5 [7680/60000 (13%)]       Loss: 0.000888
Train Epoch: 5 [8320/60000 (14%)]       Loss: 0.001256
Train Epoch: 5 [8960/60000 (15%)]       Loss: 0.000202
Train Epoch: 5 [9600/60000 (16%)]       Loss: 0.001705
Train Epoch: 5 [10240/60000 (17%)]      Loss: 0.000958
Train Epoch: 5 [10880/60000 (18%)]      Loss: 0.000676
Train Epoch: 5 [11520/60000 (19%)]      Loss: 0.002135
Train Epoch: 5 [12160/60000 (20%)]      Loss: 0.004779
Train Epoch: 5 [12800/60000 (21%)]      Loss: 0.000448
Train Epoch: 5 [13440/60000 (22%)]      Loss: 0.004633
Train Epoch: 5 [14080/60000 (23%)]      Loss: 0.000458
Train Epoch: 5 [14720/60000 (25%)]      Loss: 0.000703
Train Epoch: 5 [15360/60000 (26%)]      Loss: 0.001159
Train Epoch: 5 [16000/60000 (27%)]      Loss: 0.007025
Train Epoch: 5 [16640/60000 (28%)]      Loss: 0.002218
Train Epoch: 5 [17280/60000 (29%)]      Loss: 0.001387
Train Epoch: 5 [17920/60000 (30%)]      Loss: 0.010037
Train Epoch: 5 [18560/60000 (31%)]      Loss: 0.000348
Train Epoch: 5 [19200/60000 (32%)]      Loss: 0.001255
Train Epoch: 5 [19840/60000 (33%)]      Loss: 0.025103
Train Epoch: 5 [20480/60000 (34%)]      Loss: 0.003568
Train Epoch: 5 [21120/60000 (35%)]      Loss: 0.000137
Train Epoch: 5 [21760/60000 (36%)]      Loss: 0.000760
Train Epoch: 5 [22400/60000 (37%)]      Loss: 0.003502
Train Epoch: 5 [23040/60000 (38%)]      Loss: 0.029284
Train Epoch: 5 [23680/60000 (39%)]      Loss: 0.001276
Train Epoch: 5 [24320/60000 (41%)]      Loss: 0.030836
Train Epoch: 5 [24960/60000 (42%)]      Loss: 0.006574
Train Epoch: 5 [25600/60000 (43%)]      Loss: 0.008493
Train Epoch: 5 [26240/60000 (44%)]      Loss: 0.020253
Train Epoch: 5 [26880/60000 (45%)]      Loss: 0.026145
Train Epoch: 5 [27520/60000 (46%)]      Loss: 0.000325
Train Epoch: 5 [28160/60000 (47%)]      Loss: 0.003906
Train Epoch: 5 [28800/60000 (48%)]      Loss: 0.008448
Train Epoch: 5 [29440/60000 (49%)]      Loss: 0.003100
Train Epoch: 5 [30080/60000 (50%)]      Loss: 0.004825
Train Epoch: 5 [30720/60000 (51%)]      Loss: 0.002534
Train Epoch: 5 [31360/60000 (52%)]      Loss: 0.001649
Train Epoch: 5 [32000/60000 (53%)]      Loss: 0.004588
Train Epoch: 5 [32640/60000 (54%)]      Loss: 0.000632
Train Epoch: 5 [33280/60000 (55%)]      Loss: 0.011770
Train Epoch: 5 [33920/60000 (57%)]      Loss: 0.001889
Train Epoch: 5 [34560/60000 (58%)]      Loss: 0.006371
Train Epoch: 5 [35200/60000 (59%)]      Loss: 0.003406
Train Epoch: 5 [35840/60000 (60%)]      Loss: 0.003683
Train Epoch: 5 [36480/60000 (61%)]      Loss: 0.004477
Train Epoch: 5 [37120/60000 (62%)]      Loss: 0.001394
Train Epoch: 5 [37760/60000 (63%)]      Loss: 0.000608
Train Epoch: 5 [38400/60000 (64%)]      Loss: 0.001213
Train Epoch: 5 [39040/60000 (65%)]      Loss: 0.000249
Train Epoch: 5 [39680/60000 (66%)]      Loss: 0.003591
Train Epoch: 5 [40320/60000 (67%)]      Loss: 0.020278
Train Epoch: 5 [40960/60000 (68%)]      Loss: 0.007069
Train Epoch: 5 [41600/60000 (69%)]      Loss: 0.006638
Train Epoch: 5 [42240/60000 (70%)]      Loss: 0.002554
Train Epoch: 5 [42880/60000 (71%)]      Loss: 0.000070
Train Epoch: 5 [43520/60000 (72%)]      Loss: 0.020195
Train Epoch: 5 [44160/60000 (74%)]      Loss: 0.019447
Train Epoch: 5 [44800/60000 (75%)]      Loss: 0.001946
Train Epoch: 5 [45440/60000 (76%)]      Loss: 0.001599
Train Epoch: 5 [46080/60000 (77%)]      Loss: 0.000333
Train Epoch: 5 [46720/60000 (78%)]      Loss: 0.001696
Train Epoch: 5 [47360/60000 (79%)]      Loss: 0.000553
Train Epoch: 5 [48000/60000 (80%)]      Loss: 0.002307
Train Epoch: 5 [48640/60000 (81%)]      Loss: 0.001422
Train Epoch: 5 [49280/60000 (82%)]      Loss: 0.000312
Train Epoch: 5 [49920/60000 (83%)]      Loss: 0.002286
Train Epoch: 5 [50560/60000 (84%)]      Loss: 0.001461
Train Epoch: 5 [51200/60000 (85%)]      Loss: 0.002731
Train Epoch: 5 [51840/60000 (86%)]      Loss: 0.000743
Train Epoch: 5 [52480/60000 (87%)]      Loss: 0.052643
Train Epoch: 5 [53120/60000 (88%)]      Loss: 0.005123
Train Epoch: 5 [53760/60000 (90%)]      Loss: 0.005136
Train Epoch: 5 [54400/60000 (91%)]      Loss: 0.005147
Train Epoch: 5 [55040/60000 (92%)]      Loss: 0.001530
Train Epoch: 5 [55680/60000 (93%)]      Loss: 0.001608
Train Epoch: 5 [56320/60000 (94%)]      Loss: 0.010844
Train Epoch: 5 [56960/60000 (95%)]      Loss: 0.003777
Train Epoch: 5 [57600/60000 (96%)]      Loss: 0.010776
Train Epoch: 5 [58240/60000 (97%)]      Loss: 0.004469
Train Epoch: 5 [58880/60000 (98%)]      Loss: 0.005965
Train Epoch: 5 [59520/60000 (99%)]      Loss: 0.001397
Epoch in  55.7s

Test set: Average loss: 0.0295, Accuracy: 9909/10000 (99%)

Done

trholding Oct 31, 2022
Author

Today the first epoch and others are taking less than 48s!

artyom-beilis · 2022-10-31T09:45:57Z

artyom-beilis
Oct 31, 2022
Maintainer

I merged updated pytorch support to master - so you can use stable 1.13 version with privateuseone device or rename in to ocl in the 1.14 nighly.

Also I tested the setup on Windows now you can use it on Windows as well.

2 replies

trholding Oct 31, 2022
Author

This broke the build so I commented it out:

#if(NOT WIN32)
#	set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -g -O2 -Wall")
#endif()

There were some complaints from torch about requiring c++ 14 and it errored out...

artyom-beilis Oct 31, 2022
Maintainer

There were some complaints from torch about requiring c++ 14 and it errored out..

Ooops - fixed

trholding · 2022-11-02T17:43:07Z

trholding
Nov 2, 2022
Author

So I tried to make OpenAI's whisper work with dlprim and I got this error. I am unsure how to fix.

Accessing device #0:STONEY (stoney, LLVM 14.0.6, DRM 3.48, 6.0.6-arch1-1) on Clover
Failed Program Code:
=========================
#define PARAMS , __global float const *px0, ulong px0_offset, Shape strides0, __global half *py0, ulong py0_offset
#define LOADS float x0=px0[get_offset(index,strides0,px0_offset)];\
typedef float typeof_x0;\
half y0;\
typedef half typeof_y0;\
typedef half target_type;\


#define SAVES py0[get_direct_offset(index,limit,py0_offset)]=y0;\


#define CALC y0=x0;

///////////////////////////////////////////////////////////////////////////////
///
/// Copyright (c) 2021-2022 Artyom Beilis <[email protected]>
///
/// MIT License, see LICENSE.TXT
///
///////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////
///
/// Copyright (c) 2021-2022 Artyom Beilis <[email protected]>
///
/// MIT License, see LICENSE.TXT
///
///////////////////////////////////////////////////////////////////////////////
#define ACTIVATION_IDENTITY 0
#define ACTIVATION_RELU     1
#define ACTIVATION_TANH     2
#define ACTIVATION_SIGMOID  3
#define ACTIVATION_RELU6    4

#ifndef dtype
#define dtype float
#define dtype2 float2
#define dtype4 float4
#define DTYPE_MAX FLT_MAX
#define DTYPE_MIN FLT_MIN
#endif


#ifndef ACTIVATION
#define ACTIVATION ACTIVATION_IDENTITY
#endif

#if ACTIVATION == ACTIVATION_IDENTITY
#   define ACTIVATION_F(x) (x)
#   define ACTIVATION_FINV(y,dy) (dy)
#   define ACTIVATION_NAME identity
#elif ACTIVATION == ACTIVATION_RELU
#   define ACTIVATION_F(x) (max((x),(dtype)(0)))
#   define ACTIVATION_FINV(y,dy)  ((y>0)?dy:0)
#   define ACTIVATION_NAME relu
#elif ACTIVATION == ACTIVATION_TANH
#   define ACTIVATION_F(x) (tanh((x)))
#   define ACTIVATION_FINV(y,dy) ((1-(y)*(y))*(dy))
#   define ACTIVATION_NAME tanh 
#elif ACTIVATION == ACTIVATION_SIGMOID
#   define ACTIVATION_F(x) ((dtype)(1) / ((dtype)(1) + exp(-(x))))
#   define ACTIVATION_FINV(y,dy) ((y)*(1-(y))*(dy))
#   define ACTIVATION_NAME sigmoid
#elif ACTIVATION == ACTIVATION_RELU6
#   define ACTIVATION_F(x) (min(max((x),(dtype)(0)),(dtype)(6)))
#   define ACTIVATION_FINV(y,dy)  ((0<y && y<6)?dy:0)
#   define ACTIVATION_NAME relu6
#else
#   error "Unknown activation"
#endif 



///////////////////////////////////////////////////////////////////////////////
///
/// Copyright (c) 2021-2022 Artyom Beilis <[email protected]>
///
/// MIT License, see LICENSE.TXT
///
///////////////////////////////////////////////////////////////////////////////
typedef struct Shape {
    ulong s[DIMS];
} Shape;


inline Shape get_pos_broadcast(Shape limits)
{
    Shape r;
#if DIMS <= 1
    r.s[0] = get_global_id(0);
#elif DIMS == 2
    r.s[0] = get_global_id(1);      
    r.s[1] = get_global_id(0);      
#elif DIMS == 3    
    r.s[0] = get_global_id(2);      
    r.s[1] = get_global_id(1);      
    r.s[2] = get_global_id(0);      
#elif DIMS == 4    
    r.s[0] = get_global_id(2);      
    r.s[1] = get_global_id(1);      
    r.s[2] = get_global_id(0) / limits.s[3];      
    r.s[3] = get_global_id(0) % limits.s[3];      
#elif DIMS == 5    
    r.s[0] = get_global_id(2);      
    r.s[1] = get_global_id(1) / limits.s[2];      
    r.s[2] = get_global_id(1) % limits.s[2];      
    r.s[3] = get_global_id(0) / limits.s[4];      
    r.s[4] = get_global_id(0) % limits.s[4];      
#elif DIMS == 6    
    r.s[0] = get_global_id(2) / limits.s[1];      
    r.s[1] = get_global_id(2) % limits.s[1];      
    r.s[2] = get_global_id(1) / limits.s[3];      
    r.s[3] = get_global_id(1) % limits.s[3];      
    r.s[4] = get_global_id(0) / limits.s[5];      
    r.s[5] = get_global_id(0) % limits.s[5];      
#elif DIMS == 7
    r.s[0] = get_global_id(2) / limits.s[1];      
    r.s[1] = get_global_id(2) % limits.s[1];      
    r.s[2] = get_global_id(1) / limits.s[3];      
    r.s[3] = get_global_id(1) % limits.s[3];      
    r.s[4] = get_global_id(0) / (limits.s[5]*limits.s[6]);      
    ulong s56 = get_global_id(0) % (limits.s[5]*limits.s[6]);
    r.s[5] = s56 / limits.s[6];      
    r.s[6] = s56 % limits.s[6];      
#elif DIMS == 8
    r.s[0] = get_global_id(2) / limits.s[1];      
    r.s[1] = get_global_id(2) % limits.s[1];      

    r.s[2] = get_global_id(1) / (limits.s[3]*limits.s[4]);      
    ulong s34 = get_global_id(1) % (limits.s[3]*limits.s[4]);
    r.s[3] = s34 / limits.s[4];      
    r.s[4] = s34 % limits.s[4];      

    r.s[5] = get_global_id(0) / (limits.s[6]*limits.s[7]);      
    ulong s67 = get_global_id(0) % (limits.s[6]*limits.s[7]);
    r.s[6] = s67 / limits.s[7];      
    r.s[7] = s67 % limits.s[7];      
#else
#error "Unsupported dim"
#endif
    return r;
}


inline ulong get_offset(Shape s,Shape strides,ulong offset)
{
    ulong r = offset;
    #pragma unroll
    for(int i=0;i<DIMS;i++) {
        r+= s.s[i]*strides.s[i];
    }
    return r;
}

inline ulong get_direct_offset(Shape s,Shape sizes,ulong offset)
{
    ulong index = 0;
    #pragma unroll
    for(int i=0;i<DIMS-1;i++) {
        index += s.s[i];
        index *= sizes.s[i+1];
    }
    index += s.s[DIMS-1] + offset;
    return index;
}

#define get_pos(limits) get_pos_broadcast(limits)

inline bool valid_pos(Shape pos,Shape limits)
{
    #pragma unroll
    for(int i=0;i<DIMS;i++)
        if(pos.s[i] >= limits.s[i])
            return 0;
    return 1;

}

__kernel void exec(Shape limit   PARAMS)
{
    Shape index = get_pos(limit);
    if(!valid_pos(index,limit)) {
        return;
    }
    LOADS
    CALC
    SAVES
}


=========================

Traceback (most recent call last):
  File "/home/skunkwork/.local/bin/whisper", line 8, in <module>
    sys.exit(cli())
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/transcribe.py", line 311, in cli
    result = transcribe(model, audio_path, temperature=temperature, **args)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/transcribe.py", line 181, in transcribe
    segment = pad_or_trim(mel[:, seek:], N_FRAMES).to(model.device).to(dtype)
RuntimeError: Failed to build program source pointwise_broadcast with parameters -DDIMS=1     log:
For device: STONEY (stoney, LLVM 14.0.6, DRM 3.48, 6.0.6-arch1-1)
input.cl:184:5: error: declaring variable of type '__private half' is not allowed
input.cl:4:6: note: expanded from macro 'LOADS'

clBuildErrorCode: -11

13 replies

trholding Nov 3, 2022
Author

I tried without really understanding how it works and failed miserably :-|

I know the code is not correct as I had no idea what I am doing (10000 monkeys with typewriters in my head).

    // {"schema": "aten::gelu.out(Tensor self, *, str approximate='none', Tensor(a!) out) -> Tensor(a!)", "dispatch": "True", "default": "False"}
    Tensor & gelu_out(const Tensor & self, const c10::string_view approximate , Tensor & out)
    {
        GUARD;
        Tensor self_c = self.contiguous();
        dlprim::Tensor x=todp(self_c);
        dlprim::Tensor y=todp(out);
        if (approximate == "none") {
  		//Not sure what should come here
  		dlprim::core::pointwise_operation({x},{y},{},"y0 = 0.5 * (1.0 + tanh(sqrt(2.0/3.141592) * (x0 + 0.044715^3.0)));",getExecutionContext(self));
  	} else if (approximate == "tanh") {
  		dlprim::core::pointwise_operation({x},{y},{},"y0 = 0.5 * (1.0 + tanh(sqrt(2.0/3.141592) * (x0 + 0.044715^3.0)));",getExecutionContext(self));
  	} 
        sync_if_needed(self.device());
        return out;
    }

Errors:

$ whisper elon.mp3 --model tiny.en --fp16 False
Accessing device #0:STONEY (stoney, LLVM 14.0.6, DRM 3.48, 6.0.6-arch1-1) on Clover
Failed Program Code:
=========================
#define PARAMS , __global dtype const *px0, ulong px0_offset , __global dtype *py0, ulong py0_offset 
#define LOADS dtype x0=px0[index + px0_offset]; dtype y0;
#define SAVES py0[index]=y0; 
#define CALC y0 = 0.5 * (1.0 + tanh(sqrt(2.0/3.141592) * (x0 + 0.044715^3.0)));
///////////////////////////////////////////////////////////////////////////////
///
/// Copyright (c) 2021-2022 Artyom Beilis <[email protected]>
///
/// MIT License, see LICENSE.TXT
///
///////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////
///
/// Copyright (c) 2021-2022 Artyom Beilis <[email protected]>
///
/// MIT License, see LICENSE.TXT
///
///////////////////////////////////////////////////////////////////////////////
#define ACTIVATION_IDENTITY 0
#define ACTIVATION_RELU     1
#define ACTIVATION_TANH     2
#define ACTIVATION_SIGMOID  3
#define ACTIVATION_RELU6    4

#ifndef dtype
#define dtype float
#define dtype2 float2
#define dtype4 float4
#define DTYPE_MAX FLT_MAX
#define DTYPE_MIN FLT_MIN
#endif


#ifndef ACTIVATION
#define ACTIVATION ACTIVATION_IDENTITY
#endif

#if ACTIVATION == ACTIVATION_IDENTITY
#   define ACTIVATION_F(x) (x)
#   define ACTIVATION_FINV(y,dy) (dy)
#   define ACTIVATION_NAME identity
#elif ACTIVATION == ACTIVATION_RELU
#   define ACTIVATION_F(x) (max((x),(dtype)(0)))
#   define ACTIVATION_FINV(y,dy)  ((y>0)?dy:0)
#   define ACTIVATION_NAME relu
#elif ACTIVATION == ACTIVATION_TANH
#   define ACTIVATION_F(x) (tanh((x)))
#   define ACTIVATION_FINV(y,dy) ((1-(y)*(y))*(dy))
#   define ACTIVATION_NAME tanh 
#elif ACTIVATION == ACTIVATION_SIGMOID
#   define ACTIVATION_F(x) ((dtype)(1) / ((dtype)(1) + exp(-(x))))
#   define ACTIVATION_FINV(y,dy) ((y)*(1-(y))*(dy))
#   define ACTIVATION_NAME sigmoid
#elif ACTIVATION == ACTIVATION_RELU6
#   define ACTIVATION_F(x) (min(max((x),(dtype)(0)),(dtype)(6)))
#   define ACTIVATION_FINV(y,dy)  ((0<y && y<6)?dy:0)
#   define ACTIVATION_NAME relu6
#else
#   error "Unknown activation"
#endif 



__kernel void exec(ulong total  PARAMS)
{
    ulong index=get_global_id(0);
    if(index>=total)
        return;
    LOADS
    CALC
    SAVES
}

=========================

Traceback (most recent call last):
  File "/home/skunkwork/.local/bin/whisper", line 8, in <module>
    sys.exit(cli())
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/transcribe.py", line 307, in cli
    result = transcribe(model, audio_path, temperature=temperature, **args)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/transcribe.py", line 181, in transcribe
    result: DecodingResult = decode_with_fallback(segment)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/transcribe.py", line 117, in decode_with_fallback
    decode_result = model.decode(segment, options)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/decoding.py", line 701, in decode
    result = DecodingTask(model, options).run(mel)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/decoding.py", line 617, in run
    audio_features: Tensor = self._get_audio_features(mel)  # encoder forward pass
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/decoding.py", line 561, in _get_audio_features
    audio_features = self.model.encoder(mel)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1423, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/skunkwork/.local/lib/python3.10/site-packages/whisper/model.py", line 148, in forward
    x = F.gelu(self.conv1(x))
RuntimeError: Failed to build program source pointwise with parameters -Ddtype=float     log:
For device: STONEY (stoney, LLVM 14.0.6, DRM 3.48, 6.0.6-arch1-1)
input.cl:70:5: warning: double precision constant requires cl_khr_fp64, casting to single precision
input.cl:4:19: note: expanded from macro 'CALC'
input.cl:70:5: warning: double precision constant requires cl_khr_fp64, casting to single precision
input.cl:4:26: note: expanded from macro 'CALC'
input.cl:70:5: warning: double precision constant requires cl_khr_fp64, casting to single precision
input.cl:4:42: note: expanded from macro 'CALC'
input.cl:70:5: warning: double precision constant requires cl_khr_fp64, casting to single precision
input.cl:4:46: note: expanded from macro 'CALC'
input.cl:70:5: warning: double precision constant requires cl_khr_fp64, casting to single precision
input.cl:4:64: note: expanded from macro 'CALC'
input.cl:70:5: warning: double precision constant requires cl_khr_fp64, casting to single precision
input.cl:4:73: note: expanded from macro 'CALC'
input.cl:70:5: error: invalid operands to binary expression ('float' a
clBuildErrorCode: -11

I think I've to study dlprim and torch code a bit deeper.

artyom-beilis Nov 4, 2022
Maintainer

^ in C (and OpenCL) isn't power operator - it is XOR...

trholding Nov 4, 2022
Author

oops :) Thank you :)

I live next to a construction site, sometimes I can't think clearly when they rev up the JCBs, drills till late night. I'll fix that and check.

trholding Nov 4, 2022
Author

Okay I get this final error after the pow fix.

return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: Running sums are expected to be present

artyom-beilis Nov 4, 2022
Maintainer

I opened discussion about transformer gaps: #16

artyom-beilis · 2022-12-03T06:56:45Z

artyom-beilis
Dec 3, 2022
Maintainer

Did you clone the repository recursively? Artyom On Saturday, December 3, 2022, 08:41:56 AM GMT+2, Evgeny Kurnevsky ***@***.***> wrote: Just tried to build the latest version but got: pytorch-dlprim> CMake Error: The following variables are used in this project, but they are set to NOTFOUND. pytorch-dlprim> Please set them or make sure they are set and tested correctly in the CMake files: pytorch-dlprim> DLPRIM_INC pytorch-dlprim> used as include directory in directory /build/source pytorch-dlprim> used as include directory in directory /build/source pytorch-dlprim> used as include directory in directory /build/source pytorch-dlprim> DLPRIM_LIB pytorch-dlprim> linked by target "pt_ocl" in directory /build/source pytorch-dlprim> pytorch-dlprim> CMake Error in CMakeLists.txt: pytorch-dlprim> Found relative path while evaluating include directories of "pt_ocl": pytorch-dlprim> "DLPRIM_INC-NOTFOUND" — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

1 reply

kurnevsky Dec 3, 2022

Sorry for noise, I deleted my comment, but it seems you didn't see it via email :)
I actually forgot to update derivation hash, so the old revision of pytorch_dlprim was used.

FAQ: Can I use this as a drop in replacement for pytorch? #12

trholding Oct 14, 2022

Replies: 11 comments · 60 replies

artyom-beilis Oct 14, 2022 Maintainer

trholding Oct 17, 2022 Author

trholding Oct 18, 2022 Author

trholding Oct 23, 2022 Author

trholding Oct 23, 2022 Author

trholding Oct 24, 2022 Author

artyom-beilis Oct 23, 2022 Maintainer

trholding Oct 23, 2022 Author

trholding Oct 24, 2022 Author

trholding Oct 24, 2022 Author

trholding Oct 24, 2022 Author

artyom-beilis Oct 24, 2022 Maintainer

artyom-beilis Oct 24, 2022 Maintainer

trholding Oct 25, 2022 Author

artyom-beilis Oct 24, 2022 Maintainer

trholding Oct 25, 2022 Author

artyom-beilis Oct 25, 2022 Maintainer

artyom-beilis Oct 25, 2022 Maintainer

artyom-beilis Oct 24, 2022 Maintainer

trholding Oct 25, 2022 Author

artyom-beilis Oct 25, 2022 Maintainer

artyom-beilis Oct 26, 2022 Maintainer

artyom-beilis Oct 29, 2022 Maintainer

trholding Oct 30, 2022 Author

trholding Oct 30, 2022 Author

artyom-beilis Oct 30, 2022 Maintainer

trholding Oct 30, 2022 Author

artyom-beilis Oct 29, 2022 Maintainer

trholding Oct 30, 2022 Author

trholding Oct 31, 2022 Author

artyom-beilis Oct 31, 2022 Maintainer

trholding Oct 31, 2022 Author

artyom-beilis Oct 31, 2022 Maintainer

trholding Nov 2, 2022 Author

trholding Nov 3, 2022 Author

artyom-beilis Nov 4, 2022 Maintainer

trholding Nov 4, 2022 Author

trholding Nov 4, 2022 Author

artyom-beilis Nov 4, 2022 Maintainer

artyom-beilis Dec 3, 2022 Maintainer

kurnevsky Dec 3, 2022

trholding
Oct 14, 2022

Replies: 11 comments 60 replies

artyom-beilis
Oct 14, 2022
Maintainer

trholding Oct 17, 2022
Author

trholding Oct 18, 2022
Author

trholding Oct 23, 2022
Author

trholding Oct 23, 2022
Author

trholding Oct 24, 2022
Author

artyom-beilis
Oct 23, 2022
Maintainer

trholding Oct 23, 2022
Author

trholding Oct 24, 2022
Author

trholding Oct 24, 2022
Author

trholding Oct 24, 2022
Author

artyom-beilis
Oct 24, 2022
Maintainer

artyom-beilis
Oct 24, 2022
Maintainer

trholding Oct 25, 2022
Author

artyom-beilis
Oct 24, 2022
Maintainer

trholding Oct 25, 2022
Author

artyom-beilis Oct 25, 2022
Maintainer

artyom-beilis Oct 25, 2022
Maintainer

artyom-beilis
Oct 24, 2022
Maintainer

trholding Oct 25, 2022
Author

artyom-beilis Oct 25, 2022
Maintainer

artyom-beilis
Oct 26, 2022
Maintainer

artyom-beilis Oct 29, 2022
Maintainer

trholding Oct 30, 2022
Author

trholding Oct 30, 2022
Author

artyom-beilis Oct 30, 2022
Maintainer

trholding Oct 30, 2022
Author

artyom-beilis
Oct 29, 2022
Maintainer

trholding Oct 30, 2022
Author

trholding Oct 31, 2022
Author

artyom-beilis
Oct 31, 2022
Maintainer

trholding Oct 31, 2022
Author

artyom-beilis Oct 31, 2022
Maintainer

trholding
Nov 2, 2022
Author

trholding Nov 3, 2022
Author

artyom-beilis Nov 4, 2022
Maintainer

trholding Nov 4, 2022
Author

trholding Nov 4, 2022
Author

artyom-beilis Nov 4, 2022
Maintainer

artyom-beilis
Dec 3, 2022
Maintainer