Mocha Backends¶
A backend in Mocha is a component that carries out the actual numerical computation. Mocha is designed to support multiple backends, and switching between different backends should be almost transparent to the rest of the world.
There is a DefaultBackend
defined which is a typealias for one of the following
backends, depending on availability. By default, GPUBackend
is preferred if
CUDA is available, falling back to the CPUBackend
otherwise.
Pure Julia CPU Backend¶
A pure Julia CPU backend is implemented in Julia. This backend is reasonably fast by making heavy use of Julia’s built-in BLAS matrix computation library and performance annotations to help the LLVM-based JIT compiler produce high performance instructions.
A pure Julia CPU backend can be instantiated by calling the constructor
CPUBackend()
. Because there is no external dependency, it should run on any
platform that runs Julia.
If you have many cores in your computer, you can play with the number of threads used by Julia’s BLAS matrix computation library by:
blas_set_num_threads(N)
Depending on the problem size and a lot of other factors, using larger N is not necessarily faster.
CPU Backend with Native Extension¶
Mocha comes with C++ implementations of some bottleneck computations for the CPU backend. In order to use the native extension, you need to build the native code first (if it is not built automatically when installing the package).
Pkg.build("Mocha")
After successfully building the native extension, it can be enabled by setting the following environment variable. In bash or zsh, execute
export MOCHA_USE_NATIVE_EXT=true
before running Mocha. You can also set the environment variable inside the Julia code:
ENV["MOCHA_USE_NATIVE_EXT"] = "true"
using Mocha
Note you need to set the environment variable before loading the Mocha module. Otherwise Mocha will not load the native extension sub-module at all.
The native extension uses OpenMP to do parallel
computation on Linux. The number of OpenMP threads used can be controlled by
the OMP_NUM_THREADS
environment variable. Note that this variable is not specific
to Mocha. If you have other programs that use OpenMP, setting this environment
variable in a shell will also affect the programs started subsequently. If you
want to restrict the effect to Mocha, simply set the variable in the Julia code:
ENV["OMP_NUM_THREADS"] = 1
Note that setting it to 1 disables the OpenMP parallelization. Depending on the problem size and a lot of other factors, using multi-thread OpenMP parallelization is not necessarily faster because of the overhead of multi-threads.
The parameter for the number of threads used by the BLAS library applies to the CPU backend with native extension, too.
OpenMP on Mac OS X¶
When compiling the native extension on Mac OS X, you will get a warning that OpenMP is disabled. This is because currently clang, the built-in compiler for OS X, does not officially support OpenMP yet. If you want to try OpenMP on OS X, please refer to Clang-OMP and compile manually (see below).
Native Extension on Windows¶
The native extension does not support Windows because the automatic building script does not work on Windows. However, the native code themselve does not use any OS specific features. If you have a compiler installed on Windows, you can try to compile the native extension manually. However, I have not tested the native extension on Windows personally.
Compile Native Extension Manually¶
The native code is located in the deps
directory of Mocha. Use
Pkg.dir("Mocha")
to find out where Mocha is installed. You should compile it as a shared library
(DLL on Windows). However, currently the filename for the library is hard-coded
to be libmochaext.so
, with a .so
extension, regardless of the underlying
OS.
CUDA Backend¶
GPUs have been shown to be very effective at training large scale deep neural networks. NVidia® recently released a GPU accelerated library of primitives for deep neural networks called cuDNN. Mocha implementes a CUDA backend by combining cuDNN, cuBLAS and plain CUDA kernels.
In order to use the CUDA backend, you need to have a CUDA-compatible GPU device. The CUDA toolkit needs to be installed in order to compile the Mocha CUDA kernels. cuBLAS is included in the CUDA distribution. But cuDNN needs to be installed separately. You can obtain cuDNN from Nvidia’s website by registering as a CUDA developer for free.
Note
Mocha is tested on CUDA 8.0 and cuDNN 5.1 on Linux. Since cuDNN typically do not keep backward compatibility in the APIs, it is not guaranteed to run on different versions.
Before using the CUDA backend, the Mocha kernels needs to be compiled. The kernels
are located in src/cuda/kernels
. Please use Pkg.dir("Mocha")
to find out
where Mocha is installed on your system. We have included a Makefile for
convenience, but if you don’t have make
installed, the command for compiling is
as simple as
nvcc -ptx kernels.cu
After compiling the kernels, you can now start to use the CUDA backend by
setting the environment variable MOCHA_USE_CUDA
. For example:
ENV["MOCHA_USE_CUDA"] = "true"
using Mocha
backend = GPUBackend()
init(backend)
# ...
shutdown(backend)
Note that instead of instantiating a CPUBackend
, you now construct
a GPUBackend
. The environment variable needs to be set before loading
Mocha. It is designed to use conditional loading so that the pure CPU backend
can still run on machines which don’t have a GPU device or don’t have the CUDA
library installed. If you have multiple GPU devices on one node, the environment
variable MOCHA_CUDA_DEVICE
can be used to specify the device ID to use. The
default device ID is 0
.
Recompiling Kernels¶
When you upgrade Mocha to a higher version, the source code for some CUDA kernel implementations might have changed. Mocha will compile the timestamps for the compiled kernel and the source files. An error will be raised if the compiled kernel file is found to be older than the kernel source files. Simply following the procedures above to compile the kernel again will solve this problem.