Tensorflow 1.12 GPU mac installation

If like me you felt very frustrated by that line in Tensorflow doc : There is no GPU support for macOS. and you wanted to prove it wrong, welcome abroad.

There are several other gist lying around explaining how to do some or all of it on older version of CUDA/Tensorflow, so I felt a need to explain it all over with the lastest versions.

References :

Environment

OSX High Sierra 10.13.6
NVIDIA GeForce GT 650M 1024 Mo
CUDA 10.0
cuDNN 7.4.1
No NCCL
Bazel 0.18.1
Python 3.6.7 (Conda env)
Xcode 9.4
TensorFlow 1.12.0

Prerequisite

Install CUDA and cuDNN normally (if you have problem for the CUDA installation, see this guide)

Download tensorflow release from github or
```
$ git clone https://github.com/tensorflow/tensorflow.git
$ git checkout r1.12
```
Install all needed Tensorflow dependencies listed in official doc :
- pip
- six
- numpy
- wheel
- mock
- keras_applications
- keras_preprocessing
If you use a virtualenv or a conda env, install them in it (and deactivate/activate your env to update the correct pip for later use).

Install Bazel from Homebrew

Move inside tensorflow folder

Official Tensorflow build from source instructions tells you to run this command :
```
bazel test -c opt -- //tensorflow/... -//tensorflow/compiler/... -//tensorflow/contrib/lite/...
```
but after 2 hours of seeing lots of errors, I got tired of it and decided to make without. Feel free to do as you want.
Prevent compilation error

In ./tensorflow/core/kernels/split_lib_gpu.cu.cc and ./tensorflow/core/kernels/concat_lib_gpu_impl.cu.cc

Replace
```
-     extern __shared__ __align__(sizeof(T)) unsigned char smem[];
+     extern __shared__ unsigned char smem[];
```
Prevent NCCL errors

Since NCCL is no longer available on OSX, you can't use it with Tensorflow. Fortunately, its use is optional. Unfortunately, since Tensorflow-gpu isn't officially supported on OSX, the check to avoid including it is only done for Windows, so you need to tweek some bazel conf file a little.

In ./tensorflow/contrib/nccl/BUILD

Comment out
```
gpu_srcs = if_not_windows_cuda([
-   "kernels/nccl_manager.cc",
-   "kernels/nccl_manager.h",
-   "kernels/nccl_ops.cc",
+   #"kernels/nccl_manager.cc",
+   #"kernels/nccl_manager.h",
+   #"kernels/nccl_ops.cc",
]),
```
Prevent not found lib error

In ./third_party/gpus/cuda/BUILD.tpl and ./third_party/toolchains/gpus/cuda/BUILD (one of them might not be needed but I didn't test without modifying both)

Remove
```
-   linkopts = ["-lgomp"],
+   linkopts = [],
```

Config

Launch the configure script to customize the settings. The only important part are the choice of CUDA and cuDNN versions, the compute capability corresponding to your graphic card (mine is an old one so only 3.0) and to say no to the use of clang as CUDA compiler (even though the host compiler will still be clang, as an alias to gcc).

For all the rest, including the paths, I used the defaults, but adjust as needed for your build.

$ ./configure
You have bazel 0.18.1-homebrew installed.
Please specify the location of python. [Default is ~/.conda/envs/VenvName/bin/python]:


Found possible Python library paths:
  ~/.conda/envs/VenvName/lib/python3.6/site-packages
Please input the desired Python library path to use.  Default is [~/.conda/envs/VenvName/lib/python3.6/site-packages]

Do you wish to build TensorFlow with Apache Ignite support? [Y/n]: n
No Apache Ignite support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 9.0]: 10.0


Please specify the location where CUDA 10.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:


Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]:


Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:


Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,7.0]: 3.0


Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:


Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:


Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
    --config=mkl         	# Build with MKL support.
    --config=monolithic  	# Config for mostly static monolithic build.
    --config=gdr         	# Build with GDR support.
    --config=verbs       	# Build with libverbs support.
    --config=ngraph      	# Build with Intel nGraph support.
Configuration finished

Build away !

You can now build Tensorflow the traditionnal way, hopefully without error.

# use of MKL is optional
# you have time for a coffee (or beer whichever)
$ bazel build --config=mkl --config=opt //tensorflow/tools/pip_package:build_pip_package
$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
# now is the time to activate your virtual or conda env if not done yet
$ pip install --ignore-installed --upgrade /tmp/tensorflow_pkg/`ls /tmp/tensorflow_pkg/ | grep tensorflow`

Resolve SIP causing import error

If you get an error like @rpath/libcu-something image not found when you try to run Tensorflow code using your GPU, the problem comes from the SIP in OSX preventing python subprocess from getting access to your DYLD_LIBRARY_PATH environment variable. You can test that you have this issue by launching a python interpreter and running import os; os.getenv('DYLD_LIBRARY_PATH') which should return blank. You can resolve it by running these two commands, as per this answer :
```
$ install_name_tool \
-change @rpath/libcusolver.10.0.dylib /usr/local/cuda/lib/libcusolver.10.0.dylib \
-change @rpath/libcudart.10.0.dylib /usr/local/cuda/lib/libcudart.10.0.dylib \
-change @rpath/libcublas.10.0.dylib /usr/local/cuda/lib/libcublas.10.0.dylib  **please-locate-this**/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so

$ install_name_tool\
-change @rpath/libcudart.10.0.dylib /usr/local/cuda/lib/libcudart.10.0.dylib\
-change @rpath/libcublas.10.0.dylib /usr/local/cuda/lib/libcublas.10.0.dylib\
-change @rpath/libcudnn.7.dylib /usr/local/cuda/lib/libcudnn.7.dylib\
-change @rpath/libcufft.10.0.dylib /usr/local/cuda/lib/libcufft.10.0.dylib\
-change @rpath/libcurand.10.0.dylib /usr/local/cuda/lib/libcurand.10.0.dylib\
-change @rpath/libcudart.10.0.dylib /usr/local/cuda/lib/libcudart.10.0.dylib **please-locate-this**/lib/python3.6/site-packages/tensorflow/libtensorflow_framework.so
```
The two directories you need to locate are in your conda packages folder if you use conda. For me this was in ~/miniconda3/pkgs/tensorflow-base-1.12.0-mkl_py36h70e0e9a_0/. If you already created a conda env, you might need to rerun both commands on the _pywrap_tensorflow_internal.soand python/_pywrap_tensorflow_internal.so that are also present in your env folder, for me this was in ~/.conda/envs/EnvName. This will make Tensorflow use the real path instead of relying on @rpath. Changes the path /usr/local/cuda/lib/ if needed.

Et voilà ! You can now test some Tensorflow programs to see if all is well (but launch your Python interpreter outside of the Tensorflow directory to avoid the scare of an immediate error).

Chadys/tensorflow-1.12-cuda-10.0-mac-install.md

Tensorflow 1.12 GPU mac installation

Environment