TensorFlow

TensorFlow is a Deep Learning Framework developed by Google. This wiki will show you how to classify handwritten digits with it.

Here is how to do this with Nut (installation instructions):

cd examples/tensorflow  # This folder contains a `nut.yml` file with proper configuration for TensorFlow
nut train-mnist         # Train a neural network to classify MNIST dataset

You should get the following result:

$ nut train-mnist
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:900] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties: 
name: GeForce GTX TITAN X
major: 5 minor: 2 memoryClockRate (GHz) 1.076
pciBusID 0000:01:00.0
Total memory: 12.00GiB
Free memory: 11.61GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:717] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:01:00.0)
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 1.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 2.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 4.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 8.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 16.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 32.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 64.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 128.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 256.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 512.0KiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 1.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 2.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 4.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 8.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 16.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 32.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 64.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 128.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 256.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 512.00MiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 1.00GiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 2.00GiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 4.00GiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 8.00GiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 16.00GiB
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:73] Allocating 11.03GiB bytes.
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:83] GPU 0 memory begins at 0xb06c80000 extends to 0xdc8888b34
Initialized!
Step 0 (epoch 0.00), 23.1 ms
Minibatch loss: 12.054, learning rate: 0.010000
Minibatch error: 90.6%
Validation error: 84.6%
Step 100 (epoch 0.12), 10.3 ms
Minibatch loss: 3.301, learning rate: 0.010000
Minibatch error: 4.7%
Validation error: 7.4%
...

Congratulations: the neural network is training :) And it will automatically use your GPUs if your environment can support it.

To use the GPUs with Nut, you need to:

1. setup the drivers of your graphic cards
2. install CUDA (required by Caffe)
3. install nvidia-docker-plugin
4. Make sure that nvidia-docker-plugin is running when you call Nut. You can check that curl -s http://0.0.0.0:3476/v1.0/gpu/info displays information about your graphic card.

Keep in mind that Nut communicates with nvidia-docker-plugin to enable GPUs in your environments, so it has to run when you call Nut. Also, due to containers limitation, Docker does not support graphic cards neither on OSX, nor on Windows.

train-mnist is a macro defined in nut.yml:

syntax_version: "5"
project_name: tensorflow
based_on:
  docker_image: b.gcr.io/tensorflow/tensorflow:latest-devel-gpu  # this image is pulled automatically from Docker Hub
enable_nvidia_devices: true  # enable GPU support. You can leave it even if you don't have a GPU: you will just get an error message.
container_working_directory: /nut/tensorflow
mount:
  main:
    - .
    - /nut/tensorflow
macros:
  train-mnist:
    usage: test TensorFlow by training on MNIST dataset
    actions:
    - python -m tensorflow.models.image.mnist.convolutional
  run:
    usage: start tensorflow interactive session
    actions:
    - python -i -c 'import tensorflow as tf; sess = tf.InteractiveSession()'

If you run nut in examples/tensorflow, you can see the list of all macros that have been defined:

NAME:
   nut - the development environment, containerized

USAGE:
   nut [global options] command [command options] [arguments...]
   
VERSION:
   0.1.0 dev
   
COMMANDS:
   run          macro: start tensorflow interactive session
   train-mnist  macro: test TensorFlow by training on MNIST dataset

So you can start a TensorFlow interactive session with nut run. You can also define a new macro train to train your own model.

Thanks to containers, you installed TensorFlow and all of its dependencies without messing up with the configuration of your computer. The docker image takes some space on your hard drive though. Good news is: if you don't need Caffe daily, you can simply remove it with docker rmi b.gcr.io/tensorflow/tensorflow:latest-devel-gpu. And next time you use Nut for this project, the Docker image will be pulled from Docker Hub automatically.