Crowd Sourced Deep Learning GPU Benchmarks from the Community

We open sourced the benchmarking code we use at Lambda Labs so that anybody can reproduce the benchmarks that we publish or run their own. We encourage people to email us with their results and will continue to publish those results here. You can run the code and email benchmarks@lambdalabs.com or tweet @LambdaAPI. This is the official page for all Lambda Community Benchmarks.

How to get your results published here

Component        | Version
-----------------|------------
CPU              | $(cat /proc/cpuinfo | grep 'model name' | uniq | awk -F: '{ print $2 }')
Distro	         | $(lsb_release -d)
Kernel Version   | $(uname -r)
Kernel Arch      | $(uname -m)
GPU              | $(sudo lspci | grep VGA\ compat | head -n1)
Tensorflow       | $(python -c 'import tensorflow;print(tensorflow.__version__)')
NVIDIA Driver    | $(head -n1 /proc/driver/nvidia/version | awk '{ print $8 }')
CUDA	         | $(nvcc --version | tail -n 1 | grep Cuda | awk '{ print $6 }')
cuDNN	         | 7.3.0.29
Python	         | $(python --version)

Copy the above and paste into template.txt. Then run the code below to output your table.

cat > template.txt
CTRL-V (paste in)
CTRL-D (end file)
for line in $(cat template.txt); do eval "echo \"$line\""; done

Crowd Sourced Results

Here are the results that have been submitted to us by third parties.

SUMMARY - Mike Metral - 1080 Ti

model input size param mem feat. mem flops
resnet-50 224 x 224 98 MB 103 MB 4 BFLOPs
resnet-152 224 x 224 230 MB 219 MB 11 BFLOPs
inception-v3 299 x 299 91 MB 89 MB 6 BFLOPs
vgg-vd-19 224 x 224 548 MB 63 MB 20 BFLOPs
alexnet 227 x 227 233 MB 3 MB 1.5 BFLOPs
ssd-300 300 x 300 100 MB 116 MB 31 GFLOPS

syn-replicated-fp32-1gpus

Config v2-GeForce_GTX_1080_Ti
resnet50 221.33
resnet152 84.99
inception3 142.51
inception4 60.11
vgg16 142.39
alexnet 2868.88
ssd300 112.22

syn-parameter_server-fp32-1gpus

Config v2-GeForce_GTX_1080_Ti
resnet50 221.24
resnet152 85.04
inception3 142.39
inception4 60.12
vgg16 142.17
alexnet 2870.47
ssd300 112.14

syn-replicated-fp16-1gpus

Config v2-GeForce_GTX_1080_Ti
resnet50 275.24
resnet152 99.76
inception3 161.39
inception4 64.63
vgg16 153.03
alexnet 2981.33
ssd300 126.42

syn-parameter_server-fp16-1gpus

Config v2-GeForce_GTX_1080_Ti
resnet50 275.78
resnet152 100.20
inception3 160.48
inception4 65.22
vgg16 156.34
alexnet 3022.28
ssd300 127.33

HARDWARE / SOFTWARE

Component Version
Distro Ubuntu 18.04.1
Kernel 4.18.5 x86_64
GPU / Compute Capacity NVIDIA GeForce GTX 1080 TI - 6.1
Tensorflow v1.11.0
NVIDIA 410.57
CUDA 10.0.130_410.48
cuDNN 7.3.0.29
NCCL 2.3.5
GCC Ubuntu 6.4.0-17ubuntu1
Python 3.6.6
Bazel 0.16.1
!-- Intercom -->