list.rst 4.28 KB
Newer Older
1
.. frameworks/validated/list.rst: 
2

3 4 5
#################################
Validated workloads by framework
#################################
6

7 8
We validated performance [#f1]_ for the following TensorFlow\* and MXNet\* 
workloads:
9

10 11 12
* :ref:`tensorflow_valid`
* :ref:`mxnet_valid`
* :ref:`onnx_valid`
13
* :doc:`../../project/extras/testing_latency.rst`
14 15

.. _tensorflow_valid:
16

17 18 19
TensorFlow 
==========

20
.. csv-table::
L.S. Cook's avatar
L.S. Cook committed
21
   :header: "TensorFlow Workload", "Genre of Deep Learning"
22 23 24
   :widths: 27, 53
   :escape: ~

25 26 27 28
   Resnet50 v1, Image recognition
   Resnet50 v2, Image recognition
   Inception V3, Image recognition
   Inception V4, Image recognition
29 30
   Inception-ResNetv2, Image recognition
   MobileNet v1, Image recognition
31 32
   MobileNet v2, Image recognition
   VGG16, Image recognition
33 34
   SSD-VGG16, Object detection
   SSD-MobileNetv1, Object detection
35
   R-FCN, Object detection
36 37
   Faster RCNN, Object detection
   Yolo v2, Object detection
38
   Transformer-LT, Language translation
39 40 41 42 43 44
   Wide & Deep, Recommender system
   NCF, Recommender system
   U-Net, Image segmentation
   DCGAN, Generative adversarial network
   DRAW, Image generation
   A3C, Reinforcement learning
45

46 47 48

.. _mxnet_valid:

49 50
MXNet
=====
51

52
.. csv-table::
L.S. Cook's avatar
L.S. Cook committed
53
   :header: "MXNet Workload", "Genre of Deep Learning"
54 55
   :widths: 27, 53
   :escape: ~
56

57 58
   Resnet50 v1, Image recognition
   Resnet50 v2, Image recognition
59
   DenseNet-121, Image recognition
60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
   InceptionV3, Image recognition
   InceptionV4, Image recognition
   Inception-ResNetv2, Image recognition
   MobileNet v1, Image recognition
   SqueezeNet v1 and v1.1, Image recognition
   VGG16, Image recognition
   Faster RCNN, Object detection
   SSD-VGG16, Object detection
   GNMT, Language translation
   Transformer-LT, Language translation
   Wide & Deep, Recommender system
   WaveNet, Speech generation
   DeepSpeech2, Speech recognition
   DCGAN, Generative adversarial network
   A3C, Reinforcement learning

76 77 78

.. _onnx_valid:

79
ONNX
80
====
81

L.S. Cook's avatar
L.S. Cook committed
82
Additionally, we validated the following workloads are functional through 
83
`nGraph ONNX importer`_. ONNX models can be downloaded from the `ONNX Model Zoo`_.
84 85

.. csv-table::
L.S. Cook's avatar
L.S. Cook committed
86
   :header: "ONNX Workload", "Genre of Deep Learning"
87 88 89
   :widths: 27, 53
   :escape: ~

90
   ResNet-50, Image recognition
91
   ResNet-50-v2, Image recognition
92 93 94
   DenseNet-121, Image recognition
   Inception-v1, Image recognition
   Inception-v2, Image recognition
95
   Mobilenet, Image recognition
96 97 98 99 100 101 102 103 104
   Shufflenet, Image recognition
   SqueezeNet, Image recognition
   VGG-19, Image recognition
   ZFNet-512, Image recognition
   MNIST, Image recognition
   Emotion-FERPlus, Image recognition
   BVLC AlexNet, Image recognition
   BVLC GoogleNet, Image recognition
   BVLC CaffeNet, Image recognition
105 106
   BVLC R-CNN ILSVRC13, Object detection
   ArcFace, Face Detection and Recognition
107 108


109 110
.. important:: Please see Intel's `Optimization Notice`_ for details on disclaimers. 

L.S. Cook's avatar
L.S. Cook committed
111
.. rubric:: Footnotes
112

Leona C's avatar
Leona C committed
113
.. [#f1] Benchmarking performance of DL systems is a young discipline; it is a
L.S. Cook's avatar
L.S. Cook committed
114 115 116 117 118
   good idea to be vigilant for results based on atypical distortions in the 
   configuration parameters. Every topology is different, and performance 
   changes can be attributed to multiple causes. Also watch out for the word 
   "theoretical" in comparisons; actual performance should not be compared to 
   theoretical performance.
119 120 121 122




L.S. Cook's avatar
L.S. Cook committed
123 124
.. _Optimization Notice: https://software.intel.com/en-us/articles/optimization-notice
.. _nGraph ONNX importer: https://github.com/NervanaSystems/ngraph-onnx/blob/master/README.md
125
.. _ONNX Model Zoo: https://github.com/onnx/models
L.S. Cook's avatar
L.S. Cook committed
126 127 128 129 130 131 132 133 134 135

.. Notice revision #20110804: Intel's compilers may or may not optimize to the same degree for 
   non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. 
   These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. 
   Intel does not guarantee the availability, functionality, or effectiveness of any optimization 
   on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this 
   product are intended for use with Intel microprocessors. Certain optimizations not specific 
   to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the 
   applicable product User and Reference Guides for more information regarding the specific 
   instruction sets covered by this notice.
136