onnx_integ.rst 3.04 KB
Newer Older
1
.. frameworks/onnx_integ.rst:
2

3 4 5 6 7

ONNX Support
============


Leona C's avatar
Leona C committed
8 9 10
nGraph is able to import and execute ONNX models. Models are converted to 
nGraph's internal representation and converted to ``Function`` objects, which 
can be compiled and executed on one of nGraph's backends.
11

Leona C's avatar
Leona C committed
12 13 14 15 16 17 18 19
You can use nGraph's Python API to run an ONNX model and nGraph can be used 
as an ONNX backend using the add-on package `nGraph-ONNX <ngraph_onnx>`_.


.. note:: In order to support ONNX, nGraph must be built with the 
   ``NGRAPH_ONNX_IMPORT_ENABLE`` flag. See `Building nGraph-ONNX 
   <ngraph_onnx_building>`_ for more information. All nGraph packages 
   published on PyPI are built with ONNX support.
20 21 22 23 24


Installation
------------

Leona C's avatar
Leona C committed
25 26
To prepare your environment to use nGraph and ONNX, install the Python packages
for nGraph, ONNX and NumPy:
27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

::

    $ pip install ngraph-core onnx numpy


Importing an ONNX model
-----------------------

You can download models from the `ONNX Model Zoo <onnx_model_zoo_>`_.
For example ResNet-50:

::

    $ wget https://s3.amazonaws.com/download.onnx/models/opset_9/resnet50.tar.gz
    $ tar -xzvf resnet50.tar.gz


Leona C's avatar
Leona C committed
45 46
Use the following Python commands to convert the downloaded model to an nGraph 
``Function``:
47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

.. code-block:: python

    # Import ONNX and load an ONNX file from disk
    >>> import onnx
    >>> onnx_protobuf = onnx.load('resnet50/model.onnx')

    # Convert ONNX model to an ngraph model
    >>> from ngraph.impl.onnx_import import import_onnx_model
    >>> ng_function = import_onnx_model(onnx_protobuf.SerializeToString())

    # The importer returns a list of ngraph models for every ONNX graph output:
    >>> print(ng_function)
    <Function: 'resnet50' ([1, 1000])>


Leona C's avatar
Leona C committed
63 64
This creates an nGraph ``Function`` object, which can be used to execute a 
computation on a chosen backend.
65 66 67 68

Running a computation
---------------------

Leona C's avatar
Leona C committed
69 70 71 72
You can now create an nGraph ``Runtime`` backend and use it to compile your 
``Function`` to a backend-specific ``Computation`` object. Finally, you can 
execute your model by calling the created ``Computation`` object with input 
data:
73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93

.. code-block:: python

    # Using an nGraph runtime (CPU backend) create a callable computation object
    >>> import ngraph as ng
    >>> runtime = ng.runtime(backend_name='CPU')
    >>> resnet_on_cpu = runtime.computation(ng_function)
    >>> print(resnet_on_cpu)
    <Computation: resnet50(Parameter_269)>

    # Load an image (or create a mock as in this example)
    >>> import numpy as np
    >>> picture = np.ones([1, 3, 224, 224], dtype=np.float32)

    # Run computation on the picture:
    >>> resnet_on_cpu(picture)
    [array([[2.16105007e-04, 5.58412226e-04, 9.70510227e-05, 5.76671446e-05,
             7.45318757e-05, 4.80892748e-04, 5.67404088e-04, 9.48728994e-05,
             ...


Leona C's avatar
Leona C committed
94 95
Find more information about nGraph and ONNX in the 
`nGraph-ONNX <ngraph_onnx>`_ GitHub repository.
96 97 98 99 100


.. _ngraph_onnx: https://github.com/NervanaSystems/ngraph-onnx/
.. _ngraph_onnx_building: https://github.com/NervanaSystems/ngraph-onnx/blob/master/BUILDING.md
.. _onnx_model_zoo: https://github.com/onnx/models