index.rst 3.97 KB
Newer Older
1
.. backends/index.rst
L.S. Cook's avatar
L.S. Cook committed
2

3 4 5
#####################
Working with Backends
#####################
6

7
* :ref:`what_is_backend`
Leona C's avatar
Leona C committed
8
* :ref:`how_to_use`
Leona C's avatar
Leona C committed
9
* :ref:`miscellaneous_resources`
10 11


12 13
.. _what_is_backend:

14 15
What is a backend?
------------------
L.S. Cook's avatar
L.S. Cook committed
16

17
In the nGraph Compiler stack, what we call a *backend* is responsible for 
18 19 20 21 22 23
function execution and value allocation. A backend can be used to 
:doc:`carry out computations<../core/constructing-graphs/execute>` 
from a framework on a CPU, GPU, or ASIC; it can also be used with an 
*Interpreter* mode, which is primarily intended for testing, to analyze a 
program, or to help a framework developer customize targeted solutions. 

24 25 26
nGraph also provides a way to use the advanced tensor compiler PlaidML 
as a backend; you can learn more about this backend and how to build it 
from source in our documentation: :ref:`ngraph_plaidml_backend`.
27

Leona C's avatar
Leona C committed
28 29 30 31 32 33 34 35
.. csv-table::
   :header: "Backend", "Current nGraph support", "Future nGraph support"
   :widths: 35, 10, 10

   Intel® Architecture Processors (CPUs), Yes, Yes
   Intel® Nervana™ Neural Network Processor™ (NNPs), Yes, Yes
   AMD\* GPUs, Yes, Some

L.S. Cook's avatar
L.S. Cook committed
36

Leona C's avatar
Leona C committed
37 38 39 40 41 42 43
Each backend must define a function ``ngraph_register_${backend}_backend`` 
that registers a backend constructor function and ensures that 
initializations are performed. An example that includes initializations 
can be found in the ``ngraph/src/runtime/cpu/cpu_backend.cpp`` file. See 
also: :ref:`backend_api_macros`.


Leona C's avatar
Leona C committed
44
.. _how_to_use:
L.S. Cook's avatar
L.S. Cook committed
45

Leona C's avatar
Leona C committed
46 47
How to use?
-----------
L.S. Cook's avatar
L.S. Cook committed
48

Leona C's avatar
Leona C committed
49 50 51 52
#. Create a ``Backend``; think of it as a compiler. 
#. A ``Backend`` can then produce an ``Executable`` by calling ``compile``. 
#. A single iteration of the executable is executed by calling the ``call``
   method on the ``Executable`` object.
53

Leona C's avatar
Leona C committed
54 55
.. figure:: ../graphics/execution-interface.png
   :width: 650px
56

Leona C's avatar
Leona C committed
57
   The execution interface for nGraph 
58

Leona C's avatar
Leona C committed
59 60
The nGraph execution API for ``Executable`` objects is a simple, five-method 
interface; each backend implements the following five functions:
61 62


Leona C's avatar
Leona C committed
63 64 65 66 67 68 69 70
* The ``create_tensor()`` method allows the bridge to create tensor objects 
  in host memory or an accelerator's memory.
* The ``write()`` and ``read()`` methods are used to transfer raw data into 
  and out of tensors that reside in off-host memory.
* The ``compile()`` method instructs the backend to prepare an nGraph function 
  for later execution.
* And, finally, the ``call()`` method is used to invoke an nGraph function 
  against a particular set of tensors.
Leona C's avatar
Leona C committed
71 72 73 74 75 76 77 78 79 80 81 82


.. _miscellaneous_resources: 

Miscellaneous resources
=======================

Additional resources for device or framework-specific configurations:

OpenCL
------

Leona C's avatar
Leona C committed
83 84 85
OpenCL is needed for the :doc:`plaidml-ng-api/index`; this is not needed if 
you have only a CPU backend.  

Leona C's avatar
Leona C committed
86 87 88 89 90 91 92 93 94 95 96 97 98
#. Install the latest Linux driver for your system. You can find a list 
   of drivers at https://software.intel.com/en-us/articles/opencl-drivers;
   You may need to install `OpenCL SDK`_ in case of an ``libOpenCL.so`` absence.

#. Any user added to "video" group: 

   .. code-block:: console 

      sudo usermod –a –G video <user_id>

   may, for example, be able to find details at the ``/sys/module/[system]/parameters/`` location. 


Leona C's avatar
Leona C committed
99
nGraph Bridge from TensorFlow\*
Leona C's avatar
Leona C committed
100 101 102 103 104 105
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When specified as the generic backend -- either manually or automatically 
from a framework --  ``NGRAPH`` defaults to CPU, and it also allows for 
additional device configuration or selection. 

Leona C's avatar
Leona C committed
106 107 108
Because nGraph can select backends, specifying the ``INTELGPU`` 
backend as a runtime environment variable also works if one is 
present in your system: 
Leona C's avatar
Leona C committed
109 110 111 112 113 114 115 116 117 118

:envvar:`NGRAPH_TF_BACKEND="INTELGPU"`

An `axpy.py example`_ is optionally available to test; outputs will vary 
depending on the parameters specified. 

.. code-block:: console

   NGRAPH_TF_BACKEND="INTELGPU" python3 axpy.py

Leona C's avatar
Leona C committed
119 120
* ``NGRAPH_INTELGPU_DUMP_FUNCTION`` -- dumps nGraph’s functions 
  in dot format.
Leona C's avatar
Leona C committed
121 122 123 124


.. _axpy.py example: https://github.com/tensorflow/ngraph-bridge/blob/master/examples/axpy.py
.. _OpenCL SDK: https://software.intel.com/en-us/opencl-sdk