Skip to content

N
ngraph
Commit 5f0391d3 authored by Leona C's avatar Leona C Committed by Scott Cyphers
Browse files
Options

Add instructions for building the NGRAPH_PLAIDML backend, (#3206) 

* Add instructions for building the NGRAPH_PLAIDML backend, ref on latest ToC

* Fix typo

* Reword paragraph to reference ngraph-plaidml build instruction page

* PR review suggestions added
parent 9b748d2c
Hide whitespace changes
Inline Side-by-side
Showing with 214 additions and 267 deletions
doc/sphinx/source/backends/index.rst
View file @ 5f0391d3
.. backends/index.rst
#################################
Developer Resources for Backends
#################################
#####################
Working with Backends
#####################
* :ref:`what_is_backend`
* :ref:`how_to_use`
......@@ -11,17 +11,19 @@ Developer Resources for Backends
.. _what_is_backend:
What's a backend?
-----------------
What is a backend?
------------------
In the nGraph Compiler stack, what we call a *backend* is responsible for
function execution and value allocation. A backend can be used to
:doc:`carry out a programmed computation<../core/constructing-graphs/execute>`
from a framework on a CPU or GPU; or it can be used with an *Interpreter* mode,
which is primarily intended for testing, to analyze a program, or to help a
framework developer customize targeted solutions. Experimental APIs to support
current and future nGraph Backends are also available; see, for example, the
section on :doc:`plaidml-ng-api/index`.
function execution and value allocation. A backend can be used to
:doc:`carry out computations<../core/constructing-graphs/execute>`
from a framework on a CPU, GPU, or ASIC; it can also be used with an
*Interpreter* mode, which is primarily intended for testing, to analyze a
program, or to help a framework developer customize targeted solutions.
nGraph also provides a way to use the advanced tensor compiler PlaidML
as a backend; you can learn more about this backend and how to build it
from source in our documentation: :ref:`ngraph_plaidml_backend`.
.. csv-table::
:header: "Backend", "Current nGraph support", "Future nGraph support"
......@@ -48,7 +50,6 @@ How to use?
The execution interface for nGraph
The nGraph execution API for ``Executable`` objects is a simple, five-method
interface; each backend implements the following five functions:
......
doc/sphinx/source/buildlb.rst
View file @ 5f0391d3
.. buildlb.rst:
###############
Build and Test
Build and Test
###############
This section details how to build the C++ version of the nGraph Library, which
is targeted toward developers working on kernel-specific operations,
optimizations, or on deep learning solutions that leverage custom backends.
* :ref:`default_ngflags`
* :ref:`ngraph_plaidml_backend`
* :ref:`ubuntu`
* :ref:`centos`
There are a few common paths to take when manually building the |project|
from source code. Today nGraph supports various developers working on all
parts of the :abbr:`Deep Learning (DL)` stack, and the way you decide to
build or install components ought to depend on the capabilities of your
hardware, and how you intend to use it.
A "from scratch" source-code build of the nGraph Library enables the CPU,
``Interpreter``, and unit tests by default. See :ref:`default_ngflags`
for more detail.
Prerequisites
=============
A "from scratch" source-code build that defaults to the PlaidML backend
contains rich algorithm libraries akin to those that were previously available
only to developers willing to spend extensive time writing, testing, and
customizing kernels. An ``NGRAPH_PLAIDML`` dist can function like a framework
that lets developers compose, train, and even deploy :abbr:`DL (Deep Learning)`
models in their preferred language on neural networks of any size. This is
a good option if, for example, you are working on a laptop with a high-end
GPU that you want to use for compute. See :ref:`ngraph_plaidml_backend`
for instructions on how to build.
In either case, there are some prerequisites that your system will need
to build from sources.
.. _prerequisites:
Release |release| of |project| supports Linux\*-based systems with the following
packages and prerequisites:
Prerequisites
-------------
.. csv-table::
:header: "Operating System", "Compiler", "Build System", "Status", "Additional Packages"
......@@ -27,71 +44,36 @@ packages and prerequisites:
Ubuntu 16.04 or 18.04 (LTS) 64-bit, Clang 3.9, CMake 3.5.1 + GNU Make, supported, ``build-essential cmake clang-3.9 clang-format-3.9 git curl zlib1g zlib1g-dev libtinfo-dev unzip autoconf automake libtool``
Clear Linux\* OS for Intel® Architecture version 28880, Clang 8.0, CMake 3.14.2, experimental, bundles ``machine-learning-basic c-basic python-basic python-basic-dev dev-utils``
Other configurations may work, but should be considered experimental with
limited support. On Ubuntu 16.04 with gcc-5.4.0 or clang-3.9, for example, we
recommend adding ``-DNGRAPH_USE_PREBUILT_LLVM=TRUE`` to the cmake command in
step 4 below. This fetches a pre-built tarball of LLVM+Clang from llvm.org,
and it will substantially reduce build time.
If using ``gcc`` version 4.8, it may be necessary to add symlinks from ``gcc``
to ``gcc-4.8``, and from ``g++`` to ``g++-4.8``, in your :envvar:`PATH`, even
if you explicitly specify the ``CMAKE_C_COMPILER`` and ``CMAKE_CXX_COMPILER``
flags when building. (**Do NOT** supply the ``-DNGRAPH_USE_PREBUILT_LLVM``
flag in this case, because the prebuilt tarball supplied on llvm.org is not
compatible with a gcc 4.8-based build.)
The ``default`` build
---------------------
Running ``cmake`` with no build flags defaults to the following settings; see
the ``CMakeLists.txt`` file for other experimental options' details:
.. code-block:: console
-- NGRAPH_UNIT_TEST_ENABLE: ON
-- NGRAPH_TOOLS_ENABLE: ON
-- NGRAPH_CPU_ENABLE: ON
-- NGRAPH_INTELGPU_ENABLE: OFF
-- NGRAPH_GPU_ENABLE: OFF
-- NGRAPH_INTERPRETER_ENABLE: ON
-- NGRAPH_NOP_ENABLE: ON
-- NGRAPH_GPUH_ENABLE: OFF
-- NGRAPH_GENERIC_CPU_ENABLE: OFF
-- NGRAPH_DEBUG_ENABLE: OFF # Set to "ON" to enable logging
-- NGRAPH_ONNX_IMPORT_ENABLE: OFF
-- NGRAPH_DEX_ONLY: OFF
-- NGRAPH_CODE_COVERAGE_ENABLE: OFF
-- NGRAPH_LIB_VERSIONING_ENABLE: OFF
-- NGRAPH_PYTHON_BUILD_ENABLE: OFF
-- NGRAPH_USE_PREBUILT_LLVM: OFF
-- NGRAPH_PLAIDML_ENABLE: OFF
-- NGRAPH_DISTRIBUTED_ENABLE: OFF
.. _default_ngflags:
Building nGraph from source
===========================
.. important:: The default :program:`cmake` procedure (no build flags) will
install ``ngraph_dist`` to an OS-level location like ``/usr/bin/ngraph_dist``
or ``/usr/lib/ngraph_dist``. Here we specify how to build locally to the
location of ``~/ngraph_dist`` with the cmake target ``-DCMAKE_INSTALL_PREFIX=~/ngraph_dist``.
All of the nGraph Library documentation presumes that ``ngraph_dist`` gets
installed locally. The system location can be used just as easily by customizing
paths on that system. See the :file:`ngraph/CMakeLists.txt` file to change or
customize the default CMake procedure.
installed locally. The system location can be used just as easily by
customizing paths on that system. See the :file:`ngraph/CMakeLists.txt`
file to change or customize the default CMake procedure.
* :ref:`ubuntu`
* :ref:`centos`
Build steps
-----------
.. _ubuntu:
Ubuntu LTS
~~~~~~~~~~
Ubuntu LTS build steps
----------------------
The process documented here will work on Ubuntu\* 16.04 (LTS) or on Ubuntu
18.04 (LTS).
#. Ensure you have installed the :ref:`prerequisites` for Ubuntu\*.
#. Clone the `NervanaSystems` ``ngraph`` repo:
.. code-block:: console
......@@ -109,7 +91,7 @@ The process documented here will work on Ubuntu\* 16.04 (LTS) or on Ubuntu
#. Generate the GNU Makefiles in the customary manner (from within the
``build`` directory). This command enables ONNX support in the library
and sets the target build location at ``~/ngraph_dist``, where it can be
found easily.
found easily.
.. code-block:: console
......@@ -132,30 +114,29 @@ The process documented here will work on Ubuntu\* 16.04 (LTS) or on Ubuntu
.. code-block:: console
$ make # note: make -j <N> may work, but sometimes results in out-of-memory errors if too many compilation processes are used
$ make install
$ make install
#. (Optional, requires `doxygen`_, `Sphinx`_, and `breathe`_). Run ``make html``
inside the ``doc/sphinx`` directory of the cloned source to build a copy of
the `website docs`_ locally. The low-level API docs with inheritance and
collaboration diagrams can be found inside the ``/docs/doxygen/`` directory.
See the :doc:`project/doc-contributor-README` for more details about how to
build documentation for nGraph.
build documentation for nGraph.
.. _centos:
CentOS 7.4
~~~~~~~~~~
CentOS 7.4 build steps
----------------------
The process documented here will work on CentOS 7.4.
#. Update the system with :command:`yum` and issue the following commands:
#. Ensure you have installed the :ref:`prerequisites` for CentOS\*,
and update the system with :command:`yum`.
.. code-block:: console
$ sudo yum update
$ sudo yum install zlib-devel install ncurses-libs ncurses-devel patch diffutils wget gcc-c++ make git perl-Data-Dumper
#. Install Cmake 3.4:
......@@ -180,6 +161,59 @@ The process documented here will work on CentOS 7.4.
$ make && sudo make install
.. _ngraph_plaidml_backend:
Building nGraph-PlaidML from source
===================================
The following instructions will create the ``~/ngraph_plaidml_dist``
locally:
#. Ensure you have installed the :ref:`prerequisites` for your OS.
#. Install the prerequisites for the backend. Our hybrid ``NGRAPH_PLAIDML``
backend works best with Python3 versions. We recommend that you use a
virtual environment, due to some of the difficulties that users have
seen when trying to install outside of a venv.
.. code-block:: console
$ sudo apt install python3-pip
$ pip install plaidml
$ plaidml-setup
#. Clone the source code, create and enter your build directory:
.. code-block:: console
$ git clone https://github.com/NervanaSystems/ngraph.git
$ cd ngraph && mkdir build && cd build
#. Prepare the CMake files as follows:
.. code-block:: console
$ cmake .. -DCMAKE_INSTALL_PREFIX=~/ngraph_plaidml_dist -DNGRAPH_CPU_ENABLE=OFF -DNGRAPH_PLAIDML_ENABLE=ON
#. Run :command:`make` and ``make install``. Note that if you are building
outside a local or user path, you may need to run ``make install`` as the
root user.
.. code-block:: console
$ make
$ make install
This should create the shared library ``libplaidml_backend.so`` and
nbench. Note that if you built in a virtual environment and run
``make check`` from it, the Google Test may report failures. Full
tests can be run when PlaidML devices are available at the machine
level.
For more about working with the PlaidML backend from nGraph, see our
API documentation :doc:`backends/plaidml-ng-api/index`.
macOS\* development
--------------------
......@@ -228,3 +262,4 @@ To perform unit tests on the install:
.. _ONNX: http://onnx.ai
.. _website docs: https://ngraph.nervanasys.com/docs/latest/
.. _googletest framework: https://github.com/google/googletest.git
.. _PlaidML: https://github.com/plaidml/plaidml
\ No newline at end of file
doc/sphinx/source/conf.py
View file @ 5f0391d3
......@@ -73,11 +73,11 @@ author = 'Intel Corporation'
# built documents.
#
# The short X.Y version.
version = '0.23'
version = '0.24'
# The Documentation full version, including alpha/beta/rc tags. Some features
# available in the latest code will not necessarily be documented first
release = '0.23.0'
release = '0.24.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
......
doc/sphinx/source/frameworks/getting_started.rst 0 → 100644
View file @ 5f0391d3
.. frameworks/getting_started.rst
Getting Started
###############
No matter what your level of experience with :abbr:`Deep Learning (DL)` systems
may be, nGraph provides a path to start working with the DL stack. Let's begin
with the easiest and most straightforward options.
.. figure:: ../graphics/translation-flow-to-ng-fofx.png
:width: 725px
:alt: Translation flow to nGraph function graph
The easiest way to get started is to use the latest PyPI `ngraph-tensorflow-bridge`_,
which has instructions for Linux* systems, and tips for users of Mac OS X.
You can install TensorFlow\* and nGraph to a virtual environment; otherwise, the code
will install to a system location.
.. code-block:: console
pip install tensorflow
pip install ngraph-tensorflow-bridge
.. note:: You may need to use the latest versions of ```tensorflow``` and the bridge
to get pip installs to work. See the :doc:`tensorflow_connect` file for more
detail about working with TensorFlow\*.
That's it! Now you can test the installation by running the following command:
.. code-block:: console
python -c "import tensorflow as tf; print('TensorFlow version: ',tf.__version__);import ngraph_bridge; print(ngraph_bridge.__version__)"
Output will look something like:
::
TensorFlow version: [version]
nGraph bridge version: b'[version]'
nGraph version used for this build: b'[version-rc-hash]'
TensorFlow version used for this build: v[version-hash]
CXX11_ABI flag used for this build: boolean
More detail in the `ngraph_bridge examples`_ directory.
ONNX
====
Another easy way to get started working with the :abbr:`DL (Deep Learning)`
stack is to try the examples available via `nGraph ONNX`_.
Installation
------------
To prepare your environment to use nGraph and ONNX, install the Python packages
for nGraph, ONNX and NumPy:
::
$ pip install ngraph-core onnx numpy
Now you can start exploring some of the :doc:`onnx_integ` examples.
See also nGraph's :doc:`../python_api/index`.
PlaidML
=======
See :ref:`ngraph_plaidml_backend` section on how to build the
nGraph-PlaidML.
Other integration paths
=======================
If you are considering incorporating components from the nGraph Compiler stack
in your framework or neural network design, another useful doc is the section
on :doc:`generic-configs`. Contents here are also useful if you are working on
something built-from-scratch, or on an existing framework that is less
widely-supported than the popular frameworks like TensorFlow and PyTorch.
.. _ngraph-tensorflow-bridge: https://pypi.org/project/ngraph-tensorflow-bridge
.. _ngraph ONNX: https://github.com/NervanaSystems/ngraph-onnx
.. _ngraph_bridge examples: https://github.com/tensorflow/ngraph-bridge/blob/master/examples/README.md
doc/sphinx/source/frameworks/index.rst
View file @ 5f0391d3
.. frameworks/index.rst:
.. frameworks/index.rst
Connect a framework
===================
Using Frameworks
================
.. toctree::
:maxdepth: 1
tensorflow_connect.rst
mxnet_integ.rst
onnx_integ.rst
getting_started.rst
onnx_integ.rst
paddle_integ.rst
validated/testing-latency.rst
A framework is "supported" when there is a framework :term:`bridge` that can be
cloned from one of our GitHub repos and built to connect to nGraph device
backends while maintaining the framework's programmatic or user interface. Bridges
currently exist for the TensorFlow\* and MXNet\* frameworks.
ONNX is not a framework; however, it can be used with nGraph's :doc:`../python_api/index`
to import and execute ONNX models.
.. figure:: ../graphics/whole-stack.png
:width: 733px
:alt: JiT compiling of a computation
:abbr:`Just-in-Time (JiT)` Compiling for computation. nGraph `Core` components
are colored in blue.
Once connected via the bridge, the framework can then run and train a deep
learning model with various workloads on various backends using nGraph Compiler
as an optimizing compiler available through the framework.
While a :abbr:`Deep Learning (DL)` :term:`framework` is ultimately meant for
end use by data scientists, or for deployment in cloud container environments,
nGraph Core ops and the nGraph C++ Library are designed for framework builders
themselves. We invite anyone working on new and novel frameworks or neural
network designs to explore our highly-modularized stack of components that can
be implemented or integrated in countless ways.
Please read this section if you are considering incorporating components from
the nGraph Compiler stack in your framework or neural network design. Contents
here are also useful if you are working on something built-from-scratch, or on
an existing framework that is less widely-supported than the popular frameworks
like TensorFlow and PyTorch.
.. figure:: ../graphics/translation-flow-to-ng-fofx.png
:width: 725px
:alt: Translation flow to nGraph function graph
.. _tune the workload to extract best performance: https://ai.intel.com/accelerating-deep-learning-training-inference-system-level-optimizations
.. _a few small: https://software.intel.com/en-us/articles/boosting-deep-learning-training-inference-performance-on-xeon-and-xeon-phi
tensorflow_connect.rst
doc/sphinx/source/frameworks/mxnet_integ.rst deleted 100644 → 0
View file @ 9b748d2c
.. frameworks/mxnet_integ.rst:
MXNet\* bridge
==============
.. deprecated:: 0.21.0
* See the nGraph-MXNet `Integration Guide`_ on the nGraph-MXNet repo.
* **Testing inference latency**: See the :doc:`validated/testing-latency`
doc for a fully-documented example how to compile and test latency with an
MXNet-supported model.
.. note:: The nGraph-MXNet bridge is designed to be used with trained models
only; it does not support distributed training.
.. _Integration Guide: https://github.com/NervanaSystems/ngraph-mxnet/blob/master/NGRAPH_README.md
doc/sphinx/source/frameworks/validated/list.rst
View file @ 5f0391d3
......@@ -10,8 +10,6 @@ workloads:
* :ref:`tensorflow_valid`
* :ref:`mxnet_valid`
* :ref:`onnx_valid`
* :doc:`testing-latency`
......
doc/sphinx/source/frameworks/validated/testing-latency.rst deleted 100644 → 0
View file @ 9b748d2c
.. frameworks/validated/testing-latency.rst:
Testing latency
###############
Many open-source DL frameworks provide a layer where experts in data science
can make use of optimizations contributed by machine learning engineers. Having
a common API benefits both: it simplifies deployment and makes it easier for ML
engineers working on advanced deep learning hardware to bring highly-optimized
performance to a wide range of models, especially in inference.
One DL framework with advancing efforts on graph optimizations is Apache
MXNet\*, where `Intel has contributed efforts showing`_ how to work with our
nGraph Compiler stack as an `experimental backend`_. Our approach provides
**more opportunities** to start working with different kinds of graph
optimizations **than would be available to the MXNet framework alone**, for
reasons outlined in our `introduction`_ documentation. Note that the
MXNet bridge requires trained models only; it does not support distributed
training.
.. figure:: ../../graphics/ngraph-mxnet-models.png
:width: 533px
:alt: Up to 45X faster
Up to 45X faster compilation with nGraph backend
Tutorial: Testing inference latency of ResNet-50-V2 with MXNet
==============================================================
This tutorial supports compiling MXNet with nGraph's CPU backend.
Begin by cloning MXNet from GitHub:
.. code-block:: console
git clone --recursive https://github.com/apache/incubator-mxnet
To compile run:
.. code-block:: console
cd incubator-mxnet
make -j USE_NGRAPH=1
MXNet's build system will automatically download, configure, and build the
nGraph library, then link it into ``libmxnet.so``. Once this is complete, we
recommend building a python3 virtual environment for testing, and then
install MXNet to the virtual environment:
.. code-block:: console
python3 -m venv .venv
. .venv/bin/activate
cd python
pip install -e .
cd ../
Now we're ready to use nGraph to run any model on a CPU backend. Building MXNet
with nGraph automatically enabled nGraph on your model scripts, and you
shouldn't need to do anything special. If you run into trouble, you can disable
nGraph by setting
.. code-block:: console
MXNET_SUBGRAPH_BACKEND=
If you do see trouble, please report it and we'll address it as soon as possible.
Running ResNet-50-V2 Inference
------------------------------
To show a working example, we'll demonstrate how MXNet may be used to run
ResNet-50 Inference. For ease, we'll consider the standard MXNet ResNet-50-V2
model from the `gluon model zoo`_, and we'll test with ``batch_size=1``.
Note that the nGraph-MXNet bridge supports static graphs only (dynamic graphs
are in the works); so for this example, we begin by converting the gluon model
into a static graph. Also note that any model with a saved checkpoint can be
considered a "static graph" in nGraph. For this example, we'll presume that the
model is pre-trained.
.. literalinclude:: ../../../../examples/subgraph_snippets/mxnet-gluon-example.py
:language: python
:lines: 17-32
To load the model into nGraph, we simply bind the symbol into an Executor.
.. literalinclude:: ../../../../examples/subgraph_snippets/mxnet-gluon-example.py
:language: python
:lines: 34-35
At binding, the MXNet Subgraph API finds nGraph, determines how to partition
the graph, and in the case of Resnet, sends the entire graph to nGraph for
compilation. This produces a single call to an `NNVM`_ ``NGraphSubgraphOp`` embedded
with the compiled model. At this point, we can test the model's performance.
.. literalinclude:: ../../../../examples/subgraph_snippets/mxnet-gluon-example.py
:language: python
:lines: 40-48
.. _experimental backend: https://github.com/apache/incubator-mxnet/pull/12502
.. _Intel has contributed efforts showing: https://cwiki.apache.org/confluence/display/MXNET/MXNet+nGraph+integration+using+subgraph+backend+interface
.. _introduction: http://ngraph.nervanasys.com/docs/latest/project/introduction.html
.. _gluon model zoo: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/model_zoo/vision/resnet.py#L499
.. _subgraph acceleration API: https://cwiki.apache.org/confluence/display/MXNET/Unified+integration+with+external+backend+libraries
.. _NNVM: https://github.com/dmlc/nnvm
.. _nGraph-MXNet: https://github.com/NervanaSystems/ngraph-mxnet/blob/master/README.md
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment