Commit 2d0721d5 authored by L.S. Cook's avatar L.S. Cook Committed by Robert Kimball

Update fusion doc and add ONNX build flag to buildlb doc (#1585)

* Update fusion doc and add ONNX build flag to buildlb doc

* Fix PR comments

* Final PR review comments addreswsed

* Fix link on reformmatted doc README

* Delete index.rst.save
parent c631b50b
...@@ -85,12 +85,13 @@ The process documented here will work on Ubuntu\* 16.04 (LTS) or on Ubuntu ...@@ -85,12 +85,13 @@ The process documented here will work on Ubuntu\* 16.04 (LTS) or on Ubuntu
$ mkdir build && cd build $ mkdir build && cd build
#. Generate the GNU Makefiles in the customary manner (from within the #. Generate the GNU Makefiles in the customary manner (from within the
``build`` directory). This command sets the target build location to ``build`` directory). This command enables ONNX support in the library
be ``~/ngraph_dist``, where it can be easily located. and sets the target build location at ``~/ngraph_dist``, where it can be
found easily.
.. code-block:: console .. code-block:: console
$ cmake ../ -DCMAKE_INSTALL_PREFIX=~/ngraph_dist $ cmake .. -DNGRAPH_ONNX_IMPORT_ENABLE=ON -DCMAKE_INSTALL_PREFIX=~/ngraph_dist
**Other optional build flags** -- If running ``gcc-5.4.0`` or ``clang-3.9``, **Other optional build flags** -- If running ``gcc-5.4.0`` or ``clang-3.9``,
remember that you can also append ``cmake`` with the prebuilt LLVM option remember that you can also append ``cmake`` with the prebuilt LLVM option
...@@ -157,14 +158,15 @@ The process documented here will work on CentOS 7.4. ...@@ -157,14 +158,15 @@ The process documented here will work on CentOS 7.4.
$ make && make install $ make && make install
#. Clone the `NervanaSystems` ``ngraph`` repo via HTTPS and use Cmake 3.4.3 to #. Clone the `NervanaSystems` ``ngraph`` repo via HTTPS and use Cmake 3.4.3 to
build nGraph Libraries to ``~/ngraph_dist``. build nGraph Libraries to ``~/ngraph_dist``. This command enables ONNX
support in the library (optional).
.. code-block:: console .. code-block:: console
$ cd /opt/libraries $ cd /opt/libraries
$ git clone https://github.com/NervanaSystems/ngraph.git $ git clone https://github.com/NervanaSystems/ngraph.git
$ cd ngraph && mkdir build && cd build $ cd ngraph && mkdir build && cd build
$ ~/cmake/bin/cmake .. -DCMAKE_INSTALL_PREFIX=~/ngraph_dist $ ~/cmake/bin/cmake .. -DCMAKE_INSTALL_PREFIX=~/ngraph_dist -DNGRAPH_ONNX_IMPORT_ENABLE=ON
$ make && sudo make install $ make && sudo make install
...@@ -189,8 +191,8 @@ according to those conventions. These scripts require the command ...@@ -189,8 +191,8 @@ according to those conventions. These scripts require the command
$ echo 'export PATH=$HOME/bin:$PATH' >> $HOME/.bash_profile $ echo 'export PATH=$HOME/bin:$PATH' >> $HOME/.bash_profile
Test Testing the build
==== =================
The |InG| library code base uses GoogleTest's\* `googletest framework`_ The |InG| library code base uses GoogleTest's\* `googletest framework`_
for unit tests. The ``cmake`` command from the :doc:`buildlb` guide for unit tests. The ``cmake`` command from the :doc:`buildlb` guide
......
...@@ -5,38 +5,52 @@ Optimize Graphs ...@@ -5,38 +5,52 @@ Optimize Graphs
=============== ===============
with nGraph Compiler fusions with nGraph Compiler fusions
----------------------------- ----------------------------
The nGraph Compiler is an optimizing compiler. As such, it performs a series
of optimization passes over a given function graph to translate it into a
semantically-equivalent and inherently-optimized graph with superior runtime
characteristics for any of nGraph's current or future backends. Indeed, a
framework's capability to increase training performance or to reduce inference
latency by simply adding another device of *any* specialized form factor (CPU,
GPU, VPU, or FPGA) is one of the :doc:`key benefits <../project/about>` of
developing upon a framework that uses the nGraph Compiler.
In handling a :term:`function graph`, there are many ways to describe what
happens when we translate the framework's output of ops into an nGraph
graph. :term:`Fusion` is the term we shall use in our documentation, but the the
action also can be described as: *combining*, *folding*, *collapsing*, or
*merging* of graph functions. The most common use case is to *fuse* a subgraph
from the function graph into :doc:`one of the nGraph Core ops <../ops/index>`.
Optimization passes may include algebraic simplifications, domain-specific The nGraph Compiler is an optimizing compiler. As such, it provides a way to
simplifications, and fusion. Most passes share the same mode of operation (or capture a given :term:`function graph` and perform a series of optimization
the same operational structure) and consist of two stages: passes over that graph. The result is a semantically-equivalent graph that, when
executed using any |InG| :doc:`backend <../programmable/index>`, has optimizations
#. Locating a list of potential transformation candidates (usually, subgraphs) inherent at the hardware level: superior runtime characteristics to increase
in the given graph. training performance or reduce inference latency.
#. Transforming the selected candidates into semantically-equivalent subgraphs
that run faster and/or with less memory.
Optimization passes can be programmed ahead of time if you know what your graph There are several ways to describe what happens when we capture and translate
will look like when it's ready to be executed, or the optimization passes can the framework's output of ops into an nGraph graph. :term:`Fusion` is the term
be figured out manually with *Interpreter* mode on a stateless graph. we shall use in our documentation; the action also can be described as:
*combining*, *folding*, *squashing*, *collapsing*, or *merging* of graph
functions.
Let us first consider an example. A user would like to execute a simple graph Optimization passes may include algebraic simplifications, domain-specific
simplifications, and fusion. Most passes share the same mode of operation (or
the same operational structure) and consist of various stages (each one a
:term:`step`) where a developer can experiment with the intercepted or dynamic
graph. These steps may be cycled or recycled as needed:
#. Locate a list of potentially-transformable subgraphs in the given graph.
#. Transform the selected candidates into semantically-equivalent subgraphs
that execute faster, or with less memory (or both).
#. Verify that the optimization pass performs correctly, with any or all expected
transformations, with the ``NGRAPH_SERIALIZE_TRACING`` option, which
serializes a graph in the `json` format after a pass.
#. Measure and evaluate your performance improvements with ``NGRAPH_CPU_TRACING``,
which produces timelines compatible with ``chrome://tracing``.
Optimizations can be experimented upon without using any backend by registering
a pass with pass manager (``Manager``), calling ``run_passes`` on a function, and
then inspecting the transformed graph.
Optimization passes can be programmed ahead of time if you know or can predict
what your graph will look like when it's ready to be executed (in other words:
which `ops` can be automatically translated into :doc:`nGraph Core ops <../ops/index>`).
The ``Interpreter`` is simply a backend providing reference implementations of
ngraph ops in C++, with the focus on simplicity over performance.
Example
-------
Let us first consider a simple example. A user would like to execute a graph
that describes the following arithmetic expression: that describes the following arithmetic expression:
:math:`a + b * 1` or :math:`Add(a, Mul(b, 1))` :math:`a + b * 1` or :math:`Add(a, Mul(b, 1))`
...@@ -52,14 +66,13 @@ multiplicands are multiplied by `1` (for stage 1) and to then ``transform``, ...@@ -52,14 +66,13 @@ multiplicands are multiplied by `1` (for stage 1) and to then ``transform``,
``simplify``, or ``replace`` those expressions with just their multiplicands ``simplify``, or ``replace`` those expressions with just their multiplicands
(for stage 2). (for stage 2).
To make the work of an optimization pass writer easier, the nGraph library To make the work of an optimization pass writer easier, the nGraph Library
includes facilities that enable the *finding* of relevant candidates using includes facilities that enable the *finding* of relevant candidates using
pattern matching (via ``pattern/matcher.hpp``), and the *transforming* of the pattern matching (via ``pattern/matcher.hpp``), and the *transforming* of the
original graph into a condensed version (via ``pass/graph_rewrite.hpp``). original graph into a condensed version (via ``pass/graph_rewrite.hpp``).
Let's consider each of the two in more detail and many ways they can help the Let's consider each in more detail and many ways they can help the graph
work of the optimization pass writer. optimizer.
.. toctree:: .. toctree::
...@@ -67,7 +80,4 @@ work of the optimization pass writer. ...@@ -67,7 +80,4 @@ work of the optimization pass writer.
graph-rewrite.rst graph-rewrite.rst
passes-that-use-matcher.rst passes-that-use-matcher.rst
...@@ -4,7 +4,7 @@ Overview ...@@ -4,7 +4,7 @@ Overview
======== ========
Welcome to the documentation site for nGraph™, an open-source C++ Compiler, Welcome to the documentation site for |InG|, an open-source C++ Compiler,
Library, and runtime suite for running training and inference on Library, and runtime suite for running training and inference on
:abbr:`Deep Neural Network (DNN)` models. nGraph is framework-neutral and can be :abbr:`Deep Neural Network (DNN)` models. nGraph is framework-neutral and can be
targeted for programming and deploying :abbr:`Deep Learning (DL)` applications targeted for programming and deploying :abbr:`Deep Learning (DL)` applications
...@@ -26,17 +26,13 @@ Develop without lock-in ...@@ -26,17 +26,13 @@ Develop without lock-in
:width: 650px :width: 650px
Indeed, capabilities to increase training performance or to reduce inference Being able to increase training performance or reduce inference latency by simply
latency by simply adding another device of *any* specialized form factor -- adding another device of *any* specialized form factor -- whether it be more
whether it be more compute (CPU), GPU or VPU processing power, custom ASIC or compute (CPU), GPU or VPU processing power, custom ASIC or FPGA, or a yet-to-be
FPGA, or a yet-to-be invented generation of NNP or accelerator -- are the key invented generation of NNP or accelerator -- is a key benefit for frameworks
benefits for frameworks developers working with nGraph. Our commitment to bake developers working with nGraph. Our commitment to bake flexibility into our
flexibility into our ecosystem ensures developers' freedom to design user-facing ecosystem ensures developers' freedom to design user-facing APIs for various
APIs for various hardware deployments directly into the framework. hardware deployments directly into their frameworks.
Developers working on things like edge-devices augmented by machine learning, or
large distributed training clusters, or those who simply want a framework
without restrictive lock-in to let users switch or upgrade backends quickly and
easily.
.. figure:: ../graphics/ngraph-ecosystem.png .. figure:: ../graphics/ngraph-ecosystem.png
...@@ -50,12 +46,14 @@ we've created a :doc:`How to Guide <../howto/index>` so you can learn how to ...@@ -50,12 +46,14 @@ we've created a :doc:`How to Guide <../howto/index>` so you can learn how to
create custom bridge code that can be used to create custom bridge code that can be used to
:doc:`compile and run <../howto/execute>` a training model. :doc:`compile and run <../howto/execute>` a training model.
Additionally We've recently added initial support for the `ONNX`_ format. Developers who Additionally, nGraph Library supports the `ONNX`_ format. Developers who
already have a "trained" model can use nGraph to bypass a lot of the already have a "trained" model can use nGraph to bypass much of the
framework-based complexity and :doc:`../howto/import` to test or run it framework-based complexity and :doc:`../howto/import` to test or run it
on targeted and efficient backends with our user-friendly ``ngraph_api``. on targeted and efficient backends with our user-friendly ``ngraph_api``.
With nGraph, data scientists can focus on data science rather than worrying With nGraph, data scientists can focus on data science rather than worrying
about how to adapt models to train and run efficiently on different devices. about how to adapt models to train and run efficiently on different devices.
Be sure to add the ``-DNGRAPH_ONNX_IMPORT_ENABLE=ON`` option when running `cmake`
to build the Library.
Supported platforms Supported platforms
...@@ -66,8 +64,15 @@ Supported platforms ...@@ -66,8 +64,15 @@ Supported platforms
* Intel® Nervana™ Neural Network Processor™ (NNPs), and * Intel® Nervana™ Neural Network Processor™ (NNPs), and
* NVIDIA\* CUDA (GPUs). * NVIDIA\* CUDA (GPUs).
We built the first-generation of the Intel Nervana™ NNP family of processors
last year to show that the nGraph Library can be used to train a
:abbr:`Neural Network (NN)` more quickly. The more advanced the silicon, the
more powerful a lightweight a library can be. So while we do currently support
traditional GPUs, they are not advanced silicon, and trying to scale workloads
using traditional GPU libraries is clunky and brittle with bottlenecks. Iteration
from an already-trained NN model to one that can also perform inference
computations is immensely simplified. Read more about these compute-friendly
options on the documentation for :doc:`../fusion/index`.
.. note:: The library code is under active development as we're continually .. note:: The library code is under active development as we're continually
...@@ -82,7 +87,7 @@ When Deep Learning (DL) frameworks first emerged as the vehicle for training ...@@ -82,7 +87,7 @@ When Deep Learning (DL) frameworks first emerged as the vehicle for training
models, they were designed around kernels optimized for a particular platform. models, they were designed around kernels optimized for a particular platform.
As a result, many backend details were being exposed in the model definitions, As a result, many backend details were being exposed in the model definitions,
making the adaptability and portability of DL models to other, or more advanced making the adaptability and portability of DL models to other, or more advanced
backends inherently complex and expensive. backends complex and expensive.
The traditional approach means that an algorithm developer cannot easily adapt The traditional approach means that an algorithm developer cannot easily adapt
his or her model to different backends. Making a model run on a different his or her model to different backends. Making a model run on a different
......
# nGraph python binding # Python binding for the nGraph Library
## Installation ## Build the nGraph Library (required)
### Install nGraph (Required) Follow the [build instructions] to build nGraph. When you get to the `cmake`
command, be sure to specify the option that enables ONNX support in the Library:
``` $ cmake ../ -DNGRAPH_ONNX_IMPORT_ENABLE=ON
Follow the [steps](http://ngraph.nervanasys.com/docs/latest/install.html) to build and install ngraph.
```
Clone the pybind repository
``` Next, clone the `pybind11` repository:
cd ngraph/python
git clone --recursive -b allow-nonconstructible-holders https://github.com/jagerman/pybind11.git
```
Set the environment variables $ cd ngraph/python
$ git clone --recursive -b allow-nonconstructible-holders https://github.com/jagerman/pybind11.git
```
export NGRAPH_CPP_BUILD_PATH=$HOME/ngraph_dist
export LD_LIBRARY_PATH=$HOME/ngraph_dist/lib
export DYLD_LIBRARY_PATH=$HOME/ngraph_dist/lib # (Only needed on MacOS)
export PYBIND_HEADERS_PATH=pybind11
```
Install Wrapper (python binding) Set the environment variables:
``` export NGRAPH_CPP_BUILD_PATH=$HOME/ngraph_dist
python setup.py install export LD_LIBRARY_PATH=$HOME/ngraph_dist/lib
``` export DYLD_LIBRARY_PATH=$HOME/ngraph_dist/lib # (Only needed on MacOS)
export PYBIND_HEADERS_PATH=pybind11
To run unit tests, first install additional required packages.
``` Install the wrapper (Python binding):
pip install -r test_requirements.txt
```
Then run a test. $ python setup.py install
Unit tests require additional packages be installed:
$ pip install -r test_requirements.txt
Then run a test:
$ pytest test/test_ops.py
$ pytest test/ngraph/
```
pytest test/test_ops.py
pytest test/ngraph/
```
## Running tests with tox ## Running tests with tox
[Tox](https://tox.readthedocs.io/) is a Python [virtualenv](https://virtualenv.pypa.io/) management and test command line tool. In our project it automates: [Tox] is a Python [virtualenv] management and test command line tool. In our
project it automates:
* running unit tests using [pytest](https://docs.pytest.org/) * running of unit tests with [pytest]
* checking that code style is compliant with [PEP8](https://www.python.org/dev/peps/pep-0008/) using [Flake8](http://flake8.pycqa.org/) * checking that code style is compliant with [PEP8] using [Flake8]
* static type checking using [MyPy](http://mypy.readthedocs.io) * static type checking using [MyPy]
* testing across Python 2 and 3 * testing across Python 2 and 3
Installing and running test with Tox: Installing and running test with Tox:
pip install tox $ pip install tox
tox $ tox
You can run tests using only Python 3 or 2 using the `-e` (environment) switch: You can run tests using only Python 3 or 2 using the `-e` (environment) switch:
tox -e py36 $ tox -e py36
tox -e py27 $ tox -e py27
You can check styles in a particular code directory by specifying the path: You can check styles in a particular code directory by specifying the path:
tox ngraph/ $ tox ngraph/
In case of problems, try to recreate the virtual environments by deleting the `.tox` directory: If you run into any problems, try recreating the virtual environments by
deleting the `.tox` directory:
``` $ rm -rf .tox
rm -rf .tox $ tox
tox
```
[build instructions]:http://ngraph.nervanasys.com/docs/latest/buildlb.html
[Tox]:https://tox.readthedocs.io/
[virtualenv]:https://virtualenv.pypa.io/
[pytest]:https://docs.pytest.org/
[PEP8]:https://www.python.org/dev/peps/pep-0008
[Flake8]:http://flake8.pycqa.org
[MyPy]:http://mypy.readthedocs.io
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment