about.rst

.. about: 

Overview
========


Welcome to the documentation site for nGraph™, an open-source C++ Compiler, 
Library, and runtime suite for running training and inference on 
:abbr:`Deep Neural Network (DNN)` models. nGraph is framework-neutral and can be 
targeted for programming and deploying :abbr:`Deep Learning (DL)` applications 
on the most modern compute and edge devices.   

Features
--------

:ref:`no-lockin`
:ref:`framework-flexibility`


.. _no-lockin:

Develop without lock-in
~~~~~~~~~~~~~~~~~~~~~~~

.. figure:: ../graphics/develop-without-lockin.png
   :width: 650px
  

Indeed, capabilities to increase training performance or to reduce inference 
latency by simply adding another device of *any* specialized form factor -- 
whether it be more compute (CPU), GPU or VPU processing power, custom ASIC or 
FPGA, or a yet-to-be invented generation of NNP or accelerator -- are the key 
benefits for frameworks developers working with nGraph. Our commitment to bake 
flexibility into our ecosystem ensures developers' freedom to design user-facing 
APIs for various hardware deployments directly into the framework. 
Developers working on things like edge-devices augmented by machine learning, or 
large distributed training clusters, or those who simply want a framework 
without restrictive lock-in to let users switch or upgrade backends quickly and 
easily.  


.. figure:: ../graphics/ngraph-ecosystem.png
   :width: 585px   
  
nGraph currently supports :doc:`three popular <../framework-integration-guides>` 
frameworks for :abbr:`Deep Learning (DL)` models through what we call 
a :term:`bridge` that can be integrated during the framework's build time. 
For developers working with other frameworks (even those not listed above), 
we've created a :doc:`How to Guide <../howto/index>` so you can learn how to 
create custom bridge code that can be used to 
:doc:`compile and run <../howto/execute>` a training model.  

Additionally We've recently added initial support for the `ONNX`_ format. Developers who 
already have a "trained" model can use nGraph to bypass a lot of the 
framework-based complexity and :doc:`../howto/import` to test or run it 
on targeted and efficient backends with our user-friendly ``ngraph_api``. 
With nGraph, data scientists can focus on data science rather than worrying 
about how to adapt models to train and run efficiently on different devices.


Supported platforms
--------------------


* Intel® Architecture Processors (CPUs), 
* Intel® Nervana™ Neural Network Processor™ (NNPs), and 
* NVIDIA\* CUDA (GPUs). 





.. note:: The library code is under active development as we're continually 
   adding support for more kinds of DL models and ops, framework compiler 
   optimizations, and backends. 


Why was this needed?
---------------------

When Deep Learning (DL) frameworks first emerged as the vehicle for training 
models, they were designed around kernels optimized for a particular platform. 
As a result, many backend details were being exposed in the model definitions, 
making the adaptability and portability of DL models to other, or more advanced 
backends inherently complex and expensive.

The traditional approach means that an algorithm developer cannot easily adapt 
his or her model to different backends. Making a model run on a different 
framework is also problematic because the user must separate the essence of 
the model from the performance adjustments made for the backend, translate 
to similar ops in the new framework, and finally make the necessary changes 
for the preferred backend configuration on the new framework.

We designed the Intel nGraph project to substantially reduce these kinds of 
engineering complexities. Our compiler-inspired approach means that developers 
have fewer constraints imposed by frameworks when working with their models; 
they can pick and choose only the components they need to build custom algorithms 
for advanced deep learning tasks. Furthermore, if working with a model that is 
already trained (or close to being trained), or if they wish to pivot and add a 
new layer to an existing model, the data scientist can :doc:`../howto/import` 
and start working with :doc:`../ops/index` more quickly. 


How does it work?
------------------

The *nGraph core* uses a **strongly-typed and platform-neutral stateless graph 
representation** for computations. Each node, or *op*, in the graph corresponds
to one :term:`step` in a computation, where each step produces zero or more 
tensor outputs from zero or more tensor inputs. For a more detailed dive into 
how this works, read our documentation on how to :doc:`../howto/execute`.


.. _framework-flexibility:

How do I connect it to a framework? 
------------------------------------

Currently, we offer *framework bridges* for some of the more widely-known 
:doc:`frameworks <../framework-integration-guides>`. The bridge acts as an 
intermediary between the *ngraph core* and the framework, providing a means 
to use various execution platforms. The result is a function that can be 
executed from the framework bridge.

Given that we have no way to predict how many more frameworks might be invented
for either model or framework-specific purposes, it would be nearly impossible 
for us to create bridges for every framework that currently exists (or that will 
exist in the future). Thus, the library provides a way for developers to write 
or contribute "bridge code" for various frameworks.  We welcome such 
contributions from the DL community.


How do I connect a DL training or inference model to nGraph?
-------------------------------------------------------------

Framework bridge code is *not* the only way to connect a model (function graph) 
to nGraph's :doc:`../ops/index`. We've also built an importer for models that 
have been exported from a framework and saved as serialized file, such as ONNX. 
To learn how to convert such serialized files to an nGraph model, please see 
the :doc:`../howto/import` documentation.  


What's next?
-------------
  
We developed nGraph to simplify the realization of optimized deep learning 
performance across frameworks and hardware platforms. You can read more about 
design decisions and what is tentatively in the pipeline for development in 
our `arXiv paper`_ from the 2018 SysML conference.


.. _arXiv paper: https://arxiv.org/pdf/1801.08058.pdf
.. _ONNX: http://onnx.ai 
.. _Intel® MKL-DNN: https://github.com/intel/mkl-dnn
.. _Movidius: https://developer.movidius.com/