Doc the A-ops. (#482)

* Doc the A-ops.

* Better structure for ops and the docs around them, based on cyphers branch for doc-the-a-ops

* More edits for merge into preview branch

* Update link to framework integration guide page on testing libngraph

* Better arcbackprops, review comments

* Make ``input`` consistent.

* Uniform formatting.

.gitignore

@@ -52,6 +52,9 @@ doc/source/generated
 .cache/
 nervana_aeon.egg-info/
 
+# emacs
+*~
+
 # vim
 *.swp
 *.swo

README.md

@@ -17,4 +17,4 @@ See our [install] docs for how to get started.
 For this early release, we provide framework integration guides to compile 
 MXNet and TensorFlow-based projects.  
 
-[install]: doc/sphinx/source/installation.rst
+[install]: http://ngraph.nervanasys.com/docs/cpp/installation.html
\ No newline at end of file

doc/sphinx/ngraph_theme/static/css/theme.css

@@ -2220,7 +2220,7 @@ div[class^='highlight'] pre {
   background-color: #272525;
   display: block;
   text-align: right;
-  font-size: 90%;
+  font-size: 95%;
   cursor: pointer;
   color: #27AE60;
   *zoom: 1;
@@ -2488,8 +2488,8 @@ div[class^='highlight'] pre {
   line-height: 1.0em;
 }
 .rst-content tt.literal, .rst-content tt.literal, .rst-content code.literal {
-  font-size: 101% !important;
-  color: #72a1ab;
+  font-size: 100% !important;
+  color: #528481;
   line-height: 0.91em;
 }
 .rst-content tt.xref, a .rst-content tt, .rst-content tt.xref, .rst-content code.xref, a .rst-content tt, a .rst-content code {

doc/sphinx/ngraph_theme/static/css/theme.css.backup

@@ -2489,7 +2489,7 @@ div[class^='highlight'] pre {
 }
 .rst-content tt.literal, .rst-content tt.literal, .rst-content code.literal {
   font-size: 101% !important;
-  color: #72a1ab;
+  color: #528481;
   line-height: 0.91em;
 }
 .rst-content tt.xref, a .rst-content tt, .rst-content tt.xref, .rst-content code.xref, a .rst-content tt, a .rst-content code {

doc/sphinx/source/api.rst

-.. api.rst:
-
-API 
-###
-
-.. Don't add Python APIs that will break the build.  
-
-Sections
-========
-
-
-   
-
-
-   
-
-

doc/sphinx/source/autodiff.rst

-.. autodiff.rst
-
-Autodiff 
-########
-
-
-The ``autodiff`` ... 
-
-.. TODO update for cpp
-
-
-

doc/sphinx/source/branding-notice.rst

+:orphan:
+
 .. branding-notice:
 
 

doc/sphinx/source/glossary.rst

+:orphan:
+
 .. glossary: 
 
 Glossary 

doc/sphinx/source/graph-basics.rst

@@ -3,8 +3,22 @@
 Graph Basics
 ============
 
-This section describes the basic concepts you need to know when constructing
-a graph.
+This section describes the basic concepts you need to know when 
+constructing a graph.
+
+
+Framework Bridges
+------------------
+
+Frontends (or users who require the flexibility of constructing 
+Ops directly) can utilize a set of graph construction functions 
+to construct graphs. 
+
+A framework bridge constructs a function which is compiled/optimized
+by a sequence of graph transformations that replace subgraphs of the
+computation with more optimal subgraphs. Throughout this process, ops
+represent tensor operations.
+
 
 Tensors
 -------
@@ -150,38 +164,3 @@ After the graph is constructed, we create the function, passing the
 `Function` constructor the nodes that are results and the parameters
 that are arguments.
 
-
-Defining ops
-============
-
-A framework bridge constructs a function which is compiled/optimized
-by a sequence of graph transformations that replace subgraphs of the
-computation with more optimal subgraphs. Throughout this process, ops
-represent tensor operations.
-
-*Core ops* are ops that are available and generally useful to all
-framework bridges and that can be compiled by all transformers. A
-framework bridge may define framework-specific ops to simplify graph
-construction, provided that the bridge can enable every transformer to
-replace all such ops with equivalent subgraphs composed of core
-ops. Similary, transformers may define transformer-specific ops to
-represent kernels or other intermediate operations. If a framework
-supports extending the set of ops it offers, a bridge may even expose
-transformer-specific ops to the framework user.
-
-It is easiest to define a new op by adapting an existing op. Some of
-the tasks that must be performed are:
-
-- Op constructor:
-
-  * Checking type-consistency of arguments 
-  * Specifying the result type for a call 
-
-- Serializer/Deserializer
-
-- Transformer handlers:
-
-  * Interpreter (reference) implementation of behavior. The
-    implementation should favor clarity over efficiency.
-
-

doc/sphinx/source/index.rst

@@ -22,6 +22,9 @@ of :abbr:`Deep Learning (DL)` (DL) systems. Here you will find a suite of
 components, APIs, and documentation that can be used to compile and run  
 :abbr:`Deep Neural Network (DNN)` (DNN) models defined in a variety of frameworks.  
 
+.. figure:: graphics/ngraph-hub.png  
+
+
 For this early release, we provide :doc:`framework-integration-guides` to compile 
 and run MXNet and TensorFlow-based projects.
 
@@ -32,54 +35,26 @@ Architecture CPUs (CPU), the Intel® Nervana Neural Network Processor™ (NNP),
 and NVIDIA\* GPUs. Currently-supported compiler optimizations include efficient 
 memory management and data layout abstraction. 
 
-Further overview details can be found on our :doc:`about` page. 
+Further project details can be found on our :doc:`project/about` page. 
+
+
 
 =======
 
+Sections
+=========
+
 .. toctree::
    :maxdepth: 1
-   :caption: Table Of Contents
    :name: tocmaster
+   :caption: Table of Contents
 
    installation.rst
    testing-libngraph.rst
    framework-integration-guides.rst
    graph-basics.rst
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Algorithms 
-   :name: 
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Reference API
-
-   api.rst
-   autodiff.rst
-   glossary.rst
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Ops
-
-   ops/abs.rst
-   ops/convolution.rst
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Project Docs
-
-   about.rst
-   release-notes.rst
-   code-contributor-README.rst
-
-.. toctree::
-   :maxdepth: 0
-   :hidden: 
-   
-   branding-notice.rst
-   doc-contributor-README.rst
+   ops/index.rst
+   project/index.rst
 
 
 Indices and tables
@@ -88,3 +63,4 @@ Indices and tables
    * :ref:`search`   
    * :ref:`genindex`
 
+     

doc/sphinx/source/ops/abs.rst

@@ -4,23 +4,32 @@
 Abs
 ###
 
+Description
+===========
+
 Elementwise absolute value operation.
 
-Produces a single output tensor of the same element type and shape as the input,
-where the value at each coordinate of the output is the absoloute value of the
-value at each input coordinate.
+Produces a single output tensor of the same element type and shape as ``arg``,
+where the value at each coordinate of ``output`` is the absoloute value of the
+value at each ``arg`` coordinate.
+
+Inputs
+------
 
 +-----------------+-------------------------+--------------------------------+
-| Input Name      | Element Type            | Shape                          |
+| Name            | Element Type            | Shape                          |
 +=================+=========================+================================+
-| ``input``       | Any                     | Any                            |
+| ``arg``         | Any                     | Any                            |
 +-----------------+-------------------------+--------------------------------+
 
-+------------------+-------------------------+----------------------------------------------------+
-| Output Name      | Element Type            | Shape                                              |
-+==================+=========================+====================================================+
-| ``output``       | Same as ``input``       | Same as input.                                     |
-+------------------+-------------------------+----------------------------------------------------+
+Outputs
+-------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``output``      | Same as ``arg``         | Same as ``arg``.               |
++-----------------+-------------------------+--------------------------------+
 
 
 Mathematical Definition
@@ -28,14 +37,15 @@ Mathematical Definition
 
 .. math::
 
-   output_{i_0, \ldots, i_{n-1}} = \mathrm{abs}(input_{i_0, \ldots, i_{n-1}})
+   \mathtt{output}_{i_0, \ldots, i_{n-1}} = \left|\mathtt{arg}_{i_0,
+   \ldots, i_{n-1}}\right|
 
 Backprop
 ========
 
 .. math::
 
-   \overline{input} \leftarrow \mathrm{sgn}(input)\Delta
+   \overline{\texttt{arg}} \leftarrow \Delta\ \mathrm{sgn}(\texttt{arg})
 
 
 C++ Interface
@@ -43,8 +53,3 @@ C++ Interface
 
 .. doxygenclass:: ngraph::op::Abs
    :members:
-
-Python Interface
-================
-
-is not merged yet, but could go here!

doc/sphinx/source/ops/acos.rst

+.. acos.rst:
+
+####
+Acos
+####
+
+Description
+===========
+
+Elementwise acos operation.
+
+Produces a tensor of the same element type and shape as ``arg``,
+where the value at each coordinate of ``output`` is the inverse cosine of the
+value at the corresponding coordinate of ``arg`` .
+
+Inputs
+------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``arg``         | Any                     | Any                            |
++-----------------+-------------------------+--------------------------------+
+
+Outputs
+-------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``output``      | Same as ``arg``         | Same as ``arg``.               |
++-----------------+-------------------------+--------------------------------+
+
+
+Mathematical Definition
+=======================
+
+.. math::
+
+   \texttt{output}_{i_0, \ldots, i_{n-1}} = \cos^{-1}(\texttt{arg}_{i_0, \ldots, i_{n-1}})
+
+Backprop
+========
+
+.. math::
+
+   \overline{\texttt{arg}} \leftarrow -\frac{\Delta}{\sqrt{1-\texttt{arg}^2}}
+
+
+C++ Interface
+=============
+
+.. doxygenclass:: ngraph::op::Acos
+   :members:

doc/sphinx/source/ops/add.rst

+.. add.rst:
+
+###
+Add
+###
+
+Description
+===========
+
+Elementwise add operation.
+
+Produces tensor of the same element type and shape as the two inputs,
+where the value at each coordinate of ``output`` is the sum of the
+value at the corresponding input coordinates.
+
+Inputs
+------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``arg0``        | any                     | any                            |
++-----------------+-------------------------+--------------------------------+
+| ``arg1``        | same as ``arg0``        | same as ``arg0``               |
++-----------------+-------------------------+--------------------------------+
+
+Outputs
+-------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``output``      | same as ``arg0``        | same as ``arg0``               |
++-----------------+-------------------------+--------------------------------+
+
+
+Mathematical Definition
+=======================
+
+.. math::
+
+   \texttt{output}_{i_0, \ldots, i_{n-1}} = \texttt{arg0}_{i_0, \ldots, i_{n-1}} + \texttt{arg1}_{i_0, \ldots, i_{n-1}}
+
+Backprop
+========
+
+.. math::
+
+   \overline{\texttt{arg0}} &\leftarrow \Delta \\
+   \overline{\texttt{arg1}} &\leftarrow \Delta
+
+
+C++ Interface
+=============
+
+.. doxygenclass:: ngraph::op::Add
+   :members:

doc/sphinx/source/ops/asin.rst

+.. asin.rst:
+
+####
+Asin
+####
+
+Description
+===========
+
+Elementwise asin operation.
+
+Produces a tensor of the same element type and shape as ``arg``,
+where the value at each coordinate of ``output`` is the inverse sine of the
+value at the corresponding coordinate of ``arg`` .
+
+Inputs
+------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``arg``         | Any                     | Any                            |
++-----------------+-------------------------+--------------------------------+
+
+Outputs
+-------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``output``      | Same as ``arg``         | Same as ``arg``.               |
++-----------------+-------------------------+--------------------------------+
+
+
+Mathematical Definition
+=======================
+
+.. math::
+
+   \texttt{output}_{i_0, \ldots, i_{n-1}} = \sin^{-1}(\texttt{arg}_{i_0, \ldots, i_{n-1}})
+
+Backprop
+========
+
+.. math::
+
+   \overline{\texttt{arg}} \leftarrow \frac{\Delta}{\sqrt{1-\texttt{arg}^2}}
+
+C++ Interface
+=============
+
+.. doxygenclass:: ngraph::op::Asin
+   :members:

doc/sphinx/source/ops/atan.rst

+.. atan.rst:
+
+####
+Atan
+####
+
+Description
+===========
+
+Elementwise atan operation.
+
+Produces a tensor of the same element type and shape as ``arg``,
+where the value at each coordinate of ``output`` is the inverse tangent of the
+value at the corresponding coordinate of ``arg`` .
+
+Inputs
+------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``arg``         | Any                     | Any                            |
++-----------------+-------------------------+--------------------------------+
+
+Outputs
+-------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``output``      | Same as ``arg``         | Same as ``arg``.               |
++-----------------+-------------------------+--------------------------------+
+
+
+Mathematical Definition
+=======================
+
+.. math::
+
+   \texttt{output}_{i_0, \ldots, i_{n-1}} = \tan^{-1}(\texttt{arg}_{i_0, \ldots, i_{n-1}})
+
+
+Backprop
+========
+
+.. math::
+
+   \overline{\texttt{arg}} \leftarrow \frac{\Delta}{1+\texttt{arg}^2}
+
+
+C++ Interface
+=============
+
+.. doxygenclass:: ngraph::op::Atan
+   :members:

doc/sphinx/source/ops/avg_pool.rst

+.. avg_pool.rst:
+
+#######
+AvgPool
+#######
+
+Description
+===========
+
+Average Pooling operation.
+
+Average pooling windows its input and produces an average for each window.
+
+Inputs
+------
+
++-----------------+----------------+--------------------------------+--------------------+
+| Name            | Element Type   | Shape                          | Notes              |
++=================+================+================================+====================+
+| ``data``        | Any            | :math:`(N,C,d_1,\ldots,d_n)`   | :math:`n>0, d_i>0` |
++-----------------+----------------+--------------------------------+--------------------+
+
+
+Attributes
+----------
+
++----------------------+-----------------+----------------------------------+
+| Name                 | Type            | Notes                            |
++======================+=================+==================================+
+| ``w``                | ``Shape[n]``    | Window shape. :math:`w_i\le d_i` |
++----------------------+-----------------+----------------------------------+
+| ``s``                | ``Strides[n]``  | Window strides.                  |
++----------------------+-----------------+----------------------------------+
+| ``p``                | ``Shape[n]``    | Padding below.                   |
++----------------------+-----------------+----------------------------------+
+| ``q``                | ``Shape[n]``    | Padding above.                   |
++----------------------+-----------------+----------------------------------+
+
+Outputs
+-------
+
++-----------------+-------------------------+--------------------------------+
+| Name            | Element Type            | Shape                          |
++=================+=========================+================================+
+| ``output``      | Any                     | :math:`(N,C,d'_1,\ldots,d'_n)` |
++-----------------+-------------------------+--------------------------------+
+
+
+Average pooling takes as its input a batch tensor `data` of shape
+:math:`(N,C,d_1,\ldots,d_n)` where  where :math:`N` is the batch
+size, and :math:`C > 0` is the
+number of channels (sometimes called features).  The dimensions
+:math:`(d_1,\ldots,d_n)` correspond to the shape of an
+:math:`n`-dimensional data item in a batch. For example, where
+:math:`n=2`, the data may represent a two-dimensional image. It also
+takes four attributes:
+        
+1. *window shape*,
+2. *window movement strides*, (optional)
+3. *padding below*, (optional)
+4. *padding above*, (optional).
+        
+The shape of `output` is :math:`(N,C,d'_1,\ldots,d'_n)`, where
+:math:`d'_n = \lceil \frac{p_i + d_i + q_i - w_i + 1}{s_i} \rceil`.
+        
+*In the absence of padding*, given an input data batch tensor
+ :math:`T_\textit{in}`, the output tensor is defined by the equation
+
+.. math::
+       
+   T_\textit{out}[a,c,i_1,\ldots,i_n] =
+   \frac{\sum_{j_1 = s_1 i_1, \ldots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \ldots, j_n = s_n i_n + w_n - 1}
+   T_\textit{in}[a,c,j_1,\ldots,j_n]}{\prod_{i=1}^n{w_n}}
+
+        
+*In the presence of padding*, we do not always want to divide by a
+reciprocal equal to the number of elements in the window, since some
+of the output points are determined by a window that is partly hanging
+beyond the edge of the tensor. In this case we can define the output
+via a few intermediate steps.
+        
+First define the *sum tensor* :math:`T_\textit{sum}`, with shape
+:math:`(N,C,d'_1,\ldots,d'_n)`, as follows.
+
+.. math::
+
+   T_\textit{sum}[a,c,i_1,\ldots,i_n] =
+   \frac{\sum_{j_1 = s_1 i_1, \ldots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \ldots, j_n = s_n i_n + w_n - 1}
+   \textit{val}[a,c,j_1,\ldots,j_n]}{\prod_{i=1}^n{w_n}}
+        
+where
+
+.. math::
+
+   \textit{val}[a,c,j_1,\ldots,j_n] =
+   \begin{cases}
+   T_\textit{in}[a,c,j_1,\ldots,j_n]&\text{if for all } k, p_k \le j_k < p_k + d_k\\
+   0&\text{otherwise}.
+   \end{cases}
+        
+Second, define the *divisor tensor* :math:`T_\textit{div}`, with shape :math:`(N,C,d'_1,\ldots,d'_n)`, as follows.
+
+.. math::
+
+   T_\textit{div}[a,c,i_1,\ldots,i_n] =
+   \frac{\sum_{j_1 = s_1 i_1, \ldots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \ldots, j_n = s_n i_n + w_n - 1}
+   \textit{val}[a,c,j_1,\ldots,j_n]}{\prod_{i=1}^n{w_n}}
+
+where
+
+.. math::
+
+   \textit{val}[a,c,j_1,\ldots,j_n] =
+   \begin{cases}
+   1&\text{if for all }k, p_k \le j_k < p_k + d_k\\
+   0&\text{otherwise}.
+   \end{cases}
+        
+Finally, define :math:`T_\textit{out}` as the result of elementwise
+dividing :math:`T_\textit{sum}` by :math:`T_\textit{div}`.  Note that
+at positions where :math:`T_\textit{div}` is zero, values may be
+infinity or nan.  (This corresponds to a condition where the pooling
+window is completely out of bounds, encompassing no valid values.)
+
+Backprop
+========
+
+
+C++ Interface
+=============
+
+.. doxygenclass:: ngraph::op::AvgPool
+   :members:
+

doc/sphinx/source/ops/avg_pool_backprop.rst

+.. avg_pool_backprop.rst:
+
+###############
+AvgPoolBackprop
+###############
+
+Average Pooling backprop operation.
+
+
+C++ Interface
+=============
+
+.. doxygenclass:: ngraph::op::AvgPoolBackprop
+   :members:
+
+Python Interface
+================
+
+is not merged yet, but could go here!

doc/sphinx/source/ops/convolution.rst

@@ -4,21 +4,50 @@
 Convolution
 ###########
 
+Description
+===========
+
 A batched convolution operation.
 
-Basic Operation
-===============
+Inputs
+------
 
 +-----------------+-------------------------+--------------------------------+
-| Input Name      | Element Type            | Shape                          |
+| Name            | Element Type            | Shape                          |
 +=================+=========================+================================+
 | ``image_batch`` | Any                     | ``(N, C_in, d_1, ..., d_n)``   |
 +-----------------+-------------------------+--------------------------------+
 | ``filters``     | Same as ``image_batch`` | ``(N, C_in, df_1, ..., df_n)`` |
 +-----------------+-------------------------+--------------------------------+
 
+Attributes
+----------
+
++-----------------------------+-----------------------------+---------------------------------------+
+| Name                        | Type                        | Notes                                 |
++=============================+=============================+=======================================+
+| ``window_movement_strides`` | ``Strides[n]``              | How far to slide the window along     |
+|                             |                             | each axis at each step.               |
++-----------------------------+-----------------------------+---------------------------------------+
+| ``window_dilation_strides`` | ``Strides[n]``              | Per-axis dilation to apply to the     |
+|                             |                             | filters.                              |
++-----------------------------+-----------------------------+---------------------------------------+
+| ``padding_below``           | ``Shape[n]``                | How many padding elements to add      |
+|                             |                             | below the 0-coordinate on each axis.  |
++-----------------------------+-----------------------------+---------------------------------------+
+| ``padding_above``           | ``Shape[n]``                | How manny padding elements to add     |
+|                             |                             | above the max-coordinate on each axis.|
++-----------------------------+-----------------------------+---------------------------------------+
+| ``image_dilation_strides``  | ``Strides[n]``              | Per-axis dilation to apply to the     |
+|                             |                             | image batch.                          |
++-----------------------------+-----------------------------+---------------------------------------+
+
+
+Outputs
+-------
+
 +------------------+-------------------------+----------------------------------------------------+
-| Output Name      | Element Type            | Shape                                              |
+| Name             | Element Type            | Shape                                              |
 +==================+=========================+====================================================+
 | ``features_out`` | Same as ``image_batch`` | ``(N, C_in, d_1 - df_1 + 1, ..., d_n - df_n + 1)`` |
 +------------------+-------------------------+----------------------------------------------------+
@@ -27,38 +56,6 @@ It must be the case that after dilation and padding are applied, the filter fits
 
 .. TODO image add
 
-Window Parameters
-=================
-
-+-----------------------------+-----------------------------+------------------------------------+
-| Parameter Name              | Type                        | Meaning                            |
-+=============================+=============================+====================================+
-| ``window_movement_strides`` | ``Strides`` of length ``n`` | How far to slide the window along  |
-|                             |                             | each axis at each step.            |
-+-----------------------------+                             +------------------------------------+
-| ``window_dilation_strides`` |                             | Per-axis dilation to apply to the  |
-|                             |                             | filters.                           |
-+-----------------------------+-----------------------------+------------------------------------+
-
-.. TODO: pictorial example of the effect of window movement stride.
-.. TODO: pictorial example of window before and after dilation.
-
-Image Batch Parameters
-======================
-
-+----------------------------+-----------------------------+---------------------------------------+
-| Parameter Name             | Type                        | Meaning                               |
-+============================+=============================+=======================================+
-| ``padding_below``          | ``Padding`` of length ``n`` | How many padding elements to add      |
-|                            |                             | below the 0-coordinate on each axis.  |
-+----------------------------+                             +---------------------------------------+
-| ``padding_above``          |                             | How manny padding elements to add     |
-|                            |                             | above the max-coordinate on each axis.|
-+----------------------------+-----------------------------+---------------------------------------+
-| ``image_dilation_strides`` | ``Strides`` of length ``n`` | Per-axis dilation to apply to the     |
-|                            |                             | image batch.                          |
-+----------------------------+-----------------------------+---------------------------------------+
-
 
 Mathematical Definition
 =======================

doc/sphinx/source/ops/index.rst

+.. ops/index.rst
+
+Core Ops
+========
+
+An ``Op``'s primary role is to function as a node in a directed acyclic graph 
+dependency computation graph.  
+
+*Core ops* are ops that are available and generally useful to all framework 
+bridges and that can be compiled by all transformers. A framework bridge may 
+define framework-specific ops to simplify graph construction, provided that the 
+bridge can enable every transformer to replace all such ops with equivalent 
+subgraphs composed of core ops. Similary, transformers may define 
+transformer-specific ops to represent kernels or other intermediate operations. 
+
+If a framework supports extending the set of ops it offers, a bridge may even 
+expose transformer-specific ops to the framework user.
+
+Our design philosophy is that the graph is not a script for running kernels; 
+rather, our compilation will match ``ops`` to appropriate kernels for the 
+backend(s) in use. Thus, we expect that adding of new Core ops should be 
+infrequent and that most functionality instead gets added with new functions 
+that build sub-graphs from existing core ops.   
+
+It is easiest to define a new op by adapting an existing op. Some of the tasks 
+that must be performed are:
+
+- Op constructor:
+
+  * Checking type-consistency of arguments 
+  * Specifying the result type for a call 
+
+- Serializer/Deserializer
+
+- Transformer handlers:
+
+  * Interpreter (reference) implementation of behavior. The
+    implementation should favor clarity over efficiency.
+
+
+
+Alphabetical list of Core ``ops``
+----------------------------------
+
+Not currently a comprehensive list.  
+
+
+.. toctree::
+   :maxdepth: 1
+
+   abs.rst
+   acos.rst
+   add.rst
+   asin.rst
+   atan.rst
+   avg_pool.rst
+   avg_pool_backprop.rst
+   convolution.rst

doc/sphinx/source/about.rst → doc/sphinx/source/project/about.rst

@@ -8,6 +8,8 @@ of :abbr:`Deep Learning (DL)` (DL) systems. Here you will find a suite of
 components, APIs, and documentation that can be used to compile and run  
 :abbr:`Deep Neural Network (DNN)` models defined in a variety of frameworks.  
 
+.. figure:: ../graphics/ngraph-hub.png  
+
 The nGraph library translates a framework’s representation of computations into 
 an :abbr:`Intermediate Representation (IR)` designed to promote computational 
 efficiency on target hardware. Initially-supported backends include Intel 
@@ -15,8 +17,6 @@ Architecture CPUs, the Intel® Nervana Neural Network Processor™ (NNP),
 and NVIDIA\* GPUs. Currently-supported compiler optimizations include efficient 
 memory management and data layout abstraction. 
 
-.. figure:: graphics/fig.jpeg  
-
 The *nGraph core* uses a strongly-typed and platform-neutral stateless graph 
 representation for computations. Each node, or *op*, in the graph corresponds
 to one step in a computation, where each step produces zero or more tensor

doc/sphinx/source/project/branding-notice.rst

+:orphan:
+
+.. branding-notice:
+
+
+Branding Notice
+===============
+
+The Intel® nGraph™ library is an open source project providing code and component 
+reference for many kinds of machine learning, deep learning, and DNN applications. 
+
+Documentation may include references to frontend frameworks, modules, extensions, 
+or other libraries that may be wholly or partially open source, or that may be 
+claimed as the property of others.  
+
+
+Intel nGraph library core documentation
+---------------------------------------
+
+.. note:: The branding notice below applies to code and documentation 
+   contributions intended to be added directly to Intel nGraph library core.   
+
+Use the first or most prominent usage with symbols as described below.
+
+Subsequent references on the same document, or on a file with an 
+already-present prominent form (such as Sphinx\* documentation sidebars), 
+may be done as an abbreviated form (sub-bullet items) and/or without the 
+repeated use of the trademark / branding symbols. 
+
+* Intel® Nervana™ Neural Network Processor 
+  
+  * Intel® Nervana™ NNP 
+
+* Intel® Xeon Phi™ (CPU processor)
+
+* Intel® Xeon® (CPU processor)
+  
+* Intel® nGraph™
+
+* Intel® nGraph™ library 
+
+    * nGraph library
+    * ``ngraph`` API
+    * ``ngraph`` library
+    * ``ngraph`` backend
+    * nGraph abstraction layer
+    * neon™ frontend framework 
+    
+* Intel® Math Kernel Library
+
+  * Intel® MKL
+ 
+* Intel® Math Kernel Library for Deep Neural Networks 
+
+  * Intel® MKL-DNN
+
+* Intel® Nervana™ Graph (deprecated)
+
+

doc/sphinx/source/doc-contributor-README.rst → doc/sphinx/source/project/doc-contributor-README.rst

@@ -56,14 +56,14 @@ source file (``.rst``):
 
 ::
 
-  .. literalinclude:: ../../../src/ngraph/descriptor/primary_tensor_view.cpp
+  .. literalinclude:: ../../../../src/ngraph/descriptor/primary_tensor_view.cpp
      :language: cpp
      :lines: 20-31
 
 And the raw code will render as follows
 
 
-.. literalinclude:: ../../../src/ngraph/descriptor/primary_tensor_view.cpp
+.. literalinclude:: ../../../../src/ngraph/descriptor/primary_tensor_view.cpp
    :language: cpp
    :lines: 20-31
 
@@ -86,7 +86,7 @@ line numbers, and add a caption "One way to define neon axes within the dqn_atar
 
 ::
 
-  .. literalinclude:: ../../../src/ngraph/descriptor/primary_tensor_view.cpp
+  .. literalinclude:: ../../../../src/ngraph/descriptor/primary_tensor_view.cpp
     :language: cpp
     :lines: 20-31
     :caption: 
@@ -94,7 +94,7 @@ line numbers, and add a caption "One way to define neon axes within the dqn_atar
 
 and the generated output will show readers of your helpful documentation
 
-.. literalinclude:: ../../../src/ngraph/descriptor/primary_tensor_view.cpp
+.. literalinclude:: ../../../../src/ngraph/descriptor/primary_tensor_view.cpp
    :language: cpp
    :lines: 20-31
    :caption: 

doc/sphinx/source/project/index.rst

+.. project/index.rst
+
+
+Project Docs
+============
+
+This section contains documentation about the project and how to contribute.
+
+
+.. toctree::
+   :maxdepth: 1
+
+   about.rst
+   release-notes.rst
+   code-contributor-README.rst
+   doc-contributor-README.rst
+   ../glossary.rst

doc/sphinx/source/testing-libngraph.rst

@@ -28,10 +28,11 @@ After building and installing the nGraph library to your system, the next
 logical step is to compile a framework that you can use to run a 
 training/inference model with one of the backends that are now enabled.
 
-For this early |release| release, we're providing integration guides for:
+For this early |release| release, we're providing :doc:`framework-integration-guides`, 
+for:
 
-* `MXNet`_,  
-* `TensorFlow`_, and
+* :doc:`framework-integration-guides` framework,  
+* :doc:`framework-integration-guides` framework, and
 * neon™ `frontend framework`_.
 
 Integration guides for other frameworks are tentatively forthcoming.

src/ngraph/ops/abs.hpp

@@ -31,10 +31,11 @@ namespace ngraph
         public:
             /// \brief Constructs an absolute value operation.
             ///
-            /// Output `[d1, ...]`
-            ///
             /// \param arg Node that produces the input tensor.<br>
             /// `[d1, ...]`
+            ///
+            /// Output `[d1, ...]`
+            ///
             Abs(const std::shared_ptr<Node>& arg)
                 : UnaryElementwiseArithmetic("Abs", arg)
             {

src/ngraph/ops/acos.hpp

@@ -26,23 +26,16 @@ namespace ngraph
     {
         /// \brief Elementwise inverse cosine (arccos) operation.
         ///
-        /// ## Inputs
-        ///
-        /// |       | Type                              | Description                                     |
-        /// | ----- | --------------------------------- | ----------------------------------------------- |
-        /// | `arg` | \f$N[d_1,\dots,d_n]~(n \geq 0)\f$ | A tensor of any shape and numeric element type. |
-        ///
-        /// ## Output
-        ///
-        /// | Type                   | Description                                                                             |
-        /// | ---------------------- | --------------------------------------------------------------------------------------- |
-        /// | \f$N[d_1,\dots,d_n]\f$ | The tensor \f$T\f$, where \f$T[i_1,\dots,i_n] = \arccos(\texttt{arg}[i_1,\dots,i_n])\f$ |
         class Acos : public UnaryElementwiseArithmetic
         {
         public:
             /// \brief Constructs an arccos operation.
             ///
-            /// \param arg Node that produces the input tensor.
+            /// \param arg Node that produces the input tensor.<br>
+            /// `[d1, ...]`
+            ///
+            /// Output `[d1, ...]`
+            ///
             Acos(const std::shared_ptr<Node>& arg)
                 : UnaryElementwiseArithmetic("Acos", arg)
             {

src/ngraph/ops/add.hpp

@@ -26,25 +26,18 @@ namespace ngraph
     {
         /// \brief Elementwise addition operation.
         ///
-        /// ## Inputs
-        ///
-        /// |        | Type                              | Description                                            |
-        /// | ------ | --------------------------------- | ------------------------------------------------------ |
-        /// | `arg0` | \f$N[d_1,\dots,d_n]~(n \geq 0)\f$ | A tensor of any shape and numeric element type.        |
-        /// | `arg1` | \f$N[d_1,\dots,d_n]~(n \geq 0)\f$ | A tensor of the same shape and element type as `arg0`. |
-        ///
-        /// ## Output
-        ///
-        /// | Type                   | Description                                                                                                    |
-        /// | ---------------------- | -------------------------------------------------------------------------------------------------------------- |
-        /// | \f$N[d_1,\dots,d_n]\f$ | The tensor \f$T\f$, where \f$T[i_1,\dots,i_n] = \texttt{arg0}[i_1,\dots,i_n] + \texttt{arg1}[i_1,\dots,i_n]\f$ |
         class Add : public BinaryElementwiseArithmetic
         {
         public:
             /// \brief Constructs an addition operation.
             ///
-            /// \param arg0 Node that produces the first input tensor.
-            /// \param arg1 Node that produces the second input tensor.
+            /// \param arg0 Node that produces the first input tensor.<br>
+            /// `[d0, ...]`
+            /// \param arg1 Node that produces the second input tensor.<br>
+            /// `[d0, ...]`
+            ///
+            /// Output `[d0, ...]`
+            ///
             Add(const std::shared_ptr<Node>& arg0, const std::shared_ptr<Node>& arg1)
                 : BinaryElementwiseArithmetic("Add", arg0, arg1)
             {

src/ngraph/ops/asin.hpp

@@ -26,23 +26,16 @@ namespace ngraph
     {
         /// \brief Elementwise inverse sine (arcsin) operation.
         ///
-        /// ## Inputs
-        ///
-        /// |       | Type                              | Description                                     |
-        /// | ----- | --------------------------------- | ----------------------------------------------- |
-        /// | `arg` | \f$N[d_1,\dots,d_n]~(n \geq 0)\f$ | A tensor of any shape and numeric element type. |
-        ///
-        /// ## Output
-        ///
-        /// | Type                   | Description                                                                             |
-        /// | ---------------------- | --------------------------------------------------------------------------------------- |
-        /// | \f$N[d_1,\dots,d_n]\f$ | The tensor \f$T\f$, where \f$T[i_1,\dots,i_n] = \arcsin(\texttt{arg}[i_1,\dots,i_n])\f$ |
         class Asin : public UnaryElementwiseArithmetic
         {
         public:
             /// \brief Constructs an arcsin operation.
             ///
-            /// \param arg Node that produces the input tensor.
+            /// \param arg Node that produces the input tensor.<br>
+            /// `[d1, ...]`
+            ///
+            /// Output `[d1, ...]`
+            ///
             Asin(const std::shared_ptr<Node>& arg)
                 : UnaryElementwiseArithmetic("Asin", arg)
             {

src/ngraph/ops/atan.hpp

@@ -26,23 +26,16 @@ namespace ngraph
     {
         /// \brief Elementwise inverse tangent (arctan) operation.
         ///
-        /// ## Inputs
-        ///
-        /// |       | Type                              | Description                                     |
-        /// | ----- | --------------------------------- | ----------------------------------------------- |
-        /// | `arg` | \f$N[d_1,\dots,d_n]~(n \geq 0)\f$ | A tensor of any shape and numeric element type. |
-        ///
-        /// ## Output
-        ///
-        /// | Type                   | Description                                                                             |
-        /// | ---------------------- | --------------------------------------------------------------------------------------- |
-        /// | \f$N[d_1,\dots,d_n]\f$ | The tensor \f$T\f$, where \f$T[i_1,\dots,i_n] = \arctan(\texttt{arg}[i_1,\dots,i_n])\f$ |
         class Atan : public UnaryElementwiseArithmetic
         {
         public:
             /// \brief Constructs an arctan operation.
             ///
-            /// \param arg Node that produces the input tensor.
+            /// \param arg Node that produces the input tensor.<br>
+            /// `[d1, ...]`
+            ///
+            /// Output `[d1, ...]`
+            ///
             Atan(const std::shared_ptr<Node>& arg)
                 : UnaryElementwiseArithmetic("Atan", arg)
             {

src/ngraph/ops/avg_pool.hpp

@@ -24,55 +24,21 @@ namespace ngraph
     {
         /// \brief Batched average pooling operation, with optional padding and window stride.
         ///
-        /// Average pooling takes as its input an data batch tensor of shape \f$(N,C,d_1,\dots,d_n)\f$ where \f$n > 0\f$, every \f$d_i > 0\f$, and where \f$N\f$ is
-        /// the batch size, and \f$C > 0\f$ is the number of channels (sometimes called features). The dimensions \f$(d_1,\dots,d_n)\f$ correspond to the shape of
-        /// an \f$n\f$-dimensional data item in a batch. For example, where \f$n=2\f$, the data may represent a two-dimensional image. It also takes four parameters:
-        ///
-        /// 1. <i>(the window shape)</i> a size vector \f$(w_1,\dots,w_n)\f$ where every \f$w_i \le d_i\f$; and
-        /// 2. <i>(the window movement strides, optional)</i> a vector of positive integers \f$(s_1,\dots,s_n)\f$.
-        /// 3. <i>(the padding below, optional)</i> a vector of positive integers \f$(p_1,\dots,p_n)\f$.
-        /// 4. <i>(the padding above, optional)</i> a vector of positive integers \f$(q_1,\dots,q_n)\f$.
-        ///
-        /// The output has the shape \f$(N,C,d'_1,\dots,d'_n)\f$, where \f$d'_n = \lceil \frac{p_i + d_i + q_i - w_i + 1}{s_i} \rceil\f$.
-        ///
-        /// *In the absence of padding*, given an input data batch tensor \f$T_\textit{in}\f$, the output tensor is defined by the equation
-        ///
-        /// \f[
-        ///      T_\textit{out}[a,c,i_1,\dots,i_n] = \frac{\sum_{j_1 = s_1 i_1, \dots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \dots, j_n = s_n i_n + w_n - 1} T_\textit{in}[a,c,j_1,\dots,j_n]}{\prod_{i=1}^n{w_n}}
-        /// \f]
-        ///
-        /// *In the presence of padding*, we do not always want to divide by a reciprocal equal to the number of elements in the window, since some of the output points are
-        /// determined by a window that is partly hanging beyond the edge of the tensor. In this case we can define the output via a few intermediate steps.
-        ///
-        /// First define the <i>sum tensor</i> \f$T_\textit{sum}\f$, with shape \f$(N,C,d'_1,\dots,d'_n)\f$, as follows.
-        ///
-        /// \f[
-        ///      T_\textit{sum}[a,c,i_1,\dots,i_n] = \frac{\sum_{j_1 = s_1 i_1, \dots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \dots, j_n = s_n i_n + w_n - 1} \textit{val}[a,c,j_1,\dots,j_n]}{\prod_{i=1}^n{w_n}}
-        /// \f]
-        ///
-        /// where \f$\textit{val}[a,c,j_1,\dots,j_n] = T_\textit{in}[a,c,j_1,\dots,j_n]\f$ if for all \f$k\f$, \f$p_k \le j_k < p_k + d_k\f$; else \f$0\f$.
-        ///
-        /// Second, define the <i>divisor tensor</i> \f$T_\textit{div}\f$, with shape \f$(N,C,d'_1,\dots,d'_n)\f$, as follows.
-        ///
-        /// \f[
-        ///      T_\textit{div}[a,c,i_1,\dots,i_n] = \frac{\sum_{j_1 = s_1 i_1, \dots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \dots, j_n = s_n i_n + w_n - 1} \textit{val}[a,c,j_1,\dots,j_n]}{\prod_{i=1}^n{w_n}}
-        /// \f]
-        ///
-        /// where \f$\textit{val}[a,c,j_1,\dots,j_n] = 1\f$ if for all \f$k\f$, \f$p_k \le j_k < p_k + d_k\f$; else \f$0\f$.
-        ///
-        /// Finally, define \f$T_\textit{out}\f$ as the result of elementwise dividing \f$T_\textit{sum}\f$ by \f$T_\textit{div}\f$.
-        /// Note that at positions where \f$T_\textit{div}\f$ is zero, values may be infinity or nan. (This corresponds to a condition where the pooling window is completely
-        /// out of bounds, encompassing no valid values.)
         class AvgPool : public RequiresTensorViewArgs
         {
         public:
             /// \brief Constructs a batched average pooling operation.
             ///
-            /// \param arg The node producing the input data batch tensor.
-            /// \param window_shape The window shape.
-            /// \param window_movement_strides The window movement strides.
-            /// \param padding_below The below-padding shape.
-            /// \param padding_above The above-padding shape.
+            /// \param arg The node producing the input data batch tensor.<br>
+            /// `[d1, dn]`
+            /// \param window_shape The window shape.<br>
+            /// `[n]`
+            /// \param window_movement_strides The window movement strides.<br>
+            /// `[n]`
+            /// \param padding_below The below-padding shape.<br>
+            /// `[n]`
+            /// \param padding_above The above-padding shape.<br>
+            /// `[n]`
             AvgPool(const std::shared_ptr<Node>& arg,
                     const Shape& window_shape,
                     const Strides& window_movement_strides,
@@ -81,17 +47,22 @@ namespace ngraph
 
             /// \brief Constructs a batched, unpadded average pooling operation (i.e., all padding shapes are set to 0).
             ///
-            /// \param arg The node producing the input data batch tensor.
-            /// \param window_shape The window shape.
-            /// \param window_movement_strides The window movement strides.
+            /// \param arg The node producing the input data batch tensor.<br>
+            /// `[d1, ..., dn]`
+            /// \param window_shape The window shape.<br>
+            /// `[n]`
+            /// \param window_movement_strides The window movement strides.<br>
+            /// `[n]`
             AvgPool(const std::shared_ptr<Node>& arg,
                     const Shape& window_shape,
                     const Strides& window_movement_strides);
 
             /// \brief Constructs an unstrided batched convolution operation (i.e., all window movement strides are 1 and all padding shapes are set to 0).
             ///
-            /// \param arg The node producing the input data batch tensor.
-            /// \param window_shape The window shape.
+            /// \param arg The node producing the input data batch tensor.<br>
+            /// `[d1, ..., dn]`
+            /// \param window_shape The window shape.<br>
+            /// `[n]`
             AvgPool(const std::shared_ptr<Node>& arg, const Shape& window_shape);
 
             virtual std::shared_ptr<Node> copy_with_new_args(

src/ngraph/ops/convolution.hpp

@@ -29,8 +29,6 @@ namespace ngraph
         public:
             /// \brief Constructs a batched convolution operation.
             ///
-            /// Output `[N, C_OUT, R1, ... Rf]`
-            ///
             /// \param data_batch The node producing the input data batch tensor.<br>
             /// `[N, C_IN, D1, ... Df]`
             /// \param filters The node producing the filters tensor.<br>
@@ -45,6 +43,9 @@ namespace ngraph
             /// `[f]`
             /// \param data_dilation_strides The data dilation strides.<br>
             /// `[f]`
+            ///
+            /// Output `[N, C_OUT, R1, ... Rf]`
+            ///
             Convolution(const std::shared_ptr<Node>& data_batch,
                         const std::shared_ptr<Node>& filters,
                         const Strides& window_movement_strides,
@@ -67,6 +68,9 @@ namespace ngraph
             /// `[f]`
             /// \param padding_above The padding-above sizes.<br>
             /// `[f]`
+            ///
+            /// Output `[N, C_OUT, R1, ... Rf]`
+            ///
             Convolution(const std::shared_ptr<Node>& data_batch,
                         const std::shared_ptr<Node>& filters,
                         const Strides& window_movement_strides,
@@ -84,6 +88,9 @@ namespace ngraph
             /// `[f]`
             /// \param window_dilation_strides The window dilation strides.<br>
             /// `[f]`
+            ///
+            /// Output `[N, C_OUT, R1, ... Rf]`
+            ///
             Convolution(const std::shared_ptr<Node>& data_batch,
                         const std::shared_ptr<Node>& filters,
                         const Strides& window_movement_strides,
@@ -97,6 +104,9 @@ namespace ngraph
             /// `[C_OUT, C_IN, F1, ... Ff]`
             /// \param window_movement_strides The window movement strides.<br>
             /// `[f]`
+            ///
+            /// Output `[N, C_OUT, R1, ... Rf]`
+            ///
             Convolution(const std::shared_ptr<Node>& data_batch,
                         const std::shared_ptr<Node>& filters,
                         const Strides& window_movement_strides);
@@ -107,6 +117,9 @@ namespace ngraph
             /// `[N, C_IN, D1, ... Df]`
             /// \param filters The node producing the filters tensor.<br>
             /// `[C_OUT, C_IN, F1, ... Ff]`
+            ///
+            /// Output `[N, C_OUT, R1, ... Rf]`
+            ///
             Convolution(const std::shared_ptr<Node>& data_batch,
                         const std::shared_ptr<Node>& filters);