Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in / Register
Toggle navigation
N
ngraph
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Packages
Packages
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
submodule
ngraph
Commits
1c53fd36
Commit
1c53fd36
authored
Oct 26, 2018
by
L.S. Cook
Committed by
Scott Cyphers
Oct 26, 2018
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Documentation update for BatchNorm Ops (#1927)
parent
fc5842d9
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
29 additions
and
21 deletions
+29
-21
batch_norm.rst
doc/sphinx/source/ops/batch_norm.rst
+29
-21
No files found.
doc/sphinx/source/ops/batch_norm.rst
View file @
1c53fd36
...
...
@@ -4,9 +4,6 @@
BatchNorm
#########
NOTE: This describes what the ``BatchNorm`` op should look like. The current
version will be made a CPU transformer op.
.. code-block:: cpp
BatchNorm // Produces a normalized output
...
...
@@ -50,6 +47,7 @@ Attributes
Outputs
-------
+---------------------+-------------------------+-----------------------------+
| Name | Element Type | Shape |
+=====================+=========================+=============================+
...
...
@@ -60,38 +58,48 @@ Outputs
| ``batch_variance`` | same as ``gamma`` | \(C\) |
+---------------------+-------------------------+-----------------------------+
The ``batch_mean`` and ``batch_variance`` are computed per-channel from ``input``.
The values only need to be computed if ``use_global`` is ``false`` or they are used.
The ``batch_mean`` and ``batch_variance`` outputs are computed per-channel from
``input``. The values only need to be computed if ``use_global`` is ``false``,
or if they are used.
Mathematical Definition
=======================
The axes of the input fall into two categories, positional and
channel, with channel being axis 1. For each position, there are
:math:`C` channel values, each normalized independently.
The axes of the input fall into two categories: positional and channel, with
channel being axis 1. For each position, there are :math:`C` channel values,
each normalized independently.
Normalization of a channel sample is controlled by two values:
Normalization of a channel sample is controlled by two values, the
mean :math:`\mu`, and the variance :math:`\sigma^2`, and two scaling
attributes, :math:`\gamma` and :math:`\beta`. The values for :math:`\mu`
and :math:`\sigma^2` come from either compuing the mean and variance of
``input`` or from ``global_mean`` and ``global_variance``, depending on
the value of ``use_global``.
* the mean :math:`\mu`, and
* the variance :math:`\sigma^2`;
and by two scaling attributes: :math:`\gamma` and :math:`\beta`.
The values for :math:`\mu` and :math:`\sigma^2` come either from computing the
mean and variance of ``input``, or from ``global_mean`` and ``global_variance``,
depending on the value of ``use_global``.
.. math::
y_c = \frac{x_c-\mu_c}{\sqrt{\sigma^2_c+\epsilon}}\gamma_c+\beta_c
The mean and variance can be arguments
or computed for each channel of
``input`` over the positional axes. When computed from ``input``, the
mean and variance per
channel are available as outputs.
The mean and variance can be arguments
, or they may be computed for each channel
of ``input`` over the positional axes. When computed from ``input``, the mean
and variance per-
channel are available as outputs.
Backprop
========
C++ Interface
=============
=============
=
.. doxygenclass:: ngraph::op::BatchNorm
.. doxygenclass:: ngraph::op::BatchNorm
Training
:project: ngraph
:members:
.. doxygenclass:: ngraph::op::BatchNormInference
:project: ngraph
:members:
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment