glossary.rst 7.78 KB
Newer Older
Scott Cyphers's avatar
Scott Cyphers committed
1 2
:orphan:

3 4 5 6 7 8
.. glossary: 

Glossary 
========

.. glossary::
Scott Cyphers's avatar
Scott Cyphers committed
9
   :sorted:
10

Scott Cyphers's avatar
Scott Cyphers committed
11 12 13 14 15 16 17 18 19
   backend

      A component that can execute computations.

   bridge

      A component of nGraph that acts as a backend for a framework,
      allowing the framework to define and execute computations.

L.S. Cook's avatar
L.S. Cook committed
20 21 22 23 24 25
   data-flow graph

      Data-flow graphs are used to implement deep learning models. In  
      a data-flow graph, nodes represent operations on data and edges 
      represent data flowing between those operations. 

Scott Cyphers's avatar
Scott Cyphers committed
26 27
   framework

L.S. Cook's avatar
L.S. Cook committed
28 29 30 31
      Frameworks provide expressive user-facing APIs for constructing, 
      training, validating, and deploying DL/ML models: TensorFlow\*, 
      PaddlePaddle\*, MXNet\*, PyTorch\*, and Caffe\* are all examples of 
      well-known frameworks.
Scott Cyphers's avatar
Scott Cyphers committed
32

L.S. Cook's avatar
L.S. Cook committed
33
   function graph
Scott Cyphers's avatar
Scott Cyphers committed
34

35
      The nGraph Library uses a function graph to represent an
Scott Cyphers's avatar
Scott Cyphers committed
36
      ``op``'s parameters and results.
L.S. Cook's avatar
L.S. Cook committed
37

38 39 40 41 42 43
   fusion
   
      Fusion is the fusing, combining, merging, collapsing, or refactoring
      of a graph's functional operations (``ops``) into one or more of
      nGraph's core ops.   

44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
   ISA 

      An acronym for "Instruction Set Architecture," an ISA is machine code that  
      is compatible with the underlying silicon architecture. A realization of 
      an ISA is called an *implementation*. An ISA permits multiple 
      implementations that may vary in performance, physical size, memory use or 
      reuse, and monetary cost among other things. An ISA defines everything a 
      machine-language programmer needs to know in order to program a particular 
      backend device. What an ISA defines will differ among ISAs; in general, it
      defines things like:

         - supported *data types*; 
         - physical *states* available, such as the main memory and registers; 
         - *semantics*, such as the memory consistency and addressing modes; 
         - *low-level machine instructions* that comprise a machine language; 
         - and the *input/output model*.

      Be careful to not confuse ISAs with microarchitectures.    

L.S. Cook's avatar
L.S. Cook committed
63 64
   op

Scott Cyphers's avatar
Scott Cyphers committed
65 66 67 68 69 70
      An op represents an operation. Ops are stateless and have zero
      or more inputs and zero or more outputs. Some ops have
      additional constant attributes. Every output of an op
      corresponds to a tensor and has an element type and a shape. The
      element types and shapes of the outputs of an op are determined
      by the inputs and attributes of the op.
L.S. Cook's avatar
L.S. Cook committed
71

72
   parameter
Scott Cyphers's avatar
Scott Cyphers committed
73 74 75

      In the context of a function graph, a "parameter" refers to what
      "stands in" for an argument in an ``op`` definition.
76

77 78 79 80
   quantization

      Quantization refers to the conversion of numerical data into a lower-precision representation. Quantization is often used in deep learning to reduce the time and energy needed to perform computations by reducing the size of data transfers and the number of steps needed to perform a computation. This improvement in speed and energy usage comes at a cost in terms of numerical accuracy, but deep learning models are often able to function well in spite of this reduced accuracy. 

81
   result
Scott Cyphers's avatar
Scott Cyphers committed
82 83 84

      In the context of a function graph, the term "result" refers to
      what stands in for the returned value.
L.S. Cook's avatar
L.S. Cook committed
85 86

   shape
Scott Cyphers's avatar
Scott Cyphers committed
87 88 89

      The shape of a tensor is a tuple of non-negative integers that
      represents an exclusive upper bound for coordinate values.
L.S. Cook's avatar
L.S. Cook committed
90

Scott Cyphers's avatar
Scott Cyphers committed
91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120
   shared pointer

      The C++ standard template library has the template
      ``std::shared_ptr<X>``. A shared pointer is used like an ``X*``
      pointer, but maintains a reference count to the underlying
      object. Each new shared pointer to the object increases the
      count. When a shared pointer goes out of scope, the reference
      count is decremented, and, when the count reaches 0, the
      underlying object is deleted. The function template
      ``std::make_shared<X>(...)`` can be used similarly to ``new
      X(...)``, except it returns a ``std::shared_ptr<X>`` instead of
      an ``X*``.

      If there is a chain of shared pointers from an object back to
      itself, every object in the chain is referenced, so the
      reference counts will never reach 0 and the objects will never
      be deleted.

      If ``a`` referenced ``b`` and ``b`` wanted to track all
      references to itself and shared pointers were used both
      directions, there would be a chain of pointers form ``a`` to
      itself. We avoid this by using shared pointers in only one
      direction, and raw pointers for the inverse
      direction. ``std::enabled_shared_from_this`` is a class template
      that defines a method ``shared_from_this`` that provides a
      shared pointer from a raw pointer.

      nGraph makes use of shared pointers for objects whose lifetime
      is hard to determine when they are allocated.
   
L.S. Cook's avatar
L.S. Cook committed
121
   step
Scott Cyphers's avatar
Scott Cyphers committed
122 123 124 125

      An abstract "action" that produces zero or more tensor outputs
      from zero or more tensor inputs. Steps correspond to *ops* that
      connect *nodes*.
L.S. Cook's avatar
L.S. Cook committed
126
           
Scott Cyphers's avatar
Scott Cyphers committed
127 128 129 130
   tensors

      Tensors are maps from *coordinates* to scalar values, all of the
      same type, called the *element type* of the tensor.
131

Leona C's avatar
Leona C committed
132 133
      .. figure:: graphics/classngraph_tensor_descriptor.png
         :width: 90%
134

135 136 137 138 139 140 141 142
   
   Tensorview 

      The interface backends implement for tensor use. When there are no more 
      references to the tensor view, it will be freed when convenient for the 
      backend.


143 144 145
   model description

      A description of a program's fundamental operations that are 
146 147 148 149 150
      used by a framework to generate inputs for computation. 

   export
   
      The serialized version of a trained model that can be passed to
151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194
      one of the nGraph backends for computation.

   NN

      :abbr:`NN (Neural Network)` is an acronym for "Neural Network". NN models 
      are used to simulate possible combinations of binary logic processing 
      and multi-layer (multi-dimensional) paths through which a :term:`data-flow graph` 
      may be mapped or computed. A NN does not have centralized storage; rather, 
      a NN manifests as information stored as patterns throughout the network 
      structure. NNs may be **Recurrent** (feedback loop) or **Nonrecurrent** 
      (feed-forward) with regard to the network vector.

   ANN

      :abbr:`Artificial Neural Network (ANN)`, often abbreviated as :term:`NN`. 

   RANN 

      :abbr:`Recurrent Artificial Neural Network (RANN)`, often abbreviated as 
      :term:`RNN`.


   RNN 
    
      A :abbr:`Recurrent Neural Network (RNN)` is a variety of :term:`NN` where 
      output nodes from a layer on a data-flow graph have loopback to nodes that 
      comprise an earlier layer. Since the RNN has no "centralized" storage, this 
      loopback is the means by which the ANN can "learn" or be trained. There are 
      several sub-categories of RNNs. The traditional RNN looks like: 

      :math:`s_t = tanh(dot(W,x_{t-1}) + dot(U, s_{t-1})`

      where :math:`x` is the input data, :math:`s` is the memory, and output is
      :math:`o_t = softmax(dot(V, s_t))`.  :doc:`ops/tanh`, :doc:`ops/dot`, and 
      :doc:`ops/softmax` are all nGraph :doc:`core Ops <ops/index>`.


   LSTM

      :abbr:`LSTM (Long Short-Term Memory)` is an acronym for "Long Short-Term 
      Memory". LSTMs extend on the traditional RNN by providing a number of ways 
      to "forget" the memory of the previous time step via a set of learnable 
      gates. These gates help avoid the problem of exploding or vanishing 
      gradients that occur in the traditional RNN.
195

196 197 198
   SGD

      :abbr:`Stochastic Gradient Descent (SGD)`, also known as incremental 
199 200 201 202 203 204 205 206 207 208 209 210
      gradient descent, is an iterative method for optimizing a 
      differentiable objective function.

   validated

      To provide optimizations with nGraph, we first confirm that a given 
      workload is "validated" as being functional; that is, we can
      successfully load its serialized graph as an nGraph :term:`function 
      graph`