* LayerNorm (#3630)

* Constructors.

Type prop.

Decompose LayerNorm.

Add serialize.

* Add dummy test case.

* Add dummy type prop test.

* Fix some build errors.

* Remove build errors.

* Update decompose for bprop.

* Change begin_norm_axis default value to 1.

* Style.

* Reorder class members.

* Add actual type prop tests.

* Add fprop test.

* Working on bprop test.

* Bprop tests.

* Allow flattened scale and bias.

* Add support for flattened scale and bias.

* Fix incorrect type_name.

* PlaidML: Decompose fused_op LayerNorm

* Update Backprop constructors.

* PlaidML: Add missing header file.

* Remove doc about removed param.

* Fix type prop tests.

* PlaidML: Disable unit test.

* Fix stats flattening axes bug.

* Upgrade description to type_info.

* Strided slice

* Strided slice

* default value for strides

* Added new strided slice test, enabled old tests, refactoring

* Refactoring

* Autogenerated file: dyn_replace_slice tests

* Renaming

* Fix codestyle

* Fix build on MacOS

* Fix codestyle

* Add several tests in unit_test.manifest to skip it on PlaidML

* Disable all dyn_replace_slice tests on PlaidML

* Gather:v1 was introduced

* Added support for negative axis

* Removed unsused serialization

* Code review remarks introduced

* Change returned type of get_axis method

* Code review remarks introduced

* Chnaged axis_node to scalar during transformation

* Clang style applied

* Fixed clang errors

* style

* Add RandomUniform op

* Missing files

* Make CPU compile again (RandomUniform tests still don't pass)

* Bop users of existing RNGState class and reorder some comment junk

* Try to bop RNGState in gcpu

* Add RandomUniform to CPU manifest

* Add RandomUniform to PlaidML manifest

* Add .rst for random_uniform

* Clean up junk in the .rst

* Change UniformRNGState to always use double internally

* Change weird test failure message

* Compilation issues

* Fix unsigned comparison and implicit double conversion

* Eliminate sign/unsigned compare warnings, implicit double conversion

* Missed type

* Neww int type

* omp compromise

* style

* Pad:v1 was introduced

* Fixed problem with foward declaration

* Code review remarks introduced

* Addtinal asserts and unit-test for Pad:v1 introduced

* Handled optinal arg_pad_value input

* Fixed unit tests messages and clang styles applied

* Styles applied, checking rank improved

* Updated changes.md

* Revert "Updated changes.md"

This reverts commit 8ee16e580705275ee0719648d5251021354e4d84.

* Clang warning fixed

* Unnused parameter cleanup

* style

* Move required reshape helpers to builders

* Remove warning from matmul_factory

* Rid off onnx dependency from matmul factory

* Move malmul_factory into builders

* Add implementation of fused MatMul

* Add MatMul to serializer

* Add type_prop tests

* Remove reference.

* Make more metods private

* Use protected instead of private

* Fix compilation issues

* Change construction of matmul_factory in matmul op

* Add MatMul operator into switch of is_supported_impl function

* Change transpose flags from int to bool

* Review Fix I

* Update MatMul op

* Use OutputVector instead of NodeVector in MatmulFactories

* Fix usage of OutputVector

* Convert more shared_ptrs to Outputs

* Fix comments after merge

* Fix comments after merge II

* Fix comments after merge III

* wip

* r50 test

* update test

* R50 topk calls

* Faster topk

* Finish topk implementation

* cleanup

* checkpoint

* new test working

* more unit test

* style

* wip

* fix tests

* add needed header

* Add new TopK test to plaid manifest

* Fix windows build error:

* Axes input was added to LRN

* Unit tests for axes shape check were added

* LRN node deserialization was updated

* Fixed EOF and clang style applied

* Changed Constant to Parameter type in unit tests

* Expanded LRN reference ingterface

* Fixed LRN assert description

* Fixed passing arguments

* Reference implementation for one axis

* Implementation for channel

* Implementation for hw

* working on recurence version

* Implemented recurence version for hw

* Reference implementation code refactor

* Fixed ref LRN implementation and added tests

* Added 6D unit test

* Clang styles applied

* Code review remarks introduced

* Support for dynamic shape of axes input

* Clang styles applied

* Code review remarks introduced

* Added checking if axes values are in correct range

* Clang styles applied

* Removed redundant include

* Code review remarks introduced

* First pass, remove QDot as experimental op, add it as core op, test case failing

* Change QuantizedMatmul API

* Tests for QuantizedDot passing, cleanup required

* Move test cases to backend_test.in.cpp

* Change reference dot API to take scale and zero point

* Onnx matmul integer tests passing

* Add validate_and_infer_types()

* Add more checks for QDot

* Add type prop tests

* Add interpreter support

* Add serializer support

* Disable tests in other backends

* Fix test and add codegen support

* Make axes optional argument

* Change fusion pattern

* Copyright

* merge changes

* more merge changes

* Fix more merge conflicts

* change builders to reflect new changes

* cleanup

* Add reduction_axes to the API

* Fix onnx test cases

* Remove debug comments

* Fix CI

* disable plaidml tests

* Remove comment

* Normalize BatchNorm test names

* unit test renamed so manifest must be updated

* Changed name from Normalize to Normalize2

* Changed name from Normalize to Normalize2 in tests

* Changed name of normalize source files

* Removed across spatial and channel shared params

* Removed scale and introduced input axes

* Support for axes input was introduced

* Added possibility to choose method of bias using

* Clang style applied

* Code review remarks introduced

* Code review remarks introduced

* Added python script to generate normalize_l2 test data

* Changed required image_shape type from u64 to float

* Proposal op was adjusted

* Clang styles applied

* Added missing new line

* Fixed problem with narrowing conversion

* Code review remarks

* Removed checking types of Proposal input params

* Removed unused input element type variables

* [FUSED] Change alpha attribute in Elu from dynamic to static

* Fix python API

* Fix failing python test

* Remove unnecesary test

* Fix merge artifact

* Review fix I

* Review Fix PyAPI

* Axes input was added to LRN

* Unit tests for axes shape check were added

* LRN node deserialization was updated

* Fixed EOF and clang style applied

* Changed Constant to Parameter type in unit tests

* Fixed LRN assert description

* Fixed copy_with_new_args

* Clang style applied

* Better error checking for Argmin/Argmax

Argmin and Argmax are not defined if the reduction axis has length 0.

* Simplify

* Update test/type_prop/index_reduction.cpp
Co-Authored-By: Adam Procter <adam.m.procter@intel.com>

* Update test/type_prop/index_reduction.cpp
Co-Authored-By: Adam Procter <adam.m.procter@intel.com>

* Helper function converting ONNX auto_pad into nGraph PadType.

Separate auto_pad support from explicitly provided paddings.

* Add support for more PadType values for GroupConvolutionTranspose.

* Pass auto_pad attribute value to nGraph operator.

* Helper class for generating ONNX pooling operators.

* Pass auto pad type to nGraph Convolution operator.

* Use pooling factory.

* Helper function calculating pads taking into account auto_pad attribute.

* Fix attribute type in UT ONNX models.

* Take auto_pad attribute value into account.

* Rename helper function and update doc.

* Retain old API for GroupConvoloutionTranspose

* Remove PadType::INVALID and use other approach to validate auto_pad
value.

* Style apply.

* Fix clang err on documentation style.

* Refactor get_auto_pad method.

* Fix segfaults on CentOS 7.

When using const reference the node member m_attributes had invalid
data.

* Start splitting type_prop.cpp (just a few for now, to get feedback on general pattern)

* Split out some more type_prop stuff

* A bunch more

* Move one test out of type_prop into build_graph

* Split Reverse from ReverseSequence (oops), and fix a typo I noticed in dyn_reshape.cpp

* fix EOF newline

* Style.

* Add newline at eof.