* LogSoftmax introduced

* Added LogSoftmax to serializer

* Fixed style

* Fixed CmakeLists style

* code review remarks introduced

* Code review remarks introduced

* Shape as a node in ConvBpropData (v1)

* Fix constant op creation

* Convbpropfilters dynamic shape

* Avgpool bprop dynamic shape

* Add downgrade pass

* Downgrade pass Avgpoolbackprop

* Shape checks and shape relevancy

* Dyn avgpoolbprop test case

* Revert "Dyn avgpoolbprop test case"

This reverts commit b094f2f99d00023593c667ff16cf060a586ad16a.

* Revert "Avgpool bprop dynamic shape"

This reverts commit ef8a4d197577c0d277d634baa9d6d082adcddae0.

* Revert "Downgrade pass Avgpoolbackprop"

This reverts commit 3f31f0fea8fe6e79bd958cbaa7cb8495008a795f.

* Call shape relevancy

* Revert Avgpoolbprop changes

* mark nodes as dynamic

* Add opset transform tests

* Change API

* Adjust strides

* Use size

* Merge PR 3776

* add shape relevance

* Add convbprop_data test case

* Add convbprop_filter test case

* Use is_constant helper

* Remove opset pdpd test case

* resolve conflict

* add checks for validate and infer

* Type prop tests

* Rename input arg -> data

* Update node validation and shape propagation to support data rank and number of repeats to be different.

* Add type_prop tests

* Add UT for new tile cases

* style

* Modify CPU backend to support the new case.

* Fix backend selection in test

* Add new line back.

* DynSlice was adjusted to specyfication

* Fixed documentation

* Fixed styles

* Fixed clang warning

* Introduced StridedSlice as separate op

* revert dyn_slice

* Fixed StrideSlice implementation

* Enable downgrade pass in dynamic backend

* Implemented upgrade/downgrade pass

* Added serialization

* Styles applied

* Styles applied. Part.2

* Updated dynamic backends list

* Move StridedSlice outside experimental

* Make Slice as v0 of StridedSlice

* Styles applied

* Styles applied

* Fixed clang error

* Code review remarks introduced

* Move GenerateMask up in Opset0Downgrade to keep alphabetical sort

* Style apply

* unfold attributes

* Remove unnecesary if

* Rename run() -> lstm_pass()

* Unify usage of LSTMForward for one and bi directional LSTM

* Unify LSTMForward return values for one and bi directional LSTM

* Dirty moving LSTMForward into fused directory

* Accept lstm direction as string instead of enum

* Fused op which uses decompose_op in onnx_importer

* Rename LSTMForward -> LSTMSequence

* Split LSTMSequence to cpp and hpp

* Remove LSTMDirection enum

* Add getters for class fields

* Adjust constructors

* Add direction validation.

* Add support of LSTMSequence op in serializer

* Reorder fused op input order

* Style fix

* Fix for reorder of inputs

* Use NodeTypeInfo instead of static string

* Node -> value in doc

* Add doc for prepare_input method

* Fix shape inference

* Use enum instead of string for direction

* Add Type prop unit test

* Fix style

* Support auto broadcast in FakeQuantize op

* style fix

* add pdpd unit test to plaidml manifest

* - Added support for v1 Broadcast op specification
- Added upgrade/downgrade conversions between v0 and v1

* Added unit test for pdpd broadcast

* Make numpy default autobroadcast type and some style fixes

* Added support in Dynamic wrapper for dyn elimination and copied over unit tests from DynBroadcast

* Addressed PR feedback

* Addressed PR feedback on documentation

* Implemented downgrade pass

* Using Pad:v1 in onnx_importer

* Downgrade transformation doc fixed

* Added downgrade pass for reduce_prod

* Using ReduceProd:v1 in onnx_importer

* Added ReduceProd:1 downgrade support

* Clang styles applied

* Downgrade pass added for all backends

* Added downgrade pass for reduce_prod

* Using ReduceProd:v1 in onnx_importer

* Added ReduceProd:1 downgrade support

* Clang styles applied

* Apply suggestions from code review

Changed pad_opset_pass to opset_downgrade
Co-Authored-By: Tomasz Socha <tomasz.socha@intel.com>

* Fix names

* Apply suggestions from code review

Removed redundant include, renamed tests
Co-Authored-By: Tomasz Socha <tomasz.socha@intel.com>

* Changed order of passes

* Changed way of op casting

* Changed order of passes

* Changed downgrade pass order of CPU backend

* Changed downgrade pass order of CPU backend

* styles applied

* [SPEC] Fixed NormalizeL2 (#3695)

* Fixed NormalizeL2

* style applied

* Fixed axes rank checking

* removed redundant comma

* Set keep_dims by function argument

* Improved error message

* Fixed casting method

* fix styles

* styles applied

* Fixed normalize tests

* unit tests names refactor
Co-Authored-By: Michał Karzyński <postrational@users.noreply.github.com>

* Disable not supported PlaidML tests

* Added missing EOF

* Cyphers/lastwarns (#3640)

* Build changes for -Wall

* Use properties

* Remove no-zero-as-null-pointer-constant

* add ##__VAR_ARGS__ workaround

* enable gnu-zero-variadic-macro-arguments warning

* fix gnu-zero-variadic-macro-arguments

* use PrintToDummyParamName

* remove ##__VA_ARGS__ workaround

* fix ##__VA_ARGS__ and enable gnu-zero-variadic-macro-arguments

* handle windows build

* use alternative fix to support VS compiler

* fix merge issue

* remove variadic macros from kernel_selectors

* replace remaining ##__VA_ARGS__

* fix missing braces

* Remove some clang warnings

* Warnings cleanup

* Implemented downgrade pass

* Using Pad:v1 in onnx_importer

* Downgrade transformation doc fixed

* Downgrade pass added for all backends

* Apply suggestions from code review

Changed pad_opset_pass to opset_downgrade
Co-Authored-By: Tomasz Socha <tomasz.socha@intel.com>

* Changed order of passes

* Changed downgrade pass order of CPU backend

* Code review remarks introduced

* add check for dynamic dimensions

* to_shape() -> rank()

* Delete unnecessary check

* Added in DeduceDynSlice suite to check dynamic dimension

* minor codestyle fix

* c-cast -> static_cast

* LayerNorm (#3630)

* Constructors.

Type prop.

Decompose LayerNorm.

Add serialize.

* Add dummy test case.

* Add dummy type prop test.

* Fix some build errors.

* Remove build errors.

* Update decompose for bprop.

* Change begin_norm_axis default value to 1.

* Style.

* Reorder class members.

* Add actual type prop tests.

* Add fprop test.

* Working on bprop test.

* Bprop tests.

* Allow flattened scale and bias.

* Add support for flattened scale and bias.

* Fix incorrect type_name.

* PlaidML: Decompose fused_op LayerNorm

* Update Backprop constructors.

* PlaidML: Add missing header file.

* Remove doc about removed param.

* Fix type prop tests.

* PlaidML: Disable unit test.

* Fix stats flattening axes bug.

* Upgrade description to type_info.

* Strided slice

* Strided slice

* default value for strides

* Added new strided slice test, enabled old tests, refactoring

* Refactoring

* Autogenerated file: dyn_replace_slice tests

* Renaming

* Fix codestyle

* Fix build on MacOS

* Fix codestyle

* Add several tests in unit_test.manifest to skip it on PlaidML

* Disable all dyn_replace_slice tests on PlaidML

* Gather:v1 was introduced

* Added support for negative axis

* Removed unsused serialization

* Code review remarks introduced

* Change returned type of get_axis method

* Code review remarks introduced

* Chnaged axis_node to scalar during transformation

* Clang style applied

* Fixed clang errors

* style

* Add RandomUniform op

* Missing files

* Make CPU compile again (RandomUniform tests still don't pass)

* Bop users of existing RNGState class and reorder some comment junk

* Try to bop RNGState in gcpu

* Add RandomUniform to CPU manifest

* Add RandomUniform to PlaidML manifest

* Add .rst for random_uniform

* Clean up junk in the .rst

* Change UniformRNGState to always use double internally

* Change weird test failure message

* Compilation issues

* Fix unsigned comparison and implicit double conversion

* Eliminate sign/unsigned compare warnings, implicit double conversion

* Missed type

* Neww int type

* omp compromise

* style

* Pad:v1 was introduced

* Fixed problem with foward declaration

* Code review remarks introduced

* Addtinal asserts and unit-test for Pad:v1 introduced

* Handled optinal arg_pad_value input

* Fixed unit tests messages and clang styles applied

* Styles applied, checking rank improved

* Updated changes.md

* Revert "Updated changes.md"

This reverts commit 8ee16e580705275ee0719648d5251021354e4d84.

* Clang warning fixed

* Unnused parameter cleanup

* style

* Move required reshape helpers to builders

* Remove warning from matmul_factory

* Rid off onnx dependency from matmul factory

* Move malmul_factory into builders

* Add implementation of fused MatMul

* Add MatMul to serializer

* Add type_prop tests

* Remove reference.

* Make more metods private

* Use protected instead of private

* Fix compilation issues

* Change construction of matmul_factory in matmul op

* Add MatMul operator into switch of is_supported_impl function

* Change transpose flags from int to bool

* Review Fix I

* Update MatMul op

* Use OutputVector instead of NodeVector in MatmulFactories

* Fix usage of OutputVector

* Convert more shared_ptrs to Outputs

* Fix comments after merge

* Fix comments after merge II

* Fix comments after merge III

* wip

* r50 test

* update test

* R50 topk calls

* Faster topk

* Finish topk implementation

* cleanup

* checkpoint

* new test working

* more unit test

* style

* wip

* fix tests

* add needed header

* Add new TopK test to plaid manifest

* Fix windows build error:

* Axes input was added to LRN

* Unit tests for axes shape check were added

* LRN node deserialization was updated

* Fixed EOF and clang style applied

* Changed Constant to Parameter type in unit tests

* Expanded LRN reference ingterface

* Fixed LRN assert description

* Fixed passing arguments

* Reference implementation for one axis

* Implementation for channel

* Implementation for hw

* working on recurence version

* Implemented recurence version for hw

* Reference implementation code refactor

* Fixed ref LRN implementation and added tests

* Added 6D unit test

* Clang styles applied

* Code review remarks introduced

* Support for dynamic shape of axes input

* Clang styles applied

* Code review remarks introduced

* Added checking if axes values are in correct range

* Clang styles applied

* Removed redundant include

* Code review remarks introduced

* First pass, remove QDot as experimental op, add it as core op, test case failing

* Change QuantizedMatmul API

* Tests for QuantizedDot passing, cleanup required

* Move test cases to backend_test.in.cpp

* Change reference dot API to take scale and zero point

* Onnx matmul integer tests passing

* Add validate_and_infer_types()

* Add more checks for QDot

* Add type prop tests

* Add interpreter support

* Add serializer support

* Disable tests in other backends

* Fix test and add codegen support

* Make axes optional argument

* Change fusion pattern

* Copyright

* merge changes

* more merge changes

* Fix more merge conflicts

* change builders to reflect new changes

* cleanup

* Add reduction_axes to the API

* Fix onnx test cases

* Remove debug comments

* Fix CI

* disable plaidml tests

* Remove comment

* Normalize BatchNorm test names

* unit test renamed so manifest must be updated

* Changed name from Normalize to Normalize2

* Changed name from Normalize to Normalize2 in tests

* Changed name of normalize source files

* Removed across spatial and channel shared params

* Removed scale and introduced input axes

* Support for axes input was introduced

* Added possibility to choose method of bias using

* Clang style applied

* Code review remarks introduced

* Code review remarks introduced

* Added python script to generate normalize_l2 test data

* Changed required image_shape type from u64 to float

* Proposal op was adjusted

* Clang styles applied

* Added missing new line

* Fixed problem with narrowing conversion

* Code review remarks

* Removed checking types of Proposal input params

* Removed unused input element type variables

* [FUSED] Change alpha attribute in Elu from dynamic to static

* Fix python API

* Fix failing python test

* Remove unnecesary test

* Fix merge artifact

* Review fix I

* Review Fix PyAPI

* Axes input was added to LRN

* Unit tests for axes shape check were added

* LRN node deserialization was updated

* Fixed EOF and clang style applied

* Changed Constant to Parameter type in unit tests

* Fixed LRN assert description

* Fixed copy_with_new_args

* Clang style applied

* Better error checking for Argmin/Argmax

Argmin and Argmax are not defined if the reduction axis has length 0.

* Simplify

* Update test/type_prop/index_reduction.cpp
Co-Authored-By: Adam Procter <adam.m.procter@intel.com>

* Update test/type_prop/index_reduction.cpp
Co-Authored-By: Adam Procter <adam.m.procter@intel.com>

* Helper function converting ONNX auto_pad into nGraph PadType.

Separate auto_pad support from explicitly provided paddings.

* Add support for more PadType values for GroupConvolutionTranspose.

* Pass auto_pad attribute value to nGraph operator.

* Helper class for generating ONNX pooling operators.

* Pass auto pad type to nGraph Convolution operator.

* Use pooling factory.

* Helper function calculating pads taking into account auto_pad attribute.

* Fix attribute type in UT ONNX models.

* Take auto_pad attribute value into account.

* Rename helper function and update doc.

* Retain old API for GroupConvoloutionTranspose

* Remove PadType::INVALID and use other approach to validate auto_pad
value.

* Style apply.

* Fix clang err on documentation style.

* Refactor get_auto_pad method.

* Fix segfaults on CentOS 7.

When using const reference the node member m_attributes had invalid
data.

* Start splitting type_prop.cpp (just a few for now, to get feedback on general pattern)

* Split out some more type_prop stuff

* A bunch more

* Move one test out of type_prop into build_graph

* Split Reverse from ReverseSequence (oops), and fix a typo I noticed in dyn_reshape.cpp

* fix EOF newline

* Style.

* Add newline at eof.