* Lin quant ONNX operators

* Add EOL at EOF

* Update licence year

* Cleanup namespaces

* Add missing headers

* Check if group is correct

* Remove Check in dequantize

* Remove TODO

* Add converting zero point to data element type

* Remove unused scale variable

* Removing zero point

* Remove make_ng_conv_bias helper function

* Add test for Quantize Linear

* Remove dead code

* Add test for Dequantize Linear

* Add Quant Conv test

* Convert models to prototxt

* Remove artifact test

* Skip test on INTERPRETER

* Style check

* Remove stupid skipping

* Enable test skipping in onnx_backend tests

* Skip GPU tests on quantized operators

* Review fix

* Handle models in prototxt format.
Signed-off-by: Adam Rogowiec <adam.rogowiec@intel.com>

* Convert Onnx models to text format.

* Tool for conversion between protobuf ONNX models in binary and text format.
Signed-off-by: Adam Rogowiec <adam.rogowiec@intel.com>

* Update ONNX test models text format.

* Convert ONNX LSTM model to text format.

* Rename and make second argument optional.

* Unify models metadata.

* Remove onnx binary models.

* Review: fix comments.

* Fix not yet updated models extension in UTs.

* Add UT with binary protobuf models.

* Clang format.

* Remove failing UT in prototxt since it has problems with CI.

* Inhibit logging protobuf errors.

* Revert "Remove failing UT in prototxt since it has problems with CI."

This reverts commit 94741a8c4594f3cc2ebdca428fa40b94a79240c2.

* Remove LogSilencer from onnx importer api and remove respective UT.

* Conversion script updates

* Reuse memory for CPU backend.

* Use NGRAPH_REUSE_MEMORY to enable memory reuse.

* Add a test.

* Move make_function to test_tools.cpp.

* Add more comments.

* Address PR Feedback: add a method to CPU backend.

* *Add a member to CPUOpAnnotations to remove redundant code.

*Overload compile function for CPU backend.

* Move make_function out of test_tools.

* Address PR Feedback.

* Use modified liveness analysis in CPUMemoryAssignment pass.

* Use lambda expression.

* Fix style error.

* Check if any user of the tensor has destructive io when building tensor alias map.

* Fix a bug.

* Check if tensor has multiple users.

* Allow tensor alias for destructive oi node.

* Update multiple_users_tensor set along the chain of in place ops.

* No tensor alias if input is parameter or constant.

* Use buffer sets in cpu memory assignment,
tensors sharing the same memory buffer are put into the same set.

* Add more checks and do not combine sets when allowing destructive oi.

* Style fix.

* Do no allow destructive oi if the input tensor uses function input memory.

Update set label.

* Add unit tests.

* Style fix.

* Get the correct size for memcpy when the input is padded.

* Style fix.

* Address PR feedback.

* Address PR feedback.

* Move make_function in cpu_test after #if 0 and before the disabled test.

* Add utility functions.

Use iterator.

Rename variables.

* Add pass attributes and move cpu memory assignment to common passes (#2504)

* Utility functions for calculating Lp norm.

* Use functor object as a reduction operation.

* Use new api of make_ng_reduction_op.

* Use utility norm functions for reduction operations.

* Onnx GlobalLpPool operator.

* Ensure correct shapes after lp_norm reduction.

* Remove unused function overload.

* Fix shapes and tensor types.

* Unit tests.

* Update comments.

* Update supported ops status table.

* Fix: take absolute value of input tensor elements.

* UT: with odd value p-norm.

* Fix: move taking abs value into respective lp-norm functions.

* Fix clang -Wdocumentation-unknown-command error.

* Update supported op status table with new Jira ticket for Erf op.

* Update supported_ops status table.

* Update interface of make_ng_reduction_op - accept std::function object.

* Update to use new make_ng_reduction_op api.

* Remove unused header.

* Fix errors on CentOS.

* - Added reorder support for rnn weights_layer/iter

* i) fixed compilation issues ii) working but still observing precision error

* i) fixed failing rnn unit test for DEX ii) refactored workspace in RNN mkldnn emitter

* i) added support for src reorder to TNC from NTC

* reorder support for rnn output fron NTC to TNC

* - added support for rnn weight reorder ldgoi -> ldigo
- code refactor for lstm/rnn kernel in mkldnn emitter

* - refactor rnn mkldnnn kernel, change variable names

* fix RNN codegen kernel

* disbale layer rnn fusion pass, to test CI

* method to validate recurrent rnn inputs

* add correlated macthes for Recurrent RNN PM

* - simplify reorder logic for rnn_weights
- fix graph pattern for fusing rnn cell across time steps

* do weights reorders in rnn timesteps fusion

* refactored LSTM graph pass

* - Bug fix for finding the lstm inputs determenstically
- Refactored LSTM graph pass to single pass
- made changes to LSTM RNN time step fusion graph pass

* - use replace_node instead of replace_output in Lstm_step_wise fusion graph pass

* fix compilation error

* Fix GNMT rnn fusion

* check if the node is in use before replacing in RNN graph passes

*  i) fix style ii) fix topo sort issue in RNN graph pass

* style fix

* fix bug in simplify_concat pass

* replaces Lstm1 -> {GOE1, GOE2} -> {Slice1, Slice2} -> Concat -> Lstm2 with Lstm1 -> Lstm2

* cse for convert layout

* addressed PR comments

* - optimization pass to remove  Lstm1 -> {GOE1, GOE2} -> {Slice1, Slice2} -> Lstm2
- conditional fusing of LSTM cells only for the decoder

* made changes to multi layer RNN fusion callback

* fix asserts in RNN op

* - added support to fuse layers when slc=dlc for RNN cells
- bug fix on the sanity checks for RNN Op

* - support RNN layer fusion till slc = dlc
- bug fixes in multi layer rnn fusion call back

* capture reshape in the RNN weights

* Addressed PR comments

* - added comments in multi layer PM call back
- fuse only if slc == DLC across layers

* restore deleted 3_lstm_cell_forward.json file

* fix typo

* fix failing unit tets

* When processing in place slice, do not change the offset of the slice node if the argument pointer comes from function input.

* Address PR feedback: process in place slice after propagating in place input.

* Set INTERMEDIATE role before propagating in place input.

* Do not add temporaries to the variable name map before propagating in place input in codegen.

* Fix a bug in codegen.

* Fix a bug in codegen slice.

* reenable disabled rnn unit test

* fix compiler error

* - bug fix in the slicing logic for the layer fused rnn cell
- fix failing rnn unit test

* - Addressed PR comments
- removed redundant checks from the rnn graph pass
- simplified rnn call back replace node logic

* - added new multilayer rnn *.json file
- fix test case

* [PRIVATE BRANCH] Style fixes (#2080)

* Style fixes

* change order of lstm gates

* WIP bi rnn

* [PRIVATE BRANCH] Jbobba/rnn fusion review (#2113)

* Style fixes for single-layer RNN fusion

* Style fixes to multi-layer RNN

* added callback routine for bi-directional rnn

* fix rnn op ctor, rnn mkldnn emitter to accomodate bi directional rnn

* style fix

* added helper function for rnn's to query direction and cell_type

* fix clang error

* - unit test case for bi rnn fusion
- style fix

* - updated bi-rnn graph pass to handle reverse and reverse_seq ops in the predicate
- added bi-rnn inter v/s cpu unit test case
- add support to in mkldnn_utils to create_md with tnc/ntc format

* - added enum type to deduce rnn_type

* Addressed PR comments
    - handle reshapes from {t, n, c} to {n, t, c} in the graph pass

* fix style

* fix clang error

* fix style

* i) move enum specific to rnn to seperate header

* [ONNX] Overriding custom ops

* Add UT

* Style Check

* Review & style fix

* [ONNX] Enable OneHot operation

* Add UT

* Style check

* Change converts

* Change assert messages

* Update comments

* Update onehot.cpp

* [ONNX] Add support for ONNX sinh op

* [ONNX] Test the sinh op and improve an error msg about not supported op version

* [ONNX] Fix the failing tests by moving sinh to opset 1

* [ONNX] Apply clang-format to fix failing CI builds

* [ONNX] Add and test ONNX cosh op

* [ONNX] Add and test ONNX cosh op

* [ONNX] Add and test ONNX sign op

* [ONNX] Test the sign operator with more challenging values

* [ONNX] Update of supported ops documentation table

* [ONNX] Add null node.

Optional inputs in ONNX standard are represented by empty string.
We need a placeholder to keep information which inputs are not provided.

* Rename class null_node -> NullNode

* Remove unnecesary validate_and_infer_types method

* Add <memory> header

* Change name != "" -> !name.empty()

* Change constructor

* Little description

* Change node type in NullNode

* Add is_null() method

* Docstring

* Add UT

* Use override

* Style check

* Update null_node.cpp

* [ONNX] Move make_ng_constant to Tensor class

* [ONNX] Add workaround for initializers without corresponding inputs

* Fix malformed test models

* clang-format

* Avoid creating multiple constants

* Code review comments

* [ONNX] Add support for ONNX sinh op

* [ONNX] Test the sinh op and improve an error msg about not supported op version

* [ONNX] Fix the failing tests by moving sinh to opset 1

* [ONNX] Apply clang-format to fix failing CI builds

* [ONNX] Add and test ONNX cosh op

* [ONNX] Add and test ONNX cosh op

* -   check to verify if the data_slices shares the same weights

* add the serialized graph

* - explicitly fuse the data slices, so all the parameter partitioned by slices are in contigous memory location
- fixes all the failing test cases

* Enable support for group attribute.

* UT for ConvTranspose with groups.

* Validate group attribute value.

* Move helper function to unnamed namespace.

* Access values with bounds checking.

* Fix spelling.

* UT for Softplus ONNX operator testing edge cases.

* Rename UT model name.

* Handle overflows.

* Add UT for ininite values and check them correctly.

* Update values in comment

* Add broadcasting to variadic OP in opset 8.

* Apply style format.

* Update onnx_import.cpp

* - Added reorder support for rnn weights_layer/iter

* i) fixed compilation issues ii) working but still observing precision error

* i) fixed failing rnn unit test for DEX ii) refactored workspace in RNN mkldnn emitter

* i) added support for src reorder to TNC from NTC

* reorder support for rnn output fron NTC to TNC

* - added support for rnn weight reorder ldgoi -> ldigo
- code refactor for lstm/rnn kernel in mkldnn emitter

* - refactor rnn mkldnnn kernel, change variable names

* fix RNN codegen kernel

* disbale layer rnn fusion pass, to test CI

* method to validate recurrent rnn inputs

* add correlated macthes for Recurrent RNN PM

* - simplify reorder logic for rnn_weights
- fix graph pattern for fusing rnn cell across time steps

* do weights reorders in rnn timesteps fusion

* refactored LSTM graph pass

* - Bug fix for finding the lstm inputs determenstically
- Refactored LSTM graph pass to single pass
- made changes to LSTM RNN time step fusion graph pass

* - use replace_node instead of replace_output in Lstm_step_wise fusion graph pass

* fix compilation error

* Fix GNMT rnn fusion

* check if the node is in use before replacing in RNN graph passes

*  i) fix style ii) fix topo sort issue in RNN graph pass

* style fix

* fix bug in simplify_concat pass

* replaces Lstm1 -> {GOE1, GOE2} -> {Slice1, Slice2} -> Concat -> Lstm2 with Lstm1 -> Lstm2

* cse for convert layout

* addressed PR comments

* - optimization pass to remove  Lstm1 -> {GOE1, GOE2} -> {Slice1, Slice2} -> Lstm2
- conditional fusing of LSTM cells only for the decoder

* made changes to multi layer RNN fusion callback

* fix asserts in RNN op

* - added support to fuse layers when slc=dlc for RNN cells
- bug fix on the sanity checks for RNN Op

* - support RNN layer fusion till slc = dlc
- bug fixes in multi layer rnn fusion call back

* capture reshape in the RNN weights

* Addressed PR comments

* - added comments in multi layer PM call back
- fuse only if slc == DLC across layers

* restore deleted 3_lstm_cell_forward.json file

* fix typo

* fix failing unit tets

* When processing in place slice, do not change the offset of the slice node if the argument pointer comes from function input.

* Address PR feedback: process in place slice after propagating in place input.

* Set INTERMEDIATE role before propagating in place input.

* Do not add temporaries to the variable name map before propagating in place input in codegen.

* Fix a bug in codegen.

* Fix a bug in codegen slice.

* reenable disabled rnn unit test

* fix compiler error

* - bug fix in the slicing logic for the layer fused rnn cell
- fix failing rnn unit test

* - Addressed PR comments
- removed redundant checks from the rnn graph pass
- simplified rnn call back replace node logic

* - added new multilayer rnn *.json file
- fix test case

* [PRIVATE BRANCH] Style fixes (#2080)

* Style fixes

* change order of lstm gates

* [PRIVATE BRANCH] Jbobba/rnn fusion review (#2113)

* Style fixes for single-layer RNN fusion

* Style fixes to multi-layer RNN

* style fix

* disable GPU test

* Unit tests for conv2d causing errors.

* UT for conv3D_bias

* Fix padding order.

`padding below` in nGraph terminology means padding added at the beginning
of the axis. Whereas `padding above` means padding added at the end of
the axis.

* Rename test to sth more descriptive.

* Apply clang-format.

* Fix handling of `SAME_UPPER/LOWER` auto_pads mode for convolution/pooling ops.

* Fix order of padding_below/above.
Signed-off-by: Adam Rogowiec <adam.rogowiec@intel.com>

* Fix error in calculating output data shape.

* Add static keyword for helper function.

* Fix MatMul for cases where left hand side is 1D vector.

- Add unit-test for this case.

* Add new line at the end of file.

* Log warning when dealing with scalars

* Apply clang-format

* Review: fix spelling, rename test model.

* Add ArgMin operator

* Add ArgMax and a basic test case

* Rename variables

* Apply workaround for problems with Reshape on i64

* Review comments

* Review comments

* onnx: add information about a domain to operators set
Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>

* onnx: updates after review
Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>

* onnx: update comments in the code
Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>

* onnx: fix bug in node's description method
Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>

* onnx: fix CentOS compilation
Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>

* onnx: more after review changes
Signed-off-by: Artur Wojcik <artur.wojcik@intel.com>

* split bn into bn_inference bn_training

* fix warnings

* Add GPU support for the new BN ops (#1569)

* Add GPU support and change batchnorm_globalstats test to use BNInference.

* Changed test back to using BNTraining for global stats and updated cudnn backend to account for it.

* Fix issues in merge with master.

* Formatting.

* CPU fixes

* remove 5-arg training BN for now

* more fixes

* python batchnorm changes

* fix onnx_import

* fix a call BatchNormInference c-tor

* yet another fix to BatchNormInference c-tor

* AND yet another fix to batchnorm_inference c-tor

* ops.py

* address adam's feedback

* Remove unnecessary parameter/argument.

* remove batch_norm_training_relu_with_global_stats

* remove bn_relu (training)

* [ONNX] Assert all op types supported

* Apply clang-format

* Address code review comments

* Fix #include statements

* Update ONNX Squeeze Op implementation to conform with doc. Add unit test.

* Apply code-format.

* Correct attribute value type.

* Change used loop structure.

* Modified version of loops.

- Without erase and with minimal computation time complexity.

* Run CI

* reshape sinking working on mnist_conv

* forgot to add reshape_sinking files

* refactoring of binary case

* Quantize/Dequantize case, fix add case, add assert

* address bob and scott's feedback

* debug

* fix a bug where reshapes are removed too early

* adding nGraph Quantize op

* unit test failing for floating point exception

* unit test working in float

* unit test working in uint8

* improved type checking and polished unit test - passing

* quantized axes working

* inclusive project method

* add round mode

* TODO cleanup

* code format

* adding serializer support - fails build

* add serializer support

* make CPU quantize op work; new tests for int8, clamp)

* fix build failure

* fix GPU build issue

* fix GPU unit test manifest

* use quantized offset

* add is_quantized field to element::Type

* add reduce function to coordinate.hpp

* [ONNX] Non-linear operators

* Review fix pt. 1

* Review fix pt. 2

* Non-linear tests

* style check

* Exception fix

* Test fix

* Add missing header.

* Test for ReduceSum

* Simple tests for reductions

- L1/L2/LogSum/LogSumExp/Max/Mean/Min/Prod/SumSquare.

* Add floating point literal suffix

* Fix typo

* [ONNX] Shape operator

* Review fix pt. 1

* Style check

* Move reshape utils down to reshape namespace.

* Reshape operation.

* Reshape operator binding.

* Error fixes.

* Reshape unit tests.

* Move flatten utility function to reshape namespace.

* Fix unused catched exception object

* Add Constant support for int64

* Review fix.

* clang-format

* Review fix part 2.

* Enable output shape as a second node input (only Constant).

* Unit test for "dynamic" output shape (from Constant node).

* Review fixes.

* Make sure second Reshape op input is Constant node.

* [ONNX] Concat operator

* Style fix