Primitive CPU Executor for managing thread pools and executing kernels across partitioned resources (#1858)

* Add arena support to DEX kernels

* Add CPUExecutionContext to pass executor information to kernels

* Removed eigen_thread_pool. Functionality moved to cpu_executor

* Moved MKLDNN engine to CPU executor

* Added TBB arenas for node affinitization

* Fixed missing arena in one hot kernel

* Style fixes

* remove 5arg BNTraining

* Remove 5-arg BNWithStats from gpu/op.

* Change GetOutputElement to use delayed validation

* Cleanup

* Update "JSON for Modern C++" to a version that officially supports gcc 4.8

* "JSON for Modern C++" is header only and does not need to be configured.

* Implement partial shape/type propagation for Convolution; fail for want of unit tests

* Implement unit tests for partial shapes/types for Convolution

* Implement partial shape/type validation for BatchNormInference, BatchNormTraining, BatchNormTrainingBackprop with unit tests

* Formatting

* Update CPU and GPU backends to deal with elimination of BatchNormBase

* Update BatchNormTrainingWithStats to use templated emit_BatchNorm function

* Restore five-argument BatchNormTraining ctor for now; #1901 will eliminate it

* Replace enum for input indices with symbolic constants

* Update intelgpu/visualize_tree.cpp to deal with removal of BatchNormBase

* generate mask

* add codegen+dex

* states in context

* CPU dropout

* remove dead code

* remove dead code

* remove state.cpp

* change perms, add state.cpp

* address bobs feedback

* restore igpu unit-test manifest after a bad merge

* better error msgs

* throw on GPUs to keep a compiler happy

* address more feedback

* fix tests

* graph_rewriter until fixed point

* fix a bug + comment

* move comment

* move rewritten=true into process_node conditional

* create unique_ptr backend

* unit test cleanup

* address more code that was recently added

* change from reference to pointer when passing backend to reduce the number of lines changed.

* fix build error

* fix python wrapper

* style

* more specific treatment for unique_ptr

* [WIP] Draft for matmul

* Numpy-style broadcasting for matrix multiplication.

* 3D matrix multiplication with one big Dot/slice/concat.

* Generic ND matmul implementation using slice/dot/concat pattern.

* Code formatting.

* remove unused header

* Add missing header

* Utility reshape-like functions.

* Use utility functions.

* Review comments.

* Code format

* Use if/else instead of ternary operator for readability.

* Remove unused function overloading

* Utility function expanding tensor shape with empty axes.

* Use helper functions.

* Use type for auto variable initializer to fix Centos build

* Fix Centos build errors.

* Initial implementation of GroupConv+BatchNorm fusion

* Added GroupConv+BatchNorm with Relu fusion

* Added changes to fuse with BoundedRelu

* Changed BoundedRelu to Relu

* Added test; Code cleanup

* Code formatting

* Removed dead code

* Added test cases and other misc

* Bug fix in group conv callback and general cleanup

* Address PR feedback

* Minor edit to comment. MKLDNN divides both input and output channels by groups

* Style fixes and PR feedback

* Add ArgMin operator

* Add ArgMax and a basic test case

* Rename variables

* Apply workaround for problems with Reshape on i64

* Review comments

* Review comments

* wip

* wip

* simple hybrid test harness

* cleanup

* disable unit test in progress

* Add PlaidML backend

* CR comments

Used m_ prefix for members; removed trailing underscores
Updated license headers
Moved associated header inclusions to project blocks
Wrapped comments to 100 chars
Added missing newlines between functions
Removed nested namespaces in operation implementations

* Add earhart to CODEOWNERS

* Rebase updates

* style

* fresh commit for the changes

* Working topk on ndims for nvGPU

* fix

* clang

* Added unit test, improved kernel hash and Bob's comment

* int64 test+clang

* Moved argReduce and topk tests to a separate file

* TopK unsupported for IntelGPU

* addressed Fenglei and Chris's comments

* addressed Fenglei and Chris's comments

* IntelGPU backend: Profile data improved

* PR1932. Comments addressed

* Move to TBB2019 and bug fix to capture functor

* Change to use TBB release tag

* remove lightweight from codegen

* Enable TBB flow graph tracing

* add split concat

* fix bug

* fix bug

* fix bug

* add test

* fix test bug

* add comments

* format

* return intead of check processed

* remove .back() since it's not vector anymore.

* format

* change to paramter tests based on Geoff's comments

* types-> type

* change split size to 256