1. 25 Oct, 2019 1 commit
    • Pruthvi's avatar
      Softmax + cross Entropy fusion for numerical Stabilization (#3669) · 1e2a3f34
      Pruthvi authored
      * - WIP fusion pattern for softmax + cross entropy
      
      * fix compiler error
      
      *  make summation axis integer for the fusion pattern
      
      * - Fusion pattern for sigmoid cross entropy bprop
      
      * WIP callback implementation for fused sigmod+crossentropy fprop
      
      * - implemented fprop softmax+crossentropy as single layer for numerical
      stabilization
      - added broadcasting nodes to fix elementwise assertions
      
      * Added unit test case for functionality test
      
      * Move the softmax + crossentropy fusion pass to core
      
      * i) style fix ii)added missing header
      
      * - Added new Fused Op for Softmax + CrossEntropy
      - moved the decomposition to the Softmax + CrossEntropy FusedOp
      
      * - Add SoftmaxCrossEntropy for fused tablegen
      - Add serializer support for SoftmaxCrossEntropy
      - fix documentation
      
      * Added missing json file for unit test case
      
      * Addressed PR comment
      
      * Addressed PR comments
      
      * - Fix fusion string
      
      * - Style fix
      
      * - Added Bprop for Softmax + crossEntropy
      
      * - added SoftmaxCrossEntropy support when soft_lable is provided
      - serailizer and deserializer support for SoftmaxCrossEntropyBprop
      
      * - Added support in decompose_op for SM+CE bprop when ignore_mask is specified
      
      * Updated Doc strinng
      
      * - unit test case for SoftmaxCrossEntropy backprop with soft lables
      - fixed decompose_op bug in bprop
      
      * - if soft_label=true, capture pattern only if the labels dont have one
      hot encoding
      
      * - SoftmaxCrossEntropyBprop Support if ignore_index is specified
      
      * add serialized files for unit test
      
      * - fix softmax + CE pattern bug
      - fix softmax + CE decompose_op() bug
      
      * - change reduction_axes to int64_t type in fprop and bprop ctor
      
      * - add soft_labels and ignore_index attribute to SM+CE fprop ctor
      
      * - addition asserts in unit test to ensure SM + CE fprop and bprop fusion is successful
      
      * - move reduction_axis computation to decompose_op from ctor to relax on
      dynamic shapes
      
      * Addressd PR Comments
      
      * - suppprt for SM+CE for ignore_index and softmax=false
      
      * - test case for SM+CE fprop with ignore_mask, soft_labels=false
      - fix bug in decompose_op
      
      * - refactor unit test case
      
      * - fix PDPD unit test
      
      * broadcast delta if shape mismatches
      
      * -fix bdcast issue in decompose_op
      1e2a3f34