• Pruthvi's avatar
    Softmax + cross Entropy fusion for numerical Stabilization (#3669) · 1e2a3f34
    Pruthvi authored
    * - WIP fusion pattern for softmax + cross entropy
    
    * fix compiler error
    
    *  make summation axis integer for the fusion pattern
    
    * - Fusion pattern for sigmoid cross entropy bprop
    
    * WIP callback implementation for fused sigmod+crossentropy fprop
    
    * - implemented fprop softmax+crossentropy as single layer for numerical
    stabilization
    - added broadcasting nodes to fix elementwise assertions
    
    * Added unit test case for functionality test
    
    * Move the softmax + crossentropy fusion pass to core
    
    * i) style fix ii)added missing header
    
    * - Added new Fused Op for Softmax + CrossEntropy
    - moved the decomposition to the Softmax + CrossEntropy FusedOp
    
    * - Add SoftmaxCrossEntropy for fused tablegen
    - Add serializer support for SoftmaxCrossEntropy
    - fix documentation
    
    * Added missing json file for unit test case
    
    * Addressed PR comment
    
    * Addressed PR comments
    
    * - Fix fusion string
    
    * - Style fix
    
    * - Added Bprop for Softmax + crossEntropy
    
    * - added SoftmaxCrossEntropy support when soft_lable is provided
    - serailizer and deserializer support for SoftmaxCrossEntropyBprop
    
    * - Added support in decompose_op for SM+CE bprop when ignore_mask is specified
    
    * Updated Doc strinng
    
    * - unit test case for SoftmaxCrossEntropy backprop with soft lables
    - fixed decompose_op bug in bprop
    
    * - if soft_label=true, capture pattern only if the labels dont have one
    hot encoding
    
    * - SoftmaxCrossEntropyBprop Support if ignore_index is specified
    
    * add serialized files for unit test
    
    * - fix softmax + CE pattern bug
    - fix softmax + CE decompose_op() bug
    
    * - change reduction_axes to int64_t type in fprop and bprop ctor
    
    * - add soft_labels and ignore_index attribute to SM+CE fprop ctor
    
    * - addition asserts in unit test to ensure SM + CE fprop and bprop fusion is successful
    
    * - move reduction_axis computation to decompose_op from ctor to relax on
    dynamic shapes
    
    * Addressd PR Comments
    
    * - suppprt for SM+CE for ignore_index and softmax=false
    
    * - test case for SM+CE fprop with ignore_mask, soft_labels=false
    - fix bug in decompose_op
    
    * - refactor unit test case
    
    * - fix PDPD unit test
    
    * broadcast delta if shape mismatches
    
    * -fix bdcast issue in decompose_op
    1e2a3f34