• Pruthvi's avatar
    Softmax + cross Entropy fusion for numerical Stabilization (#3669) · 1e2a3f34
    Pruthvi authored
    * - WIP fusion pattern for softmax + cross entropy
    
    * fix compiler error
    
    *  make summation axis integer for the fusion pattern
    
    * - Fusion pattern for sigmoid cross entropy bprop
    
    * WIP callback implementation for fused sigmod+crossentropy fprop
    
    * - implemented fprop softmax+crossentropy as single layer for numerical
    stabilization
    - added broadcasting nodes to fix elementwise assertions
    
    * Added unit test case for functionality test
    
    * Move the softmax + crossentropy fusion pass to core
    
    * i) style fix ii)added missing header
    
    * - Added new Fused Op for Softmax + CrossEntropy
    - moved the decomposition to the Softmax + CrossEntropy FusedOp
    
    * - Add SoftmaxCrossEntropy for fused tablegen
    - Add serializer support for SoftmaxCrossEntropy
    - fix documentation
    
    * Added missing json file for unit test case
    
    * Addressed PR comment
    
    * Addressed PR comments
    
    * - Fix fusion string
    
    * - Style fix
    
    * - Added Bprop for Softmax + crossEntropy
    
    * - added SoftmaxCrossEntropy support when soft_lable is provided
    - serailizer and deserializer support for SoftmaxCrossEntropyBprop
    
    * - Added support in decompose_op for SM+CE bprop when ignore_mask is specified
    
    * Updated Doc strinng
    
    * - unit test case for SoftmaxCrossEntropy backprop with soft lables
    - fixed decompose_op bug in bprop
    
    * - if soft_label=true, capture pattern only if the labels dont have one
    hot encoding
    
    * - SoftmaxCrossEntropyBprop Support if ignore_index is specified
    
    * add serialized files for unit test
    
    * - fix softmax + CE pattern bug
    - fix softmax + CE decompose_op() bug
    
    * - change reduction_axes to int64_t type in fprop and bprop ctor
    
    * - add soft_labels and ignore_index attribute to SM+CE fprop ctor
    
    * - addition asserts in unit test to ensure SM + CE fprop and bprop fusion is successful
    
    * - move reduction_axis computation to decompose_op from ctor to relax on
    dynamic shapes
    
    * Addressd PR Comments
    
    * - suppprt for SM+CE for ignore_index and softmax=false
    
    * - test case for SM+CE fprop with ignore_mask, soft_labels=false
    - fix bug in decompose_op
    
    * - refactor unit test case
    
    * - fix PDPD unit test
    
    * broadcast delta if shape mismatches
    
    * -fix bdcast issue in decompose_op
    1e2a3f34
Name
Last commit
Last update
.ci Loading commit data...
cmake Loading commit data...
contrib/docker Loading commit data...
doc Loading commit data...
licenses Loading commit data...
maint Loading commit data...
python Loading commit data...
src Loading commit data...
test Loading commit data...
.clang-format Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.travis.yml Loading commit data...
ABOUT.md Loading commit data...
CMakeLists.txt Loading commit data...
CODEOWNERS Loading commit data...
CONTRIB.md Loading commit data...
LICENSE Loading commit data...
README.md Loading commit data...
VERSION.in Loading commit data...
changes.md Loading commit data...
ecosystem-overview.md Loading commit data...