• Frank Barchard's avatar
    ubsan overflow fix for multiply by 0x01010101 · 9e39c1f2
    Frank Barchard authored
    This is an UBSan error reported by libjingle
    
    [ RUN      ] WebRtcVideoFrameTest.ConvertToYUY2BufferStride
    [000:000] (videoframe.cc:375): Validate frame passed. format: I420 bpp: 12 size: 1280x720 bytes: 1382400 expected: 1382400 sample[0..3]: 73, 73, 73, 73
    ../../chromium/src/third_party/libyuv/source/row_gcc.cc:2903:25: runtime error: signed integer overflow: 128 * 16843009 cannot be represented in type 'int'
    [8/614] WebRtcVideoFrameTest.ConvertToYUY2BufferStride returned/aborted with exit code 1 (32 ms)
    [9/614] WebRtcVideoFrameTest.ConvertToYUY2BufferInverted (29 ms)
    Note: Google Test filter = WebRtcVideoFrameTest.ConvertToYUY2BufferInverted
    
    The source is uint8 and the multiply is by 0x01010101 to replicate the byte to 4 bytes.
    Changing the constant to 0x01010101u should avoid overflow.
    
    R=harryjin@google.com
    TBR=harryjin@google.com
    BUG=libyuv:563
    
    Review URL: https://codereview.chromium.org/1657533005 .
    9e39c1f2
Name
Last commit
Last update
..
compare.cc Loading commit data...
compare_common.cc Loading commit data...
compare_gcc.cc Loading commit data...
compare_neon.cc Loading commit data...
compare_neon64.cc Loading commit data...
compare_win.cc Loading commit data...
convert.cc Loading commit data...
convert_argb.cc Loading commit data...
convert_from.cc Loading commit data...
convert_from_argb.cc Loading commit data...
convert_jpeg.cc Loading commit data...
convert_to_argb.cc Loading commit data...
convert_to_i420.cc Loading commit data...
cpu_id.cc Loading commit data...
mjpeg_decoder.cc Loading commit data...
mjpeg_validate.cc Loading commit data...
planar_functions.cc Loading commit data...
rotate.cc Loading commit data...
rotate_any.cc Loading commit data...
rotate_argb.cc Loading commit data...
rotate_common.cc Loading commit data...
rotate_gcc.cc Loading commit data...
rotate_mips.cc Loading commit data...
rotate_neon.cc Loading commit data...
rotate_neon64.cc Loading commit data...
rotate_win.cc Loading commit data...
row_any.cc Loading commit data...
row_common.cc Loading commit data...
row_gcc.cc Loading commit data...
row_mips.cc Loading commit data...
row_neon.cc Loading commit data...
row_neon64.cc Loading commit data...
row_win.cc Loading commit data...
scale.cc Loading commit data...
scale_any.cc Loading commit data...
scale_argb.cc Loading commit data...
scale_common.cc Loading commit data...
scale_gcc.cc Loading commit data...
scale_mips.cc Loading commit data...
scale_neon.cc Loading commit data...
scale_neon64.cc Loading commit data...
scale_win.cc Loading commit data...
video_common.cc Loading commit data...