Commit e057e83a authored by Michael Niedermayer's avatar Michael Niedermayer

swscale/output: Fix integer overflow in yuv2rgb_write_full() with out of range input

Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int'
Fixes: ticket8293

Found-by: Suhwan
Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
parent 49ba1879
...@@ -1850,9 +1850,9 @@ static av_always_inline void yuv2rgb_write_full(SwsContext *c, ...@@ -1850,9 +1850,9 @@ static av_always_inline void yuv2rgb_write_full(SwsContext *c,
Y -= c->yuv2rgb_y_offset; Y -= c->yuv2rgb_y_offset;
Y *= c->yuv2rgb_y_coeff; Y *= c->yuv2rgb_y_coeff;
Y += 1 << 21; Y += 1 << 21;
R = Y + V*c->yuv2rgb_v2r_coeff; R = (unsigned)Y + V*c->yuv2rgb_v2r_coeff;
G = Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff; G = (unsigned)Y + V*c->yuv2rgb_v2g_coeff + U*c->yuv2rgb_u2g_coeff;
B = Y + U*c->yuv2rgb_u2b_coeff; B = (unsigned)Y + U*c->yuv2rgb_u2b_coeff;
if ((R | G | B) & 0xC0000000) { if ((R | G | B) & 0xC0000000) {
R = av_clip_uintp2(R, 30); R = av_clip_uintp2(R, 30);
G = av_clip_uintp2(G, 30); G = av_clip_uintp2(G, 30);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment