ALSA: compress: fix an integer overflow check

I previously added an integer overflow check here but looking at it now,
it's still buggy.

The bug happens in snd_compr_allocate_buffer().  We multiply
".fragments" and ".fragment_size" and that doesn't overflow but then we
save it in an unsigned int so it truncates the high bits away and we
allocate a smaller than expected size.

Fixes: b35cc8225845 ('ALSA: compress_core: integer overflow in snd_compr_allocate_buffer()')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Takashi Iwai <tiwai@suse.de>

diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index 7403f348ed1425733ce77735c8dff37d1610ad4d..89028fab64fdd27bd75698e651fdc3b26b424941 100644 (file)
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -491,7 +491,7 @@ static int snd_compress_check_input(struct snd_compr_params *params)
 {
        /* first let's check the buffer parameter's */
        if (params->buffer.fragment_size == 0 ||
-                       params->buffer.fragments > SIZE_MAX / params->buffer.fragment_size)
+           params->buffer.fragments > INT_MAX / params->buffer.fragment_size)
                return -EINVAL;
 
        /* now codec parameters */