drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl()
authorDan Carpenter <dan.carpenter@oracle.com>
Wed, 16 May 2018 14:00:26 +0000 (17:00 +0300)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Wed, 20 Jun 2018 19:03:00 +0000 (04:03 +0900)
commit96ef5c44149479f13b8943a556f4193d126e1222
tree242b68e60754e97957b7a13086612df43d741d5b
parent77a60e752ec8d2b0b507d55f7a5aa36e6c8c15e7
drm/dumb-buffers: Integer overflow in drm_mode_create_ioctl()

[ Upstream commit 2b6207291b7b277a5df9d1aab44b56815a292dba ]

There is a comment here which says that DIV_ROUND_UP() and that's where
the problem comes from.  Say you pick:

args->bpp = UINT_MAX - 7;
args->width = 4;
args->height = 1;

The integer overflow in DIV_ROUND_UP() means "cpp" is UINT_MAX / 8 and
because of how we picked args->width that means cpp < UINT_MAX / 4.

I've fixed it by preventing the integer overflow in DIV_ROUND_UP().  I
removed the check for !cpp because it's not possible after this change.
I also changed all the 0xffffffffU references to U32_MAX.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20180516140026.GA19340@mwanda
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
drivers/gpu/drm/drm_dumb_buffers.c