mm: fix wrong vmap address calculations with odd NR_CPUS values

Commit db64fe02258f ("mm: rewrite vmap layer") introduced code that does
address calculations under the assumption that VMAP_BLOCK_SIZE is a
power of two.  However, this might not be true if CONFIG_NR_CPUS is not
set to a power of two.

Wrong vmap_block index/offset values could lead to memory corruption.
However, this has never been observed in practice (or never been
diagnosed correctly); what caught this was the BUG_ON in vb_alloc() that
checks for inconsistent vmap_block indices.

To fix this, ensure that VMAP_BLOCK_SIZE always is a power of two.

BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=31572
Reported-by: Pavel Kysilka <goldenfish@linuxsoft.cz>
Reported-by: Matias A. Fonzo <selk@dragora.org>
Signed-off-by: Clemens Ladisch <clemens@ladisch.de>
Signed-off-by: Stefan Richter <stefanr@s5r6.in-berlin.de>
Cc: Nick Piggin <npiggin@suse.de>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Krzysztof Helt <krzysztof.h1@poczta.fm>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: 2.6.28+ <stable@kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 464621d18eb249fde1403741e0f75b193aa0c18a..7ef0903058eeb110246b350aa02154257a3202b6 100644 (file)
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -725,9 +725,10 @@ static void free_unmap_vmap_area_addr(unsigned long addr)
 #define VMAP_BBMAP_BITS_MIN    (VMAP_MAX_ALLOC*2)
 #define VMAP_MIN(x, y)         ((x) < (y) ? (x) : (y)) /* can't use min() */
 #define VMAP_MAX(x, y)         ((x) > (y) ? (x) : (y)) /* can't use max() */
-#define VMAP_BBMAP_BITS                VMAP_MIN(VMAP_BBMAP_BITS_MAX,           \
-                                       VMAP_MAX(VMAP_BBMAP_BITS_MIN,   \
-                                               VMALLOC_PAGES / NR_CPUS / 16))
+#define VMAP_BBMAP_BITS                \
+               VMAP_MIN(VMAP_BBMAP_BITS_MAX,   \
+               VMAP_MAX(VMAP_BBMAP_BITS_MIN,   \
+                       VMALLOC_PAGES / roundup_pow_of_two(NR_CPUS) / 16))
 
 #define VMAP_BLOCK_SIZE                (VMAP_BBMAP_BITS * PAGE_SIZE)