slub: correctly bootstrap boot caches

After we create a boot cache, we may allocate from it until it is bootstraped.
This will move the page from the partial list to the cpu slab list. If this
happens, the loop:

	list_for_each_entry(p, &n->partial, lru)

that we use to scan for all partial pages will yield nothing, and the pages
will keep pointing to the boot cpu cache, which is of course, invalid. To do
that, we should flush the cache to make sure that the cpu slab is back to the
partial list.

Signed-off-by: Glauber Costa <glommer@parallels.com>
Reported-by:  Steffen Michalke <StMichalke@web.de>
Tested-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Acked-by: Christoph Lameter <cl@linux.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Tejun Heo <tj@kernel.org>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

diff --git a/mm/slub.c b/mm/slub.c
index 6184b0821f7e49e3b872e30c424b8e51668df120..aa0728daf8bb37c503c6518b7dd1ec414fdaef80 100644 (file)
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3552,6 +3552,12 @@ static struct kmem_cache * __init bootstrap(struct kmem_cache *static_cache)
 
        memcpy(s, static_cache, kmem_cache->object_size);
 
+       /*
+        * This runs very early, and only the boot processor is supposed to be
+        * up.  Even if it weren't true, IRQs are not up so we couldn't fire
+        * IPIs around.
+        */
+       __flush_cpu_slab(s, smp_processor_id());
        for_each_node_state(node, N_NORMAL_MEMORY) {
                struct kmem_cache_node *n = get_node(s, node);
                struct page *p;