UPSTREAM: arm64: mm: move pgd_cache initialisation to pgtable_cache_init

Initialising the suppport for EFI runtime services requires us to
allocate a pgd off the back of an early_initcall. On systems where the
PGD_SIZE is smaller than PAGE_SIZE (e.g. 64k pages and 48-bit VA), the
pgd_cache isn't initialised at this stage, and we panic with a NULL
dereference during boot:

  Unable to handle kernel NULL pointer dereference at virtual address 00000000

  __create_mapping.isra.5+0x84/0x350
  create_pgd_mapping+0x20/0x28
  efi_create_mapping+0x5c/0x6c
  arm_enable_runtime_services+0x154/0x1e4
  do_one_initcall+0x8c/0x190
  kernel_init_freeable+0x84/0x1ec
  kernel_init+0x10/0xe0
  ret_from_fork+0x10/0x50

This patch fixes the problem by initialising the pgd_cache earlier, in
the pgtable_cache_init callback, which sounds suspiciously like what it
was intended for.

Reported-by: Dennis Chen <dennis.chen@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Bug: 30369029
Patchset: rework-pagetable

(cherry picked from commit 39b5be9b4233a9f212b98242bddf008f379b5122)
Signed-off-by: Jeff Vander Stoep <jeffv@google.com>
Change-Id: I124f03d19299be93124af35641294bf73c13bb22

diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 51f382b7e5c701a3480765e8fd6ad4b0116ed077..69d2e2f86bce3f65a76637e9ab3cd9251d6f3d16 100644 (file)
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -680,7 +680,8 @@ extern int kern_addr_valid(unsigned long addr);
 
 #include <asm-generic/pgtable.h>
 
-#define pgtable_cache_init() do { } while (0)
+void pgd_cache_init(void);
+#define pgtable_cache_init     pgd_cache_init
 
 /*
  * On AArch64, the cache coherency is handled via the set_pte_at() function.

diff --git a/arch/arm64/mm/pgd.c b/arch/arm64/mm/pgd.c
index cb3ba1b812e74dcd1acbc167756d60da331d105f..ae11d4e03d0e68d7f0fe621f1c9d313fcab09127 100644 (file)
--- a/arch/arm64/mm/pgd.c
+++ b/arch/arm64/mm/pgd.c
@@ -46,14 +46,14 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd)
                kmem_cache_free(pgd_cache, pgd);
 }
 
-static int __init pgd_cache_init(void)
+void __init pgd_cache_init(void)
 {
+       if (PGD_SIZE == PAGE_SIZE)
+               return;
+
        /*
         * Naturally aligned pgds required by the architecture.
         */
-       if (PGD_SIZE != PAGE_SIZE)
-               pgd_cache = kmem_cache_create("pgd_cache", PGD_SIZE, PGD_SIZE,
-                                             SLAB_PANIC, NULL);
-       return 0;
+       pgd_cache = kmem_cache_create("pgd_cache", PGD_SIZE, PGD_SIZE,
+                                     SLAB_PANIC, NULL);
 }
-core_initcall(pgd_cache_init);