MIPS: mm: Don't clobber $1 on XPA TLB refill

For XPA kernels build_update_entries() uses $1 (at) as a scratch
register, but doesn't arrange for it to be preserved, so it will always
be clobbered by the TLB refill exception. Although this register
normally has a very short lifetime that doesn't cross memory accesses,
TLB refills due to instruction fetches (either on a page boundary or
after preemption) could clobber live data, and its easy to reproduce
the clobber with a little bit of assembler code.

Note that the use of a hardware page table walker will partly mask the
problem, as the TLB refill handler will not always be invoked.

This is fixed by avoiding the use of the extra scratch register. The
pte_high parts (going into the lower half of the EntryLo registers) are
loaded and manipulated separately so as to keep the PTE pointer around
for the other halves (instead of storing in the scratch register), and
the pte_low parts (going into the high half of the EntryLo registers)
are masked with 0x00ffffff using an ext instruction (instead of loading
0x00ffffff into the scratch register and AND'ing).

[paul.burton@imgtec.com:
  - Rebase atop other TLB work.
  - Use ext instead of an sll, srl sequence.
  - Use cpu_has_xpa instead of #ifdefs.
  - Modify commit subject to include "mm".]

Fixes: c5b367835cfc ("MIPS: Add support for XPA.")
Signed-off-by: James Hogan <james.hogan@imgtec.com>
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/13120/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

diff --git a/arch/mips/mm/tlbex.c b/arch/mips/mm/tlbex.c
index db4adf9cc65c44bd206326afd811e980d8d82d06..81d42c33f8820592cf8d8aea427cb74536833ab4 100644 (file)
--- a/arch/mips/mm/tlbex.c
+++ b/arch/mips/mm/tlbex.c
@@ -1014,26 +1014,22 @@ static void build_update_entries(u32 **p, unsigned int tmp, unsigned int ptep)
        if (config_enabled(CONFIG_XPA)) {
                int pte_off_even = sizeof(pte_t) / 2;
                int pte_off_odd = pte_off_even + sizeof(pte_t);
-               const int scratch = 1; /* Our extra working register */
-
-               uasm_i_addu(p, scratch, 0, ptep);
 
                uasm_i_lw(p, tmp, pte_off_even, ptep); /* even pte */
                UASM_i_ROTR(p, tmp, tmp, ilog2(_PAGE_GLOBAL));
                UASM_i_MTC0(p, tmp, C0_ENTRYLO0);
 
-               uasm_i_lw(p, ptep, pte_off_odd, ptep); /* odd pte */
-               UASM_i_ROTR(p, ptep, ptep, ilog2(_PAGE_GLOBAL));
-               UASM_i_MTC0(p, ptep, C0_ENTRYLO1);
-
-               uasm_i_lw(p, tmp, 0, scratch);
-               uasm_i_lw(p, ptep, sizeof(pte_t), scratch);
-               uasm_i_lui(p, scratch, 0xff);
-               uasm_i_ori(p, scratch, scratch, 0xffff);
-               uasm_i_and(p, tmp, scratch, tmp);
-               uasm_i_and(p, ptep, scratch, ptep);
+               uasm_i_lw(p, tmp, 0, ptep);
+               uasm_i_ext(p, tmp, tmp, 0, 24);
                uasm_i_mthc0(p, tmp, C0_ENTRYLO0);
-               uasm_i_mthc0(p, ptep, C0_ENTRYLO1);
+
+               uasm_i_lw(p, tmp, pte_off_odd, ptep); /* odd pte */
+               UASM_i_ROTR(p, tmp, tmp, ilog2(_PAGE_GLOBAL));
+               UASM_i_MTC0(p, tmp, C0_ENTRYLO1);
+
+               uasm_i_lw(p, tmp, sizeof(pte_t), ptep);
+               uasm_i_ext(p, tmp, tmp, 0, 24);
+               uasm_i_mthc0(p, tmp, C0_ENTRYLO1);
                return;
        }