MIPS: tlbex: Fix potential HTW race on TLBL/M/S handlers

There is a potential race when probing the TLB in TLBL/M/S exception
handlers for a matching entry. Between the time we hit a TLBL/S/M
exception and the time we get to execute the TLBP instruction, the
HTW may have replaced the TLB entry we are interested in hence the TLB
probe may fail. However, in the existing handlers, we never checked the
status of the TLBP (ie check the result in the C0/Index register). We
fix this by adding such a check when the core implements the HTW. If
we couldn't find a matching entry, we return back and try again.

Signed-off-by: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: Markos Chandras <markos.chandras@imgtec.com>
Reviewed-by: James Hogan <james.hogan@imgtec.com>
Cc: <stable@vger.kernel.org> # v3.17+
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/8599/
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>

diff --git a/arch/mips/mm/tlbex.c b/arch/mips/mm/tlbex.c
index b5f228e7eae6144565e34c74bf6f86e5d973a760..e3328a96e80909b758a8d619b6a0f8398399d2da 100644 (file)
--- a/arch/mips/mm/tlbex.c
+++ b/arch/mips/mm/tlbex.c
@@ -1872,8 +1872,16 @@ build_r4000_tlbchange_handler_head(u32 **p, struct uasm_label **l,
        uasm_l_smp_pgtable_change(l, *p);
 #endif
        iPTE_LW(p, wr.r1, wr.r2); /* get even pte */
-       if (!m4kc_tlbp_war())
+       if (!m4kc_tlbp_war()) {
                build_tlb_probe_entry(p);
+               if (cpu_has_htw) {
+                       /* race condition happens, leaving */
+                       uasm_i_ehb(p);
+                       uasm_i_mfc0(p, wr.r3, C0_INDEX);
+                       uasm_il_bltz(p, r, wr.r3, label_leave);
+                       uasm_i_nop(p);
+               }
+       }
        return wr;
 }