x86/ldt: Simplify the LDT switching logic

Originally, Linux reloaded the LDT whenever the prev mm or the next
mm had an LDT. It was changed in 2002 in:

  0bbed3beb4f2 ("[PATCH] Thread-Local Storage (TLS) support")

(commit from the historical tree), like this:

-		/* load_LDT, if either the previous or next thread
-		 * has a non-default LDT.
+		/*
+		 * load the LDT, if the LDT is different:
		 */
-		if (next->context.size+prev->context.size)
+		if (unlikely(prev->context.ldt != next->context.ldt))
			load_LDT(&next->context);

The current code is unlikely to avoid any LDT reloads, since different
mms won't share an LDT.

When we redo lazy mode to stop flush IPIs without switching to
init_mm, though, the current logic would become incorrect: it will
be possible to have real_prev == next but nonetheless have a stale
LDT descriptor.

Simplify the code to update LDTR if either the previous or the next
mm has an LDT, i.e. effectively restore the historical logic..
While we're at it, clean up the code by moving all the ifdeffery to
a header where it belongs.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Borislav Petkov <bp@suse.de>
Acked-by: Rik van Riel <riel@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/2a859ac01245f9594c58f9d0a8b2ed8a7cd2507e.1498022414.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index 1458f530948b663e2d71ef71bc75fb18f99b3307..ecfcb6643c9b4502fa8d89b99dd8b351a9665b73 100644 (file)
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -93,6 +93,32 @@ static inline void load_mm_ldt(struct mm_struct *mm)
 #else
        clear_LDT();
 #endif
+}
+
+static inline void switch_ldt(struct mm_struct *prev, struct mm_struct *next)
+{
+#ifdef CONFIG_MODIFY_LDT_SYSCALL
+       /*
+        * Load the LDT if either the old or new mm had an LDT.
+        *
+        * An mm will never go from having an LDT to not having an LDT.  Two
+        * mms never share an LDT, so we don't gain anything by checking to
+        * see whether the LDT changed.  There's also no guarantee that
+        * prev->context.ldt actually matches LDTR, but, if LDTR is non-NULL,
+        * then prev->context.ldt will also be non-NULL.
+        *
+        * If we really cared, we could optimize the case where prev == next
+        * and we're exiting lazy mode.  Most of the time, if this happens,
+        * we don't actually need to reload LDTR, but modify_ldt() is mostly
+        * used by legacy code and emulators where we don't need this level of
+        * performance.
+        *
+        * This uses | instead of || because it generates better code.
+        */
+       if (unlikely((unsigned long)prev->context.ldt |
+                    (unsigned long)next->context.ldt))
+               load_mm_ldt(next);
+#endif
 
        DEBUG_LOCKS_WARN_ON(preemptible());
 }

diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 2a5e851f203569f7344d890c9766adc4635736b6..b2485d69f7c2c593d921bc2267c2b898ae0a4da6 100644 (file)
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -148,25 +148,9 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
                     real_prev != &init_mm);
        cpumask_clear_cpu(cpu, mm_cpumask(real_prev));
 
-       /* Load per-mm CR4 state */
+       /* Load per-mm CR4 and LDTR state */
        load_mm_cr4(next);
-
-#ifdef CONFIG_MODIFY_LDT_SYSCALL
-       /*
-        * Load the LDT, if the LDT is different.
-        *
-        * It's possible that prev->context.ldt doesn't match
-        * the LDT register.  This can happen if leave_mm(prev)
-        * was called and then modify_ldt changed
-        * prev->context.ldt but suppressed an IPI to this CPU.
-        * In this case, prev->context.ldt != NULL, because we
-        * never set context.ldt to NULL while the mm still
-        * exists.  That means that next->context.ldt !=
-        * prev->context.ldt, because mms never share an LDT.
-        */
-       if (unlikely(real_prev->context.ldt != next->context.ldt))
-               load_mm_ldt(next);
-#endif
+       switch_ldt(real_prev, next);
 }
 
 /*