sched/x86: Do not clear PREEMPT_NEED_RESCHED on preempt count reset
authorMartin Schwidefsky <schwidefsky@de.ibm.com>
Mon, 7 Nov 2016 13:01:00 +0000 (14:01 +0100)
committerIngo Molnar <mingo@kernel.org>
Wed, 16 Nov 2016 09:29:04 +0000 (10:29 +0100)
commitf285144f81e814f39342dbf5321d6ba939890b1b
treed9194b57593b5767efab27870bfda51e730b4798
parent527b0a76f41d062381adbb55c8eb61e32cb0bfc9
sched/x86: Do not clear PREEMPT_NEED_RESCHED on preempt count reset

The per-cpu preempt count of x86 contains two values, the actual preempt
count and the inverted PREEMPT_NEED_RESCHED bit. If a corrupted preempt
count is detected the preempt_count_set() function is used to reset the
preempt count.

In case the inverted PREEMPT_NEED_RESCHED bit is zero at the time of the
reset, the preemption indication is lost. Use raw_cpu_cmpxchg_4() to reset
only the count part and leave the PREEMPT_NEED_RESCHED bit as it is.

This improves the kernel's behavior when it runs into preempt count leaks
and tries to fix them up.

Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1478523660-733-1-git-send-email-schwidefsky@de.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
arch/x86/include/asm/preempt.h