sched: fix SMP migration latencies

fix SMP migration latencies: the vruntimes of different CPUs are
at incompatible offsets so they have to be fixed up when migrating
a task across CPUs.

Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

diff --git a/kernel/sched.c b/kernel/sched.c
index 0f0cf374c7756855964aa9cb6a4870901533bd42..4ad789d268fe4959d7a24789f06a0c711af516a1 100644 (file)
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -992,6 +992,9 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
        if (p->se.block_start)
                p->se.block_start -= clock_offset;
 #endif
+       if (likely(new_rq->cfs.min_vruntime))
+               p->se.vruntime -= old_rq->cfs.min_vruntime -
+                                               new_rq->cfs.min_vruntime;
 
        __set_task_cpu(p, new_cpu);
 }