cpufreq / sched: SCHED_CPUFREQ_IOWAIT flag to indicate iowait condition

Testing indicates that it is possible to improve performace
significantly without increasing energy consumption too much by
teaching cpufreq governors to bump up the CPU performance level if
the in_iowait flag is set for the task in enqueue_task_fair().

For this purpose, define a new cpufreq_update_util() flag
SCHED_CPUFREQ_IOWAIT and modify enqueue_task_fair() to pass that
flag to cpufreq_update_util() in the in_iowait case.  That generally
requires cpufreq_update_util() to be called directly from there,
because update_load_avg() may not be invoked in that case.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Looks-good-to: Steve Muckle <smuckle@linaro.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>

diff --git a/include/linux/sched.h b/include/linux/sched.h
index b0fa726b7f31a625cb25667e55cb4d919a2271d4..98fe95fea30cb0656c4a90e60a71028e7059c536 100644 (file)
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -3471,6 +3471,7 @@ static inline unsigned long rlimit_max(unsigned int limit)
 
 #define SCHED_CPUFREQ_RT       (1U << 0)
 #define SCHED_CPUFREQ_DL       (1U << 1)
+#define SCHED_CPUFREQ_IOWAIT   (1U << 2)
 
 #define SCHED_CPUFREQ_RT_DL    (SCHED_CPUFREQ_RT | SCHED_CPUFREQ_DL)
 

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 5d558cc91f08eb3538265e0426536ba289b669b1..a5cd07b25aa147119133b9f3dce2691af00e3cc3 100644 (file)
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4500,6 +4500,14 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
        struct cfs_rq *cfs_rq;
        struct sched_entity *se = &p->se;
 
+       /*
+        * If in_iowait is set, the code below may not trigger any cpufreq
+        * utilization updates, so do it here explicitly with the IOWAIT flag
+        * passed.
+        */
+       if (p->in_iowait)
+               cpufreq_update_this_cpu(rq, SCHED_CPUFREQ_IOWAIT);
+
        for_each_sched_entity(se) {
                if (se->on_rq)
                        break;