From: Lai Jiangshan <laijs@cn.fujitsu.com>
Date: Tue, 6 Mar 2012 09:57:34 +0000 (+0800)
Subject: rcu: Improve srcu_readers_active_idx()'s cache locality
X-Git-Url: https://git.stricted.de/?a=commitdiff_plain;h=dc87917501e324701dbfb249def44054b5220187;p=GitHub%2FLineageOS%2FG12%2Fandroid_kernel_amlogic_linux-4.9.git

rcu: Improve srcu_readers_active_idx()'s cache locality

Expand the calls to srcu_readers_active_idx() from srcu_readers_active()
inline.  This change improves cache locality by interating over the CPUs
once rather than twice.

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---

diff --git a/kernel/srcu.c b/kernel/srcu.c
index e0139a274856..a43211c92863 100644
--- a/kernel/srcu.c
+++ b/kernel/srcu.c
@@ -193,7 +193,14 @@ static bool srcu_readers_active_idx_check(struct srcu_struct *sp, int idx)
  */
 static int srcu_readers_active(struct srcu_struct *sp)
 {
-	return srcu_readers_active_idx(sp, 0) + srcu_readers_active_idx(sp, 1);
+	int cpu;
+	unsigned long sum = 0;
+
+	for_each_possible_cpu(cpu) {
+		sum += ACCESS_ONCE(per_cpu_ptr(sp->per_cpu_ref, cpu)->c[0]);
+		sum += ACCESS_ONCE(per_cpu_ptr(sp->per_cpu_ref, cpu)->c[1]);
+	}
+	return sum;
 }
 
 /**