selinux: de-crapify avc cache stat code generation

You can turn off the avc cache stats, but distributions seem to not do
that (perhaps because several performance tuning how-to's talk about the
avc cache statistics).

Which is sad, because the code it generates is truly horrendous, with
the statistics update being sandwitched between get_cpu/put_cpu which in
turn causes preemption disables etc.  We're talking ten+ instructions
just to increment a per-cpu variable in some pretty hot code.

Fix the craziness by just using 'this_cpu_inc()' instead.  Suddenly we
only need a single 'inc' instruction to increment the statistics.  This
is quite noticeable in the incredibly hot avc_has_perm_noaudit()
function (which triggers all the statistics by virtue of doing an
avc_lookup() call).

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/security/selinux/avc.c b/security/selinux/avc.c
index 1d027e29ce8d5400c9d4a681d52f2f28025aacd2..5971e30e8239c90f107f768a47f5c1963d9efb3e 100644 (file)
--- a/security/selinux/avc.c
+++ b/security/selinux/avc.c
@@ -38,11 +38,7 @@
 #define AVC_CACHE_RECLAIM              16
 
 #ifdef CONFIG_SECURITY_SELINUX_AVC_STATS
-#define avc_cache_stats_incr(field)                            \
-do {                                                           \
-       per_cpu(avc_cache_stats, get_cpu()).field++;            \
-       put_cpu();                                              \
-} while (0)
+#define avc_cache_stats_incr(field)    this_cpu_inc(avc_cache_stats.field)
 #else
 #define avc_cache_stats_incr(field)    do {} while (0)
 #endif