From 3f9d35b9514da6757ca98831372518f9eeb71b33 Mon Sep 17 00:00:00 2001 From: Eric Dumazet Date: Thu, 11 Nov 2010 14:05:08 -0800 Subject: [PATCH] atomic: add atomic_inc_not_zero_hint() Followup of perf tools session in Netfilter WorkShop 2010 In the network stack we make high usage of atomic_inc_not_zero() in contexts we know the probable value of atomic before increment (2 for udp sockets for example) Using a special version of atomic_inc_not_zero() giving this hint can help processor to use less bus transactions. On x86 (MESI protocol) for example, this avoids entering Shared state, because "lock cmpxchg" issues an RFO (Read For Ownership) akpm: Adds a new include/linux/atomic.h. This means that new code should henceforth include linux/atomic.h and not asm/atomic.h. The presence of include/linux/atomic.h will in fact cause checkpatch.pl to warn about use of asm/atomic.h. The new include/linux/atomic.h becomes the place where arch-neutral atomic_t code should be placed. [akpm@linux-foundation.org: coding-style fixes] Signed-off-by: Eric Dumazet Cc: Christoph Lameter Cc: Ingo Molnar Cc: Andi Kleen Cc: Arnaldo Carvalho de Melo Cc: David Miller Cc: "Paul E. McKenney" Cc: Nick Piggin Reviewed-by: "Paul E. McKenney" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/atomic.h | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 include/linux/atomic.h diff --git a/include/linux/atomic.h b/include/linux/atomic.h new file mode 100644 index 000000000000..96c038e43d66 --- /dev/null +++ b/include/linux/atomic.h @@ -0,0 +1,37 @@ +#ifndef _LINUX_ATOMIC_H +#define _LINUX_ATOMIC_H +#include + +/** + * atomic_inc_not_zero_hint - increment if not null + * @v: pointer of type atomic_t + * @hint: probable value of the atomic before the increment + * + * This version of atomic_inc_not_zero() gives a hint of probable + * value of the atomic. This helps processor to not read the memory + * before doing the atomic read/modify/write cycle, lowering + * number of bus transactions on some arches. + * + * Returns: 0 if increment was not done, 1 otherwise. + */ +#ifndef atomic_inc_not_zero_hint +static inline int atomic_inc_not_zero_hint(atomic_t *v, int hint) +{ + int val, c = hint; + + /* sanity test, should be removed by compiler if hint is a constant */ + if (!hint) + return atomic_inc_not_zero(v); + + do { + val = atomic_cmpxchg(v, c, c + 1); + if (val == c) + return 1; + c = val; + } while (c); + + return 0; +} +#endif + +#endif /* _LINUX_ATOMIC_H */ -- 2.20.1