mm: avoid false sharing of mm_counter
authorKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Fri, 5 Mar 2010 21:41:40 +0000 (13:41 -0800)
committerLinus Torvalds <torvalds@linux-foundation.org>
Sat, 6 Mar 2010 19:26:24 +0000 (11:26 -0800)
commit34e55232e59f7b19050267a05ff1226e5cd122a5
tree6b94e776e87d2a2fe1ceca7c5606901575323900
parentd559db086ff5be9bcc259e5aa50bf3d881eaf1d1
mm: avoid false sharing of mm_counter

Considering the nature of per mm stats, it's the shared object among
threads and can be a cache-miss point in the page fault path.

This patch adds per-thread cache for mm_counter.  RSS value will be
counted into a struct in task_struct and synchronized with mm's one at
events.

Now, in this patch, the event is the number of calls to handle_mm_fault.
Per-thread value is added to mm at each 64 calls.

 rough estimation with small benchmark on parallel thread (2threads) shows
 [before]
     4.5 cache-miss/faults
 [after]
     4.0 cache-miss/faults
 Anyway, the most contended object is mmap_sem if the number of threads grows.

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Lee Schermerhorn <lee.schermerhorn@hp.com>
Cc: David Rientjes <rientjes@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Documentation/filesystems/proc.txt
fs/exec.c
include/linux/mm.h
include/linux/mm_types.h
include/linux/sched.h
kernel/exit.c
mm/memory.c