Commit | Line | Data |
---|---|---|
a560aa48 PZ |
1 | |
2 | LOCK STATISTICS | |
3 | ||
4 | - WHAT | |
5 | ||
6 | As the name suggests, it provides statistics on locks. | |
7 | ||
8 | - WHY | |
9 | ||
10 | Because things like lock contention can severely impact performance. | |
11 | ||
12 | - HOW | |
13 | ||
14 | Lockdep already has hooks in the lock functions and maps lock instances to | |
15 | lock classes. We build on that. The graph below shows the relation between | |
16 | the lock functions and the various hooks therein. | |
17 | ||
18 | __acquire | |
19 | | | |
20 | lock _____ | |
21 | | \ | |
22 | | __contended | |
23 | | | | |
24 | | <wait> | |
25 | | _______/ | |
26 | |/ | |
27 | | | |
28 | __acquired | |
29 | | | |
30 | . | |
31 | <hold> | |
32 | . | |
33 | | | |
34 | __release | |
35 | | | |
36 | unlock | |
37 | ||
38 | lock, unlock - the regular lock functions | |
39 | __* - the hooks | |
40 | <> - states | |
41 | ||
42 | With these hooks we provide the following statistics: | |
43 | ||
44 | con-bounces - number of lock contention that involved x-cpu data | |
45 | contentions - number of lock acquisitions that had to wait | |
46 | wait time min - shortest (non-0) time we ever had to wait for a lock | |
47 | max - longest time we ever had to wait for a lock | |
48 | total - total time we spend waiting on this lock | |
49 | acq-bounces - number of lock acquisitions that involved x-cpu data | |
50 | acquisitions - number of times we took the lock | |
51 | hold time min - shortest (non-0) time we ever held the lock | |
52 | max - longest time we ever held the lock | |
53 | total - total time this lock was held | |
54 | ||
55 | From these number various other statistics can be derived, such as: | |
56 | ||
57 | hold time average = hold time total / acquisitions | |
58 | ||
59 | These numbers are gathered per lock class, per read/write state (when | |
60 | applicable). | |
61 | ||
62 | It also tracks 4 contention points per class. A contention point is a call site | |
63 | that had to wait on lock acquisition. | |
64 | ||
f5754bfd FR |
65 | - CONFIGURATION |
66 | ||
67 | Lock statistics are enabled via CONFIG_LOCK_STATS. | |
68 | ||
a560aa48 PZ |
69 | - USAGE |
70 | ||
f5754bfd FR |
71 | Enable collection of statistics: |
72 | ||
73 | # echo 1 >/proc/sys/kernel/lock_stat | |
74 | ||
75 | Disable collection of statistics: | |
76 | ||
77 | # echo 0 >/proc/sys/kernel/lock_stat | |
78 | ||
a560aa48 PZ |
79 | Look at the current lock statistics: |
80 | ||
81 | ( line numbers not part of actual output, done for clarity in the explanation | |
82 | below ) | |
83 | ||
84 | # less /proc/lock_stat | |
85 | ||
c7e78cff | 86 | 01 lock_stat version 0.3 |
a560aa48 PZ |
87 | 02 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
88 | 03 class name con-bounces contentions waittime-min waittime-max waittime-total acq-bounces acquisitions holdtime-min holdtime-max holdtime-total | |
89 | 04 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
90 | 05 | |
c7e78cff PZ |
91 | 06 &mm->mmap_sem-W: 233 538 18446744073708 22924.27 607243.51 1342 45806 1.71 8595.89 1180582.34 |
92 | 07 &mm->mmap_sem-R: 205 587 18446744073708 28403.36 731975.00 1940 412426 0.58 187825.45 6307502.88 | |
93 | 08 --------------- | |
94 | 09 &mm->mmap_sem 487 [<ffffffff8053491f>] do_page_fault+0x466/0x928 | |
95 | 10 &mm->mmap_sem 179 [<ffffffff802a6200>] sys_mprotect+0xcd/0x21d | |
96 | 11 &mm->mmap_sem 279 [<ffffffff80210a57>] sys_mmap+0x75/0xce | |
97 | 12 &mm->mmap_sem 76 [<ffffffff802a490b>] sys_munmap+0x32/0x59 | |
98 | 13 --------------- | |
99 | 14 &mm->mmap_sem 270 [<ffffffff80210a57>] sys_mmap+0x75/0xce | |
100 | 15 &mm->mmap_sem 431 [<ffffffff8053491f>] do_page_fault+0x466/0x928 | |
101 | 16 &mm->mmap_sem 138 [<ffffffff802a490b>] sys_munmap+0x32/0x59 | |
102 | 17 &mm->mmap_sem 145 [<ffffffff802a6200>] sys_mprotect+0xcd/0x21d | |
103 | 18 | |
104 | 19 ............................................................................................................................................................................................... | |
105 | 20 | |
106 | 21 dcache_lock: 621 623 0.52 118.26 1053.02 6745 91930 0.29 316.29 118423.41 | |
107 | 22 ----------- | |
108 | 23 dcache_lock 179 [<ffffffff80378274>] _atomic_dec_and_lock+0x34/0x54 | |
109 | 24 dcache_lock 113 [<ffffffff802cc17b>] d_alloc+0x19a/0x1eb | |
110 | 25 dcache_lock 99 [<ffffffff802ca0dc>] d_rehash+0x1b/0x44 | |
111 | 26 dcache_lock 104 [<ffffffff802cbca0>] d_instantiate+0x36/0x8a | |
112 | 27 ----------- | |
113 | 28 dcache_lock 192 [<ffffffff80378274>] _atomic_dec_and_lock+0x34/0x54 | |
114 | 29 dcache_lock 98 [<ffffffff802ca0dc>] d_rehash+0x1b/0x44 | |
115 | 30 dcache_lock 72 [<ffffffff802cc17b>] d_alloc+0x19a/0x1eb | |
116 | 31 dcache_lock 112 [<ffffffff802cbca0>] d_instantiate+0x36/0x8a | |
a560aa48 PZ |
117 | |
118 | This excerpt shows the first two lock class statistics. Line 01 shows the | |
119 | output version - each time the format changes this will be updated. Line 02-04 | |
c7e78cff | 120 | show the header with column descriptions. Lines 05-18 and 20-31 show the actual |
a560aa48 | 121 | statistics. These statistics come in two parts; the actual stats separated by a |
c7e78cff | 122 | short separator (line 08, 13) from the contention points. |
a560aa48 | 123 | |
c7e78cff | 124 | The first lock (05-18) is a read/write lock, and shows two lines above the |
a560aa48 | 125 | short separator. The contention points don't match the column descriptors, |
c7e78cff PZ |
126 | they have two: contentions and [<IP>] symbol. The second set of contention |
127 | points are the points we're contending with. | |
a560aa48 | 128 | |
cdad7220 | 129 | The integer part of the time values is in us. |
a560aa48 PZ |
130 | |
131 | View the top contending locks: | |
132 | ||
133 | # grep : /proc/lock_stat | head | |
134 | &inode->i_data.tree_lock-W: 15 21657 0.18 1093295.30 11547131054.85 58 10415 0.16 87.51 6387.60 | |
135 | &inode->i_data.tree_lock-R: 0 0 0.00 0.00 0.00 23302 231198 0.25 8.45 98023.38 | |
136 | dcache_lock: 1037 1161 0.38 45.32 774.51 6611 243371 0.15 306.48 77387.24 | |
137 | &inode->i_mutex: 161 286 18446744073709 62882.54 1244614.55 3653 20598 18446744073709 62318.60 1693822.74 | |
138 | &zone->lru_lock: 94 94 0.53 7.33 92.10 4366 32690 0.29 59.81 16350.06 | |
139 | &inode->i_data.i_mmap_lock: 79 79 0.40 3.77 53.03 11779 87755 0.28 116.93 29898.44 | |
140 | &q->__queue_lock: 48 50 0.52 31.62 86.31 774 13131 0.17 113.08 12277.52 | |
141 | &rq->rq_lock_key: 43 47 0.74 68.50 170.63 3706 33929 0.22 107.99 17460.62 | |
142 | &rq->rq_lock_key#2: 39 46 0.75 6.68 49.03 2979 32292 0.17 125.17 17137.63 | |
143 | tasklist_lock-W: 15 15 1.45 10.87 32.70 1201 7390 0.58 62.55 13648.47 | |
144 | ||
145 | Clear the statistics: | |
146 | ||
147 | # echo 0 > /proc/lock_stat |