udp: add busylocks in RX path
authorEric Dumazet <edumazet@google.com>
Thu, 8 Dec 2016 19:41:54 +0000 (11:41 -0800)
committerDavid S. Miller <davem@davemloft.net>
Sat, 10 Dec 2016 03:12:21 +0000 (22:12 -0500)
commit4b272750dbe6f92a8d39a0ee1c7bd50d6cc1a2c8
tree133fce068fcb1964ef08ae66ef51f5301d0862ad
parentd96dac145492ff54e32967dc486e2a90e51664ea
udp: add busylocks in RX path

Idea of busylocks is to let producers grab an extra spinlock
to relieve pressure on the receive_queue spinlock shared by consumer.

This behavior is requested only once socket receive queue is above
half occupancy.

Under flood, this means that only one producer can be in line
trying to acquire the receive_queue spinlock.

These busylock can be allocated on a per cpu manner, instead of a
per socket one (that would consume a cache line per socket)

This patch considerably improves UDP behavior under stress,
depending on number of NIC RX queues and/or RPS spread.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
net/ipv4/udp.c