bpf: make programs see skb->data == L2 for ingress and egress
authorAlexei Starovoitov <ast@plumgrid.com>
Thu, 4 Jun 2015 17:11:53 +0000 (10:11 -0700)
committerDavid S. Miller <davem@davemloft.net>
Sun, 7 Jun 2015 09:01:33 +0000 (02:01 -0700)
commit3431205e03977aaf32bce6d4b16fb8244b510056
tree0d9405acb8ac34a6bc5ad93a9b9ae674c4d3879e
parent98da81a426a76c351f3c2854d5bc31f17bba3194
bpf: make programs see skb->data == L2 for ingress and egress

eBPF programs attached to ingress and egress qdiscs see inconsistent skb->data.
For ingress L2 header is already pulled, whereas for egress it's present.
This is known to program writers which are currently forced to use
BPF_LL_OFF workaround.
Since programs don't change skb internal pointers it is safe to do
pull/push right around invocation of the program and earlier taps and
later pt->func() will not be affected.
Multiple taps via packet_rcv(), tpacket_rcv() are doing the same trick
around run_filter/BPF_PROG_RUN even if skb_shared.

This fix finally allows programs to use optimized LD_ABS/IND instructions
without BPF_LL_OFF for higher performance.
tc ingress + cls_bpf + samples/bpf/tcbpf1_kern.o
       w/o JIT   w/JIT
before  20.5     23.6 Mpps
after   21.8     26.6 Mpps

Old programs with BPF_LL_OFF will still work as-is.

We can now undo most of the earlier workaround commit:
a166151cbe33 ("bpf: fix bpf helpers to use skb->mac_header relative offsets")

Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
net/core/filter.c
net/sched/act_bpf.c
net/sched/cls_bpf.c
samples/bpf/tcbpf1_kern.c