IB/rxe: Cache dst in QP instead of getting it for each send
authoryonatanc <yonatanc@mellanox.com>
Thu, 20 Apr 2017 17:55:56 +0000 (20:55 +0300)
committerDoug Ledford <dledford@redhat.com>
Fri, 21 Apr 2017 14:45:02 +0000 (10:45 -0400)
commit4ed6ad1eb30e20baf819ae4090b7978519beb3a9
treeb78c635bd1ad59fbf25d95f034a05d8f2e0911ed
parentcee2688e3cd60e0d1ab2f049e31cf11fa3f62540
IB/rxe: Cache dst in QP instead of getting it for each send

In RC QP there is no need to resolve the outgoing interface
for each packet, as this does not change during QP life cycle.

Instead cache the interface on the socket and use that one.
This improves performance by 12% by sparing redundant
calls to rxe_find_route.

ib_send_bw -d rxe0  -x 1 -n 9000 -e  -s $((1024 * 1024 )) -l 100

----------------------------------------------------------------------------------------
|        | bytes   | iterations | BW peak[MB/sec] | BW average[MB/sec] | MsgRate[Mpps] |
----------------------------------------------------------------------------------------
| before | 1048576 | 9000       | inf             | 551.21             | 0.000551      |
| after  | 1048576 | 9000       | inf             | 615.54             | 0.000616      |
----------------------------------------------------------------------------------------

Fixes: 8700e3e7c485 ("Soft RoCE driver")
Signed-off-by: Yonatan Cohen <yonatanc@mellanox.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Signed-off-by: Doug Ledford <dledford@redhat.com>
drivers/infiniband/sw/rxe/rxe_net.c
drivers/infiniband/sw/rxe/rxe_qp.c