IB/mthca: Fix max_sge value returned by query_device

The mthca driver returns the maximum number of scatter/gather entries
returned by the firmware as the max_sge value when device properties
are queried.  However, the firmware also reports a limit on the
maximum descriptor size allowed, and because mthca takes into account
the worst case send request overhead when checking whether to allow a
QP to be created, the largest number of scatter/gather entries that
can be used with mthca may be limited by the maximum descriptor size
rather than just by the actual s/g entry limit.

This means that applications cannot actually create QPs with
max_send_sge equal to the limit returned by ib_query_device().  Fix
this by checking if the maximum descriptor size imposes a lower limit
and if so returning that lower limit.

Signed-off-by: Roland Dreier <rolandd@cisco.com>

diff --git a/drivers/infiniband/hw/mthca/mthca_main.c b/drivers/infiniband/hw/mthca/mthca_main.c
index 9ebadd6e0cfba4046e4dc7309aeaf4033cc0e18f..200cf13fc9bb241f515f26a6443a363345268838 100644 (file)
--- a/drivers/infiniband/hw/mthca/mthca_main.c
+++ b/drivers/infiniband/hw/mthca/mthca_main.c
@@ -45,6 +45,7 @@
 #include "mthca_cmd.h"
 #include "mthca_profile.h"
 #include "mthca_memfree.h"
+#include "mthca_wqe.h"
 
 MODULE_AUTHOR("Roland Dreier");
 MODULE_DESCRIPTION("Mellanox InfiniBand HCA low-level driver");
@@ -200,7 +201,18 @@ static int mthca_dev_lim(struct mthca_dev *mdev, struct mthca_dev_lim *dev_lim)
        mdev->limits.gid_table_len      = dev_lim->max_gids;
        mdev->limits.pkey_table_len     = dev_lim->max_pkeys;
        mdev->limits.local_ca_ack_delay = dev_lim->local_ca_ack_delay;
-       mdev->limits.max_sg             = dev_lim->max_sg;
+       /*
+        * Need to allow for worst case send WQE overhead and check
+        * whether max_desc_sz imposes a lower limit than max_sg; UD
+        * send has the biggest overhead.
+        */
+       mdev->limits.max_sg             = min_t(int, dev_lim->max_sg,
+                                             (dev_lim->max_desc_sz -
+                                              sizeof (struct mthca_next_seg) -
+                                              (mthca_is_memfree(mdev) ?
+                                               sizeof (struct mthca_arbel_ud_seg) :
+                                               sizeof (struct mthca_tavor_ud_seg))) /
+                                               sizeof (struct mthca_data_seg));
        mdev->limits.max_wqes           = dev_lim->max_qp_sz;
        mdev->limits.max_qp_init_rdma   = dev_lim->max_requester_per_qp;
        mdev->limits.reserved_qps       = dev_lim->reserved_qps;