Re: [dpdk-dev] [PATCH v2 3/5] net/mlx5: use buffer address for LKEY search

["Nélio Laranjeiro" <nelio.laranjeiro@6wind.com>]

On Fri, Jun 30, 2017 at 12:23:31PM -0700, Yongseok Koh wrote:
> When searching LKEY, if search key is mempool pointer, the 2nd cacheline
> has to be accessed and it even requires to check whether a buffer is
> indirect per every search. Instead, using address for search key can reduce
> cycles taken. And caching the last hit entry is beneficial as well.
> 
> Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
> ---
>  drivers/net/mlx5/mlx5_mr.c   | 17 ++++++++++++++---
>  drivers/net/mlx5/mlx5_rxtx.c | 39 +++++++++++++++++++++------------------
>  drivers/net/mlx5/mlx5_rxtx.h |  4 +++-
>  drivers/net/mlx5/mlx5_txq.c  |  3 +--
>  4 files changed, 39 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c
> index 0a3638460..287335179 100644
> --- a/drivers/net/mlx5/mlx5_mr.c
> +++ b/drivers/net/mlx5/mlx5_mr.c
> @@ -265,18 +266,28 @@ txq_mp2mr_iter(struct rte_mempool *mp, void *arg)
>  	struct txq_mp2mr_mbuf_check_data data = {
>  		.ret = 0,
>  	};
> +	uintptr_t start;
> +	uintptr_t end;
>  	unsigned int i;
>  
>  	/* Register mempool only if the first element looks like a mbuf. */
>  	if (rte_mempool_obj_iter(mp, txq_mp2mr_mbuf_check, &data) == 0 ||
>  			data.ret == -1)
>  		return;
> +	if (mlx5_check_mempool(mp, &start, &end) != 0) {
> +		ERROR("mempool %p: not virtually contiguous",
> +		      (void *)mp);
> +		return;
> +	}
>  	for (i = 0; (i != RTE_DIM(txq_ctrl->txq.mp2mr)); ++i) {
> -		if (unlikely(txq_ctrl->txq.mp2mr[i].mp == NULL)) {
> +		struct ibv_mr *mr = txq_ctrl->txq.mp2mr[i].mr;
> +
> +		if (unlikely(mr == NULL)) {
>  			/* Unknown MP, add a new MR for it. */
>  			break;
>  		}
> -		if (txq_ctrl->txq.mp2mr[i].mp == mp)
> +		if (start >= (uintptr_t)mr->addr &&
> +		    end <= (uintptr_t)mr->addr + mr->length)
>  			return;
>  	}
>  	txq_mp2mr_reg(&txq_ctrl->txq, mp, i);

 if (start >= (uintptr_t)mr->addr &&
     end <= (uintptr_t)mr->addr + mr->length)

Is this expected to have a memory region bigger than the memory pool
space?  I mean I was expecting to see strict equality in the addresses.

Regards,

-- 
Nélio Laranjeiro
6WIND