Re: [dpdk-dev] [PATCH 3/5] net/mlx5: use buffer address for LKEY search

["Nélio Laranjeiro" <nelio.laranjeiro@6wind.com>]

On Wed, Jun 28, 2017 at 04:04:01PM -0700, Yongseok Koh wrote:
> When searching LKEY, if search key is mempool pointer, the 2nd cacheline
> has to be accessed and it even requires to check whether a buffer is
> indirect per every search. Instead, using address for search key can reduce
> cycles taken. And caching the last hit entry is beneficial as well.
> 
> Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
> ---
>  drivers/net/mlx5/mlx5_mr.c   |  2 ++
>  drivers/net/mlx5/mlx5_rxtx.c | 37 ++++++++++++++++++++-----------------
>  drivers/net/mlx5/mlx5_rxtx.h |  3 +++
>  3 files changed, 25 insertions(+), 17 deletions(-)
> 
>[...]
> diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
> index f9b738b4e..a86f41abc 100644
> --- a/drivers/net/mlx5/mlx5_rxtx.h
> +++ b/drivers/net/mlx5/mlx5_rxtx.h
> @@ -267,10 +267,13 @@ struct txq {
>  	volatile uint32_t *cq_db; /* Completion queue doorbell. */
>  	volatile void *bf_reg; /* Blueflame register. */
>  	struct {
> +		uintptr_t start; /* Start address of MR */
> +		uintptr_t end;   /* End address of MR */

Extra space between the ';' and the comment.

>  		const struct rte_mempool *mp; /* Cached Memory Pool. */
>  		struct ibv_mr *mr; /* Memory Region (for mp). */
>  		uint32_t lkey; /* htonl(mr->lkey) */
>  	} mp2mr[MLX5_PMD_TX_MP_CACHE]; /* MP to MR translation table. */
> +	uint16_t mr_cache_idx; /* Index of last hit entry. */
>  	struct rte_mbuf *(*elts)[]; /* TX elements. */
>  	struct mlx5_txq_stats stats; /* TX queue counters. */
>  } __rte_cache_aligned;
> -- 
> 2.11.0

The pointer to the mempool should be removed from this structure, it was
useful in the dataplane to retrieve the lkey, but with this new
implementation it becomes useless.
This also helps to keep the memory footprint of this array.  The control
plane can spend some cycles to retrieve the start/end addresses of the
mempool to compare them.

Thanks,

-- 
Nélio Laranjeiro
6WIND