Re: [dpdk-dev] [PATCH v2] common/mlx5: fix Rx/Tx queue doorbell record size

[Raslan Darawsheh <rasland@mellanox.com>]

Hi,
> -----Original Message-----
> From: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> Sent: Tuesday, July 21, 2020 2:11 PM
> To: dev@dpdk.org
> Cc: Matan Azrad <matan@mellanox.com>; Raslan Darawsheh
> <rasland@mellanox.com>; Thomas Monjalon <thomas@monjalon.net>;
> Dekel Peled <dekelp@mellanox.com>; stable@dpdk.org
> Subject: [PATCH v2] common/mlx5: fix Rx/Tx queue doorbell record size
> 
> When Rx/Tx queue was being created with DevX the allocated
> doorbell record size was only uint64_t. That was definitely
> less than size of CPU cacheline and it might have happened the
> doorbell records attached to different queues handled by
> different cores were allocated within same cacheline. It might
> have caused the contention on doorbell record writing.
> 
> This patch extends the allocated memory size for doorbell
> record to cacheline size.
> 
> Fixes: 21cae8580fd0 ("net/mlx5: allocate door-bells via DevX")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> Acked-by: Matan Azrad <matan@mellanox.com>
> 
> ---
> v2: - rebase
>     - clarify commit message, register is replaced with record
> 
>  drivers/common/mlx5/mlx5_common.c |  2 +-
>  drivers/common/mlx5/mlx5_common.h | 10 ++++++----
>  2 files changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/common/mlx5/mlx5_common.c
> b/drivers/common/mlx5/mlx5_common.c
> index 79cd5ba..193e73b 100644
> --- a/drivers/common/mlx5/mlx5_common.c
> +++ b/drivers/common/mlx5/mlx5_common.c
> @@ -187,7 +187,7 @@ static inline void mlx5_cpu_id(unsigned int level,
>  	page->dbr_bitmap[i] |= (UINT64_C(1) << j);
>  	page->dbr_count++;
>  	*dbr_page = page;
> -	return (((i * 64) + j) * sizeof(uint64_t));
> +	return (i * CHAR_BIT * sizeof(uint64_t) + j) * MLX5_DBR_SIZE;
>  }
> 
>  /**
> diff --git a/drivers/common/mlx5/mlx5_common.h
> b/drivers/common/mlx5/mlx5_common.h
> index 5b9b7bd..ef7b61b 100644
> --- a/drivers/common/mlx5/mlx5_common.h
> +++ b/drivers/common/mlx5/mlx5_common.h
> @@ -214,10 +214,12 @@ enum mlx5_class {
>  	MLX5_CLASS_VDPA = RTE_BIT64(1),
>  };
> 
> -#define MLX5_DBR_PAGE_SIZE 4096 /* Must be >= 512. */
> -#define MLX5_DBR_SIZE 8
> -#define MLX5_DBR_PER_PAGE (MLX5_DBR_PAGE_SIZE / MLX5_DBR_SIZE)
> -#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / 64)
> +#define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE
> +#define MLX5_DBR_PER_PAGE 64
> +/* Must be >= CHAR_BIT * sizeof(uint64_t) */
> +#define MLX5_DBR_PAGE_SIZE (MLX5_DBR_PER_PAGE * MLX5_DBR_SIZE)
> +/* Page size must be >= 512. */
> +#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / (CHAR_BIT *
> sizeof(uint64_t)))
> 
>  struct mlx5_devx_dbr_page {
>  	/* Door-bell records, must be first member in structure. */
> --
> 1.8.3.1

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh