From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 267A1A0526 for ; Tue, 21 Jul 2020 13:11:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D7CF41BFE9; Tue, 21 Jul 2020 13:11:34 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id F37701BFE9 for ; Tue, 21 Jul 2020 13:11:33 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with SMTP; 21 Jul 2020 14:11:31 +0300 Received: from pegasus12.mtr.labs.mlnx (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06LBBVkn018127; Tue, 21 Jul 2020 14:11:31 +0300 Received: from pegasus12.mtr.labs.mlnx (localhost [127.0.0.1]) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7) with ESMTP id 06LBBVJu031564; Tue, 21 Jul 2020 11:11:31 GMT Received: (from viacheslavo@localhost) by pegasus12.mtr.labs.mlnx (8.14.7/8.14.7/Submit) id 06LBBUqc031563; Tue, 21 Jul 2020 11:11:30 GMT X-Authentication-Warning: pegasus12.mtr.labs.mlnx: viacheslavo set sender to viacheslavo@mellanox.com using -f From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: matan@mellanox.com, rasland@mellanox.com, thomas@monjalon.net, dekelp@mellanox.com, stable@dpdk.org Date: Tue, 21 Jul 2020 11:11:29 +0000 Message-Id: <1595329889-31516-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1595165997-12225-1-git-send-email-viacheslavo@mellanox.com> References: <1595165997-12225-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-stable] [PATCH v2] common/mlx5: fix Rx/Tx queue doorbell record size X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" When Rx/Tx queue was being created with DevX the allocated doorbell record size was only uint64_t. That was definitely less than size of CPU cacheline and it might have happened the doorbell records attached to different queues handled by different cores were allocated within same cacheline. It might have caused the contention on doorbell record writing. This patch extends the allocated memory size for doorbell record to cacheline size. Fixes: 21cae8580fd0 ("net/mlx5: allocate door-bells via DevX") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- v2: - rebase - clarify commit message, register is replaced with record drivers/common/mlx5/mlx5_common.c | 2 +- drivers/common/mlx5/mlx5_common.h | 10 ++++++---- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 79cd5ba..193e73b 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -187,7 +187,7 @@ static inline void mlx5_cpu_id(unsigned int level, page->dbr_bitmap[i] |= (UINT64_C(1) << j); page->dbr_count++; *dbr_page = page; - return (((i * 64) + j) * sizeof(uint64_t)); + return (i * CHAR_BIT * sizeof(uint64_t) + j) * MLX5_DBR_SIZE; } /** diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index 5b9b7bd..ef7b61b 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -214,10 +214,12 @@ enum mlx5_class { MLX5_CLASS_VDPA = RTE_BIT64(1), }; -#define MLX5_DBR_PAGE_SIZE 4096 /* Must be >= 512. */ -#define MLX5_DBR_SIZE 8 -#define MLX5_DBR_PER_PAGE (MLX5_DBR_PAGE_SIZE / MLX5_DBR_SIZE) -#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / 64) +#define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE +#define MLX5_DBR_PER_PAGE 64 +/* Must be >= CHAR_BIT * sizeof(uint64_t) */ +#define MLX5_DBR_PAGE_SIZE (MLX5_DBR_PER_PAGE * MLX5_DBR_SIZE) +/* Page size must be >= 512. */ +#define MLX5_DBR_BITMAP_SIZE (MLX5_DBR_PER_PAGE / (CHAR_BIT * sizeof(uint64_t))) struct mlx5_devx_dbr_page { /* Door-bell records, must be first member in structure. */ -- 1.8.3.1