From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C66EEA0542; Thu, 13 Feb 2020 13:44:47 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 59D0E1BFE5; Thu, 13 Feb 2020 13:44:31 +0100 (CET) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 24F701BFB5; Thu, 13 Feb 2020 13:44:30 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B37BD1063; Thu, 13 Feb 2020 04:44:29 -0800 (PST) Received: from net-arm-thunderx2-04.shanghai.arm.com (net-arm-thunderx2-04.shanghai.arm.com [10.169.40.184]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 33DCC3F6CF; Thu, 13 Feb 2020 04:44:22 -0800 (PST) From: Gavin Hu To: dev@dpdk.org Cc: nd@arm.com, Phil Yang , david.marchand@redhat.com, thomas@monjalon.net, rasland@mellanox.com, drc@linux.vnet.ibm.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, matan@mellanox.com, shahafs@mellanox.com, viacheslavo@mellanox.com, jerinj@marvell.com, Honnappa.Nagarahalli@arm.com, ruifeng.wang@arm.com, joyce.kong@arm.com, steve.capper@arm.com, stable@dpdk.org Date: Thu, 13 Feb 2020 20:38:54 +0800 Message-Id: <20200213123854.203566-7-gavin.hu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200213123854.203566-1-gavin.hu@arm.com> References: <20200213123854.203566-1-gavin.hu@arm.com> Subject: [dpdk-dev] [PATCH RFC v1 6/6] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Phil Yang PMD Rx queue descriptor contains two mlx5_mprq_buf fields, which are the multi-packet RQ buffer header pointers. It uses the common rte_atomic_XXX functions to make sure the refcnt access is atomic. The common rte_atomic_XXX functions are full barriers on aarch64. Optimized it with one-way barrier to improve performance. Fixes: 7d6bf6b866b8 ("net/mlx5: add Multi-Packet Rx support") Cc: stable@dpdk.org Suggested-by: Gavin Hu Signed-off-by: Phil Yang Reviewed-by: Gavin Hu --- drivers/net/mlx5/mlx5_rxq.c | 2 +- drivers/net/mlx5/mlx5_rxtx.c | 16 +++++++++------- drivers/net/mlx5/mlx5_rxtx.h | 2 +- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 2d1b153a3..765bb1af5 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1535,7 +1535,7 @@ mlx5_mprq_buf_init(struct rte_mempool *mp, void *opaque_arg, memset(_m, 0, sizeof(*buf)); buf->mp = mp; - rte_atomic16_set(&buf->refcnt, 1); + __atomic_store_n(&buf->refcnt, 1, __ATOMIC_RELAXED); for (j = 0; j != strd_n; ++j) { shinfo = &buf->shinfos[j]; shinfo->free_cb = mlx5_mprq_buf_free_cb; diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 5eea932d4..0e7519c56 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1592,10 +1592,11 @@ mlx5_mprq_buf_free_cb(void *addr __rte_unused, void *opaque) { struct mlx5_mprq_buf *buf = opaque; - if (rte_atomic16_read(&buf->refcnt) == 1) { + if (__atomic_load_n(&buf->refcnt, __ATOMIC_RELAXED) == 1) { rte_mempool_put(buf->mp, buf); - } else if (rte_atomic16_add_return(&buf->refcnt, -1) == 0) { - rte_atomic16_set(&buf->refcnt, 1); + } else if (unlikely(__atomic_sub_fetch(&buf->refcnt, 1, + __ATOMIC_RELAXED) == 0)) { + __atomic_store_n(&buf->refcnt, 1, __ATOMIC_RELAXED); rte_mempool_put(buf->mp, buf); } } @@ -1676,7 +1677,8 @@ mlx5_rx_burst_mprq(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) if (consumed_strd == strd_n) { /* Replace WQE only if the buffer is still in use. */ - if (rte_atomic16_read(&buf->refcnt) > 1) { + if (__atomic_load_n(&buf->refcnt, + __ATOMIC_RELAXED) > 1) { mprq_buf_replace(rxq, rq_ci & wq_mask, strd_n); /* Release the old buffer. */ mlx5_mprq_buf_free(buf); @@ -1766,9 +1768,9 @@ mlx5_rx_burst_mprq(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n) void *buf_addr; /* Increment the refcnt of the whole chunk. */ - rte_atomic16_add_return(&buf->refcnt, 1); - MLX5_ASSERT((uint16_t)rte_atomic16_read(&buf->refcnt) <= - strd_n + 1); + __atomic_add_fetch(&buf->refcnt, 1, __ATOMIC_ACQUIRE); + MLX5_ASSERT(__atomic_load_n(&buf->refcnt, + __ATOMIC_RELAXED) <= strd_n + 1); buf_addr = RTE_PTR_SUB(addr, headroom_sz); /* * MLX5 device doesn't use iova but it is necessary in a diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index d32c4e430..1f453fe09 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -78,7 +78,7 @@ struct rxq_zip { /* Multi-Packet RQ buffer header. */ struct mlx5_mprq_buf { struct rte_mempool *mp; - rte_atomic16_t refcnt; /* Atomically accessed refcnt. */ + uint16_t refcnt; /* Atomically accessed refcnt. */ uint8_t pad[RTE_PKTMBUF_HEADROOM]; /* Headroom for the first packet. */ struct rte_mbuf_ext_shared_info shinfos[]; /* -- 2.17.1