From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 495B0A0540; Mon, 31 Oct 2022 17:09:19 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 120904281C; Mon, 31 Oct 2022 17:08:55 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2085.outbound.protection.outlook.com [40.107.243.85]) by mails.dpdk.org (Postfix) with ESMTP id 58BB0427F3 for ; Mon, 31 Oct 2022 17:08:53 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Uw+ufOm1t9W1ZymjOzFPtr0h1uIproa82G0B8z4Spwp77soLtyCQp8Lhd3kTyHvwiAt3z8uWgRflyFzE5lQ65EZ/y7MRcx8k4UEjKT0CaoaahLHlmkf9W8pdP8abHNN5Zq26nKunVk/v+ZEz3r9e8iXSIHmJ6PIt1EIL+jwEdvjtFTbEQdK+aBx1yzOSDXjZQ7t5wuIailFtc088dt/iIgiH/usRZ+2x86LPFdvQLEKUqBExIzWGqsyvrTHdicJL3hjy2ElJy0y8SKjuaqAzStX0wc1PvSGFkUAd+4zwJBMCzB9OBK4fF7RE9NSqpxiy4nh1bJEOFzpFiyKiUylvYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LNO9/5ja7ILi+ymsQhgBxCMbfWC/te54y+GXqj0pecA=; b=hG7VIBO6LWDUJ9MgJbkWUluFKSZBpq4MshQdnDlcRwEQuAYzCicy/eNYcXhBmZnFaT33F1syfC9qzADP6GOyNFMjrJHV4d2Fbdl267iLMNKQhO4n1hS320sJxmBsv4uPgVSsZNCcNOYLh9fbouOyadZ0rHZ4OeUbfbZeYSn8CkjKwHHaZ++ALiJmHgQ5S1I83UG2OWBKSBsZyJ9JbkpAQhBVYJrSNgbQLJwZ6GoVM0CaH5rfTtY/WeGgHo9a8yFM17kaRFdol+tKYU1/9RJe+CYFWYCRyhhhcL7VNKhGf9hu/1wxnhXW8lowWUHLyIf/cTRLjd02l2xdaeHAqPN9yQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNO9/5ja7ILi+ymsQhgBxCMbfWC/te54y+GXqj0pecA=; b=jG7jdn+MeNZq6pzS3LZzUH0Fsyx2EiFQIxkh+O94mzZxmQnemsCWENdyhF9honPS1AVqRtc8IPEQyG0ncDFRIt36ZPNtze39YlFitQV8dcGxSD0K8G8wsY81/gz72c8EkUZm+Qa9YIJAL5Wr3udN6a7Jly3DuEDwb8EMZ8TXYuJmoaEHRKf1rD3jGwQrCAl+yHIYSDcZT/2MEgBtRV92RMfoyJ/5WuFoloZV36zLfIJdd8HmAWeFnyRUNIM/N9l+6Sw9WLOWF0ZVkaBCI6iX7FpDNzFDjgahnG2Wu/8kyVjscbds2fpqyLKFX2jbEPqawah+BsXld2M1krTJ57xHzA== Received: from BN9PR03CA0075.namprd03.prod.outlook.com (2603:10b6:408:fc::20) by SA1PR12MB7223.namprd12.prod.outlook.com (2603:10b6:806:2bc::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.28; Mon, 31 Oct 2022 16:08:51 +0000 Received: from BN8NAM11FT004.eop-nam11.prod.protection.outlook.com (2603:10b6:408:fc:cafe::f5) by BN9PR03CA0075.outlook.office365.com (2603:10b6:408:fc::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.19 via Frontend Transport; Mon, 31 Oct 2022 16:08:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by BN8NAM11FT004.mail.protection.outlook.com (10.13.176.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5769.14 via Frontend Transport; Mon, 31 Oct 2022 16:08:51 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Mon, 31 Oct 2022 09:08:39 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Mon, 31 Oct 2022 09:08:38 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29 via Frontend Transport; Mon, 31 Oct 2022 09:08:37 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko , Xiaoyu Min Subject: [PATCH 5/5] net/mlx5: assert for enough space in counter rings Date: Mon, 31 Oct 2022 18:08:24 +0200 Message-ID: <20221031160824.330200-6-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031160824.330200-1-michaelba@nvidia.com> References: <20221031160824.330200-1-michaelba@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT004:EE_|SA1PR12MB7223:EE_ X-MS-Office365-Filtering-Correlation-Id: 33bf08db-4a27-4b26-b9d0-08dabb5a3396 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5NbTVyDdYorBb5B1lnbfL51ap3wkxNDC9YFHboYqG+BuJQZosyeuGrMm9ouC5wBU8AQiIM6NgxQB+2JL7kM4R96fWaRWLPRaU+RcQxbr1DnaujL03o/1VnMX84YiUVEuKOsFnYGRAlZLBCj7cfGOtcCbhwqfczDZ20Vd8pMAoMOmJCTOXX2z8N4iKMbzFPTsKCy+oVkv2yjrEmzkiifl2mqrZRIyYKBITwHO0yl4VV6ymQ2ftmA5JYUcZEFdbSyNyzIF/RnY4W7LhOYttog2j8zvP7VQvnojjdZCS0l5D37ulUUkbTKoBxBl8OkLRS4l5xk6ruJl4RpEgHkRsNSHnKOsjvosB8lExTCX4P/qOj+P/IG4Yc6i6lIbIKmm/i7070uiEL2UCFMxALekm9bOqYpOsQwNvUsdItTuktwEBquCh4InHjTFp2kHuP7bAbof9HjCl9DzfmQ2U8IQMTf0MfutMR+Z3Ars8OHOmp928fNbRV2wgxj8vZtDAtIM1aPP7w9FR5EGfiIaGXefX4fqgJGsNyrzrn+9PUzDONCw3FHohMyR8BVBfghtUhdmmYHtm1uLHWTk9eywKo1DhABxdR0SlBZnFhCuI709Ki3O4L7F8KkIAjAuWNvRt/pHfEU/SC3ukj4TMIb8WCgKgx5QkEkYfEzkMvAWVVSsTmvYXLTckuTDIFw+rNSsKQ6lgr72RDEhYgUvxRNXUtOG+Luq8g9AfdZFhcFeRRj38XL6cMf5eYz+thEh3IZ7EhKFWO9y9vK0phZHb+qD363E8GmgZg== X-Forefront-Antispam-Report: CIP:216.228.118.233; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc7edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(136003)(346002)(376002)(396003)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(40460700003)(2616005)(82310400005)(5660300002)(478600001)(47076005)(426003)(8936002)(7696005)(36860700001)(40480700001)(82740400003)(1076003)(6916009)(186003)(41300700001)(356005)(36756003)(7636003)(54906003)(4326008)(30864003)(2906002)(83380400001)(70586007)(6286002)(336012)(316002)(86362001)(70206006)(26005)(6666004)(107886003)(55016003)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Oct 2022 16:08:51.1438 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 33bf08db-4a27-4b26-b9d0-08dabb5a3396 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.118.233]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT004.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7223 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is a by-design assumption in the code that the global counter rings can contain all the port counters. So, enqueuing to these global rings should always succeed. Add assertions to help for debugging this assumption. In addition, change mlx5_hws_cnt_pool_put() function to return void due to those assumptions. Signed-off-by: Michael Baum Acked-by: Matan Azrad Acked-by: Xiaoyu Min --- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- drivers/net/mlx5/mlx5_hws_cnt.c | 25 ++++---- drivers/net/mlx5/mlx5_hws_cnt.h | 106 +++++++++++++++++--------------- 3 files changed, 72 insertions(+), 61 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 2d275ad111..54a0afe45f 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -7874,7 +7874,7 @@ flow_hw_action_handle_destroy(struct rte_eth_dev *dev, uint32_t queue, * time to update the AGE. */ mlx5_hws_age_nb_cnt_decrease(priv, age_idx); - ret = mlx5_hws_cnt_shared_put(priv->hws_cpool, &act_idx); + mlx5_hws_cnt_shared_put(priv->hws_cpool, &act_idx); break; case MLX5_INDIRECT_ACTION_TYPE_CT: ret = flow_hw_conntrack_destroy(dev, act_idx, error); diff --git a/drivers/net/mlx5/mlx5_hws_cnt.c b/drivers/net/mlx5/mlx5_hws_cnt.c index b8ce69af57..24c01eace0 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.c +++ b/drivers/net/mlx5/mlx5_hws_cnt.c @@ -58,13 +58,14 @@ __hws_cnt_id_load(struct mlx5_hws_cnt_pool *cpool) static void __mlx5_hws_cnt_svc(struct mlx5_dev_ctx_shared *sh, - struct mlx5_hws_cnt_pool *cpool) + struct mlx5_hws_cnt_pool *cpool) { struct rte_ring *reset_list = cpool->wait_reset_list; struct rte_ring *reuse_list = cpool->reuse_list; uint32_t reset_cnt_num; struct rte_ring_zc_data zcdr = {0}; struct rte_ring_zc_data zcdu = {0}; + uint32_t ret __rte_unused; reset_cnt_num = rte_ring_count(reset_list); do { @@ -72,17 +73,19 @@ __mlx5_hws_cnt_svc(struct mlx5_dev_ctx_shared *sh, mlx5_aso_cnt_query(sh, cpool); zcdr.n1 = 0; zcdu.n1 = 0; - rte_ring_enqueue_zc_burst_elem_start(reuse_list, - sizeof(cnt_id_t), reset_cnt_num, &zcdu, - NULL); - rte_ring_dequeue_zc_burst_elem_start(reset_list, - sizeof(cnt_id_t), reset_cnt_num, &zcdr, - NULL); + ret = rte_ring_enqueue_zc_burst_elem_start(reuse_list, + sizeof(cnt_id_t), + reset_cnt_num, &zcdu, + NULL); + MLX5_ASSERT(ret == reset_cnt_num); + ret = rte_ring_dequeue_zc_burst_elem_start(reset_list, + sizeof(cnt_id_t), + reset_cnt_num, &zcdr, + NULL); + MLX5_ASSERT(ret == reset_cnt_num); __hws_cnt_r2rcpy(&zcdu, &zcdr, reset_cnt_num); - rte_ring_dequeue_zc_elem_finish(reset_list, - reset_cnt_num); - rte_ring_enqueue_zc_elem_finish(reuse_list, - reset_cnt_num); + rte_ring_dequeue_zc_elem_finish(reset_list, reset_cnt_num); + rte_ring_enqueue_zc_elem_finish(reuse_list, reset_cnt_num); reset_cnt_num = rte_ring_count(reset_list); } while (reset_cnt_num > 0); } diff --git a/drivers/net/mlx5/mlx5_hws_cnt.h b/drivers/net/mlx5/mlx5_hws_cnt.h index 338ee4d688..030dcead86 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.h +++ b/drivers/net/mlx5/mlx5_hws_cnt.h @@ -116,7 +116,7 @@ enum { HWS_AGE_CANDIDATE_INSIDE_RING, /* * AGE assigned to flows but it still in ring. It was aged-out but the - * timeout was changed, so it in ring but stiil candidate. + * timeout was changed, so it in ring but still candidate. */ HWS_AGE_AGED_OUT_REPORTED, /* @@ -182,7 +182,7 @@ mlx5_hws_cnt_id_valid(cnt_id_t cnt_id) * * @param cpool * The pointer to counter pool - * @param index + * @param iidx * The internal counter index. * * @return @@ -231,32 +231,32 @@ __hws_cnt_query_raw(struct mlx5_hws_cnt_pool *cpool, cnt_id_t cnt_id, } /** - * Copy elems from one zero-copy ring to zero-copy ring in place. + * Copy elements from one zero-copy ring to zero-copy ring in place. * * The input is a rte ring zero-copy data struct, which has two pointer. * in case of the wrapper happened, the ptr2 will be meaningful. * - * So this rountin needs to consider the situation that the address given by + * So this routine needs to consider the situation that the address given by * source and destination could be both wrapped. * First, calculate the first number of element needs to be copied until wrapped * address, which could be in source or destination. * Second, copy left number of element until second wrapped address. If in first * step the wrapped address is source, then this time it must be in destination. - * and vice-vers. - * Third, copy all left numbe of element. + * and vice-versa. + * Third, copy all left number of element. * * In worst case, we need copy three pieces of continuous memory. * * @param zcdd - * A pointer to zero-copy data of dest ring. + * A pointer to zero-copy data of destination ring. * @param zcds * A pointer to zero-copy data of source ring. * @param n - * Number of elems to copy. + * Number of elements to copy. */ static __rte_always_inline void __hws_cnt_r2rcpy(struct rte_ring_zc_data *zcdd, struct rte_ring_zc_data *zcds, - unsigned int n) + unsigned int n) { unsigned int n1, n2, n3; void *s1, *s2, *s3; @@ -291,22 +291,23 @@ static __rte_always_inline int mlx5_hws_cnt_pool_cache_flush(struct mlx5_hws_cnt_pool *cpool, uint32_t queue_id) { - unsigned int ret; + unsigned int ret __rte_unused; struct rte_ring_zc_data zcdr = {0}; struct rte_ring_zc_data zcdc = {0}; struct rte_ring *reset_list = NULL; struct rte_ring *qcache = cpool->cache->qcache[queue_id]; + uint32_t ring_size = rte_ring_count(qcache); - ret = rte_ring_dequeue_zc_burst_elem_start(qcache, - sizeof(cnt_id_t), rte_ring_count(qcache), &zcdc, - NULL); - MLX5_ASSERT(ret); + ret = rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), + ring_size, &zcdc, NULL); + MLX5_ASSERT(ret == ring_size); reset_list = cpool->wait_reset_list; - rte_ring_enqueue_zc_burst_elem_start(reset_list, - sizeof(cnt_id_t), ret, &zcdr, NULL); - __hws_cnt_r2rcpy(&zcdr, &zcdc, ret); - rte_ring_enqueue_zc_elem_finish(reset_list, ret); - rte_ring_dequeue_zc_elem_finish(qcache, ret); + ret = rte_ring_enqueue_zc_burst_elem_start(reset_list, sizeof(cnt_id_t), + ring_size, &zcdr, NULL); + MLX5_ASSERT(ret == ring_size); + __hws_cnt_r2rcpy(&zcdr, &zcdc, ring_size); + rte_ring_enqueue_zc_elem_finish(reset_list, ring_size); + rte_ring_dequeue_zc_elem_finish(qcache, ring_size); return 0; } @@ -323,7 +324,7 @@ mlx5_hws_cnt_pool_cache_fetch(struct mlx5_hws_cnt_pool *cpool, struct rte_ring_zc_data zcdu = {0}; struct rte_ring_zc_data zcds = {0}; struct mlx5_hws_cnt_pool_caches *cache = cpool->cache; - unsigned int ret; + unsigned int ret, actual_fetch_size __rte_unused; reuse_list = cpool->reuse_list; ret = rte_ring_dequeue_zc_burst_elem_start(reuse_list, @@ -334,7 +335,9 @@ mlx5_hws_cnt_pool_cache_fetch(struct mlx5_hws_cnt_pool *cpool, rte_ring_dequeue_zc_elem_finish(reuse_list, 0); free_list = cpool->free_list; ret = rte_ring_dequeue_zc_burst_elem_start(free_list, - sizeof(cnt_id_t), cache->fetch_sz, &zcdf, NULL); + sizeof(cnt_id_t), + cache->fetch_sz, + &zcdf, NULL); zcds = zcdf; list = free_list; if (unlikely(ret == 0)) { /* no free counter. */ @@ -344,8 +347,10 @@ mlx5_hws_cnt_pool_cache_fetch(struct mlx5_hws_cnt_pool *cpool, return -ENOENT; } } - rte_ring_enqueue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), - ret, &zcdc, NULL); + actual_fetch_size = ret; + ret = rte_ring_enqueue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), + ret, &zcdc, NULL); + MLX5_ASSERT(ret == actual_fetch_size); __hws_cnt_r2rcpy(&zcdc, &zcds, ret); rte_ring_dequeue_zc_elem_finish(list, ret); rte_ring_enqueue_zc_elem_finish(qcache, ret); @@ -378,15 +383,14 @@ __mlx5_hws_cnt_pool_enqueue_revert(struct rte_ring *r, unsigned int n, * * @param cpool * A pointer to the counter pool structure. + * @param queue + * A pointer to HWS queue. If null, it means put into common pool. * @param cnt_id * A counter id to be added. - * @return - * - 0: Success; object taken - * - -ENOENT: not enough entry in pool */ -static __rte_always_inline int -mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, - uint32_t *queue, cnt_id_t *cnt_id) +static __rte_always_inline void +mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, + cnt_id_t *cnt_id) { unsigned int ret = 0; struct rte_ring_zc_data zcdc = {0}; @@ -404,25 +408,29 @@ mlx5_hws_cnt_pool_put(struct mlx5_hws_cnt_pool *cpool, qcache = cpool->cache->qcache[*queue]; if (unlikely(qcache == NULL)) { ret = rte_ring_enqueue_elem(cpool->wait_reset_list, cnt_id, - sizeof(cnt_id_t)); + sizeof(cnt_id_t)); MLX5_ASSERT(ret == 0); - return ret; + return; } ret = rte_ring_enqueue_burst_elem(qcache, cnt_id, sizeof(cnt_id_t), 1, NULL); if (unlikely(ret == 0)) { /* cache is full. */ + struct rte_ring *reset_list = cpool->wait_reset_list; + wb_num = rte_ring_count(qcache) - cpool->cache->threshold; MLX5_ASSERT(wb_num < rte_ring_count(qcache)); __mlx5_hws_cnt_pool_enqueue_revert(qcache, wb_num, &zcdc); - rte_ring_enqueue_zc_burst_elem_start(cpool->wait_reset_list, - sizeof(cnt_id_t), wb_num, &zcdr, NULL); - __hws_cnt_r2rcpy(&zcdr, &zcdc, wb_num); - rte_ring_enqueue_zc_elem_finish(cpool->wait_reset_list, wb_num); + ret = rte_ring_enqueue_zc_burst_elem_start(reset_list, + sizeof(cnt_id_t), + wb_num, &zcdr, NULL); + MLX5_ASSERT(ret == wb_num); + __hws_cnt_r2rcpy(&zcdr, &zcdc, ret); + rte_ring_enqueue_zc_elem_finish(reset_list, ret); /* write-back THIS counter too */ - ret = rte_ring_enqueue_burst_elem(cpool->wait_reset_list, - cnt_id, sizeof(cnt_id_t), 1, NULL); + ret = rte_ring_enqueue_burst_elem(reset_list, cnt_id, + sizeof(cnt_id_t), 1, NULL); } - return ret == 1 ? 0 : -ENOENT; + MLX5_ASSERT(ret == 1); } /** @@ -482,15 +490,17 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, return 0; } ret = rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), 1, - &zcdc, NULL); + &zcdc, NULL); if (unlikely(ret == 0)) { /* local cache is empty. */ rte_ring_dequeue_zc_elem_finish(qcache, 0); /* let's fetch from global free list. */ ret = mlx5_hws_cnt_pool_cache_fetch(cpool, *queue); if (unlikely(ret != 0)) return ret; - rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), - 1, &zcdc, NULL); + ret = rte_ring_dequeue_zc_burst_elem_start(qcache, + sizeof(cnt_id_t), 1, + &zcdc, NULL); + MLX5_ASSERT(ret == 1); } /* get one from local cache. */ *cnt_id = (*(cnt_id_t *)zcdc.ptr1); @@ -504,8 +514,10 @@ mlx5_hws_cnt_pool_get(struct mlx5_hws_cnt_pool *cpool, uint32_t *queue, ret = mlx5_hws_cnt_pool_cache_fetch(cpool, *queue); if (unlikely(ret != 0)) return ret; - rte_ring_dequeue_zc_burst_elem_start(qcache, sizeof(cnt_id_t), - 1, &zcdc, NULL); + ret = rte_ring_dequeue_zc_burst_elem_start(qcache, + sizeof(cnt_id_t), 1, + &zcdc, NULL); + MLX5_ASSERT(ret == 1); *cnt_id = *(cnt_id_t *)zcdc.ptr1; iidx = mlx5_hws_cnt_iidx(cpool, *cnt_id); } @@ -553,17 +565,13 @@ mlx5_hws_cnt_shared_get(struct mlx5_hws_cnt_pool *cpool, cnt_id_t *cnt_id, return 0; } -static __rte_always_inline int +static __rte_always_inline void mlx5_hws_cnt_shared_put(struct mlx5_hws_cnt_pool *cpool, cnt_id_t *cnt_id) { - int ret; uint32_t iidx = mlx5_hws_cnt_iidx(cpool, *cnt_id); cpool->pool[iidx].share = 0; - ret = mlx5_hws_cnt_pool_put(cpool, NULL, cnt_id); - if (unlikely(ret != 0)) - cpool->pool[iidx].share = 1; /* fail to release, restore. */ - return ret; + mlx5_hws_cnt_pool_put(cpool, NULL, cnt_id); } static __rte_always_inline bool -- 2.25.1