From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40058.outbound.protection.outlook.com [40.107.4.58]) by dpdk.org (Postfix) with ESMTP id 01E7D235; Mon, 23 Jul 2018 20:00:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fCyGkOFOO1oSBmqqXDwU0MTtSczALjazPUKvJD0ARB8=; b=YvNPEZRrtZQxlWUfuEOy0frbqQzNZxJrLSEEdSrYazAK4AyEDPMDR8pvE/cTUDVEIPiawzPfIS7gjBlYpLaOgUSGUDQU8hEoU3PkPkgm9urD7iGII3O1B9H2tdh6QLlxlYxSlKquGrPLlkdukIdnvU1yVRbCD5dOjO6XqtsNm1o= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by VI1PR0501MB2048.eurprd05.prod.outlook.com (2603:10a6:800:36::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.973.16; Mon, 23 Jul 2018 18:00:43 +0000 From: Yongseok Koh To: shahafs@mellanox.com Cc: dev@dpdk.org, Yongseok Koh , stable@dpdk.org Date: Mon, 23 Jul 2018 11:00:10 -0700 Message-Id: <20180723180010.3575-1-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: DM5PR21CA0034.namprd21.prod.outlook.com (2603:10b6:3:ed::20) To VI1PR0501MB2048.eurprd05.prod.outlook.com (2603:10a6:800:36::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 05e66435-4ddb-4cbe-97ea-08d5f0c63603 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600073)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0501MB2048; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 3:Egyr/F6EZYX5TG/4ng4rp7SZrFI9nkfhCK4l36xNh2ip5oLtKuXl3vYnSwND8jj5wP+gl66sqfNtM2QLQEUIQ/1apekUeCBqcbgbOFuNqsOM8Uqe47K2wxGU6tQ9/Tsgbu/jmQ8rNzyKfNWzAblp0rI4Zt+XMzLZjRFk9WleG8OsKuVG974JBJe5gkZ2wKCfM5d4Sw3ChRYvRYangp2vBVgqdvZ4CIkOYCcp1DvztteJO2Yhp7lTsL7Z+vdnGLoq; 25:2ykUfMLvPkj/wAL1OD3Lx5ZzrNNAc5K1VRhPrg0nHeV384yu1IBmAVZuIOfrNUAymS2pQmxv3v6hgPbITBbhfjrY48yKgNlRYW+7DtsAa1/ZAmYFVUDDwTLR/q/RpbZ8ilPIZFX6tFuIfWY45yoYyS0TCtLemW3eIvDXUYuBcI74CNavLK0feyfJalLCvY6rgb645oK9JXzLRU8SCm5cogLeKukaKMWC1+LIVc1jNSzvr69h8iI7mgrofDrHYVcdwF+NVK/p6qD/vXV4CBusK6FkBdHWiTnKv/2IGv2NJz/1VyEVFDylB1hKfI+lAPCEWVxf1FCXcZgY+MhO2XO9Sw==; 31:BGmoNcQmF3BywlziGThueGvt9tutCmNR/11RZjeSHEuNmmXc6Skdcph+XXBWXPR5qTiIul5qioku3pR+Ws2t3PTKZuEl62sM+z0H8fOoOisL8E4zJqUEYohEAlO6xdbJTOu6rNQ5jY8lgD3X2IBxyeofo2CrcXA2EQqjHYk9rgdlKQjDX4RPKS9ChjAGTdYREFskjVcoZAKstfJHdjpKaM62ZkJ49cNqagV3FdfQWAU= X-MS-TrafficTypeDiagnostic: VI1PR0501MB2048: X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 20:PxtnSF/WKb0ZNeK+ebpl8waG85nKpGHPMtsF7Z5c19cABF1cZQ3WtAVRwZ2YlxQX2+0fTc3ZU94XCq7GM0icwck4niZu4u7vd0RM9UX+DLSOxANmBYS6k2palhFIA25hcOyxkR6BUjBxpvsQ5DeqnoLM8+J3Z0mUkVyvAm9nQpYWe4K0N4y4b1kMyylF522LF4+aJ0CFvvS6NF1ZY0MVDQ3rjDa/Mqwttoyjnu6YAP+q2yF5SZYvDsclH1Ol4Lbg+ZjgYBFyBJqVJiU7hmu8Pzs8BxK8tSpSNSe8T/OzQvkJ4tu94KTiGbBdwzn3cvjiRF5yKYihxt9Ex3OmBBnLhNS0DdYXG1IfD4Wh9wdSwPZezU2OQuA1E0q0dB2rJP76vOwDaLcpHpMEQcO3CVE8NtpWaGuE4ybT6QtQ+dYOnZbUv9vynm1pWljI5t6koMmm8cwBlrddASCb7qtpzdp9EdmLYVyQbKyBR0iWagn7Uh+x1WyNrNuqvTSZ5ydfANzM; 4:Qe51OwxuZuoHLoGem3PUJ2l1QHXgyldW6bBr22LUE7GeQa+L3Q5jDDiKtJH3xuurOiBexEP1MMZneNrFehaws/0RhRRtsO2Vc8p4Jls4RmFWDb+nz7gBlGr+3KoXmj6U5gBEPF45kGHUzkFMhyftcMLr1pk6Yxx1bwW8nvwWlJR3Kmg2NWE9eA48WDf41jycAiQ4+GE25zo+hEMb9Ghy0IcxM4Jh4Svo9IO1apYMNGIHkHwRGLpc75kxLsfCy2FxglNbnV+DtSkxzfrW+pa2lw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231311)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123558120)(20161123562045)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:VI1PR0501MB2048; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB2048; X-Forefront-PRVS: 0742443479 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39860400002)(376002)(366004)(346002)(396003)(136003)(199004)(189003)(8676002)(50226002)(478600001)(6666003)(69596002)(6636002)(14444005)(386003)(81166006)(81156014)(68736007)(2616005)(956004)(7696005)(52116002)(16526019)(51416003)(476003)(25786009)(50466002)(4326008)(47776003)(34206002)(26005)(48376002)(53936002)(36756003)(6116002)(105586002)(106356001)(1857600001)(2351001)(55016002)(97736004)(7736002)(66066001)(21086003)(486006)(8936002)(2906002)(316002)(3846002)(86362001)(16586007)(2361001)(450100002)(1076002)(305945005)(5660300001)(575784001)(37006003); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2048; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0501MB2048; 23:61C1R8m5kFwUg33+aVLFwxVn9HXPquMaaHYanp/?= =?us-ascii?Q?vwcK3NMa02WL2Scbf02WzwJJuOvW6CDbWjv0U/hnq7s4ZUBkmgNyTMnXX0e+?= =?us-ascii?Q?k2MZyuJyhrBdm0P5PQcbA+uoCq30djSy/0Die9+Lw0cKsUuX+wdfQXf8I6kY?= =?us-ascii?Q?M0Xz2JSNP4BTjY2Xt1ccm8BZVc/Q4UixSPuNd4JpGNc75i7c3fjEJ55a9OSq?= =?us-ascii?Q?/eCVEzzopDCSKHUKq31ZMirrgQR1+fWg5/gYdfr8zFmUR1bzciL+HFka27Za?= =?us-ascii?Q?lmqIMpgcT9DPeJZS3AssU671mXHL6OWxAwPzd6cvDCSTthqIkVpQcozwIOg+?= =?us-ascii?Q?BPRjfgUl5H0HYQYqCq4jwnAU0KPhTpz6elqNdJ3FXtJyDKxYyoJ14hMrQsJA?= =?us-ascii?Q?iWDL0OlVEdxJUXWoMbmTflruZC/aFRzpl89syAtk9WT5tjCpQw36QCoyUKqa?= =?us-ascii?Q?3UarNkw6xtvjX04Xc7J4IY70rdovLYPpeKs1aNJEIppOmonUDdwpkuhJDPjS?= =?us-ascii?Q?pqhcF1ov3kJd9t/vmD+yu4Z/bn6+fzqLC6bp5sebkJTTscA1c1B4xoCff4zH?= =?us-ascii?Q?I5xGGCQiWTKRjqVbrAMG9102rHYT2KbaiPFwZGrz5zwoJ5FlLfvj66w+fJO7?= =?us-ascii?Q?K6ksUxhTGpoiDqSYkjtJQsgZ3rhiW723pi+OPQ5EOMcR1tNIV+Z7n+NtNMnV?= =?us-ascii?Q?TB0IsXU1vfmT71kguJ7NNs/h5L24W7f89uszO+CpKMopJJzhpcAZ2qLYobXN?= =?us-ascii?Q?yr82D/kIJBL+XD7Gpfi20K2VStU3/5Na1ua296cyExKVjfqyLu9c6X8sbZIo?= =?us-ascii?Q?jXVknbjpS/HgShYCMpjrRgJ2SQK2gaD1oCS+tI4Z5SzDRAD6Qkvv5/yN9Lyv?= =?us-ascii?Q?8+A7ilWu0HfbDo4C/0OvC9W3NleVEBIsE8kQMhSeovcldjT49gfzwev3ExUV?= =?us-ascii?Q?symOelz7s2AqEowOrNUpijgQrQD0xGRvowseMgzDVFNf6Y7th1WSDZDpK6v/?= =?us-ascii?Q?fCqJF8W5Z8x3db7z94pRFHhcdSCvB5LC+2+J38KM5sbUJgS8/EyS1QUJQ/Ug?= =?us-ascii?Q?mVPVNm4sMjyvWv0HjzXyu8s6za9cpIo4jj9r2f7/d5H7OpB9dp7Zi6/IgyE6?= =?us-ascii?Q?sXVn+Z/Vs1HHRnmJp2BOhXAba8jGK0bfOvAcWiQcw3yNqkDbU50InSh+iyRX?= =?us-ascii?Q?MAzCpUd1UUAMoWsoT+iuSlyYWk5jkF9bLOCNmVKPukiop06bIAW+ydjtXWQ8?= =?us-ascii?Q?MQT4V+bttAoCtbr5LjcZW1XYSAR/ay9jJpjnMMP7IvxE5CZ/GAppt45EQxOr?= =?us-ascii?Q?isSS6xGWHSMZwODUmyZW7rqI=3D?= X-Microsoft-Antispam-Message-Info: hATfZh4rvm45wIob0oNg/Pztqi/OnnxNq7LdzpoeELdfsIaJLsWDlhWzctOOaJakTkqq6xf5vS6W4W1q0mITOTTnBGsa6nEB1ULVxR5/Qln6IFmaWW7eAHEtQ8RxEWW6q5/yVT9/vYBH/gomQPBwWNA6Om7Rn7r/ndgBkjZYrcFn6ea9nW3CXL9A1KSwEPZFNSENRAKem2UYRVjWZ/xJFFqBLsalEjK+diNXSKlSqacQ6PRKyM9FVkuplwr63QJdpblmkZ89VGvlCdWL58EfMZVCwzy45UKk8oP7nxACai+TRG+lMueaEYvFilEqZXHzexkBi3Rn0lVKlRRxot3k9oSNz3FFVeBu8upluMMB9cI= X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2048; 6:1Z9y5S/8wJDa5ZcqrWSZ7UlWbo90HqX0CuFZAGOtlCM/lRQYLMPxql4/MpLMj2Rod7KZ1Rg8QkIdIG0GZGxeOMl8E7cnSbnnW/cQQQIF2uMoVO/07t0MssyScIOzNBhyTwcEIoM4jOeI+540phvHEcfebjrI41gQNiBWPAwR7unqn/lrxgKEttbIWf36wTpQBPml+TIRM0adHELYiTDtp4MvzTN3tpb9NX0BDUqTYw6KAJ9GUgLW1SpeH611igxcW9OBs21pctyO1Gr9THYeUUdNJEylb5aMeDl3V333aLcVJAtaMRnFLl4uqpWzXFnJY3qWr6OgNUwjId70nOt90T+SJ441E+maKC5bVH8tSyveEE9TaaQvCcJD1+hOK5f4vKSaIJkdFlvbh5CRujStu6bJaICcLzUPxmIfAuMNRw+t5/ZdYBn4Cstv4KsA8pOuJOtoyKsHdkSEJryjXliK0g==; 5:/trwoDeoLoZ4dNtL6wvEZI1pJGx4ILVCDxnAoG4+E3+G6uwrsABkjhwd/1lLYhmmCHjddqVfZx68VJ0hchDFCLBm9KrwZysoiWmIntFur6DSahgdPP6jeDqwtRiEByXUMXyBaPuGpREtn3q9uLGi3vnJxIoiLDDtpqTDHxFLax4=; 7:natEuuku/72HDFf51o3TMej6STEdm90wrD+p3poVFX7REhya02PgxFYjgTm6LlezHcvH1JPUBapLImqUGvRXHaxYQNUEOiuP9bkdfkrKDcrz9Wv/iJ2ItOFbwh/kXRaNDlSlsoskMBSWMmxuiU7ZNCpTJLbOCxgFds+OnQwMVuDsECbGrPp35lvGO/DXAAz3FbF1fS52SnApUp7glpyjuZIIdzQuPEmraZvvpIgZ5Ec1rf1xQ6BvO9APSFW7LpNr SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jul 2018 18:00:43.2720 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 05e66435-4ddb-4cbe-97ea-08d5f0c63603 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2048 Subject: [dpdk-dev] [PATCH] net/mlx5: fix assert for Tx completion queue count X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2018 18:00:46 -0000 There should be at least one Tx CQE remained if Tx WQ and txq->elts[] have available slots to send a packet because the size of Tx CQ is exactly calculated from the size of other resources. As it is guaranteed, it is checked by an assertion. max_elts is checked after the assertion for Tx CQ. If no slot is available in txq->elts[], the assertion would be wrong. Fixes: 2eefbec531c7 ("net/mlx5: add missing sanity checks for Tx completion queue") Fixes: 6ce84bd88919 ("net/mlx5: add enhanced multi-packet send for ConnectX-5") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Xueming Li --- drivers/net/mlx5/mlx5_rxtx.c | 28 ++++++++-------------------- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 14 ++++---------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 14 ++++---------- 3 files changed, 16 insertions(+), 40 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 52a1074a8..2d14f8a6e 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -504,8 +504,6 @@ mlx5_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); if (unlikely(!max_wqe)) return 0; @@ -817,14 +815,13 @@ mlx5_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) /* Check whether completion threshold has been reached. */ comp = txq->elts_comp + i + j + k; if (comp >= MLX5_TX_COMP_THRESH) { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ last_wqe->ctrl2 = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ last_wqe->ctrl3 = txq->elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp = comp; } @@ -943,8 +940,6 @@ mlx5_tx_burst_mpw(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); if (unlikely(!max_wqe)) return 0; @@ -1033,14 +1028,13 @@ mlx5_tx_burst_mpw(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) if (comp >= MLX5_TX_COMP_THRESH) { volatile struct mlx5_wqe *wqe = mpw.wqe; + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ wqe->ctrl[2] = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ wqe->ctrl[3] = elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp = comp; } @@ -1172,8 +1166,6 @@ mlx5_tx_burst_mpw_inline(void *dpdk_txq, struct rte_mbuf **pkts, /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); do { struct rte_mbuf *buf = *(pkts++); uintptr_t addr; @@ -1330,14 +1322,13 @@ mlx5_tx_burst_mpw_inline(void *dpdk_txq, struct rte_mbuf **pkts, if (comp >= MLX5_TX_COMP_THRESH) { volatile struct mlx5_wqe *wqe = mpw.wqe; + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ wqe->ctrl[2] = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ wqe->ctrl[3] = elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp = comp; } @@ -1461,8 +1452,6 @@ txq_burst_empw(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); if (unlikely(!max_wqe)) return 0; @@ -1618,15 +1607,14 @@ txq_burst_empw(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, (1 << txq->wqe_n) / MLX5_TX_COMP_THRESH_INLINE_DIV) { volatile struct mlx5_wqe *wqe = mpw.wqe; + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ wqe->ctrl[2] = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ wqe->ctrl[3] = elts_head; txq->elts_comp = 0; txq->mpw_comp = txq->wqe_ci; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp += j; } diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 3ebf34823..b37b7381c 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -107,8 +107,6 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, assert(elts_n > pkts_n); mlx5_tx_complete(txq); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); if (unlikely(!pkts_n)) return 0; for (n = 0; n < pkts_n; ++n) { @@ -176,12 +174,11 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, txq->elts_comp += (uint16_t)(elts_head - txq->elts_head); txq->elts_head = elts_head; if (txq->elts_comp >= MLX5_TX_COMP_THRESH) { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); wqe->ctrl[2] = rte_cpu_to_be_32(8); wqe->ctrl[3] = txq->elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } #ifdef MLX5_PMD_SOFT_COUNTERS txq->stats.opackets += n; @@ -245,8 +242,6 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, assert(elts_n > pkts_n); mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); pkts_n = RTE_MIN((unsigned int)RTE_MIN(pkts_n, max_wqe), max_elts); if (unlikely(!pkts_n)) @@ -282,11 +277,10 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, if (txq->elts_comp + pkts_n < MLX5_TX_COMP_THRESH) { txq->elts_comp += pkts_n; } else { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request a completion. */ txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif comp_req = 8; } /* Fill CTRL in the header. */ diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index f0d888831..54b3783cf 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -107,8 +107,6 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, assert(elts_n > pkts_n); mlx5_tx_complete(txq); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); if (unlikely(!pkts_n)) return 0; for (n = 0; n < pkts_n; ++n) { @@ -177,12 +175,11 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, txq->elts_comp += (uint16_t)(elts_head - txq->elts_head); txq->elts_head = elts_head; if (txq->elts_comp >= MLX5_TX_COMP_THRESH) { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); wqe->ctrl[2] = rte_cpu_to_be_32(8); wqe->ctrl[3] = txq->elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } #ifdef MLX5_PMD_SOFT_COUNTERS txq->stats.opackets += n; @@ -244,8 +241,6 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, assert(elts_n > pkts_n); mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); pkts_n = RTE_MIN((unsigned int)RTE_MIN(pkts_n, max_wqe), max_elts); assert(pkts_n <= MLX5_DSEG_MAX - nb_dword_in_hdr); @@ -283,11 +278,10 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, if (txq->elts_comp + pkts_n < MLX5_TX_COMP_THRESH) { txq->elts_comp += pkts_n; } else { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request a completion. */ txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif comp_req = 8; } /* Fill CTRL in the header. */ -- 2.11.0