From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0061.outbound.protection.outlook.com [104.47.1.61]) by dpdk.org (Postfix) with ESMTP id 087EF98 for ; Fri, 27 Jul 2018 04:32:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rEv6Z1hLY1z/W39Yvvp7JhOQ/HLMofWl4svTKjHhjIA=; b=quA3VdZ0zd0hX51+E0UcoeTI6feSdZH/zBRsiexFiW4T4Jbi5gBy4E1qETQ+4ToDLcGKMEOxk8CXjvwHu50yHqPKZXWKDU4v0XERoNjzgnDHdOj8m41JdeE1nE3pVTlT1o7YcFv2fgl3NEq3dIo4BG7CjFY/vU2kY/A95VxDAr4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by AM5PR0501MB2035.eurprd05.prod.outlook.com (2603:10a6:203:1a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.973.21; Fri, 27 Jul 2018 02:32:36 +0000 From: Yongseok Koh To: Yongseok Koh Cc: Xueming Li , dpdk stable Date: Thu, 26 Jul 2018 19:32:05 -0700 Message-Id: <20180727023205.38767-10-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180727023205.38767-1-yskoh@mellanox.com> References: <20180727023205.38767-1-yskoh@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: MWHPR15CA0056.namprd15.prod.outlook.com (2603:10b6:301:4c::18) To AM5PR0501MB2035.eurprd05.prod.outlook.com (2603:10a6:203:1a::21) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5a8bd2ab-1556-44bf-b33f-08d5f36937e5 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM5PR0501MB2035; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2035; 3:QnZH74HHB92wu5UR/+pG20vt/Lfiy15oRLKHj7OmYk+nzD3MCOp32mzueMTDtVbsvnoaUyAPNdU6QEzPtuLZDT0KAKBxGef8HOfgnwazsYKCXp09I8XZ6ETQZE5Zhq7Z8F8abROAHmPrNsG4gVtowkYvKupS2/HuJmTiF3ATtIV/0ac5wUdl9lAi8aMMu9t5Zlro5mZpRyB65ZL7QKp/SJzHcfQ65JMMgS3sReH52KKWoMT2x8ywpnwqOFzt7GQt; 25:OwJ5h3xXqlYmsE2MaqR/k32IsekA5CxZKeqS2ouUKPa6HIXfxu/4EyCGcRD29HUYb4Q9jEjtdDbKd8/kGQWMj6kOuJzdbkLoTXs0hGgvF82W6lUR47twmauvfxc4L6iOq9qGva3NU7YFXbQVD2lxdbmdOqlelJU/R7miyJE1FS2eHh46s16sxW9SHGuJCnn52QilQMfFOBG+QS/C6zLsI0uKGmqWcl9dfYBra0B2xZ92rj13kCKRoX/O8bExGyqlFOF8kodBAD6qL5ANXnTkifpR5WK9tYgk24MKmPGHozomUyYGA3nheo1t/xN0OJuvQs9C1/wSjyQU/YFe5FSKYQ==; 31:v3lwc0SQCpjxRl9k3o7I1Am4rUQey0u595AzciQvkNdG2OZziWHZk15M550QnBYL2NKXyObHDFHYBnLtHv3Jzk6vLfLo5LcWTcwhu8+y6VT53IKJ4RPHSlHJZvHr+1lvXY/SDZ/kCjyJWW3vqTaOrV85YcpkyGVIx6xefADIwg8chdjvkN8yaz3P+cNdgg8zN2rF18wqD6MCneY0ESDgftjH3znPYHoMS6f3SgHSmFw= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2035: X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2035; 20:TvdcEyGJsQID8RgWvt08pazAkK2psmU3+AdsLftyoAwVtGHBW5AUNvzB1GoO0W7WyUCRUZo5w5Tf/M23kN7wk8wTCSLhtwjhjn8FgZPF2fx76KTln+khIiSg0UrJHOMn71bRap0G5tk0WZR0PRj9YqY0G9gktb9IhP8JGVRg9Ps+qou2e42y3eX91W2/rVEHYbvCDheUofRg1DIFiTwosiZQP+i1UdW6zk8wW6mqFfL218PX4zX2MINM1fH4Mi3eheg03tK6wxBJEMv/lj3wgdJxtLhAU8oCoIvhFg44449O2xpUImWDUsxL1xASKxTTEP9UFUWsz0wNCjIGm+GGE+peO17DdGxbtFIxJH02DqGNU8IzGYK3OP4n0e9bMh35K0Hbu219ZPmfKl5+XXq6GRPcgd7SPxnvWvd+1pH1O0VZOw9m0EF4f0deuqc/0LQs52USaCnglTVyEpdFDQLVpdHaaECT7zTrOlb5ShbEhfaB5Y3LfvIWrZZiXpfydXpr; 4:kxJeGw4u1+DGdPy4j1xCZHapo9LL+HBLfVW6uxt5vLYbw+/DQB0aHnZ3cGZOq5NSokD4mxY4e4cvAi5HPkm1SDwTZ2Np+crmOutMMJAmCi2E+pDxMBrmZTRxqLH44PTt72rMMr5zZsL3iFhY4QNYGkMkkXeT2GSQcD1ONQAay0Y7qiZhNA3bQxO2a/aAMLk374Q7rOLphwiPbsCLu82+5w3TThz/depOvh9Hf0PT2uJgz3ETTuBiZNaMFNXbEVldc0WPCH7j+qbF8EvaJsBvzQ== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231311)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123562045)(20161123560045)(20161123564045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:AM5PR0501MB2035; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0501MB2035; X-Forefront-PRVS: 07467C4D33 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(136003)(366004)(376002)(346002)(39850400004)(396003)(199004)(189003)(69234005)(28163001)(4326008)(50466002)(48376002)(54906003)(1076002)(1720100001)(14444005)(966005)(25786009)(6666003)(53376002)(5660300001)(66066001)(47776003)(105586002)(86362001)(106356001)(478600001)(575784001)(51416003)(386003)(53546011)(3846002)(7696005)(37006003)(76176011)(52116002)(6116002)(316002)(16586007)(446003)(97736004)(81156014)(305945005)(26005)(6306002)(55016002)(11346002)(8936002)(68736007)(81166006)(50226002)(21086003)(8676002)(36756003)(7736002)(53936002)(476003)(6862004)(2616005)(6200100001)(486006)(69596002)(7049001)(2906002)(956004)(16526019); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2035; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2035; 23:+3O8oEqE+ZBqBuKqEmEo3jgsoI//dcJ847bXLEq?= =?us-ascii?Q?0CGn5qXbOwwK2DkWG7ntVKhzqYmmOvgrOF1CllmqTTNeRUAXN/CNJuQfwQAv?= =?us-ascii?Q?vKYs1L2odtgEdGJGFECV+/THidI5TItPf8io3mfx1NPu9eM4yt6dH46oNX0s?= =?us-ascii?Q?aFfhpPscG2BiAHZUpFDRVlbnLcLT9P9FD/R81tbVy8CRoCEgOhySGvZ93yHx?= =?us-ascii?Q?WxLrQY472aMDJ4pGi5odVa/3BXlgPOiQSLfmlvCrRhVlYIt8C7EjXRfNS2pg?= =?us-ascii?Q?YLBaxSERfVPo171iUsRDVYMTCBbGyisFpEbOk++QPYVPbQ/F9kYEqw0PxVdQ?= =?us-ascii?Q?MR0bdIQ8Y0MAmM1Kgpzjuxy2hNe0Q5hncZqTn3xMV863Hqp7xbb5nhA1ywTY?= =?us-ascii?Q?UI7/bULhCPxdOAvHH/4Sme+VrScoWiu0gBIUzlPzNpOM8PMLk6uQqGKRPhQZ?= =?us-ascii?Q?nvcCnc+8hhvHrZePbdIRRDvrRJ5fAthGQOWrhs8dmRV5cxUI/Yor0amc7n5K?= =?us-ascii?Q?guiL3QGeVldGdiEokhi9krc6rFbjbReSDT+Jic2G1NbWpT/G8QtXOIK3Zuw7?= =?us-ascii?Q?DuZR+HSiRTYdYryOmEmrebv8otS6R7CVxMW2SskiCEu5a98NNRkdSl4IVhqg?= =?us-ascii?Q?19nAXgDpxbAoN7zf39oJv5Z4qp0t03WTRlJi9XnU063Pp6SykU5cmEsazU1h?= =?us-ascii?Q?wnzjCxcqG9Wk8xApnbKg+DDUDx1nEhbK0r/a2JxqGROt5pl1KkRIf0dPCs5G?= =?us-ascii?Q?YO23zjU8UzaCcCzRMpQhQkePgnpLHGAKv+YCWE8xD1v1uXMs2RdUrConj8bp?= =?us-ascii?Q?YMFnfNyNXDBFecW7Zl7cDfUeiQkHtTcUUr+Ect+2/ufGe8u+o0VqIe5pDJP5?= =?us-ascii?Q?YnjSB1QlatrUSVM9uhMwgH6vlF7EPMtAAC1zB6vyzSNwsFdsRB38XrsUJF9n?= =?us-ascii?Q?e7K59QYZ3Of78+ApXGwodIjUsDaQykipQrgSZwY379QigHmD+4AKcroTzLel?= =?us-ascii?Q?VQyHqL2a2Wb3xYIu8RzYqTdC7Mhyl66T6xv3xSj9xAm9eky69MkrI30aTd8y?= =?us-ascii?Q?2GjmO3vIeiHaD1u1p4YMYoO9zO5HxlFDvZFs4eLJagaCgux1Lksmcz5/hY/+?= =?us-ascii?Q?hVns4TtetP+tYhu5YDiQIfHSJ+vp9Circ8pb7Bea5KtSA9/BgB1te1nl7q86?= =?us-ascii?Q?BzDSu4x77LEhu5k7NBmVaZYKcovMyMI7pQg9w4qsbkQEuBnAUQVMoC7DoLd+?= =?us-ascii?Q?PmcOFct5yWhzUencQDDQlQ8NGi1Zw3QgEWRhKQNDrJduE1SWLrTidPgstIFB?= =?us-ascii?Q?MrK+mDnZnrbTHocKY3t06Rl6kBKpH/12PGipvuJICzxSRNF0St8fyZxiSaJy?= =?us-ascii?Q?lMta/bGbryVpXSQpH5PH5WmmvKYiqP+/DEkPQbTApQv43TURq0GCfVLc66/k?= =?us-ascii?Q?STuPW5b4Dx110P2vi/bAGM6XiSLUUr57aEtObLG46y+QAR7Ekez2nH7x1Rlw?= =?us-ascii?Q?ZPb6wk7UaX0hHfw=3D=3D?= X-Microsoft-Antispam-Message-Info: UM4fiOMAZv88kLSfKU69bjIotuwuPyPxzcL8MbUC6q1A/CvOaKb42R8NGHWOEmHfvTk4TvZFVMMyptcOTgSihZc9QgH+mgE3yOWwX68cjXJS98HWM7nWe4zi6CylBAVvPYCUZvgnMF1cI59VPNq3dD7goZpprK+fQuUHTQ1x1lTfxeQPO39/HIGh0SdHjP8wEZo94i0wK4jRPfBGCsjTn7ftAaPCD7nVtX7Mmxh4cp1NnTpsutFVokRkrJ7HHSjCnmIFfk12jVZGUE3ZdXTuaB/FcEpGxgyISrQOCdOEra8r34Gq3/kydgU54AC1TFmGJQYybgmOhG8IwASiWNcc55P+xa6mGQsfn/kEaicF390= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2035; 6:gqfQuJ1ldELX2vajo2mlGPl7VKClAenhxGVGhcuzA0Afnq34xp4adVSzHj2Sf/Or7npv7Y23oARmldVqYwWOzrM7qTQcLix6OaFRuCFIqh2EfJ+3JGo8esuK7DWhaNveQvp+vWEwEc3j46vlwaeb6SuFBe/rkcxoluBQrhgn2mQ8t6hSqOGRqu/vzTkkbiJtfuU0fGk9sgXNwtLys3Ne2x18fc/9vrhcRANJQl2ctoOOs3RptxbaJyIiS4DcUHBHctHqQvS2as0QbEuE8O/7ivPoy4cczM5pyQLj1TBp0thj1wJOYJk7zcugfy7BEASxBs4QDNuPcAZKeyQcsmQB1Y7Zp7oIT8eLc9NWsbKIufvm6RI21H9IWu7Ffl1A1SiZgH3hNiaiRo4tVxSZkVx+Ao+Re7YJsfjLqah2uPOFM4pG+pEJ2EuVIYya4GZCzOVWMy+tdnXT0IMWk7Gf6129tw==; 5:rKx9RNSYwX061q7VmawuUgzvGuHXBoDpxeOyF49paD2CiX1mK3TewrnNFJWzmQ2v7Lnahj98Rb9ZZ3xhbembtBNRLnwoPBBoc2MRP5vvnJRwbb17NZ0zErPTydFrI09jtvvyIG2VYFnwxgr68upzT0CEbPRSiFtDzfLIilV2L5w=; 7:8uUdAt0o6wA+PACNNDHwK18ocQWLdP3fAVPvn0bN63TTH9H4RYax3R3Dluvh0mcUDhmSro8F8nktwVDwBt2jL8baD6VtqzHbBX6Qz8nRK0+ks4lp5BaE9R652K1H289ULn8xhcbaYCrMiOB2et19Y6QtQID/81OT+GwksURGUZEBGC7MIX8B3rYaWuUPZlm2Jg8MBydti2IliBG9BbcM+ytdpwXwN4GWUwj+7qLWr2/ijW9DniF3EK709z0ZlMIa SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jul 2018 02:32:36.8941 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5a8bd2ab-1556-44bf-b33f-08d5f36937e5 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2035 Subject: [dpdk-stable] patch 'net/mlx5: fix assert for Tx completion queue count' has been queued to LTS release 17.11.4 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2018 02:32:40 -0000 Hi, FYI, your patch has been queued to LTS release 17.11.4 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 07/28/18. So please shout if anyone has objections. Thanks. Yongseok --- >>From 3bf985f340ab3ebe9318331cbd42ed05e08781ab Mon Sep 17 00:00:00 2001 From: Yongseok Koh Date: Mon, 23 Jul 2018 11:00:10 -0700 Subject: [PATCH] net/mlx5: fix assert for Tx completion queue count [ upstream commit c618e7e82bfed1da91f1dbabdd06e0a39fdb7b78 ] There should be at least one Tx CQE remained if Tx WQ and txq->elts[] have available slots to send a packet because the size of Tx CQ is exactly calculated from the size of other resources. As it is guaranteed, it is checked by an assertion. max_elts is checked after the assertion for Tx CQ. If no slot is available in txq->elts[], the assertion would be wrong. Fixes: 2eefbec531c7 ("net/mlx5: add missing sanity checks for Tx completion queue") Fixes: 6ce84bd88919 ("net/mlx5: add enhanced multi-packet send for ConnectX-5") Signed-off-by: Yongseok Koh Acked-by: Xueming Li --- drivers/net/mlx5/mlx5_rxtx.c | 28 ++++++++-------------------- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 14 ++++---------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 14 ++++---------- 3 files changed, 16 insertions(+), 40 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 38f036c4b..1bbce3b75 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -401,8 +401,6 @@ mlx5_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); if (unlikely(!max_wqe)) return 0; @@ -742,14 +740,13 @@ next_wqe: /* Check whether completion threshold has been reached. */ comp = txq->elts_comp + i + j + k; if (comp >= MLX5_TX_COMP_THRESH) { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ last_wqe->ctrl2 = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ last_wqe->ctrl3 = txq->elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp = comp; } @@ -868,8 +865,6 @@ mlx5_tx_burst_mpw(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); if (unlikely(!max_wqe)) return 0; @@ -958,14 +953,13 @@ mlx5_tx_burst_mpw(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) if (comp >= MLX5_TX_COMP_THRESH) { volatile struct mlx5_wqe *wqe = mpw.wqe; + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ wqe->ctrl[2] = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ wqe->ctrl[3] = elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp = comp; } @@ -1097,8 +1091,6 @@ mlx5_tx_burst_mpw_inline(void *dpdk_txq, struct rte_mbuf **pkts, /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); do { struct rte_mbuf *buf = *(pkts++); uintptr_t addr; @@ -1255,14 +1247,13 @@ mlx5_tx_burst_mpw_inline(void *dpdk_txq, struct rte_mbuf **pkts, if (comp >= MLX5_TX_COMP_THRESH) { volatile struct mlx5_wqe *wqe = mpw.wqe; + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ wqe->ctrl[2] = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ wqe->ctrl[3] = elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp = comp; } @@ -1385,8 +1376,6 @@ mlx5_tx_burst_empw(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) /* Start processing. */ mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); if (unlikely(!max_wqe)) return 0; @@ -1599,15 +1588,14 @@ mlx5_tx_burst_empw(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) (1 << txq->wqe_n) / MLX5_TX_COMP_THRESH_INLINE_DIV) { volatile struct mlx5_wqe *wqe = mpw.wqe; + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request completion on last WQE. */ wqe->ctrl[2] = rte_cpu_to_be_32(8); /* Save elts_head in unused "immediate" field of WQE. */ wqe->ctrl[3] = elts_head; txq->elts_comp = 0; txq->mpw_comp = txq->wqe_ci; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } else { txq->elts_comp += j; } diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 32d1e6de9..e748615ee 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -135,8 +135,6 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, assert(elts_n > pkts_n); mlx5_tx_complete(txq); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); if (unlikely(!pkts_n)) return 0; for (n = 0; n < pkts_n; ++n) { @@ -204,12 +202,11 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, txq->elts_comp += (uint16_t)(elts_head - txq->elts_head); txq->elts_head = elts_head; if (txq->elts_comp >= MLX5_TX_COMP_THRESH) { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); wqe->ctrl[2] = rte_cpu_to_be_32(8); wqe->ctrl[3] = txq->elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } #ifdef MLX5_PMD_SOFT_COUNTERS txq->stats.opackets += n; @@ -273,8 +270,6 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, assert(elts_n > pkts_n); mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); pkts_n = RTE_MIN((unsigned int)RTE_MIN(pkts_n, max_wqe), max_elts); if (unlikely(!pkts_n)) @@ -310,11 +305,10 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, if (txq->elts_comp + pkts_n < MLX5_TX_COMP_THRESH) { txq->elts_comp += pkts_n; } else { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request a completion. */ txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif comp_req = 8; } /* Fill CTRL in the header. */ diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index 1a56fca9f..7e8c9b885 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -135,8 +135,6 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, assert(elts_n > pkts_n); mlx5_tx_complete(txq); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); if (unlikely(!pkts_n)) return 0; for (n = 0; n < pkts_n; ++n) { @@ -205,12 +203,11 @@ txq_scatter_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, txq->elts_comp += (uint16_t)(elts_head - txq->elts_head); txq->elts_head = elts_head; if (txq->elts_comp >= MLX5_TX_COMP_THRESH) { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); wqe->ctrl[2] = rte_cpu_to_be_32(8); wqe->ctrl[3] = txq->elts_head; txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif } #ifdef MLX5_PMD_SOFT_COUNTERS txq->stats.opackets += n; @@ -272,8 +269,6 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, assert(elts_n > pkts_n); mlx5_tx_complete(txq); max_elts = (elts_n - (elts_head - txq->elts_tail)); - /* A CQE slot must always be available. */ - assert((1u << txq->cqe_n) - (txq->cq_pi - txq->cq_ci)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); pkts_n = RTE_MIN((unsigned int)RTE_MIN(pkts_n, max_wqe), max_elts); assert(pkts_n <= MLX5_DSEG_MAX - nb_dword_in_hdr); @@ -311,11 +306,10 @@ txq_burst_v(struct mlx5_txq_data *txq, struct rte_mbuf **pkts, uint16_t pkts_n, if (txq->elts_comp + pkts_n < MLX5_TX_COMP_THRESH) { txq->elts_comp += pkts_n; } else { + /* A CQE slot must always be available. */ + assert((1u << txq->cqe_n) - (txq->cq_pi++ - txq->cq_ci)); /* Request a completion. */ txq->elts_comp = 0; -#ifndef NDEBUG - ++txq->cq_pi; -#endif comp_req = 8; } /* Fill CTRL in the header. */ -- 2.11.0