From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1D358A0C48 for ; Wed, 16 Jun 2021 09:06:49 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DC31740140; Wed, 16 Jun 2021 09:06:48 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2072.outbound.protection.outlook.com [40.107.223.72]) by mails.dpdk.org (Postfix) with ESMTP id 4FB2E40140 for ; Wed, 16 Jun 2021 09:06:48 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IJ3LP8H0Un+gtLhGYpt64bY6KwAR4lLtsvoV5ZaZRma9jOcXejF03/DeFdeikSizPRJ7JwLLMR9Mwk4q+BELekk1E1lGKn1sAo4Rg4UAYwkD+oWUcNjjmITPtNcprJkxN5m+RinuFWIaiCH5lJxOUnmSythIHOA1u+W8S/ceE0EMjFveWZrv+cLXwbm3tMMw3vSwEjBnV4F8CGWntBbWvo4TO2o/y/tzoQ0jwL/hfwxJPHDqUNUmLJXQSlB2Ohh531sGGEud2+Ma/S02od30/AAJeRTHmpQkkcM9n3aVhf/g5L0j0GDj7InK9A8eEdQP0GSUMuLed7cr3/rRqHbFeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AZJHHR7Dgv3rYja6VF9B4TSBf1xRQlKIHaAZ32LHOrg=; b=AyW3uj8oG+1ChbvYQUvgtna/E56gpX8sELC8yMrsjlzBhxhEvKPQPLhgoo/2R0zwosin1MeKFxwDaUiQouXrTxuUcKX0Z7zhot5fvwY2BwCdEBdeISeODARp+1r/CeOwFtrv/IdJmGt8M7hpOMtLURBJrnB+eQ6aaq2NL3HyatEpPC+hMJc/OnNk0sIVC3mwyXg4c8oACF8VWLVmp1EocToItIETMIrcjKUr5DS3ciXvvCFyQfFihutInUExY4mewC24Qj2m17LSHMT6MZ3LgMUn/CFDwefl/bqUQnlFJRYKadKTdka9uXnWfRbUCUAGNPGIkpb3Z+980vu7+KGT9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=canonical.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AZJHHR7Dgv3rYja6VF9B4TSBf1xRQlKIHaAZ32LHOrg=; b=NHnauYSP+qEAxuQyuxiK25Brc1ULTaGKHutaJmxyXtiEEE25YMANzXDA+0RS1ZvvSzcowZWeN86NNQvE5N0aFtKU/vE9yJkXenxYHwsW4bI1blTOCTAk4XM2UO+tH8++2JOSKXAeHi4UpWiy4TzxhRQ+IY7tZJ9WwHLtH9jWnH9tXmUF0K1JMzbag8gjN49zTVW6mhromrMmzK1tX8oUTv1oqg2xIGOBXR64jWRjFThYKn3JzkyyZziWCV4FGI9uDCGR5CkZ3J158U4ZMaCJU8DycmepW94N9941JhRmZZKx9UEZmv//kWiIESNhJjb5XLFTitY7t1IIRl/mdMCk3g== Received: from DM6PR04CA0030.namprd04.prod.outlook.com (2603:10b6:5:334::35) by DM5PR12MB1835.namprd12.prod.outlook.com (2603:10b6:3:10c::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.23; Wed, 16 Jun 2021 07:06:47 +0000 Received: from DM6NAM11FT024.eop-nam11.prod.protection.outlook.com (2603:10b6:5:334:cafe::a2) by DM6PR04CA0030.outlook.office365.com (2603:10b6:5:334::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.20 via Frontend Transport; Wed, 16 Jun 2021 07:06:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; canonical.com; dkim=none (message not signed) header.d=none;canonical.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT024.mail.protection.outlook.com (10.13.172.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4242.16 via Frontend Transport; Wed, 16 Jun 2021 07:06:46 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 16 Jun 2021 07:06:44 +0000 From: Viacheslav Ovsiienko To: CC: , , "Viacheslav Ovsiienko" Date: Wed, 16 Jun 2021 10:06:30 +0300 Message-ID: <20210616070630.12085-1-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 12a90915-4898-4c09-8116-08d930954e19 X-MS-TrafficTypeDiagnostic: DM5PR12MB1835: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5cmBdidNm/+XpvY4BZm0slocv5ic1GMF4E6Llv4D9oXeOFLDHlnajSOpuqSKPgrJBpyd8x++9lGxy0eo95pFhiEC3CJu8v258nQw2SOucUtovmm9T+mKUTanG+YSTWdNi+rPAGD/9IGiPv+DZvc3nRR+GsZVkLwkaV+sKRlAiDAtGr1+EPID0jHZ/CoZG7PMeRU0wzvNVb27KUEAvNTPS8n2SqOUA53XH4uzmBYT6Od4tfMACTVgcvijYRynTWbupPVZakELrKjanBzwpuKv28TQYHGP/x/ol4sAJINIcNE5K/mMK6gxIdDa1bDa+iMStaaAhy4OBrGbJWa5BJs8bm8wChzwE+eaGJPWY2Z1zbSioECKxw5HpDwHCDN+S4yrbElsL+Muu6MI+vDUe1tBSOzk4B7f+QMYT+ExhdgeE8ms78EKXEuhN1mPfPC+Jmu5ewuhI5nvM5tMVKJQoUgxTHtk9kZJcNHmMWiaaFlIx08h72YU368/XNLH5ia3yZ2L0olL2ZYRAgqTJzLqgUzmjlMArFme3WZbvJz6yHggfEVT3XiHTrwWBDBGPSbcNcfQ/ZmfPTPDg77N+viLrdNN1AxZpLPWWjFRs1jiJhE6MI1tnXb4JqalqyKah68dPkJKYFuwDZ35xygspXA+y2/6AQGB0GvCXRHPWAcN+rucqT4= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(396003)(346002)(136003)(376002)(46966006)(36840700001)(36860700001)(70206006)(70586007)(6286002)(8936002)(86362001)(8676002)(7696005)(54906003)(426003)(55016002)(4326008)(316002)(1076003)(2906002)(82740400003)(186003)(83380400001)(16526019)(478600001)(336012)(5660300002)(26005)(356005)(107886003)(7636003)(47076005)(36756003)(6666004)(6916009)(82310400003)(36906005)(2616005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2021 07:06:46.7061 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 12a90915-4898-4c09-8116-08d930954e19 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT024.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1835 Subject: [dpdk-stable] [PATCH 19.11] net/mlx5: fix last completed built descriptor X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" From: Viacheslav Ovsiienko [ upstream commit 0f13d919703fb555727cc4d075272a60d2f87508 ] The routine sending packets with Multi-Packet Write method assigns the wqe_last variable with transmit descriptor (WQE - work queue entry) being built. If send queue is close to full state, the WQE has no data yet (trying to put the first packet) and there is no enough space in descriptor for the next packet the WQE is discarded and the stored wqe_last value becomes invalid - points to the discarded WQE. The mlx5_tx_burst_request_completion() routine might set the completion request flags in the WQE pointed by wqe_last, it is safe, but the next mlx5_tx_burst call uses the WQE as the first free one and request completion flags might be overwritten and completion request will be lost causing the transmit datapath malfunction. Fixes: 8b581c690a54 ("net/mlx5: move Tx complete request routine") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_rxtx.c | 51 ++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 20 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 73dbf68d2b..7c9a09c814 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2210,6 +2210,7 @@ mlx5_tx_request_completion(struct mlx5_txq_data *restrict txq, (uint16_t)(txq->wqe_ci - txq->wqe_comp) >= txq->wqe_thres)) { volatile struct mlx5_wqe *last = loc->wqe_last; + MLX5_ASSERT(last); txq->elts_comp = head; if (MLX5_TXOFF_CONFIG(INLINE)) txq->wqe_comp = txq->wqe_ci; @@ -3828,6 +3829,8 @@ mlx5_tx_sdone_empw(struct mlx5_txq_data *restrict txq, * Total size of descriptor/data in bytes. * @param slen * Accumulated statistics, data bytes sent. + * @param wqem + * The base WQE for the eMPW/MPW descriptor. * @param olx * Configured Tx offloads mask. It is fully defined at * compile time and may be used for optimization. @@ -3841,9 +3844,10 @@ mlx5_tx_idone_empw(struct mlx5_txq_data *restrict txq, struct mlx5_txq_local *restrict loc, unsigned int len, unsigned int slen, + struct mlx5_wqe *restrict wqem, unsigned int olx __rte_unused) { - struct mlx5_wqe_dseg *dseg = &loc->wqe_last->dseg[0]; + struct mlx5_wqe_dseg *dseg = &wqem->dseg[0]; assert(MLX5_TXOFF_CONFIG(INLINE)); #ifdef MLX5_PMD_SOFT_COUNTERS @@ -3870,9 +3874,10 @@ mlx5_tx_idone_empw(struct mlx5_txq_data *restrict txq, assert((len % MLX5_WSEG_SIZE) == 0); len = len / MLX5_WSEG_SIZE + 2; } - loc->wqe_last->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | len); + wqem->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | len); txq->wqe_ci += (len + 3) / 4; loc->wqe_free -= (len + 3) / 4; + loc->wqe_last = wqem; } /** @@ -4109,7 +4114,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, pkts_n -= loc->pkts_sent; for (;;) { struct mlx5_wqe_dseg *restrict dseg; - struct mlx5_wqe_eseg *restrict eseg; + struct mlx5_wqe *restrict wqem; enum mlx5_txcmp_code ret; unsigned int room, part, nlim; unsigned int slen = 0; @@ -4128,22 +4133,21 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, return MLX5_TXCMP_CODE_EXIT; if (likely(pkts_n > 1)) rte_prefetch0(*pkts); - loc->wqe_last = txq->wqes + (txq->wqe_ci & txq->wqe_m); + wqem = txq->wqes + (txq->wqe_ci & txq->wqe_m); /* * Build eMPW title WQEBB: * - Control Segment, eMPW opcode, zero DS * - Ethernet Segment, no inline */ - mlx5_tx_cseg_init(txq, loc, loc->wqe_last, 0, + mlx5_tx_cseg_init(txq, loc, wqem, 0, MLX5_OPCODE_ENHANCED_MPSW, olx); - mlx5_tx_eseg_none(txq, loc, loc->wqe_last, + mlx5_tx_eseg_none(txq, loc, wqem, olx & ~MLX5_TXOFF_CONFIG_VLAN); - eseg = &loc->wqe_last->eseg; - dseg = &loc->wqe_last->dseg[0]; + dseg = &wqem->dseg[0]; /* Store the packet length for legacy MPW. */ if (MLX5_TXOFF_CONFIG(MPW)) - eseg->mss = rte_cpu_to_be_16 - (rte_pktmbuf_data_len(loc->mbuf)); + wqem->eseg.mss = rte_cpu_to_be_16 + (rte_pktmbuf_data_len(loc->mbuf)); room = RTE_MIN(MLX5_WQE_SIZE_MAX / MLX5_WQE_SIZE, loc->wqe_free) * MLX5_WQE_SIZE - MLX5_WQE_CSEG_SIZE - @@ -4180,7 +4184,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, * We have some successfully built * packet Data Segments to send. */ - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, + slen, wqem, olx); return MLX5_TXCMP_CODE_ERROR; } /* Inline or not inline - that's the Question. */ @@ -4201,7 +4206,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, * No pointer and inline descriptor * intermix for legacy MPW sessions. */ - if (loc->wqe_last->dseg[0].bcount) + if (wqem->dseg[0].bcount) break; } } else { @@ -4249,7 +4254,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, */ if (MLX5_TXOFF_CONFIG(MPW) && part != room && - loc->wqe_last->dseg[0].bcount == RTE_BE32(0)) + wqem->dseg[0].bcount == RTE_BE32(0)) break; /* * Not inlinable VLAN packets are @@ -4279,7 +4284,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, * continue build descriptors. */ part -= room; - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, + slen, wqem, olx); return MLX5_TXCMP_CODE_EXIT; } loc->mbuf = *pkts++; @@ -4293,7 +4299,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, */ if (ret == MLX5_TXCMP_CODE_MULTI) { part -= room; - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, + slen, wqem, olx); if (unlikely(!loc->elts_free || !loc->wqe_free)) return MLX5_TXCMP_CODE_EXIT; @@ -4302,7 +4309,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, assert(NB_SEGS(loc->mbuf) == 1); if (ret == MLX5_TXCMP_CODE_TSO) { part -= room; - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, + slen, wqem, olx); if (unlikely(!loc->elts_free || !loc->wqe_free)) return MLX5_TXCMP_CODE_EXIT; @@ -4310,7 +4318,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, } if (ret == MLX5_TXCMP_CODE_SINGLE) { part -= room; - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, + slen, wqem, olx); if (unlikely(!loc->elts_free || !loc->wqe_free)) return MLX5_TXCMP_CODE_EXIT; @@ -4319,7 +4328,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, if (ret != MLX5_TXCMP_CODE_EMPW) { assert(false); part -= room; - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, + slen, wqem, olx); return MLX5_TXCMP_CODE_ERROR; } /* Check if we have minimal room left. */ @@ -4334,7 +4344,8 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, * - software parser settings * - packets length (legacy MPW only) */ - if (!mlx5_tx_match_empw(txq, eseg, loc, dlen, olx)) + if (!mlx5_tx_match_empw(txq, &wqem->eseg, + loc, dlen, olx)) break; /* Packet attributes match, continue the same eMPW. */ if ((uintptr_t)dseg >= (uintptr_t)txq->wqes_end) @@ -4348,7 +4359,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *restrict txq, part -= room; if (unlikely(!part)) return MLX5_TXCMP_CODE_EXIT; - mlx5_tx_idone_empw(txq, loc, part, slen, olx); + mlx5_tx_idone_empw(txq, loc, part, slen, wqem, olx); if (unlikely(!loc->elts_free || !loc->wqe_free)) return MLX5_TXCMP_CODE_EXIT; -- 2.18.1