From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16275A0C57; Mon, 1 Nov 2021 10:57:20 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 072B140DF6; Mon, 1 Nov 2021 10:57:20 +0100 (CET) Received: from AZHDRRW-EX02.NVIDIA.COM (azhdrrw-ex02.nvidia.com [20.64.145.131]) by mails.dpdk.org (Postfix) with ESMTP id 62C2640040 for ; Mon, 1 Nov 2021 10:57:18 +0100 (CET) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.177) by mxs.oss.nvidia.com (10.13.234.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.858.15; Mon, 1 Nov 2021 02:57:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eN2JVdcgE77Mq4IPNE8GOOv81fZn87fRBNJTveK30U3VQUt/LIAhs3JIGmfrHGk00YlI6hlBz5tbmov7ESRLYrDA/2KUt5M1BUy7fwwpB98guyiZ5Wtehd07jlN/4ZbT+4A9QsV2kKgI0jSM2r8uTmQMLJ2kLcCKnA6ikpO5Gtgsy6aQkd3Zy0dltUjRTkn88V56fxzJygwGVICwOma8EdoyE+h0wPfiMSObLqVyVQk1dLcEJSyqD5O5E21JG9M0RwlJ1Foa+3JYvYuNOzqdBfciMj1PcTQmwOUnEWt0UOdicWiEvMJIzgCOb3JbkdEJe6XrI8vDW6Gt+TZtrDDSFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BDklAR6DRIInHQ8Git9RDMJ8+uHOix4UK2vqsGlS7VI=; b=PyVqr4o+wCtrwhpJ0d6hV04Zresu2cPnO3o8/XwhpT2tjci7Fz8/lXftWkR659TaKR2CAcfYi3PApHW9Y0XcduWsKHLms5M6R7jlsX0re4k0/v6O4Yn7V9p3AClGjVcpxkvPOab3Xm1YnzZG1VN8NCE8xVXRYoko6qHGw/Dk9E93yIJNvIOShGXHCQ0VY7xm/W46HX9+1K+JX4H1u57DV0Xq+ZNXuPYfzXWEgOrPAdfAiYNxOIuCUKNWBEy0QIc0Ftixesp/pBKYKIxLkMYZQ+7oeoANs66W8HQVQheGh9YmMYQZM9oBbWwwvGQc/OmKyzGAeDbbwCWkNngRYECqSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BDklAR6DRIInHQ8Git9RDMJ8+uHOix4UK2vqsGlS7VI=; b=hbrAoLNCq/6sqBuiLGaaie7450ZCEJ2xgpoER2bK601EQI3eBDANg5nqPtub+Gq1yY+WkXpzNaO1KUA/cZBB2QbWm0skYDnviAq7EHf4y36WfZD+GonPpe5TErr8Hle/t0+wWuWXJGEcMMS9G5z84EXtygp6m2EdWAbhkonq3tFlwD+3NLoE4rEsfCS/iBM8mqDJHAcJRWNfaO9RH5jL9zvdgrZcp/eptkzzi2kTCTOOgXO41/FZVF1DS9HNxfaxDoWm18Egv/AVTtMQdY3aBAtP90zaMSR9hZuM0ChM50qEpmTq4yt9AYmgkb9T51yEsQP45zbKyMIftCfYwiE1OA== Received: from BN6PR11CA0004.namprd11.prod.outlook.com (2603:10b6:405:2::14) by CY4PR12MB1336.namprd12.prod.outlook.com (2603:10b6:903:40::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.18; Mon, 1 Nov 2021 09:57:16 +0000 Received: from BN8NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:405:2:cafe::36) by BN6PR11CA0004.outlook.office365.com (2603:10b6:405:2::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.15 via Frontend Transport; Mon, 1 Nov 2021 09:57:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT030.mail.protection.outlook.com (10.13.177.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4649.14 via Frontend Transport; Mon, 1 Nov 2021 09:57:15 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 1 Nov 2021 09:57:12 +0000 From: Dmitry Kozlyuk To: CC: Raslan Darawsheh , Matan Azrad , Viacheslav Ovsiienko Date: Mon, 1 Nov 2021 11:56:46 +0200 Message-ID: <20211101095646.840095-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f99cea13-71c0-4532-5890-08d99d1dfc27 X-MS-TrafficTypeDiagnostic: CY4PR12MB1336: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2582; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q/xi6gOzX88rxJDysRAYRlLYT52tSmfGsDkughrgD2DEb9rqOFwH+GE6+b6jWCjMavoDlcj5lTvpSAps8RkyftZR2gRj7Md1JfYSrFcdf+qAd/96+nrKhTBoWCiCjMXmUJuWIQTeWnY+uyPXRFIe/4zVT3FR6jydT7cL7mlrofkC+dKGfC9O5UuEDFj2x8fKgvatVgO7AXsTnZoclDbz112bA5yxPAghStRMFT3Lhnppvn6P1F2wA5GTDHFbOEHYN/iRBpuXjmCIH6Zli+pIYhzpRcYX57xlg/joBWPUklI5yUD6jPFvaMOD6GLMVC3I2AMQnSWqnO/h0HmPEEFEtqM55LiQJ4yP1fTL1OoGdmqWxcjaX3993psIgZovm5A9fs+LLyUxkE5wL7L1/0xgKk/BxdUcSJ9FbPWzQ3kFjgzYbuDVhnNpNBey0yF+dSjmk5E8cNKZrvnWt+7q/YJ7esSStjjCNZcoU48ZAL/M+BkTumSLd3gbjm+v1FHJmds5NXIhJxttzu6JmnP9uHaA+xT7ip1+bJT6Zp+v6shjWQ00F6CNZT2D2toIWaInWNm/5k6OcxFVQGULqxgN4SakoWmy3LmJinuq93t7rbxmhug7/sDIepiDVd5xemf6zDP/pXpIqQZRVp1frazANgasd/IXEyMjPMchKX0blG7KZKeM436JHCxDbJN5hu2SrfkwKlvHzSAmL/S/5BNpG+PAstmaf6hOqcU6878/8TLko18= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(316002)(70206006)(5660300002)(83380400001)(7636003)(356005)(7696005)(70586007)(6916009)(36906005)(8936002)(55016002)(36756003)(36860700001)(8676002)(6286002)(1076003)(508600001)(6666004)(47076005)(54906003)(86362001)(16526019)(4326008)(107886003)(426003)(26005)(2616005)(336012)(2906002)(82310400003)(186003)(83323001); DIR:OUT; SFP:1101; X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2021 09:57:15.7786 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f99cea13-71c0-4532-5890-08d99d1dfc27 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1336 Subject: [dpdk-dev] [PATCH] net/mlx5: fix split buffer Rx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Routine to lookup LKey on Rx was assuming that the mbuf address always belongs to a single mempool: the one associated with an RxQ or the MPRQ mempool. This assumption is false for split buffers case. A wrong LKey was looked up, resulting in completion errors. Modify lookup routines to lookup LKey in the mbuf->pool for non-MPRQ cases both on Rx datapath and on queue initialization. Fixes: fec28ca0e3a9 ("net/mlx5: support mempool registration") Signed-off-by: Dmitry Kozlyuk Reviewed-by: Matan Azrad Reviewed-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rx.c | 5 ++++- drivers/net/mlx5/mlx5_rx.h | 37 +++++++++++++++++++++++++++++++++++-- 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c index 258a645314..7296e81534 100644 --- a/drivers/net/mlx5/mlx5_rx.c +++ b/drivers/net/mlx5/mlx5_rx.c @@ -356,6 +356,7 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) volatile struct mlx5_wqe_data_seg *scat; uintptr_t addr; uint32_t byte_count; + uint32_t lkey; if (mlx5_rxq_mprq_enabled(rxq)) { struct mlx5_mprq_buf *buf = (*rxq->mprq_bufs)[i]; @@ -366,6 +367,7 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) 1 << rxq->strd_num_n); byte_count = (1 << rxq->strd_sz_n) * (1 << rxq->strd_num_n); + lkey = mlx5_rx_addr2mr(rxq, addr); } else { struct rte_mbuf *buf = (*rxq->elts)[i]; @@ -373,13 +375,14 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) rxq->wqes)[i]; addr = rte_pktmbuf_mtod(buf, uintptr_t); byte_count = DATA_LEN(buf); + lkey = mlx5_rx_mb2mr(rxq, buf); } /* scat->addr must be able to store a pointer. */ MLX5_ASSERT(sizeof(scat->addr) >= sizeof(uintptr_t)); *scat = (struct mlx5_wqe_data_seg){ .addr = rte_cpu_to_be_64(addr), .byte_count = rte_cpu_to_be_32(byte_count), - .lkey = mlx5_rx_addr2mr(rxq, addr), + .lkey = lkey, }; } rxq->consumed_strd = 0; diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index 4952fe1455..5903776e7a 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -266,7 +266,7 @@ uint16_t mlx5_rx_burst_mprq_vec(void *dpdk_rxq, struct rte_mbuf **pkts, static int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq); /** - * Query LKey from a packet buffer for Rx. No need to flush local caches + * Query LKey for an address on Rx. No need to flush local caches * as the Rx mempool database entries are valid for the lifetime of the queue. * * @param rxq @@ -301,7 +301,40 @@ mlx5_rx_addr2mr(struct mlx5_rxq_data *rxq, uintptr_t addr) mr_ctrl, mp, addr); } -#define mlx5_rx_mb2mr(rxq, mb) mlx5_rx_addr2mr(rxq, (uintptr_t)((mb)->buf_addr)) +/** + * Query LKey from a packet buffer for Rx. No need to flush local caches + * as the Rx mempool database entries are valid for the lifetime of the queue. + * + * @param rxq + * Pointer to Rx queue structure. + * @param mb + * Buffer to search the address of. + * + * @return + * Searched LKey on success, UINT32_MAX on no match. + * This function always succeeds on valid input. + */ +static __rte_always_inline uint32_t +mlx5_rx_mb2mr(struct mlx5_rxq_data *rxq, struct rte_mbuf *mb) +{ + struct mlx5_mr_ctrl *mr_ctrl = &rxq->mr_ctrl; + uintptr_t addr = (uintptr_t)mb->buf_addr; + struct mlx5_rxq_ctrl *rxq_ctrl; + uint32_t lkey; + + /* Linear search on MR cache array. */ + lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, + MLX5_MR_CACHE_N, addr); + if (likely(lkey != UINT32_MAX)) + return lkey; + /* + * Slower search in the mempool database on miss. + * During queue creation rxq->sh is not yet set, so we use rxq_ctrl. + */ + rxq_ctrl = container_of(rxq, struct mlx5_rxq_ctrl, rxq); + return mlx5_mr_mempool2mr_bh(&rxq_ctrl->priv->sh->cdev->mr_scache, + mr_ctrl, mb->pool, addr); +} /** * Convert timestamp from HW format to linear counter -- 2.25.1