From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 19B1DA0C4D; Mon, 8 Nov 2021 12:17:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9201C40E03; Mon, 8 Nov 2021 12:17:36 +0100 (CET) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2088.outbound.protection.outlook.com [40.107.101.88]) by mails.dpdk.org (Postfix) with ESMTP id ECB2140151 for ; Mon, 8 Nov 2021 12:17:34 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YdNbwR3fNDetOYE3GfAFYkp7u4397xtSZ7vwAAFTrqmDJa8IglnbMAc7COiQkFtX9P+nAPNusqetcJ5ZIZ21IQ5KogD4jz05lY+SHFcJAQrilE2hKwJV1vj/brMngPE+bMuxr+QZcP4jJq80qmxU0Xm60uifdKrR6pNl5GLcCnsDlQGs1FICB2p0smxMNZRL9bNESg8iaczpqhrO4CCMlWN37e98xOmkuyVG5roU4gqBTc21H3nePwoDvwaEGnsmmtPtGD3HQ5C7ISHIJaaPrFvfBB2798VdZ1PuYrjr+gHO0/sUHyagcP3MlMaJXe6x+lg5Ihj+OGxJr5QSE3IuHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xYEtqt3cANn0znb+wcv5sKb2BwY5GcFSC46gPqB1NnQ=; b=Y6ZnceUF/R1VFTVFKWnTWvOy2kp4YZdBOK8NBv30kNMegn5Vx4odJW9MEvoRbDco1qWfqgtJ1xQvxfGu4N9/gy9zucRiGCF4RoNOKVymg3yY/voPlSZVRvTwE3xytiruSjvIMYVFFFuw/gH1w7SjgYDKrvCSCIWhiXW4mlG96HVKQECs43Ch7kzwVD4EXaKrav/+MSdgBcGzEkAAxeWFGLrXfhM8lDUW0wBh7WZBfxr3C2G2P3zamqOjlUJ957GTgGXH2qepAfxGZoOMO8VUIpW9/nkHWqye/XoCCz++ZYLpQ45B/T5xniQmWo2xSC7i7B682Bl7qkwrze3U4Jadbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xYEtqt3cANn0znb+wcv5sKb2BwY5GcFSC46gPqB1NnQ=; b=c1sF5Z5Vu5OpXfE7rZ2vdV0KdHX75YwLrU7yvUfELRdM/Id7OqcVa07Is3vdrqjTP+yUaEVryIz10B0toYp3YzNT3QOlakkNYO7CTRR5CpNOSJoWUn+EwHMbvsCrbcbp26P9zm846uKF+wc/dxMrloEuc3AtGyYq2JGCtGyKOmc9ONttLD/XQZQdTh85BUTK09lxsTWnNktIKSUOGj1D39VZr7ljGMKc/KJyQFBVFog26ctS7SESLFc6T30/uMy7EIG/qCGPXUxkH/jBgJRSkV5M0/+HE4fnwB8g2p0patFfJ6KUPlRgMjabqT3QIstR8CockFSSHi3P/0lY1y2oUg== Received: from DM3PR12CA0043.namprd12.prod.outlook.com (2603:10b6:0:56::11) by BYAPR12MB4776.namprd12.prod.outlook.com (2603:10b6:a03:10d::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11; Mon, 8 Nov 2021 11:17:32 +0000 Received: from DM6NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:0:56:cafe::e2) by DM3PR12CA0043.outlook.office365.com (2603:10b6:0:56::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 11:17:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT031.mail.protection.outlook.com (10.13.172.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 11:17:31 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 11:17:29 +0000 From: Dmitry Kozlyuk To: CC: Raslan Darawsheh , Matan Azrad , Viacheslav Ovsiienko Date: Mon, 8 Nov 2021 13:17:15 +0200 Message-ID: <20211108111716.1311979-1-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211101095646.840095-1-dkozlyuk@nvidia.com> References: <20211101095646.840095-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 891b3840-4a3b-4fa5-237b-08d9a2a95b53 X-MS-TrafficTypeDiagnostic: BYAPR12MB4776: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2582; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: f+36lt7LPVNzI9F9CgvqVMgDe4VvYQ66uFuNnhDVSPNMoBKX0j9XX4VcU92xC+FIIL9tUnrmx/bSKEGyjAs2JLwc8polHM9OIH/pZ5N9ZliWhQvk67c6Olm6y0tpti00+GnalN4NZyX6mBsR4f5+58r9QBfRHaMiL5ZTi2VJpzC06NHbKUfXIF832Prm55C+5QTXdrvrusZJfXg6Ik9+1Tt2jZkxmJKYdw7magHeNCJWg5wtV6nZ2ZrysNAfKEQWh7qB37fGKudrM0Iw+eTXNEpaMUw3JUbDevMW3Kwo9jLoqo7Swnhqr/0WgiICiG0dj0q1DSS2bT5XGH/yAAV0exOlD9uKQ+okHD4QBF8isvUColAoB3Y5ZGW7QMSO2k8DKBMjBH5Tz3ea/hILuYLwfB83lc5vQDwk04k5edfin73fA3mANIjK/f8+XuupSPDNIBAFcwBMz4lNoAYIg1J4NZ+b8PID0cxkDR4SXmnssLs60HFfB5X4dtfhoTUITRQE1eMPPK5ur4RXPHZLmLq/vpxHSoKuhDYqwvNRisEzLkzyMd0bAweS2pfiGwAY7+sFbBh8y9Njz+Pth7HrAO4KRIQoMLsiC17DEMWadOC81jcum7GKQPlhL7pmLnL4J244/YyJgKYpBSao9RLX/iU3rm0geq6MvHv/DXIU7Zv0Xu96ShnKe1X0BJs619lJdlx60aYROaJunZbqGnYtGlcTnHVQG+NuP0pHNOgNwosnYoA= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(54906003)(7696005)(336012)(36860700001)(55016002)(86362001)(16526019)(186003)(8936002)(2616005)(6916009)(426003)(36756003)(82310400003)(26005)(316002)(70586007)(70206006)(508600001)(83380400001)(47076005)(8676002)(6666004)(356005)(7636003)(1076003)(2906002)(107886003)(6286002)(4326008)(5660300002)(83323001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 11:17:31.3894 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 891b3840-4a3b-4fa5-237b-08d9a2a95b53 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB4776 Subject: [dpdk-dev] [PATCH v2] net/mlx5: fix split buffer Rx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Routine to lookup LKey on Rx was assuming that the mbuf address always belongs to a single mempool: the one associated with an RxQ or the MPRQ mempool. This assumption is false for split buffers case. A wrong LKey was looked up, resulting in completion errors. Modify lookup routines to lookup LKey in the mbuf->pool for non-MPRQ cases both on Rx datapath and on queue initialization. Fixes: fec28ca0e3a9 ("net/mlx5: support mempool registration") Signed-off-by: Dmitry Kozlyuk Reviewed-by: Matan Azrad Reviewed-by: Viacheslav Ovsiienko --- v2: rebase on next-net-mlx drivers/net/mlx5/mlx5_rx.c | 5 ++++- drivers/net/mlx5/mlx5_rx.h | 37 +++++++++++++++++++++++++++++++++++-- 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c index 4d85f64acc..e8215f7381 100644 --- a/drivers/net/mlx5/mlx5_rx.c +++ b/drivers/net/mlx5/mlx5_rx.c @@ -347,6 +347,7 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) volatile struct mlx5_wqe_data_seg *scat; uintptr_t addr; uint32_t byte_count; + uint32_t lkey; if (mlx5_rxq_mprq_enabled(rxq)) { struct mlx5_mprq_buf *buf = (*rxq->mprq_bufs)[i]; @@ -357,6 +358,7 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) 1 << rxq->strd_num_n); byte_count = (1 << rxq->strd_sz_n) * (1 << rxq->strd_num_n); + lkey = mlx5_rx_addr2mr(rxq, addr); } else { struct rte_mbuf *buf = (*rxq->elts)[i]; @@ -364,13 +366,14 @@ mlx5_rxq_initialize(struct mlx5_rxq_data *rxq) rxq->wqes)[i]; addr = rte_pktmbuf_mtod(buf, uintptr_t); byte_count = DATA_LEN(buf); + lkey = mlx5_rx_mb2mr(rxq, buf); } /* scat->addr must be able to store a pointer. */ MLX5_ASSERT(sizeof(scat->addr) >= sizeof(uintptr_t)); *scat = (struct mlx5_wqe_data_seg){ .addr = rte_cpu_to_be_64(addr), .byte_count = rte_cpu_to_be_32(byte_count), - .lkey = mlx5_rx_addr2mr(rxq, addr), + .lkey = lkey, }; } rxq->consumed_strd = 0; diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index 19d194ce12..298d1b1f32 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -291,7 +291,7 @@ uint16_t mlx5_rx_burst_mprq_vec(void *dpdk_rxq, struct rte_mbuf **pkts, static int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq); /** - * Query LKey from a packet buffer for Rx. No need to flush local caches + * Query LKey for an address on Rx. No need to flush local caches * as the Rx mempool database entries are valid for the lifetime of the queue. * * @param rxq @@ -320,7 +320,40 @@ mlx5_rx_addr2mr(struct mlx5_rxq_data *rxq, uintptr_t addr) mp, addr); } -#define mlx5_rx_mb2mr(rxq, mb) mlx5_rx_addr2mr(rxq, (uintptr_t)((mb)->buf_addr)) +/** + * Query LKey from a packet buffer for Rx. No need to flush local caches + * as the Rx mempool database entries are valid for the lifetime of the queue. + * + * @param rxq + * Pointer to Rx queue structure. + * @param mb + * Buffer to search the address of. + * + * @return + * Searched LKey on success, UINT32_MAX on no match. + * This function always succeeds on valid input. + */ +static __rte_always_inline uint32_t +mlx5_rx_mb2mr(struct mlx5_rxq_data *rxq, struct rte_mbuf *mb) +{ + struct mlx5_mr_ctrl *mr_ctrl = &rxq->mr_ctrl; + uintptr_t addr = (uintptr_t)mb->buf_addr; + struct mlx5_rxq_ctrl *rxq_ctrl; + uint32_t lkey; + + /* Linear search on MR cache array. */ + lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, + MLX5_MR_CACHE_N, addr); + if (likely(lkey != UINT32_MAX)) + return lkey; + /* + * Slower search in the mempool database on miss. + * During queue creation rxq->sh is not yet set, so we use rxq_ctrl. + */ + rxq_ctrl = container_of(rxq, struct mlx5_rxq_ctrl, rxq); + return mlx5_mr_mempool2mr_bh(&rxq_ctrl->sh->cdev->mr_scache, + mr_ctrl, mb->pool, addr); +} /** * Convert timestamp from HW format to linear counter -- 2.25.1