From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2D18BA0C41; Tue, 19 Oct 2021 22:58:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E62CB4118D; Tue, 19 Oct 2021 22:57:21 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2044.outbound.protection.outlook.com [40.107.94.44]) by mails.dpdk.org (Postfix) with ESMTP id 6F45D40150 for ; Tue, 19 Oct 2021 22:56:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WKxMTt8nsbUNZLZ7ipFIQ0Hh+RC06ItEthdjpmn272qYcYJdHSCdJ5g+aeFhQu6uHjFYTNzDJgLddKOXWCNpKmNg9jeOgNd7oAjKntLgGV+7DisgCCkh8H2zu8ikvK7XVWexxiz5ozk5vLD5X7HEoH9cBuGk7XhAGjtgvLqi4ikBqrjMnNevCdCGyn9KdWefS54H367MXRY7sjrDHD9LzRtOlSOgkSOrVS9K1Slw4pohMCCWZ7pfgQwu7Gqv6dqTh0jfYtg5qxz4zEcMdM/bPfaj+4cJRh3JhO1yytxUZQ7EpZ27844QEUZ2v27I9CZ6YFtX/YE26d70irNC39871w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GnJvoj0b0e/0IV/FCoxWJ492AEP6zRbJp6UlRU2ck8g=; b=MiIB7fOhCn//sGJwSi6EXA5xLt1ay0vnE27ag0iId2GbB56J3/CFOeOwG2yjCvvAKiUJQQ76rjdNjDk1YXNz4aL1HKqdcYT4xTeW5BdMZ2/D7mt8Y5olNNRU01w54c6EIpqePpZ81GfdocBEdwNxRfGtT5ZejrEtFR+bc9/qSB1KdriWHLv/4auvSjBg0HdJa9x/laezEsWWXRKROFHjaJmps66Ve6Q3ZE9DiukfuFy1d5Cn+BRNkkzVqj6kq6dIrn+x1+KZ1kRSWVJNMnu3M+5p3MHuTEzjjYEIr2m61w5BMa8KLzao+0Yy2o0o2ZqLJbLt3qQU1xTs7ma3oOliDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GnJvoj0b0e/0IV/FCoxWJ492AEP6zRbJp6UlRU2ck8g=; b=iNQE89kyuVjfwvkqSOml8CvXzTCS2iO56Bgez+93YloozhPmLNeCtbsU7Y/NJ53VqFz7JCUYqBALPimcS4K1OXuN31TsCFOBJXZmrnv3OMavTg/chyxNBBkIWPmz21/QqN1vyVdJMNk8OXzVjk1Lz8Qixb1sBoOwEMKO3yx+lCC2iC1S1+tFVGH7xM/+Ol+oNw4hkowF8d3+F5ic4YUdCke5yqAHKjifBRhfAW75wT7P0eyQZiHFAyCSaZGtuGp1q3sirKR93UWpuS+XtvCKTVZxk7vtHcKzmrm++ojY8qzEVOyx8zuPCuFGrBlZUPkKZPBxXS15IZ7tYuwjqytp2Q== Received: from BN0PR02CA0013.namprd02.prod.outlook.com (2603:10b6:408:e4::18) by DM8PR12MB5398.namprd12.prod.outlook.com (2603:10b6:8:3f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.16; Tue, 19 Oct 2021 20:56:57 +0000 Received: from BN8NAM11FT033.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e4:cafe::12) by BN0PR02CA0013.outlook.office365.com (2603:10b6:408:e4::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.15 via Frontend Transport; Tue, 19 Oct 2021 20:56:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT033.mail.protection.outlook.com (10.13.177.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Tue, 19 Oct 2021 20:56:56 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 19 Oct 2021 20:56:48 +0000 From: To: CC: Matan Azrad , Thomas Monjalon , Michael Baum Date: Tue, 19 Oct 2021 23:55:59 +0300 Message-ID: <20211019205602.3188203-16-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211019205602.3188203-1-michaelba@nvidia.com> References: <20211006220350.2357487-1-michaelba@nvidia.com> <20211019205602.3188203-1-michaelba@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 37f8b758-df62-48fa-c203-08d99342fd31 X-MS-TrafficTypeDiagnostic: DM8PR12MB5398: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3513; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iEOg6p7QCEUe/decGAUPSehJNDzIGfstA4ihx+oNQGYVaN8lDoJJ0tJ5qp5dKHjpCs4bsICc5CYH/HXSrWE/jpiFcDe4c+iWrLaC/qJuRRtQPDMTHQ9w6wy3b7AOb4vRUD5gsh2/6tk7dJcXZujB/jvvZBdlmAJhEmmxpRw/eWoc7nSh065QpgsupXiX/cxKZiUI9PygyCf9Sb4915S/fVn5c6p0RWda2xwDvKciLvzcix7NXOz+HI8wMWmCthVjNq636/icMOfb0kCarYbTCvYQe0L/PQWNMtK+3gFgtB+asNPSQf6qfj46P9K5Q3fWY6moKJra41djSPIgIE27vVwY6oGLRAWd/arzaOaJWC/X5oTOlJEAUm9fX2IN+1NrngY0HOwArsh59CKn+AqZT1OKSRQDuUlFG+Nyp+OcXEmHN3xs4xlea/jbqKwgWyJeXJeRYTbB88uIM33+e9smkZh9bXvVPoPU2XXpWPj4b6UZyBZdf9nL33JT1jm8+f7lc5uVXAX3IS3Xn81dujHb3doWGLVnAqjVUzjRSbGeTmBwP0LcRiIV8M0a+s+s872OSocDCe5BlpOObjwffI3X+2pVA84Ep46Z8lax2+Wc7NbkzfIVmYlSi47a61569MB/jrVb9xSv788NaWPVzNkK0UitvWPxI5hzwk0KtAfjZ6FSrqn2sLr6sooCbUK9KYdloXj2+zXr9/pWvQheUewuJA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(70586007)(2616005)(316002)(6666004)(54906003)(336012)(6916009)(107886003)(6286002)(7636003)(36860700001)(70206006)(2906002)(83380400001)(47076005)(356005)(82310400003)(86362001)(2876002)(186003)(426003)(30864003)(26005)(7696005)(5660300002)(36756003)(36906005)(508600001)(8936002)(55016002)(4326008)(16526019)(1076003)(8676002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2021 20:56:56.9899 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 37f8b758-df62-48fa-c203-08d99342fd31 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT033.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5398 Subject: [dpdk-dev] [PATCH v3 15/18] common/mlx5: share MR top-half search function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Michael Baum Add function to search in local liniar cache and use it in the drivers instead of their functions. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common.h | 9 ++++ drivers/common/mlx5/mlx5_common_mr.c | 52 ++++++++++++++++++++++++ drivers/common/mlx5/version.map | 1 + drivers/compress/mlx5/mlx5_compress.c | 38 +---------------- drivers/crypto/mlx5/mlx5_crypto.c | 38 +---------------- drivers/regex/mlx5/mlx5_regex_fastpath.c | 34 +++++----------- 6 files changed, 77 insertions(+), 95 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common.h b/drivers/common/mlx5/mlx5_common.h index a863fb2b26..8df4f32aa2 100644 --- a/drivers/common/mlx5/mlx5_common.h +++ b/drivers/common/mlx5/mlx5_common.h @@ -22,6 +22,7 @@ #include "mlx5_prm.h" #include "mlx5_devx_cmds.h" #include "mlx5_common_os.h" +#include "mlx5_common_mr.h" /* Reported driver name. */ #define MLX5_PCI_DRIVER_NAME "mlx5_pci" @@ -447,6 +448,14 @@ __rte_internal bool mlx5_dev_is_pci(const struct rte_device *dev); +/* mlx5_common_mr.c */ + +__rte_internal +uint32_t +mlx5_mr_mb2mr(struct mlx5_common_device *cdev, struct mlx5_mp_id *mp_id, + struct mlx5_mr_ctrl *mr_ctrl, struct rte_mbuf *mbuf, + struct mlx5_mr_share_cache *share_cache); + /* mlx5_common_os.c */ int mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t classes); diff --git a/drivers/common/mlx5/mlx5_common_mr.c b/drivers/common/mlx5/mlx5_common_mr.c index 43dc3d88ce..4de1c25f2a 100644 --- a/drivers/common/mlx5/mlx5_common_mr.c +++ b/drivers/common/mlx5/mlx5_common_mr.c @@ -576,6 +576,8 @@ mr_find_contig_memsegs_cb(const struct rte_memseg_list *msl, * * @param pd * Pointer to pd of a device (net, regex, vdpa,...). + * @param mp_id + * Multi-process identifier, may be NULL for the primary process. * @param share_cache * Pointer to a global shared MR cache. * @param[out] entry @@ -867,6 +869,8 @@ mlx5_mr_create_primary(void *pd, * * @param pd * Pointer to pd handle of a device (net, regex, vdpa,...). + * @param mp_id + * Multi-process identifier, may be NULL for the primary process. * @param share_cache * Pointer to a global shared MR cache. * @param[out] entry @@ -874,6 +878,8 @@ mlx5_mr_create_primary(void *pd, * created. If failed to create one, this will not be updated. * @param addr * Target virtual address to register. + * @param mr_ext_memseg_en + * Configurable flag about external memory segment enable or not. * * @return * Searched LKey on success, UINT32_MAX on failure and rte_errno is set. @@ -907,6 +913,8 @@ mlx5_mr_create(void *pd, struct mlx5_mp_id *mp_id, * * @param pd * Pointer to pd of a device (net, regex, vdpa,...). + * @param mp_id + * Multi-process identifier, may be NULL for the primary process. * @param share_cache * Pointer to a global shared MR cache. * @param mr_ctrl @@ -916,6 +924,8 @@ mlx5_mr_create(void *pd, struct mlx5_mp_id *mp_id, * created. If failed to create one, this is not written. * @param addr * Search key. + * @param mr_ext_memseg_en + * Configurable flag about external memory segment enable or not. * * @return * Searched LKey on success, UINT32_MAX on no match. @@ -971,12 +981,16 @@ mr_lookup_caches(void *pd, struct mlx5_mp_id *mp_id, * * @param pd * Pointer to pd of a device (net, regex, vdpa,...). + * @param mp_id + * Multi-process identifier, may be NULL for the primary process. * @param share_cache * Pointer to a global shared MR cache. * @param mr_ctrl * Pointer to per-queue MR control structure. * @param addr * Search key. + * @param mr_ext_memseg_en + * Configurable flag about external memory segment enable or not. * * @return * Searched LKey on success, UINT32_MAX on no match. @@ -1822,3 +1836,41 @@ mlx5_mr_mempool2mr_bh(struct mlx5_mr_share_cache *share_cache, mr_ctrl->head = (mr_ctrl->head + 1) % MLX5_MR_CACHE_N; return lkey; } + +/** + * Query LKey from a packet buffer. + * + * @param cdev + * Pointer to the mlx5 device structure. + * @param mp_id + * Multi-process identifier, may be NULL for the primary process. + * @param mr_ctrl + * Pointer to per-queue MR control structure. + * @param mbuf + * Pointer to mbuf. + * @param share_cache + * Pointer to a global shared MR cache. + * + * @return + * Searched LKey on success, UINT32_MAX on no match. + */ +uint32_t +mlx5_mr_mb2mr(struct mlx5_common_device *cdev, struct mlx5_mp_id *mp_id, + struct mlx5_mr_ctrl *mr_ctrl, struct rte_mbuf *mbuf, + struct mlx5_mr_share_cache *share_cache) +{ + uint32_t lkey; + uintptr_t addr = (uintptr_t)mbuf->buf_addr; + + /* Check generation bit to see if there's any change on existing MRs. */ + if (unlikely(*mr_ctrl->dev_gen_ptr != mr_ctrl->cur_gen)) + mlx5_mr_flush_local_cache(mr_ctrl); + /* Linear search on MR cache array. */ + lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, + MLX5_MR_CACHE_N, (uintptr_t)mbuf->buf_addr); + if (likely(lkey != UINT32_MAX)) + return lkey; + /* Take slower bottom-half on miss. */ + return mlx5_mr_addr2mr_bh(cdev->pd, mp_id, share_cache, mr_ctrl, + addr, cdev->config.mr_ext_memseg_en); +} diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index abe5c12cd8..292c5ede89 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -118,6 +118,7 @@ INTERNAL { mlx5_mr_insert_cache; mlx5_mr_lookup_cache; mlx5_mr_lookup_list; + mlx5_mr_mb2mr; mlx5_free_mr_by_addr; mlx5_mr_rebuild_cache; mlx5_mr_release_cache; diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c index c36db0c062..a5cec27894 100644 --- a/drivers/compress/mlx5/mlx5_compress.c +++ b/drivers/compress/mlx5/mlx5_compress.c @@ -435,40 +435,6 @@ static struct rte_compressdev_ops mlx5_compress_ops = { .stream_free = NULL, }; -/** - * Query LKey from a packet buffer for QP. If not found, add the mempool. - * - * @param priv - * Pointer to the priv object. - * @param addr - * Search key. - * @param mr_ctrl - * Pointer to per-queue MR control structure. - * @param ol_flags - * Mbuf offload features. - * - * @return - * Searched LKey on success, UINT32_MAX on no match. - */ -static __rte_always_inline uint32_t -mlx5_compress_addr2mr(struct mlx5_compress_priv *priv, uintptr_t addr, - struct mlx5_mr_ctrl *mr_ctrl, uint64_t ol_flags) -{ - uint32_t lkey; - - /* Check generation bit to see if there's any change on existing MRs. */ - if (unlikely(*mr_ctrl->dev_gen_ptr != mr_ctrl->cur_gen)) - mlx5_mr_flush_local_cache(mr_ctrl); - /* Linear search on MR cache array. */ - lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, - MLX5_MR_CACHE_N, addr); - if (likely(lkey != UINT32_MAX)) - return lkey; - /* Take slower bottom-half on miss. */ - return mlx5_mr_addr2mr_bh(priv->cdev->pd, 0, &priv->mr_scache, mr_ctrl, - addr, !!(ol_flags & EXT_ATTACHED_MBUF)); -} - static __rte_always_inline uint32_t mlx5_compress_dseg_set(struct mlx5_compress_qp *qp, volatile struct mlx5_wqe_dseg *restrict dseg, @@ -478,8 +444,8 @@ mlx5_compress_dseg_set(struct mlx5_compress_qp *qp, uintptr_t addr = rte_pktmbuf_mtod_offset(mbuf, uintptr_t, offset); dseg->bcount = rte_cpu_to_be_32(len); - dseg->lkey = mlx5_compress_addr2mr(qp->priv, addr, &qp->mr_ctrl, - mbuf->ol_flags); + dseg->lkey = mlx5_mr_mb2mr(qp->priv->cdev, 0, &qp->mr_ctrl, mbuf, + &qp->priv->mr_scache); dseg->pbuf = rte_cpu_to_be_64(addr); return dseg->lkey; } diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index 6cf6889d21..1105d3fcd5 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -303,40 +303,6 @@ mlx5_crypto_get_block_size(struct rte_crypto_op *op) } } -/** - * Query LKey from a packet buffer for QP. If not found, add the mempool. - * - * @param priv - * Pointer to the priv object. - * @param addr - * Search key. - * @param mr_ctrl - * Pointer to per-queue MR control structure. - * @param ol_flags - * Mbuf offload features. - * - * @return - * Searched LKey on success, UINT32_MAX on no match. - */ -static __rte_always_inline uint32_t -mlx5_crypto_addr2mr(struct mlx5_crypto_priv *priv, uintptr_t addr, - struct mlx5_mr_ctrl *mr_ctrl, uint64_t ol_flags) -{ - uint32_t lkey; - - /* Check generation bit to see if there's any change on existing MRs. */ - if (unlikely(*mr_ctrl->dev_gen_ptr != mr_ctrl->cur_gen)) - mlx5_mr_flush_local_cache(mr_ctrl); - /* Linear search on MR cache array. */ - lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, - MLX5_MR_CACHE_N, addr); - if (likely(lkey != UINT32_MAX)) - return lkey; - /* Take slower bottom-half on miss. */ - return mlx5_mr_addr2mr_bh(priv->cdev->pd, 0, &priv->mr_scache, mr_ctrl, - addr, !!(ol_flags & EXT_ATTACHED_MBUF)); -} - static __rte_always_inline uint32_t mlx5_crypto_klm_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp, struct rte_mbuf *mbuf, struct mlx5_wqe_dseg *klm, @@ -350,8 +316,8 @@ mlx5_crypto_klm_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp, *remain -= data_len; klm->bcount = rte_cpu_to_be_32(data_len); klm->pbuf = rte_cpu_to_be_64(addr); - klm->lkey = mlx5_crypto_addr2mr(priv, addr, &qp->mr_ctrl, - mbuf->ol_flags); + klm->lkey = mlx5_mr_mb2mr(priv->cdev, 0, &qp->mr_ctrl, mbuf, + &priv->mr_scache); return klm->lkey; } diff --git a/drivers/regex/mlx5/mlx5_regex_fastpath.c b/drivers/regex/mlx5/mlx5_regex_fastpath.c index 575b639752..8817e2e074 100644 --- a/drivers/regex/mlx5/mlx5_regex_fastpath.c +++ b/drivers/regex/mlx5/mlx5_regex_fastpath.c @@ -123,26 +123,12 @@ set_wqe_ctrl_seg(struct mlx5_wqe_ctrl_seg *seg, uint16_t pi, uint8_t opcode, * Searched LKey on success, UINT32_MAX on no match. */ static inline uint32_t -mlx5_regex_addr2mr(struct mlx5_regex_priv *priv, struct mlx5_mr_ctrl *mr_ctrl, - struct rte_mbuf *mbuf) +mlx5_regex_mb2mr(struct mlx5_regex_priv *priv, struct mlx5_mr_ctrl *mr_ctrl, + struct rte_mbuf *mbuf) { - uintptr_t addr = rte_pktmbuf_mtod(mbuf, uintptr_t); - uint32_t lkey; - - /* Check generation bit to see if there's any change on existing MRs. */ - if (unlikely(*mr_ctrl->dev_gen_ptr != mr_ctrl->cur_gen)) - mlx5_mr_flush_local_cache(mr_ctrl); - /* Linear search on MR cache array. */ - lkey = mlx5_mr_lookup_lkey(mr_ctrl->cache, &mr_ctrl->mru, - MLX5_MR_CACHE_N, addr); - if (likely(lkey != UINT32_MAX)) - return lkey; - /* Take slower bottom-half on miss. */ - return mlx5_mr_addr2mr_bh(priv->cdev->pd, 0, &priv->mr_scache, mr_ctrl, - addr, !!(mbuf->ol_flags & EXT_ATTACHED_MBUF)); + return mlx5_mr_mb2mr(priv->cdev, 0, mr_ctrl, mbuf, &priv->mr_scache); } - static inline void __prep_one(struct mlx5_regex_priv *priv, struct mlx5_regex_hw_qp *qp_obj, struct rte_regex_ops *op, struct mlx5_regex_job *job, @@ -194,7 +180,7 @@ prep_one(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, struct mlx5_klm klm; klm.byte_count = rte_pktmbuf_data_len(op->mbuf); - klm.mkey = mlx5_regex_addr2mr(priv, &qp->mr_ctrl, op->mbuf); + klm.mkey = mlx5_regex_mb2mr(priv, &qp->mr_ctrl, op->mbuf); klm.address = rte_pktmbuf_mtod(op->mbuf, uintptr_t); __prep_one(priv, qp_obj, op, job, qp_obj->pi, &klm); qp_obj->db_pi = qp_obj->pi; @@ -317,6 +303,7 @@ prep_regex_umr_wqe_set(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, uint32_t len = 0; struct mlx5_klm *mkey_klm = NULL; struct mlx5_klm klm; + uintptr_t addr; while (left_ops--) rte_prefetch0(op[left_ops]); @@ -360,11 +347,12 @@ prep_regex_umr_wqe_set(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, klm.mkey = rte_cpu_to_be_32 (qp->jobs[mkey_job_id].imkey->id); while (mbuf) { + addr = rte_pktmbuf_mtod(mbuf, uintptr_t); /* Build indirect mkey seg's KLM. */ - mkey_klm->mkey = mlx5_regex_addr2mr - (priv, &qp->mr_ctrl, mbuf); - mkey_klm->address = rte_cpu_to_be_64 - (rte_pktmbuf_mtod(mbuf, uintptr_t)); + mkey_klm->mkey = mlx5_regex_mb2mr(priv, + &qp->mr_ctrl, + mbuf); + mkey_klm->address = rte_cpu_to_be_64(addr); mkey_klm->byte_count = rte_cpu_to_be_32 (rte_pktmbuf_data_len(mbuf)); /* @@ -380,7 +368,7 @@ prep_regex_umr_wqe_set(struct mlx5_regex_priv *priv, struct mlx5_regex_qp *qp, klm.byte_count = scatter_size; } else { /* The single mubf case. Build the KLM directly. */ - klm.mkey = mlx5_regex_addr2mr(priv, &qp->mr_ctrl, mbuf); + klm.mkey = mlx5_regex_mb2mr(priv, &qp->mr_ctrl, mbuf); klm.address = rte_pktmbuf_mtod(mbuf, uintptr_t); klm.byte_count = rte_pktmbuf_data_len(mbuf); } -- 2.25.1