From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AC7EA034F; Wed, 31 Mar 2021 09:38:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D91C8140DF1; Wed, 31 Mar 2021 09:37:56 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by mails.dpdk.org (Postfix) with ESMTP id 8ED04140DF1 for ; Wed, 31 Mar 2021 09:37:55 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from suanmingm@nvidia.com) with SMTP; 31 Mar 2021 10:37:52 +0300 Received: from nvidia.com (mtbc-r640-03.mtbc.labs.mlnx [10.75.70.8]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 12V7boeE002108; Wed, 31 Mar 2021 10:37:51 +0300 From: Suanming Mou To: orika@nvidia.com Cc: dev@dpdk.org, viacheslavo@nvidia.com, matan@nvidia.com, rasland@nvidia.com Date: Wed, 31 Mar 2021 10:37:45 +0300 Message-Id: <20210331073749.1382377-1-suanmingm@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210309235732.3952418-1-suanmingm@nvidia.com> References: <20210309235732.3952418-1-suanmingm@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v4 0/4] regex/mlx5: support scattered mbuf X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The scattered mbuf was not supported in mlx5 RegEx driver. This patch set adds the support of scattered mbuf by UMR WQE. UMR(User-Mode Memory Registration) WQE can present data buffers scattered within multiple mbufs with single indirect mkey. Take advantage of the UMR WQE, scattered mbuf in one operation can be presented to an indirect mkey. The RegEx which only accepts one mkey can now process the whole scattered mbuf in one operation. The maximum scattered mbuf can be supported in one UMR WQE is now defined as 64. The mbufs from multiple operations can be combined into one UMR WQE as well if there is enough space in the KLM array, since the operations can address their own mbuf's content by the mkey's address and length. However, one operation's scattered mbuf's can't be placed in two different UMR WQE's KLM array, if the UMR WQE's KLM does not has enough free space for one operation, the extra UMR WQE will be engaged. In case the UMR WQE's indirect mkey will be over wrapped by the SQ's WQE move, the mkey's index used by the UMR WQE should be the index of last the RegEX WQE in the operations. As one operation consumes one WQE set, build the RegEx WQE by reverse helps address the mkey more efficiently. Once the operations in one burst consumes multiple mkeys, when the mkey KLM array is full, the reverse WQE set index will always be the last of the new mkey's for the new UMR WQE. In GGA mode, the SQ WQE's memory layout becomes UMR/NOP and RegEx WQE by interleave. The UMR and RegEx WQE can be called as WQE set. The SQ's pi and ci will also be increased as WQE set not as WQE. For operations don't have scattered mbuf, uses the mbuf's mkey directly, the WQE set combination is NOP + RegEx. For operations have scattered mubf but share the UMR WQE with others, the WQE set combination is NOP + RegEx. For operations complete the UMR WQE, the WQE set combination is UMR + RegEx. v4: 1. git log improvement. v3: 1. Move testregex.rst change to the correct commit. 2. Code rebase to the latest version. v2: 1. Check mbuf multiple seg by nb_segs. 2. Add ops prefetch. 3. Allocate ops and mbuf memory together in test application. 4. Fix ci and pi in correct issue. John Hurley (1): regex/mlx5: prevent wrong calculation of free sqs in umr mode Suanming Mou (3): common/mlx5: add user memory registration bits regex/mlx5: add data path scattered mbuf process app/test-regex: support scattered mbuf input app/test-regex/main.c | 134 ++++++-- doc/guides/regexdevs/mlx5.rst | 5 + doc/guides/rel_notes/release_21_05.rst | 4 + doc/guides/tools/testregex.rst | 3 + drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_devx_cmds.c | 5 + drivers/common/mlx5/mlx5_devx_cmds.h | 3 + drivers/regex/mlx5/mlx5_regex.c | 9 + drivers/regex/mlx5/mlx5_regex.h | 26 +- drivers/regex/mlx5/mlx5_regex_control.c | 43 ++- drivers/regex/mlx5/mlx5_regex_fastpath.c | 380 +++++++++++++++++++++-- 11 files changed, 531 insertions(+), 83 deletions(-) -- 2.25.1