From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA346A0350; Tue, 22 Feb 2022 13:48:58 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0904741161; Tue, 22 Feb 2022 13:48:38 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2064.outbound.protection.outlook.com [40.107.243.64]) by mails.dpdk.org (Postfix) with ESMTP id 81C934113C for ; Tue, 22 Feb 2022 13:48:36 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dNpWPR/MlX6UA0/EQ91SRcJwAakbC+yD6Cwtq25pNqGanZ9Rp0nKPTkl8FGK9RGjVgiAGWbbAbvq8tDhPtlzrUP/S+jFRfV9qtZ/31sU9tmu5lWg0f2EMgmDxoCd1NJV9PEp5EeFf8fTq1IlMHFWZP91Pj1rLsz2ml5ZgcaP5OJAC/F1vpBFMaXDveNS4TzoduWA8jgZmMQebS9VaEXBtEyPyDL7T+py/QJ8vXFNYJFtzK2fhcGcM89Yc9ekO8l9cIYjPHDnayW+8Ki4nUUV1EGnivCJCHHkcwy+/Krc6sbLFPZmgh2qFIMZiVSzQaQlBX6e+lgsoS+Um0Zde9VcrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WyJA67ilPM5EteL9ykhRaq5TMBp3LZTtyoYQDy1GsZI=; b=TKlP+zvEi0O7ui4qyH95DBnm51G7BFIWzigZG0ciq6d0eAmi+/dSQI0yMWwFluO5jAcPpuWtwq2UHA8sY64dVpHb+1tmvqACRGB4RWB/piWwvu7gHmfp6DhKqWt4wEbTTEaJVblKGcxqkav5USvDvoXS+TOjWMl+3MEevurGCdYYz2uEXUifga+S/gIg7FyucgbRoEp0JIOlj9XxVQkFCxWz0F31m1J2yQZDb7KltaV0iqio9/cLJ7VVM7N4iwLBijM9RMWR+kgtCevDgh3wTkN5FEup4/9I7bqXmLYHeCJtfZZua8sON5fYE2PnBNpL38n2W8/gaHq4/ODIMsuHYQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WyJA67ilPM5EteL9ykhRaq5TMBp3LZTtyoYQDy1GsZI=; b=m1CK46AVyEo9+p2MSNHT/Lx3unQUExwjGv1tnjYVA2vDXUXrmy6SaAUVQ0qcoMd+m5F+LRRPDtjQYnqrH0gDJUo2cVNpmjgvK6jTq8e6uigr0BG4cXI8hKAxQ4CD0YLAHTpY0ZOLPRgMEYGNrVi2lAmQlXvxRmfdpS5wURb352JUmUOJSjqoI35HjCx8XBumF0dwLsnqyEyUlMIp0ZNSbA1pkjr8r/vn3ZnaJYdCgoeHGctn0gy5aTPOhGYnI7NYk4MXS6IijltVkhvYGePVmcLSe25h76ZpxWMWhiumxcCNUrXk6Ts3JrxNOxhKEu8OJu2ScNqzESb+pbfFdNAcuQ== Received: from MW4PR03CA0360.namprd03.prod.outlook.com (2603:10b6:303:dc::35) by MWHPR12MB1152.namprd12.prod.outlook.com (2603:10b6:300:7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.19; Tue, 22 Feb 2022 12:48:34 +0000 Received: from CO1NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:303:dc:cafe::6e) by MW4PR03CA0360.outlook.office365.com (2603:10b6:303:dc::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.22 via Frontend Transport; Tue, 22 Feb 2022 12:48:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT020.mail.protection.outlook.com (10.13.174.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4995.15 via Frontend Transport; Tue, 22 Feb 2022 12:48:33 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 22 Feb 2022 12:48:33 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Tue, 22 Feb 2022 04:48:31 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9 via Frontend Transport; Tue, 22 Feb 2022 04:48:30 -0800 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Subject: [PATCH 5/5] doc: update doorbell mapping parameter name in mlx5 guide Date: Tue, 22 Feb 2022 14:48:15 +0200 Message-ID: <20220222124815.2587851-6-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220222124815.2587851-1-michaelba@nvidia.com> References: <20220222124815.2587851-1-michaelba@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3c56e482-eb57-44fb-898d-08d9f601a309 X-MS-TrafficTypeDiagnostic: MWHPR12MB1152:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gnSU6zR8QlasN1MU+pbIGJP9BpvSo85KvsaQ1ym+9GjMpQqdMi/zQzalbx5BD+XAdOyzOl2o5Lz95hndxSVZ9aiGHwJdJPr6VjEL2h3qM7BiaVTdnM0s0ozzzRJEXUj/sE75NcXGeMkzZBmzzHKhygJKT+Xdu5yI9HShnQA2zHwU2t7K5MVp8l2IeTM6hTQFKpJ5ZjXpooBi0IVrOuspHwyjkanPBO/o5UgWXXmeYAA2rDtsPXaDrjzR4LpB2bFjbZHAcskuGgIhWYbYATI2ShYae+4SZyvrpRqPUXxsE5L9iJUNtUwoQPbp45nVFZ8ILKNIU+0YijnHGxWKKK5c9GSZEkFnt1H4aJQ+uhULZjGj5BlXYUcYtzRojMea1UAC7B//BXhmJB9S6UHAD8S19GoIxYI+AYdSm49iuNOigRlXTW185KDLNBSW5ylkKl1ECpH5KKCeUdniyzLs2yDzhg/7IKkwB1taC3KpQJFjOby0vv29l8pw9j9uXedT9iWrllPwp7xHJmTTP34517z4zFZv1tGwVPPs+/z/8smYYw5KxjjsivZc40xODzXYtxtmYa2r353Y0B7VRq/PheaiS7wKvTUMFu/0c7qUZR2A8/60jCzkDn8fsDY3pvibVy/gF7lLOXMmUwp0WJZlbZ0OwHilskPvZd/rWf+ytKWiNeLalM5l5zmvfeb0QXAPwkWSqSRMqZYvi/dS+2ghoFudow== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(426003)(70586007)(70206006)(47076005)(336012)(508600001)(8676002)(4326008)(30864003)(2906002)(83380400001)(55016003)(36860700001)(36756003)(54906003)(6916009)(316002)(356005)(1076003)(26005)(186003)(6286002)(107886003)(2616005)(40460700003)(86362001)(6666004)(8936002)(7696005)(5660300002)(81166007)(82310400004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2022 12:48:33.8784 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3c56e482-eb57-44fb-898d-08d9f601a309 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1152 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The "tx_db_nc" devarg forces doorbell register mapping to non-cached region eliminating the extra write memory barrier. This argument was used in creating the UAR for Tx and thus affected its performance. Recently [1] its use has been extended to all UAR creation in all mlx5 drivers, and now its name is no longer so accurate. This patch changes its name to "sq_db_nc" to suit any send queue that uses it. The old name will still work for backward compatibility. [1] commit 5dfa003db53f ("common/mlx5: fix post doorbell barrier") Signed-off-by: Michael Baum Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 34 ++----------------- doc/guides/platform/mlx5.rst | 39 ++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.c | 2 +- drivers/common/mlx5/mlx5_common.c | 31 ++++++++++++----- drivers/common/mlx5/mlx5_common_defs.h | 8 ++--- drivers/net/mlx5/linux/mlx5_verbs.c | 2 +- drivers/net/mlx5/mlx5_devx.c | 2 +- 7 files changed, 72 insertions(+), 46 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 8eb67efcf3..968aac01d2 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -814,37 +814,9 @@ for an additional list of options shared with other mlx5 drivers. - ``tx_db_nc`` parameter [int] - The rdma core library can map doorbell register in two ways, depending on the - environment variable "MLX5_SHUT_UP_BF": - - - As regular cached memory (usually with write combining attribute), if the - variable is either missing or set to zero. - - As non-cached memory, if the variable is present and set to not "0" value. - - The type of mapping may slightly affect the Tx performance, the optimal choice - is strongly relied on the host architecture and should be deduced practically. - - If ``tx_db_nc`` is set to zero, the doorbell is forced to be mapped to regular - memory (with write combining), the PMD will perform the extra write memory barrier - after writing to doorbell, it might increase the needed CPU clocks per packet - to send, but latency might be improved. - - If ``tx_db_nc`` is set to one, the doorbell is forced to be mapped to non - cached memory, the PMD will not perform the extra write memory barrier - after writing to doorbell, on some architectures it might improve the - performance. - - If ``tx_db_nc`` is set to two, the doorbell is forced to be mapped to regular - memory, the PMD will use heuristics to decide whether write memory barrier - should be performed. For bursts with size multiple of recommended one (64 pkts) - it is supposed the next burst is coming and no need to issue the extra memory - barrier (it is supposed to be issued in the next coming burst, at least after - descriptor writing). It might increase latency (on some hosts till next - packets transmit) and should be used with care. - - If ``tx_db_nc`` is omitted or set to zero, the preset (if any) environment - variable "MLX5_SHUT_UP_BF" value is used. If there is no "MLX5_SHUT_UP_BF", - the default ``tx_db_nc`` value is zero for ARM64 hosts and one for others. + This parameter name is deprecated and ignored. + The new name for this parameter is ``sq_db_nc``. + See :ref:`common driver options `. - ``tx_pp`` parameter [int] diff --git a/doc/guides/platform/mlx5.rst b/doc/guides/platform/mlx5.rst index 0fd5e6604d..d073c213ca 100644 --- a/doc/guides/platform/mlx5.rst +++ b/doc/guides/platform/mlx5.rst @@ -600,3 +600,42 @@ and below are the arguments supported by the common mlx5 layer. from system by default, without explicit rte memory flag. By default, the PMD will set this value to 0. + +- ``sq_db_nc`` parameter [int] + + The rdma core library can map doorbell register in two ways, + depending on the environment variable "MLX5_SHUT_UP_BF": + + - As regular cached memory (usually with write combining attribute), + if the variable is either missing or set to zero. + - As non-cached memory, if the variable is present and set to not "0" value. + + The same doorbell mapping approach is implemented directly by PMD + in UAR generation for queues created with DevX. + + The type of mapping may slightly affect the send queue performance, + the optimal choice strongly relied on the host architecture + and should be deduced practically. + + If ``sq_db_nc`` is set to zero, the doorbell is forced to be mapped to + regular memory (with write combining), the PMD will perform the extra write + memory barrier after writing to doorbell, it might increase the needed CPU + clocks per packet to send, but latency might be improved. + + If ``sq_db_nc`` is set to one, the doorbell is forced to be mapped to non + cached memory, the PMD will not perform the extra write memory barrier after + writing to doorbell, on some architectures it might improve the performance. + + If ``sq_db_nc`` is set to two, the doorbell is forced to be mapped to + regular memory, the PMD will use heuristics to decide whether a write memory + barrier should be performed. For bursts with size multiple of recommended one + (64 pkts) it is supposed the next burst is coming and no need to issue the + extra memory barrier (it is supposed to be issued in the next coming burst, + at least after descriptor writing). It might increase latency (on some hosts + till the next packets transmit) and should be used with care. + The PMD uses heuristics only for Tx queue, for other semd queues the doorbell + is forced to be mapped to regular memory as same as ``sq_db_nc`` is set to 0. + + If ``sq_db_nc`` is omitted, the preset (if any) environment variable + "MLX5_SHUT_UP_BF" value is used. If there is no "MLX5_SHUT_UP_BF", the + default ``sq_db_nc`` value is zero for ARM64 hosts and one for others. \ No newline at end of file diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 0d3e24e04e..a752d79e8e 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -630,7 +630,7 @@ mlx5_config_doorbell_mapping_env(int dbnc) setenv(MLX5_SHUT_UP_BF, MLX5_SHUT_UP_BF_DEFAULT, 1); else setenv(MLX5_SHUT_UP_BF, - dbnc == MLX5_TXDB_NCACHED ? "1" : "0", 1); + dbnc == MLX5_SQ_DB_NCACHED ? "1" : "0", 1); return value; } diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index 96906d3f39..8cf391df13 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -35,10 +35,17 @@ uint8_t haswell_broadwell_cpu; /* * Device parameter to force doorbell register mapping - * to non-cahed region eliminating the extra write memory barrier. + * to non-cached region eliminating the extra write memory barrier. + * Deprecated, ignored (Name changed to sq_db_nc). */ #define MLX5_TX_DB_NC "tx_db_nc" +/* + * Device parameter to force doorbell register mapping + * to non-cached region eliminating the extra write memory barrier. + */ +#define MLX5_SQ_DB_NC "sq_db_nc" + /* In case this is an x86_64 intel processor to check if * we should use relaxed ordering. */ @@ -255,11 +262,17 @@ mlx5_common_args_check_handler(const char *key, const char *val, void *opaque) DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val); return -rte_errno; } - if (strcmp(key, MLX5_TX_DB_NC) == 0) { - if (tmp != MLX5_TXDB_CACHED && - tmp != MLX5_TXDB_NCACHED && - tmp != MLX5_TXDB_HEURISTIC) { - DRV_LOG(ERR, "Invalid Tx doorbell mapping parameter."); + if (strcmp(key, MLX5_TX_DB_NC) == 0) + DRV_LOG(WARNING, + "%s: deprecated parameter, converted to queue_db_nc", + key); + if (strcmp(key, MLX5_SQ_DB_NC) == 0 || + strcmp(key, MLX5_TX_DB_NC) == 0) { + if (tmp != MLX5_SQ_DB_CACHED && + tmp != MLX5_SQ_DB_NCACHED && + tmp != MLX5_SQ_DB_HEURISTIC) { + DRV_LOG(ERR, + "Invalid Send Queue doorbell mapping parameter."); rte_errno = EINVAL; return -rte_errno; } @@ -293,6 +306,7 @@ mlx5_common_config_get(struct mlx5_kvargs_ctrl *mkvlist, RTE_DEVARGS_KEY_CLASS, MLX5_DRIVER_KEY, MLX5_TX_DB_NC, + MLX5_SQ_DB_NC, MLX5_MR_EXT_MEMSEG_EN, MLX5_SYS_MEM_EN, MLX5_MR_MEMPOOL_REG_EN, @@ -317,7 +331,8 @@ mlx5_common_config_get(struct mlx5_kvargs_ctrl *mkvlist, DRV_LOG(DEBUG, "mr_ext_memseg_en is %u.", config->mr_ext_memseg_en); DRV_LOG(DEBUG, "mr_mempool_reg_en is %u.", config->mr_mempool_reg_en); DRV_LOG(DEBUG, "sys_mem_en is %u.", config->sys_mem_en); - DRV_LOG(DEBUG, "Tx doorbell mapping parameter is %d.", config->dbnc); + DRV_LOG(DEBUG, "Send Queue doorbell mapping parameter is %d.", + config->dbnc); return ret; } @@ -1231,7 +1246,7 @@ mlx5_devx_alloc_uar(struct mlx5_common_device *cdev) for (retry = 0; retry < MLX5_ALLOC_UAR_RETRY; ++retry) { #ifdef MLX5DV_UAR_ALLOC_TYPE_NC /* Control the mapping type according to the settings. */ - uar_mapping = (cdev->config.dbnc == MLX5_TXDB_NCACHED) ? + uar_mapping = (cdev->config.dbnc == MLX5_SQ_DB_NCACHED) ? MLX5DV_UAR_ALLOC_TYPE_NC : MLX5DV_UAR_ALLOC_TYPE_BF; #else /* diff --git a/drivers/common/mlx5/mlx5_common_defs.h b/drivers/common/mlx5/mlx5_common_defs.h index ca80cd8d29..68b700dc0b 100644 --- a/drivers/common/mlx5/mlx5_common_defs.h +++ b/drivers/common/mlx5/mlx5_common_defs.h @@ -34,10 +34,10 @@ /* Default PMD specific parameter value. */ #define MLX5_ARG_UNSET (-1) -/* MLX5_TX_DB_NC supported values. */ -#define MLX5_TXDB_CACHED 0 -#define MLX5_TXDB_NCACHED 1 -#define MLX5_TXDB_HEURISTIC 2 +/* MLX5_SQ_DB_NC supported values. */ +#define MLX5_SQ_DB_CACHED 0 +#define MLX5_SQ_DB_NCACHED 1 +#define MLX5_SQ_DB_HEURISTIC 2 /* Fields of memory mapping type in offset parameter of mmap() */ #define MLX5_UAR_MMAP_CMD_SHIFT 8 diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index 73c44138de..dfbc5a1e08 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -926,7 +926,7 @@ mlx5_txq_ibv_uar_init(struct mlx5_txq_ctrl *txq_ctrl, void *bf_reg) DRV_LOG(ERR, "Failed to get mem page size"); rte_errno = ENOMEM; } - txq->db_heu = priv->sh->cdev->config.dbnc == MLX5_TXDB_HEURISTIC; + txq->db_heu = priv->sh->cdev->config.dbnc == MLX5_SQ_DB_HEURISTIC; txq->db_nc = mlx5_db_map_type_get(uar_mmap_offset, page_size); ppriv->uar_table[txq->idx].db = bf_reg; #ifndef RTE_ARCH_64 diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index e57787cfec..f18b18b1a2 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1326,7 +1326,7 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) txq_data->qp_db = &txq_obj->sq_obj.db_rec[MLX5_SND_DBR]; *txq_data->qp_db = 0; txq_data->qp_num_8s = txq_obj->sq_obj.sq->id << 8; - txq_data->db_heu = sh->cdev->config.dbnc == MLX5_TXDB_HEURISTIC; + txq_data->db_heu = sh->cdev->config.dbnc == MLX5_SQ_DB_HEURISTIC; txq_data->db_nc = sh->tx_uar.dbnc; /* Change Send Queue state to Ready-to-Send. */ ret = mlx5_txq_devx_modify(txq_obj, MLX5_TXQ_MOD_RST2RDY, 0); -- 2.25.1