From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 11EBB42996; Thu, 20 Apr 2023 12:09:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8282B42D0D; Thu, 20 Apr 2023 12:08:47 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2052.outbound.protection.outlook.com [40.107.244.52]) by mails.dpdk.org (Postfix) with ESMTP id 6F84742D0D for ; Thu, 20 Apr 2023 12:08:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XII8n2G8eKJo9RsBdolub9202Xpk35xZToEOnfhkms1ee08mQtmYtTXEHZUHCynxPxbjCJCnbzvYA6iQpTjhw/accKbO6yIFU39TMignNXLNei53zYSAZXMvbn+597hknV/+xZ2MhN0gMGNhXJurIy4ZJ0YzApZ7ToM/EkYPAQtOjCCXdez/zME52jSH1EmOX137YCWX3jGOwV9Zk/pdWEW5E7xtg5tRn7raCxTcDbBk7CaiC3bwbC5UqHDP/YE0QwnmNvS83eslNsl8Gz0eMvYllsELMjdGOR6a9NupymbfhQzHxySBv67ZO6UE9Zlidyir+iQ6HpwG7x6AEHzB6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Icy7qC9l3YGvJpWT/MTmoKegRGSenyh01nxBThNoFEE=; b=nDiBk4LjS2Oi2Y0eeW633B08PWe7VEff6SYPTy7VJGsRzC2IjXc00qST8nuO5LA7DMkO0yoPzBtjf5B9AC7fF82NmIGY6k0vo45KXzTnT8Sj6p7pCErLGFFTTZJb3s0QpZkVW6o7qVwfImbMAFsqG3Ff5za14jpe5aewo7JVV4xZh+Mg+5iUDwoC86/DOJzsbVld+hGA7wczH5/f7HnHgDqgf6/nJUQUmAIQkhtHHh0T/sBws2RxTYyNmuuQASEZ2ttPplpxc2h7dAGZXaZwN16U6aE9tNNWJ9Vyb4COkZrewS4TLFyojdN+w3WyXuUPASeaX8dVST3r7j7P6/+3QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Icy7qC9l3YGvJpWT/MTmoKegRGSenyh01nxBThNoFEE=; b=Qicbr5qqwDQHsVbpOtGzh7meIUAf3O56jvbr/CiW8zerwLh+uCNliK0F9OjseLr4ybu3Af1FkrZUxXxGu8AnS4GIWDwg4ESPkBDpZ20ouvRydqV1tHz+i0AbiMXE/2o/W42Lh2F89Zc1x4AVtxnpMtxf6HYiuQ6tGCl35mazrJf5ghY7PKGD/cvQ1l+N4J0VtFuUDX/kh3VSWfPn1Md+UKOZ5+DTvHWRwxpkMV4rjDuw4y1BHPronM80LQf0JbmL6XM2zIsJXJRQIto9Dc/rbfh+GCyXLsheEyi5vO0aCXcjHUePGT+MfJi9GtP6G2gs+nUyvir6mB6nWAmtKV5YlA== Received: from BN9PR03CA0431.namprd03.prod.outlook.com (2603:10b6:408:113::16) by DM4PR12MB6301.namprd12.prod.outlook.com (2603:10b6:8:a5::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.22; Thu, 20 Apr 2023 10:08:43 +0000 Received: from BN8NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:408:113:cafe::68) by BN9PR03CA0431.outlook.office365.com (2603:10b6:408:113::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend Transport; Thu, 20 Apr 2023 10:08:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT063.mail.protection.outlook.com (10.13.177.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend Transport; Thu, 20 Apr 2023 10:08:43 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Thu, 20 Apr 2023 03:08:31 -0700 Received: from nvidia.com (10.126.230.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Thu, 20 Apr 2023 03:08:30 -0700 From: Viacheslav Ovsiienko To: Subject: [RFC 4/5] net/mlx5: add comprehensive send completion trace Date: Thu, 20 Apr 2023 13:08:02 +0300 Message-ID: <20230420100803.494-5-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230420100803.494-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.126.230.37] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT063:EE_|DM4PR12MB6301:EE_ X-MS-Office365-Filtering-Correlation-Id: 4b22a6e9-993c-4ce5-843d-08db4187391a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: n0Ng164oWOSQSIN934ZjUfiClJDIhyFCaz0DRWQo1At2BQcNWJS9igVDpkfu0QEjHu+qxK4pWXNPtRdcKvEssywMMjmvLX5sidnYe0h5GeujdbwaFxbx5RyyGK0/TeKoez+4Zs57r7Xgtg5gYYvE/vR5HuIZGrXcq43yv1F0yC0Kqhcudz7ZTIrnfNAO+hVtZfcuJjL/oqYrka/GSP9IoQa4uLm5x+qVNmpDJdWDfdNwEQaZKT2lbmyL6r/YJ3uSiI0/wMpmkihaQrR4IGv82sozRGppJGIjlt1I2ImZXjipBYcVClJIvT580cJ1MnFKufwebOovk01ZbQNJJFvVz+Ldpm4nHIeFV6yNtlsklmi1hV3TAx406X0HRW1IzF/U1hOslVAnfVmBxxz7gH/cdUNgGTT66vbHlg8njVW3asl0uTFz7OPF6hP1pC2bF4mQN6Xr06+Qhgz7kF8YU9kKkTPWFuIIYAeOl1nd0GE7gFpde2hDkXUk6wIVxCfLQsIamW+v3zjptQFgRD9p76tedybG2TDJvMsntJX5qP0XUCf9jEI9+K6y2r95GPXQTyEuZLkIeCt3s9GnFysdEnw0Zl5phsHijpMaPdIE1XAE2eyhriJB8cEPJ0zw/Mw0eaopj77OamY3GHcByUgYN0NtgBPExY/nNU73M2Q+xyxBAuemGZv2eLHexcO7A8PZjvRSe08Cgb87EuI4v4Jjc/CPuvNpbscqjve9svmDjr1CmF2y/gnPcXBYkv6B1RoyB0HuucFlfShNz1r8/5Kj4jUXFY/XPAyMUxxAcAXk0hZl1fk= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(396003)(136003)(39860400002)(376002)(451199021)(36840700001)(40470700004)(46966006)(34020700004)(47076005)(83380400001)(478600001)(36860700001)(2616005)(7696005)(55016003)(40480700001)(1076003)(26005)(6666004)(6916009)(316002)(356005)(186003)(336012)(426003)(70206006)(82740400003)(70586007)(16526019)(6286002)(5660300002)(8936002)(2906002)(8676002)(40460700003)(7636003)(41300700001)(86362001)(82310400005)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2023 10:08:43.5455 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4b22a6e9-993c-4ce5-843d-08db4187391a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6301 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is the demand to trace the send completions of every WQE if time scheduling is enabled. The patch extends the size of completion queue and requests completion on every issued WQE in the send queue. As the result hardware provides CQE on each completed WQE and driver is able to fetch completion timestamp for dedicated operation. The add code is under conditional compilation RTE_ENABLE_TRACE_FP flag and does not impact the release code. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_verbs.c | 8 +++- drivers/net/mlx5/mlx5_devx.c | 8 +++- drivers/net/mlx5/mlx5_tx.h | 63 +++++++++++++++++++++++++++-- 3 files changed, 71 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index 67a7bec22b..f3f717f17b 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -968,8 +968,12 @@ mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx) rte_errno = EINVAL; return -rte_errno; } - cqe_n = desc / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + if (__rte_trace_point_fp_is_enabled() && + txq_data->offloads & RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) + cqe_n = UINT16_MAX / 2 - 1; + else + cqe_n = desc / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; txq_obj->cq = mlx5_glue->create_cq(priv->sh->cdev->ctx, cqe_n, NULL, NULL, 0); if (txq_obj->cq == NULL) { diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 4369d2557e..5082a7e178 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1465,8 +1465,12 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) MLX5_ASSERT(ppriv); txq_obj->txq_ctrl = txq_ctrl; txq_obj->dev = dev; - cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + if (__rte_trace_point_fp_is_enabled() && + txq_data->offloads & RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) + cqe_n = UINT16_MAX / 2 - 1; + else + cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; log_desc_n = log2above(cqe_n); cqe_n = 1UL << log_desc_n; if (cqe_n > UINT16_MAX) { diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index 7f624de58e..9f29df280f 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -728,6 +728,54 @@ mlx5_tx_request_completion(struct mlx5_txq_data *__rte_restrict txq, } } +/** + * Set completion request flag for all issued WQEs. + * This routine is intended to be used with enabled fast path tracing + * and send scheduling on time to provide the detailed report in trace + * for send completions on every WQE. + * + * @param txq + * Pointer to TX queue structure. + * @param loc + * Pointer to burst routine local context. + * @param olx + * Configured Tx offloads mask. It is fully defined at + * compile time and may be used for optimization. + */ +static __rte_always_inline void +mlx5_tx_request_completion_trace(struct mlx5_txq_data *__rte_restrict txq, + struct mlx5_txq_local *__rte_restrict loc, + unsigned int olx) +{ + uint16_t head = txq->elts_comp; + + while (txq->wqe_comp != txq->wqe_ci) { + volatile struct mlx5_wqe *wqe; + uint32_t wqe_n; + + MLX5_ASSERT(loc->wqe_last); + wqe = txq->wqes + (txq->wqe_comp & txq->wqe_m); + if (wqe == loc->wqe_last) { + head = txq->elts_head; + head += MLX5_TXOFF_CONFIG(INLINE) ? + 0 : loc->pkts_sent - loc->pkts_copy; + txq->elts_comp = head; + } + /* Completion request flag was set on cseg constructing. */ +#ifdef RTE_LIBRTE_MLX5_DEBUG + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head | + (wqe->cseg.opcode >> 8) << 16; +#else + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head; +#endif + /* A CQE slot must always be available. */ + MLX5_ASSERT((txq->cq_pi - txq->cq_ci) <= txq->cqe_s); + /* Advance to the next WQE in the queue. */ + wqe_n = rte_be_to_cpu_32(wqe->cseg.sq_ds) & 0x3F; + txq->wqe_comp += RTE_ALIGN(wqe_n, 4) / 4; + } +} + /** * Build the Control Segment with specified opcode: * - MLX5_OPCODE_SEND @@ -754,7 +802,7 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, struct mlx5_wqe *__rte_restrict wqe, unsigned int ds, unsigned int opcode, - unsigned int olx __rte_unused) + unsigned int olx) { struct mlx5_wqe_cseg *__rte_restrict cs = &wqe->cseg; @@ -763,8 +811,12 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, opcode = MLX5_OPCODE_TSO | MLX5_OPC_MOD_MPW << 24; cs->opcode = rte_cpu_to_be_32((txq->wqe_ci << 8) | opcode); cs->sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); - cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << - MLX5_COMP_MODE_OFFSET); + if (MLX5_TXOFF_CONFIG(TXPP) && __rte_trace_point_fp_is_enabled()) + cs->flags = RTE_BE32(MLX5_COMP_ALWAYS << + MLX5_COMP_MODE_OFFSET); + else + cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << + MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); if (__rte_trace_point_fp_is_enabled() && !loc->pkts_sent) rte_pmd_mlx5_trace_tx_entry(txq->port_id, txq->idx); @@ -3662,7 +3714,10 @@ mlx5_tx_burst_tmpl(struct mlx5_txq_data *__rte_restrict txq, if (unlikely(loc.pkts_sent == loc.pkts_loop)) goto burst_exit; /* Request CQE generation if limits are reached. */ - mlx5_tx_request_completion(txq, &loc, olx); + if (MLX5_TXOFF_CONFIG(TXPP) && __rte_trace_point_fp_is_enabled()) + mlx5_tx_request_completion_trace(txq, &loc, olx); + else + mlx5_tx_request_completion(txq, &loc, olx); /* * Ring QP doorbell immediately after WQE building completion * to improve latencies. The pure software related data treatment -- 2.18.1