From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 75D7D42E47; Tue, 11 Jul 2023 17:16:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B39642B8E; Tue, 11 Jul 2023 17:16:00 +0200 (CEST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2062.outbound.protection.outlook.com [40.107.212.62]) by mails.dpdk.org (Postfix) with ESMTP id 7778F42B8C for ; Tue, 11 Jul 2023 17:15:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M6pMtnR+JpWBwHLy21iuO31BITFX0Issgh5C76bjL1aSCnc7MJ0QOahq/URu9UiSm2+kBpZooJJEHmS1tTI6y1LhdoRJK8yw6zRSgB+Hh8Z9i4etbVf1UJraQb+HaRk60qJhSCMLivSv5d51YDKxW+jE0bqih/qawQsoqUF+n5Hz5BahgIQPaCV+GwguzuDiMa7wPFXzIIrlJNa1fjh4koBWSec9YBoxdUMjZLSaG4cIuSqv1it/DN2rs8z05xyZTcnXs33XCom818+v49mAGOXBZUA8hm+WKTmNiF9QmdEpowTEUp2WZcNdjsrqbCw8vkPIlOQoOe2t08nU6cAc8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ekiuEmhkhIXAob/bCijX7JcQnFsnmwJcWNJJdYyszIY=; b=H3BPeROVFet62t/ICn4UmAmikoTBUO73VbzVe9YsLquLb+BSeXy/nEXl+QfxeDUuUTMZWjq0weP0vixyl8ofwzi/5/RYrSiHE8IGJbMez7ZsqawITY0vEFIvT+jo4tJYo/nXiOpnHdFwwzzPRxLrA/lzRLJ2/ZCbrZ9mG1ecgviLasal/JmzWlnveAuUSZ82RRDATwyXqS9fotIuDGH7rYWyxu8Xb7hwS9s0z9IvKu3Yoz+RZ4Sgs+KG7rOICZDHfu+Th0PfrEzUZgL59iPvLCRoX2tL74tMtyHS9RJEKQ96QvG3H82d106mVx+Q+NLClw2hJynglJ9SH3EL1Kh1pQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ekiuEmhkhIXAob/bCijX7JcQnFsnmwJcWNJJdYyszIY=; b=UuqWrmouiNOd8MVQvAjTWu2vRvJYVDFrVqQ0zyJv8JQBob0tVwhKzf8biy41dpzrIscKm7jQnnrlIlc6qbE6pCr3hP0wusKFW7dWHbCu73Syh3RIseSt10MkZDlKlwQiismt4wRoE/XjWGT2DW//pfH1Ggb+KVPHBOBiJQR3rZaPsmp+M0lco1Pg1f032/EnX22jlJLoohpcyxpcf6E0JqA5iyfRJydC7iDvE2D0jtKkWjHl9S8ZyYUt7iFKZcG3c5u5cqZmsIPaewW9Hk2BLXfzarq0xoEwLaWV/4o0e4OpRAXjYAFSSTdYua/yKhU9frFEkiaL5IeRmHxeqE83dA== Received: from SJ2PR07CA0016.namprd07.prod.outlook.com (2603:10b6:a03:505::16) by CY8PR12MB8299.namprd12.prod.outlook.com (2603:10b6:930:6c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.31; Tue, 11 Jul 2023 15:15:55 +0000 Received: from DM6NAM11FT015.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:505:cafe::9c) by SJ2PR07CA0016.outlook.office365.com (2603:10b6:a03:505::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.20 via Frontend Transport; Tue, 11 Jul 2023 15:15:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT015.mail.protection.outlook.com (10.13.172.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.20 via Frontend Transport; Tue, 11 Jul 2023 15:15:55 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 11 Jul 2023 08:15:41 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 11 Jul 2023 08:15:40 -0700 From: Viacheslav Ovsiienko To: CC: , Subject: [PATCH v6 2/2] doc: add mlx5 datapath tracing feature description Date: Tue, 11 Jul 2023 18:15:21 +0300 Message-ID: <20230711151521.30175-3-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230711151521.30175-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230711151521.30175-1-viacheslavo@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT015:EE_|CY8PR12MB8299:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d56252e-ba09-44fd-628c-08db8221b91c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: x9HA23Ajce1ZLjB4Uq+WEOE4mb8WXU45gGfXuaVT4UdgZExxE+Ba3RnXZKIg3MZcmCz0HwclhRBLPRWGY1xVa+1Ku4dwSwjuFnJCqYxfGVcyrnqg7W6ECJtaosWDaLiBRdpIktt73S6uDU8TAuD/vQE04oNV87pTEATabhKTZ5U3EsJMNXNVzX0E4hqYZ7Aoab/CZ0uVACxMY/4M9a2CFCWJCWr7RPGJvnKNUKEzBOx/yxe9Wx9nA3aPA8RAOrZDxidC66DAT7Zf6rHgZJ2VZhGEVq/GLxNggoGTPJls7+DmfqjYFRcnZB8M+ExNYVBdW8hWk54vYmC0abSwYLFRbv5C5I/Y1a7u2B/+WNeg3LO2RNjwvSRpwzIx63UnM9Kt37A573fsY1NdgCRNVNBzWo+7ZdQuAFDh8vnuB2N3Ki2lxQzRKdeuuB8SvXG1+HT0NBHvzmtMyKsrbAlfYUqFyMzRnSyctjligRsxpFTLBxNuaWjMAheom0PPP9jOrxjh/1UD3A9Fycui3O/0ThDWZqi5pv5H/ejGz7fgLgjKlXZKSYdPeGlW3KUnW/nOGv8aEFLJW+2OWmadyR2LBZp/TWbVccvlu3shEhjWmYpIgPHiSTBGdotGgFzzNmLSbGB03Kbuw29G4VbNAfpVEKMYFn0/BJfp1DY2lKmr+8qJVsQejU2gDBbGUcYmBtjumEE11gDqo3H4XelKveQX+1th7VjnSP93BVtdU5pV5ZTb32pcsVtBe+mY+NtR4hq+pTDS8XbCzMUoJrJb720Qg1PAUvMoLDSZH0tcjQS12akyAZE= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(376002)(39860400002)(136003)(396003)(451199021)(40470700004)(46966006)(36840700001)(86362001)(5660300002)(8936002)(8676002)(41300700001)(316002)(82310400005)(2906002)(70206006)(70586007)(6916009)(4326008)(356005)(82740400003)(7636003)(54906003)(83380400001)(6666004)(7696005)(478600001)(36860700001)(26005)(16526019)(6286002)(186003)(2616005)(1076003)(47076005)(107886003)(966005)(40460700003)(40480700001)(36756003)(336012)(426003)(55016003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jul 2023 15:15:55.2671 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d56252e-ba09-44fd-628c-08db8221b91c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT015.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8299 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The mlx5 provides the send scheduling on specific moment of time, and for the related kind of applications it would be extremely useful to have extra debug information - when and how packets were scheduled and when the actual sending was completed by the NIC hardware (it helps application to track the internal delay issues). The patch adds the documentation for feature usage. Signed-off-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 74 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 505873ecfd..a407920555 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1923,6 +1923,80 @@ The procedure below is an example of using a ConnectX-5 adapter card (pf0) with $ echo "0000:82:00.2" >> /sys/bus/pci/drivers/mlx5_core/bind +How to trace Tx datapath +------------------------ + +The mlx5 PMD provides Tx datapath tracing capability with extra debug information: +when and how packets were scheduled +and when the actual sending was completed by the NIC hardware. + +Steps to enable Tx datapath tracing: + +#. Build DPDK application with enabled datapath tracing + + The Meson option ``--enable_trace_fp=true`` and + the C flag ``ALLOW_EXPERIMENTAL_API`` should be specified. + + .. code-block:: console + + meson configure --buildtype=debug -Denable_trace_fp=true + -Dc_args='-DRTE_LIBRTE_MLX5_DEBUG -DRTE_ENABLE_ASSERT -DALLOW_EXPERIMENTAL_API' build + +#. Configure the NIC + + If the sending completion timings are important, + the NIC should be configured to provide realtime timestamps. + The non-volatile settings parameter ``REAL_TIME_CLOCK_ENABLE`` should be configured as one. + The ``mlxconfig`` utility is part of the MFT package. + + .. code-block:: console + + mlxconfig -d /dev/mst/mt4125_pciconf0 s REAL_TIME_CLOCK_ENABLE=1 + +#. Run application with EAL parameter enabling the tracing in mlx5 Tx datapath + + By default all tracepoints are disabled. + To analyze Tx datapath and its timings: ``--trace=pmd.net.mlx5.tx``. + +#. Commit the tracing data to the storage (with ``rte_trace_save()`` API call). + +#. Install or build the ``babeltrace2`` package + + The Python script analyzing gathered trace data uses the ``babeltrace2`` library. + The package should be either installed or built from source as shown below. + + .. code-block:: console + + git clone https://github.com/efficios/babeltrace.git + cd babeltrace + ./bootstrap + ./configure -help + ./configure --disable-api-doc --disable-man-pages + --disable-python-bindings-doc --enable-python-plugins + --enable-python-binding + +#. Run analyzing script + + ``mlx5_trace.py`` is used to combine related events (packet firing and completion) + and to show the results in human-readable view. + + The analyzing script is located in the DPDK source tree: ``drivers/net/mlx5/tools``. + + It requires Python 3.6, ``babeltrace2`` package. + + The parameter of the script is the trace data folder. + + .. code-block:: console + + mlx5_trace.py /var/log/rte-2023-01-23-AM-11-52-39 + +#. Interpreting the script output data + + All the timings are given in nanoseconds. + The list of Tx bursts per port/queue is presented in the output. + Each list element contains the list of built WQEs with specific opcodes. + Each WQE contains the list of the encompassed packets to send. + Host shaper ----------- -- 2.18.1