From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BFA2EA0560; Fri, 3 Jun 2022 14:48:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 63DCD4069D; Fri, 3 Jun 2022 14:48:46 +0200 (CEST) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2060.outbound.protection.outlook.com [40.107.101.60]) by mails.dpdk.org (Postfix) with ESMTP id 89DEB40694 for ; Fri, 3 Jun 2022 14:48:44 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aGDhkCaUmhF0+cavpgb+0NHD1FE+OPRLD2tnKLPsjAj6012T3Phkc3+QSw6klhVaWWxuJssszf8frVW5mTaMlb30eRduYtxWYS41K+spWPtaNDAoXmnknl47GgeTdY6aSd3fottXeXgOb+kjsv/dTYDF2kqxR4m12qWLue1UReIADS9lJXNNbn+kD1BOSPYWx/cQz+ob9FRY7Co/Bfv6c4c5lEGqOs3KJE4Mfngm1vp8sMVfZGpVQZ0NjhpGv3msYVcItI9cMXr8BhxQV6bcqffnWmdEzI3z4chojlp3tFQvIMBbxFFWCyJk16KS5hyesW5SCStfVlEVq9M66Y8+hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0TMAatxorUd4Cu+7+Q+07YS6XRK38xjeHofZ+cCoU10=; b=HKj6WDC9YHMrs8q788CSFy96uqOOMENCant1TTS3CBpdC21fpImPsaCm45PWbL3Z+SJ+IMu84VJiIX+uGDy+hbl24LYFjnZisqcKaVIkNFZXbKW6n5Q+St/v6P9H82gYKCh+R+TY4+B3xI+3fpBLiF5bPLqVk4bl3+HL8SCvAfM+iVkkPD1m+c/WPKCeKnXu8AY9I/6V4GnYZSmnCeB22t9IiRG4SNO+ifdl3s2Ltqty+urscgViLWtCSZqHrYl4UKnRz1zT1h0S7Jx049OzeA5KxnUs9NiKIReTjd5sNHPypmKueHHg+e/rHnxHbvtTbJwglYEGq6+3Fn/MbTALuw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=smartsharesystems.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0TMAatxorUd4Cu+7+Q+07YS6XRK38xjeHofZ+cCoU10=; b=MdgjELrL4Bw/o+JzlyFcmRFI4aEmb6pZEZG6dePiw5ItxtCa8rVLy70Fmad4Cb0WKk2baktmYqd78DTki1c16FvH0nUZ/NlqcRCrtpgRJqaa/pT1V3lOopJkKPoqY6S0aqkhOUWKo6NLsyjx/jaUZCzSUMUn5A8zrFYN8LQOqFHOTERyrLfI96Vn0p9WXfqFltYzMOADPbfq0TWPTPZGEwV5MyA2/yFrV4m9FLIfNWbY/2b2fnNrZPYYmSjLQNTkfBqL3g+FScgpuzme4jFoXZ2HjmaZidPHIOmoaCy+T8QSs+4LjhypD8btOCgjRG+u1ZJemBeQgUz7Fiq75m0Fmw== Received: from DM5PR18CA0084.namprd18.prod.outlook.com (2603:10b6:3:3::22) by CY4PR1201MB0118.namprd12.prod.outlook.com (2603:10b6:910:20::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.17; Fri, 3 Jun 2022 12:48:42 +0000 Received: from DM6NAM11FT028.eop-nam11.prod.protection.outlook.com (2603:10b6:3:3:cafe::a9) by DM5PR18CA0084.outlook.office365.com (2603:10b6:3:3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.16 via Frontend Transport; Fri, 3 Jun 2022 12:48:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT028.mail.protection.outlook.com (10.13.173.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5314.12 via Frontend Transport; Fri, 3 Jun 2022 12:48:41 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 3 Jun 2022 12:48:41 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Fri, 3 Jun 2022 05:48:38 -0700 From: Spike Du To: , , , CC: , , , , Subject: [PATCH v4 0/7] introduce per-queue fill threshold and host shaper Date: Fri, 3 Jun 2022 15:48:14 +0300 Message-ID: <20220603124821.1148119-1-spiked@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220524152041.737154-1-spiked@nvidia.com> References: <20220524152041.737154-1-spiked@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1030a18d-84e2-4daf-7590-08da455f6370 X-MS-TrafficTypeDiagnostic: CY4PR1201MB0118:EE_ X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wK+o+aFdRE9h+6aQGQ56e9eVOSSxLyiRQU3n1LpXrlwhIKm+Ra46cpHQv8f+Ewsai7b25wHGP0ILbwd6XobWnFzsQcFE2ZVPg7z5g+NpZHcpksNORqEZa3tzE7DUItBSK+bvFLNx7c6T41TyLTAaEOafTvA145D+w7TmzhlLs9jWLQG1V0iGrQ0Ap/J/jnATTU/kSTzb6aK5mJ8KfMdlKuCNNIyYoBtHBPSr74ra7RDR0nooDErPmEg/u4ij9UqP2am3YmjGa2+whEocsp5Zud7DQxxnXsDLvy2EOAvRhjjqz7vTPdbwgQeFNeRh/KOl8nh5+x94cJKZL+b7BNC2IN0lLy5j0FmavOmcFzbo5zmB4m3WP202r/nK0dnZm59z4ujsNjwbchPL8lgSrQzSodS7500UHRXWrjZZVi4cPt/hnH+J2gEzOz0w1mdazp+rwg8W4zHn64Ck+qVOqjUK2W3+87uTVL4MOjUEdyPiCNVBfzwex13TqWbq3O0IQ0Ff4kZyR8SddTwS2dA/x1tccAvG9aCiPg1ZoY7hsnXUcpOfp5NxEEK4BHB6+D/PzgFVK79wG/W6T/iWl9kWHdIzwAz0cKU3IqNe2rxL4RHS2UiBVmyZk4DKneMjGCfGXmPGHy1rci8euPLLBjQ7QcFz7aaGuQA8XVpHL/OvYui0E9hsEFRbz29zdLiNzQyJ8V/9ikt6fApnr3V1nSanD+C+eA== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(8936002)(2906002)(6666004)(7696005)(336012)(26005)(6286002)(83380400001)(55016003)(356005)(36860700001)(81166007)(107886003)(186003)(16526019)(1076003)(2616005)(426003)(47076005)(4326008)(5660300002)(8676002)(86362001)(508600001)(70586007)(70206006)(40460700003)(54906003)(110136005)(316002)(36756003)(82310400005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2022 12:48:41.7620 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1030a18d-84e2-4daf-7590-08da455f6370 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT028.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0118 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Fill threshold is per RX queue attribute, when RX queue fullness reach the fill threshold limit, HW sends an event to application. Host shaper can configure shaper rate and fill_thresh-triggered for a host port. The shaper limits the rate of traffic from host port to embedded ARM rx port on Nvidia BlueField 2 NIC. If fill_thresh-triggered is enabled, a 100Mbps shaper is enabled automatically when one of the host port's Rx queues receives fill threshold event. These two features can combine to control traffic from host port to wire port for BlueField 2 NIC. The traffic flows from host to embedded ARM, then to the physical port. The work flow is on the ARM system, configure fill threshold to RX queue and enable fill_thresh-triggered flag in host shaper, after receiving fill threshold event, delay a while until RX queue is empty , then disable the shaper. We recycle this work flow to reduce RX queue drops on ARM system. Add new libethdev API to set fill threshold, add rte event RTE_ETH_EVENT_RX_FILL_THRESH to handle fill threshold event. For host shaper, because it doesn't align to existing DPDK framework and is specific to Nvidia NIC, use PMD private API. For integration with testpmd, put the private cmdline function and fill threshold event handler in mlx5 PMD directory by adding a new file mlx5_testpmd.c. Follow David Marchand's driver specific commands framework to add mlx5 specific commands. Spike Du (7): net/mlx5: add LWM support for Rxq common/mlx5: share interrupt management ethdev: introduce Rx queue based fill threshold net/mlx5: add LWM event handling support net/mlx5: support Rx queue based fill threshold net/mlx5: add private API to config host port shaper app/testpmd: add Host Shaper command app/test-pmd/cmdline.c | 68 +++++++ app/test-pmd/config.c | 21 ++ app/test-pmd/testpmd.c | 24 +++ app/test-pmd/testpmd.h | 2 + doc/guides/nics/mlx5.rst | 93 +++++++++ doc/guides/rel_notes/release_22_07.rst | 2 + drivers/common/mlx5/linux/meson.build | 13 ++ drivers/common/mlx5/linux/mlx5_common_os.c | 131 ++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.h | 11 + drivers/common/mlx5/mlx5_prm.h | 26 +++ drivers/common/mlx5/version.map | 2 + drivers/common/mlx5/windows/mlx5_common_os.h | 24 +++ drivers/net/mlx5/linux/mlx5_ethdev_os.c | 71 ------- drivers/net/mlx5/linux/mlx5_os.c | 132 +++--------- drivers/net/mlx5/linux/mlx5_socket.c | 53 +---- drivers/net/mlx5/meson.build | 4 + drivers/net/mlx5/mlx5.c | 68 +++++++ drivers/net/mlx5/mlx5.h | 12 +- drivers/net/mlx5/mlx5_devx.c | 60 +++++- drivers/net/mlx5/mlx5_devx.h | 1 + drivers/net/mlx5/mlx5_rx.c | 292 +++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rx.h | 13 ++ drivers/net/mlx5/mlx5_testpmd.c | 201 ++++++++++++++++++ drivers/net/mlx5/mlx5_testpmd.h | 26 +++ drivers/net/mlx5/mlx5_txpp.c | 28 +-- drivers/net/mlx5/rte_pmd_mlx5.h | 30 +++ drivers/net/mlx5/version.map | 2 + drivers/net/mlx5/windows/mlx5_ethdev_os.c | 22 -- drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 48 +---- lib/ethdev/ethdev_driver.h | 22 ++ lib/ethdev/rte_ethdev.c | 52 +++++ lib/ethdev/rte_ethdev.h | 72 +++++++ lib/ethdev/version.map | 2 + 33 files changed, 1320 insertions(+), 308 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_testpmd.c create mode 100644 drivers/net/mlx5/mlx5_testpmd.h -- 1.8.3.1