From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C639E45B5C; Thu, 17 Oct 2024 10:01:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AF8D4402A9; Thu, 17 Oct 2024 10:01:58 +0200 (CEST) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2085.outbound.protection.outlook.com [40.107.212.85]) by mails.dpdk.org (Postfix) with ESMTP id BFCA740261 for ; Thu, 17 Oct 2024 10:01:56 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=prM0lpvHXsK7jQXCHHkXHIqULywFOZrQlu3lxCcXAK2yBjcbUzT/tsTTKKOCq3xf7y34TXuLJPZE7hlO57aYcXnQb2mdDzOhe0J1S2blyrq0gbiuPCEfk5IlMgzPUkIZD/QV4d+oMXOYReyvskj8JBo50pGCdgDKbNSxwUrmFvQHGbNmYtcCvx3+MmbEmSWi3ptFKIKYjgjjGMcb+MiASx4ZDvuvqUPQlTarIQpuUjtgDZ6ARlvFHDwA+0v9VS557zBcPuI8Zr2pPetRuSoy32gtsfJVNRSJjpm6Nqtkvf6Enyl1a0BYClzqR2EyvLxtNuyLLvDwBswvuB7DFmLDUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y9e+0a/CjP1HTqbHsK++MG2IU46uEmKdiWwTwJYfids=; b=gi7XUHY5DGAOmbKEZDd+ud6A9ul2Bzl+kTgxl4CywIVi1bgW9jq1N7RNHLsHvVardtvWmAdsgbNzq6hpFjierDwDWxLGc/uYSVWGmn21V4Or22lcVT1BLksyA3hPHkKruMKJK/UgKJ0TexALqEfTlumCI3QrF/Vs37GJHK+KX+ADUceSVhmcJDe7LG/ssmZbkdswUUHHJRI3sRCiaa1OW9+HNIZMoaulpWm4bchlttJiRN4WbmoK4CCoNldieZboWpBNGsevqG61Sh40OUmZMLvlKziWakw6YuvvZngOzWttSqO5hk26ncOnnNPgGkHda13uWCc3VpXixzQvLdar1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y9e+0a/CjP1HTqbHsK++MG2IU46uEmKdiWwTwJYfids=; b=I/7qSP3n69NxHqyu4tSAjB97qPhjP/GfpJlnyl+vEbew4kS7rrVjt4gOMfRr33UbwifuNSLzeSZeqJ3SgAAqY71SZMhw1x167a5BCpsc+Kw7aLT7Fy6ok/LvdIjnhYQxXkpM2PdX8Q+AA3GIMjnKeCBMRBDbGc7FVoV+wxxKaiCNl+laELL96KBxEmjroUREsT+7mg8CTWszKp5RoMppzV383DnIzVdd7DGIGz3gXI+PHyBdpJBNg9QDRTcDIzmx+8asYH2X2zma4j9lriIdd8Af4pGeVHjMj6lxTE/hRcmvSxZgsogxilgYe8A+UuBf9LIyQ++BCs6SSOfommZfjA== Received: from DM4PR12MB7549.namprd12.prod.outlook.com (2603:10b6:8:10f::14) by MW4PR12MB6828.namprd12.prod.outlook.com (2603:10b6:303:209::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.18; Thu, 17 Oct 2024 08:01:50 +0000 Received: from DM4PR12MB7549.namprd12.prod.outlook.com ([fe80::36a0:7d17:4418:7c62]) by DM4PR12MB7549.namprd12.prod.outlook.com ([fe80::36a0:7d17:4418:7c62%4]) with mapi id 15.20.8069.016; Thu, 17 Oct 2024 08:01:50 +0000 From: Slava Ovsiienko To: Dariusz Sosnowski , Bing Zhao , Ori Kam , Suanming Mou , Matan Azrad CC: "dev@dpdk.org" Subject: RE: [PATCH 00/10] net/mlx5: improve MAC address and VLAN add latency Thread-Topic: [PATCH 00/10] net/mlx5: improve MAC address and VLAN add latency Thread-Index: AQHbIGpTrCedf90lHUmeaUbWFWt3+rKKlKLA Date: Thu, 17 Oct 2024 08:01:49 +0000 Message-ID: References: <20241017075738.190064-1-dsosnowski@nvidia.com> In-Reply-To: <20241017075738.190064-1-dsosnowski@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM4PR12MB7549:EE_|MW4PR12MB6828:EE_ x-ms-office365-filtering-correlation-id: edb9bd02-fab0-49b1-8b5f-08dcee81f47c x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|376014|366016|1800799024|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?mlgvqeLHHYxcNo9MWBixWATZD2mTEo2M8DN6V2HlETijEEQJ3v6LqXOYn+Oh?= =?us-ascii?Q?wiMTiZ1Ltmw8ZUw9RRKcdRvC/LFBTepAwpYb62cD0sFT1mXiGlXigx5jWG2C?= =?us-ascii?Q?UKS9AmSTLZxaGFCbjlhB7IGq8Ai2d62QgxYoBzaokjcK+zoTggg6Gr8wwFo8?= =?us-ascii?Q?PGSEqXWx/3uae8jAywGDvzlH3tJWR//ncMQf6dNgjl5d9EW6Qgz1DKlp9aXs?= =?us-ascii?Q?E2mmdm08+CrPMreNnHsK/2JE7vXgY9LSCJj+WZ5kYnzAuHww4M4HZV0wLezv?= =?us-ascii?Q?bkZNEukc4ohaOCxcSBwcNhyl/jop+4NuNEWoz1k7QHzG/3LDrGytu77vPqAW?= =?us-ascii?Q?xjakft/pePv0H46tGHXc9/Tta3D79REEXOipC8R04OPbTuWHavlyX0JaNcNH?= =?us-ascii?Q?IbijvOfqciI9ws18qVVwS4G7Vyda67Y9kNLN+GRgEAmQcSy4eb/ZWqRaB2gy?= =?us-ascii?Q?wSwhJW6eV3fNmekXP/NbqyjVnRzghFVvd8JxUq/GaMEel9djcrO2HmvwFlbu?= =?us-ascii?Q?V1KrE1pcRpJW1t3wIrxBLx1iKoJaxpMzrl63x4edBphiTvu7UizYXe3oH/pQ?= =?us-ascii?Q?avVhp1Iq/gev8iAtfkAWqq+VRLBTbe7a+stmF3jKpC3u1Gs8seyUvt1xGUZH?= =?us-ascii?Q?zvDk6ZMv5wL7i1xgg/XUIVtvXgIL4/yU0acftirN82s9VMwhoVz/JxiEDLVC?= =?us-ascii?Q?XiL8hc3AVImB3UqGoRxlG1sQszZ0VmPL3FY1yjDkrELTz06kAL8wBQSySKyq?= =?us-ascii?Q?R2ZbVUGc1IyGr4MXdOeBIRFQ8/QgF7UUURScby+N+M6/6ycyjgq6LupiVDE9?= =?us-ascii?Q?LR0yUePCsu84ZhuRMW/6XGHWEclhAizp0Px3BxefBICJnIoupls2qGsvd8gY?= =?us-ascii?Q?6DSh1WnjMWf6a26FKdwmZ16cT/vgglxivPLlGcqafs8+FXgeW+hslmfPCX8Y?= =?us-ascii?Q?2n5dATW1AHjsvA47AivVi/LM7TA6KVEnLaG/xHHNyteYilk9g/L9hf4DIavJ?= =?us-ascii?Q?FaR0JAFVUx6Qe8AZUxKjYGDt74sgQj5nTvZizOrIywGRxAdWnwRnwKvD5LMG?= =?us-ascii?Q?DhoGM2OW3U7XTcpdU2CXfjv95CKzEyw+Ki7sbGGIMSPyFbWtTaDT4UKqEm7S?= =?us-ascii?Q?96p4+pM9Jb93oBM1hXiaZZAW2P75e81/5okMEMXAqiAHVQphpCmMwMHAK/xV?= =?us-ascii?Q?EMN/ABUO3cAbsQfumfD2LrFybytRqL1C2IcBwoXWmSkWrCT4hHopTrHUkRNu?= =?us-ascii?Q?Q3xu6Q5KqmNLKhxpMNjYD2WBtTUMxP0S6ds8NvH329Qp8NPGGmpo2omnbomU?= =?us-ascii?Q?AuOjgH+9PSE91WgeMBAZGewFrOUWN+Nx99Ueca20x4LIMNckc6r1Lva1fKDz?= =?us-ascii?Q?RzGsRA4=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR12MB7549.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?FZL/5/de5rQc4vONf88YnrvFs/53gHxpmi46LHpxKiUV/dm1Esm43JDnHf8S?= =?us-ascii?Q?xSVrZ5ek/AaJAWYp3XHGW39QoWquNdzhjis0QnEBaQ7JBvD3DHwWe8AHygam?= =?us-ascii?Q?07nacRpA9D2u40s7v5dAyGOA1P2Y6U85Dw7aya4h2ZeNioi0m85d1uS120X4?= =?us-ascii?Q?Pxrom/AgceTgH2SLlIOGoiZaMcht2q9w08sY7AlIsBEk/soE0dBUnvST5dpW?= =?us-ascii?Q?DOh4g6X+HUjKs/DK594ecRZnTI8C6VSvGnSShyvqDNeP6CpN+qD69bP36j62?= =?us-ascii?Q?dU8U9MHs0iW83s9FXDLtVsQ07Gi9eOUFfR9HbiXubQMivwvQfROkwkASqWUF?= =?us-ascii?Q?LC7qBBY9hwb6jRm95Tl4FTs1XF30U1SaoEhm9FSU/AxECuKhW0hXrx5xeJWl?= =?us-ascii?Q?WTb3fAOHhndWOwgq+z0+WyVE2ezvhwHS/iI6gF4h7TCCGgYOdCK1nCzre5q8?= =?us-ascii?Q?WEsyFWvw0Fz6QayNLf0kL/vqtPC86F6HQWaL33DM9bKk7jMy8WXeLhaHciaK?= =?us-ascii?Q?tpGbhxFK6Upw3P0rhn9pLPeOrhL+9wtOdaKkXYemGQzWhCHspyIjkfIlTNpU?= =?us-ascii?Q?igdUF1EYhd6HRL9WOtuCLoi6PKtiooKMWf8Mjz1t8VowV2NTctEJUbAytF8v?= =?us-ascii?Q?0cu3NJCwqweTCrH6NNvoJxLZuHVYm3AVWLZzDSKvsu8Wqy4hsuLBuu66Zavu?= =?us-ascii?Q?gTjgdk90VSO3QvoS21/+GyUrAabXyPggJMvxSh1wAmhnytKsa4gnEKMc04tB?= =?us-ascii?Q?3x0A2ndjpT/x8+1+XBZ93CWzXaiB++nDTD3sSzG/WwFasFYRHxW+hlS9cSL7?= =?us-ascii?Q?axLxZ5LAu+O1JPlZESCaDZM9nJMdSiJcTA3+pjal2KDjKEDSvjg/YC2bcqA7?= =?us-ascii?Q?YeCkvOoJbolrL8C7nVuo2aiHNfVLZemgRRST8R1eID40HclGE2r4pDPI/i96?= =?us-ascii?Q?BDoVF2GdEFlxZzVXVL71KjPaI+f3+6ejgy48RCEu9hbsWapiUo7XGRV7H6qw?= =?us-ascii?Q?PdvSu7Et2S/cfxMqhMzLPBfQtwb0wFFq+MP8VcxUvDk4EmDH6RQKgBI/n0u6?= =?us-ascii?Q?NCUzL30QSpnXFWIcH0yPvG7U0Dg3NeuzJtd1yPC4dYMO4S6CBRGX2P18PQgx?= =?us-ascii?Q?O9V3NfnX3+RfrftYTYcVBztY4OSW7Av4eyMv0urtoBJIPp71wkd0EXuMo5AR?= =?us-ascii?Q?C4H9v2qxEaixAhacxKWm0F5ZzW/LYSDvWLKtj+MUe3xRgkDPc/ZOA+SJoq7Y?= =?us-ascii?Q?m0YwMiE9R+5ZOY/vFHc4Z9FYVRDZac0wQ+Q5J6RqhzPuU8rYxT3kxsQH7irm?= =?us-ascii?Q?0WqzDnFw7i894ZleVIPtwAY7x52EJv3RIdHSae4b7xlKLyYC9+x0WeBUKACz?= =?us-ascii?Q?Fm/bTzzb0O49Fiz4RxxhZPc9VGrewNziz9mlQ3Mj/KN3vzDv4dHqwlKs5D8W?= =?us-ascii?Q?833pSiTp5kd9zFXNx38Pm4oWsM46m00MEKsB+5xCL1J3MoNeHPlQ1iFvEHKe?= =?us-ascii?Q?lF598pUChX51RO5pUYLTIUi0TA0z7QrqgwG58EeZfbIIB5BQoTKk3YyrZI6/?= =?us-ascii?Q?9WH38GU8Jbb4WBw2jA5cQHIkFZtRIdJiclb9WL5G?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB7549.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: edb9bd02-fab0-49b1-8b5f-08dcee81f47c X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Oct 2024 08:01:49.9283 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: IZQbsJRluut5NtA1X8+nt3RbrsMe7lH2H+a3xgkDcK9fstbuedrTzvd9/gByRLsFPJS57qa7fcBXIw7JM8wRoA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6828 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org For the entire series: Acked-by: Viacheslav Ovsiienko > -----Original Message----- > From: Dariusz Sosnowski > Sent: Thursday, October 17, 2024 10:57 AM > To: Slava Ovsiienko ; Bing Zhao > ; Ori Kam ; Suanming Mou > ; Matan Azrad > Cc: dev@dpdk.org > Subject: [PATCH 00/10] net/mlx5: improve MAC address and VLAN add > latency >=20 > Whenever a new MAC address is added to the port, mlx5 PMD will: >=20 > - Add this address to `dev->data->mac_addrs[]`. > - Destroy all control flow rules. > - Recreate all control flow rules. >=20 > Similar logic is also implemented for VLAN filters. >=20 > Because of such logic, the latency of adding the new MAC address (i.e., > latency of `rte_eth_dev_mac_addr_add()` function call) is actually linear= to > number of MAC addresses already configured. > Since each operation of creating/destroying a control flow rule, involves= an > `ioctl()` syscall, on some setups the latency of adding a single MAC addr= ess > can reach ~100ms, when port is operating with >=3D 100 MAC addresses. > The same problem exists for VLAN filters (and even compounded by it). >=20 > This patchset aims to resolve these issues, by reworking how mlx5 PMD > handles adding/removing MAC addresses and VLAN filters. > Instead of recreating all control flow rules, only necessary flow rules w= ill be > created/removed on each operation, thus minimizing number of syscalls > triggered. >=20 > Summary of patches: >=20 > - Patch 1-2 - Extends existing `mlx5_hw_ctrl_flow_type` enum with special > variants, > which will be used for tracking MAC and VLAN control flow rules. > - Patch 3-4 - Refactors HWS code for control flow rule creation to allow > creation of specific control flow rules with unicast MAC/VLAN match. > Also functions are added for deletion of specific rules. > - Patch 5-6 - Prepares the control flow rules list, used by HWS flow engi= ne, > to be used by other flow engine. > Goal is to reuse the similar logic in Verbs and DV flow engines. > - Patch 7-8 - Adjusts legacy flow engines, so that unicast DMAC/VLAN cont= rol > flow rules > are added to the control flow rules list. > Also exposes functions for creating/destroying specific ones. > - Patch 9-10 - Extends `mlx5_traffic_*` interface with > `mlx5_traffic_mac_add/remove` and > `mlx5_traffic_vlan_add/remove` functions. > They are used in implementations of DPDK APIs for adding/removing MAC > addresses/VLAN filters > and their goal is to update the set of control flow rules in a minimal = number > of steps possible, > without recreating the rules. >=20 > As a result of these patches the time to add 128th MAC address, after 127= th > was added drops **from ~72 ms to ~197 us** (at least on my setup). >=20 > Dariusz Sosnowski (10): > net/mlx5: track unicast DMAC control flow rules > net/mlx5: add checking if unicast flow rule exists > net/mlx5: rework creation of unicast flow rules > net/mlx5: support destroying unicast flow rules > net/mlx5: rename control flow rules types > net/mlx5: shared init of control flow rules > net/mlx5: add legacy unicast flow rules management > net/mlx5: add legacy unicast flow rule registration > net/mlx5: add dynamic unicast flow rule management > net/mlx5: optimize MAC address and VLAN filter handling >=20 > drivers/net/mlx5/linux/mlx5_os.c | 3 + > drivers/net/mlx5/meson.build | 1 + > drivers/net/mlx5/mlx5.h | 62 +++-- > drivers/net/mlx5/mlx5_flow.c | 149 ++++++++++- > drivers/net/mlx5/mlx5_flow.h | 36 +++ > drivers/net/mlx5/mlx5_flow_hw.c | 349 ++++++++++++++++++++------ > drivers/net/mlx5/mlx5_flow_hw_stubs.c | 68 +++++ > drivers/net/mlx5/mlx5_mac.c | 41 ++- > drivers/net/mlx5/mlx5_trigger.c | 262 ++++++++++++++++++- > drivers/net/mlx5/mlx5_vlan.c | 9 +- > drivers/net/mlx5/windows/mlx5_os.c | 3 + > 11 files changed, 867 insertions(+), 116 deletions(-) create mode 10064= 4 > drivers/net/mlx5/mlx5_flow_hw_stubs.c >=20 > -- > 2.39.5