From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AFC045BC5; Thu, 24 Oct 2024 16:11:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1BBF340281; Thu, 24 Oct 2024 16:11:08 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2071.outbound.protection.outlook.com [40.107.93.71]) by mails.dpdk.org (Postfix) with ESMTP id 5F80A40265 for ; Thu, 24 Oct 2024 16:11:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ECBmRut0Mniw3KBw3u086/4dAfVZ/d0fdddJU+ANNmjRfKscMHegm3c5MKvIL8x2GdGkrQ6E373IK5HBaaJXOPtJAkyqzLXx5KOKdyHeP/WY3swDo509YHdftO7l9QCO+ZIe5nEZkte0nEWSR1mMQWZu+cXR4PV4qNGGN1gVO9gvoLIKAT4KbkP2htT4QCGd80ulaNu4xtgLUSGJ/JtI6zG605GjoiJnOZtWcB/TOfO6e/7us0yhji1fVEJ+KSP2l+kOYIbzKLCeAXjN6JpSQ5EkXkBmskla7K0LHPupzBe35fmIAYcgFLwB+uf8xSGi83bUABhAIkmGZTKjlQSFzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SbehEWPuK7zPRz5rjFB4xhqWvAXgFA8yV5HGCWSqZJI=; b=wW0OZbWSsNY35hrTdOeoVIeC1qslzB8YNcmsgX47umqIbn6+jckhicq2fLAUQxf8z7ZROAiiL4Aa1I622/Y7uPioUSVAYlbeN7M1SOhLdJrUHte2447VN2TW7kRGBYZERpYfUu2pP5VbQ4EL+vcwrugUAaZkjQ6T80IBofVw8uFbAwpSuJB/pQpjkFAgEIBcecq8QMSLNjM0PdIGx6+BjKobmAWLFERcD5DTASe60ig4mB7bi9XJogPaxIoWHxdytBvkhwuZJrvJTnWjGDuInvr7AvCy2Vsdtd8kqPu8UlAEKJS8GE6lQQnQ8/VeKZ/A3Bvg3FK/khy5Jdzp6ItZbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SbehEWPuK7zPRz5rjFB4xhqWvAXgFA8yV5HGCWSqZJI=; b=nwagx8GSzL4ovNrgv3oZttO9R3w86iadtwgGz9rcl0zZaYsGXg4Qu/jFopk2l+wbXp0Ge0kXRK/IRvbWwmeqPFv8wMxzhZr5fCflo2z6+A/nY3VvU6W+B4YvmPhVPC0IlpNyOPgAmdyzawM3ZXoawf//E3srt42jD+tJZP0haqgN9PYSem5IWFqSipAg0DNjXI3uj4F0sS3Hht0wfZnpXbzAkzH8dDkWuDGqDbLfOKBonJrKpJvnh5Mw1vCB/Dm7S/MyKTm0o7ShUKBmx7DpPIP31z6FsDkPodOZ23CdpGHBKL0FKhOhmKF1Hg9skmHHrhDc4l5f+EttLSmYNMrfjQ== Received: from CH3PR12MB8460.namprd12.prod.outlook.com (2603:10b6:610:156::15) by CY8PR12MB7122.namprd12.prod.outlook.com (2603:10b6:930:61::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.17; Thu, 24 Oct 2024 14:11:03 +0000 Received: from CH3PR12MB8460.namprd12.prod.outlook.com ([fe80::2e9d:7265:3b84:59b5]) by CH3PR12MB8460.namprd12.prod.outlook.com ([fe80::2e9d:7265:3b84:59b5%7]) with mapi id 15.20.8069.024; Thu, 24 Oct 2024 14:11:02 +0000 From: Raslan Darawsheh To: Dariusz Sosnowski , Slava Ovsiienko , Bing Zhao , Ori Kam , Suanming Mou , Matan Azrad CC: "dev@dpdk.org" Subject: Re: [PATCH v2 00/10] net/mlx5: improve MAC address and VLAN add latency Thread-Topic: [PATCH v2 00/10] net/mlx5: improve MAC address and VLAN add latency Thread-Index: AQHbJHrxBcI22phqMUav8FOrV5jliLKV9A45 Date: Thu, 24 Oct 2024 14:11:02 +0000 Message-ID: References: <20241017075738.190064-1-dsosnowski@nvidia.com> <20241022120618.512091-1-dsosnowski@nvidia.com> In-Reply-To: <20241022120618.512091-1-dsosnowski@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR12MB8460:EE_|CY8PR12MB7122:EE_ x-ms-office365-filtering-correlation-id: 8e107c38-57a7-4b39-5504-08dcf435b14c x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|366016|1800799024|376014|38070700018; x-microsoft-antispam-message-info: =?iso-8859-1?Q?8H2WpLuhrmGKn3mPpmB/6g2/JI03SRx18v2szuuwG3rNBfcwsAwEnz5X6Y?= =?iso-8859-1?Q?945rn+/XMlyaXZffuGT0lnLvQePYRdP/lcbDE6hGz8s/hq4NBB+h4BBwpY?= =?iso-8859-1?Q?Z++Z2OF1mT89tImh5xJ1FkreWkBmmtMSLdqRbA8L5zPLQ6B1p7S2n/hJoU?= =?iso-8859-1?Q?WTS0PQYXoLo/KdVJyu7LnRtLA3zTu6u38p5qtbzzz8/9Qu35nyaodvsAv+?= =?iso-8859-1?Q?osS5q2ciwhatPb2mM9flBzdiHgort8bKJVRacOKspjPa9heHJ6BkczdKJH?= =?iso-8859-1?Q?oM9L9pDBppabtwcBgTOV/uhDLAZkHnovD7Yyf97O0li6mOAQGpVIgsCLS1?= =?iso-8859-1?Q?5gmYSfXb5acoDaG0B1f7pL3OENE8ujANb/crkEh4y/e+6p9/q1V98fQdlT?= =?iso-8859-1?Q?pXm4y1sd98s9My+lDj7hm00hljPf76pOGIijbMk4+/FoYBIfsykJ4/C5p6?= =?iso-8859-1?Q?0GKtacGwE3EAzQKQC+vxNTE9bz/6nML31++BCWDM0p20IaCgi4PDL2u/5o?= =?iso-8859-1?Q?fShBVmldSA44myF1Q6ceFXlxvXvJdSckHZkPFuVXR3VpQuIOKBpwtViulF?= =?iso-8859-1?Q?ZGEqOOSHq9y0G38Uq/xI46w6qhdPAQ+qvAiCj6o9k11liXDWi7g+7V8LVa?= =?iso-8859-1?Q?JeICgtnlkjz+Np7RvoenNxumjUfzIgRRRdA0bYvmx3iqcJpKeROfYk6u+P?= =?iso-8859-1?Q?s33/+KdSS7a355KfnD39zK1GHgvyg1+OyTTKk6G6E9l39TBmvfRdrpkt5v?= =?iso-8859-1?Q?HGQO7IGadHDLYVinK0shC+FNenCkTPXUVcLDKt+vtypLCzNabiyBLb1+iT?= =?iso-8859-1?Q?OP4BIV4dqgSJnvf5CWIs2HS/YiouJwKb7jJm0ppm277nfrgKYrWH6ErLwl?= =?iso-8859-1?Q?amouyc9IRVxkcpQKJjklSFeX9YVLpDnSg6wjs5TEMVnQIVLbCCNoD4GXDE?= =?iso-8859-1?Q?zO91hF9hBSC0B1t/Lh+TdeYtX3GZGMRyX9017W6tPxlY9vsdVl8tsTMivP?= =?iso-8859-1?Q?N+1UnWbR/bsIEtobbryCT1FzuvbT6Rh7sOcgcrp/W4Q2pLADzUOC0q5zwC?= =?iso-8859-1?Q?n+gJqX0zzmxidX37/Q2mxwpchSKZe6iKfDuSjpUSRRzjjcA1UoHtXgsBPF?= =?iso-8859-1?Q?1QisV1QpRwYk5PTMyTMo8muKK4PBeljo8T+bypyxC6AOPfA0AG9My+4D5w?= =?iso-8859-1?Q?4JqWjTfZN0reXrbVaBuyue1PQOxXv5oupPoBLYAn83Ig96OaCQ0IiaMkMd?= =?iso-8859-1?Q?hVtQoRlsUgZ6mQw9nY2ciqjItLfEwsyw79vR4C5aWAFXNXy0TdxjLAWsbf?= =?iso-8859-1?Q?FuZMivftSXKTi+2T4w8vBRHOfazQbYmNCb/owumY6DOlB2ZTqDS5ejfyDL?= =?iso-8859-1?Q?6AQZloxLzTPrTZSK/TO+4E5EQvaArf4UNxSObxpu7FTSLpJaGSu5U=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8460.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(376014)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?MYvrwkza8IVFvyqiZb29Lc29/cXv4Ba4orydvWHT8+BoDKCOdjtKsUOVot?= =?iso-8859-1?Q?JO8sPDTTladzez4Jv83nnJDLlKtuYxCkuLff33A7wfA/Zm85fm5i5jtrVo?= =?iso-8859-1?Q?dA7YPEcF07Ock0QOPQzqisjJjADTXZsZCVKE4AeWOmPIB3v1ntUPCxw6XI?= =?iso-8859-1?Q?oMraZjZWUl+yk/A9sMCHAU9IaOiwAsuYtIegHBxACskVV5/iWYU/uKAO7O?= =?iso-8859-1?Q?BTCtmzqa9mzWsvO3wTdxS4H3/9g0SwyM24l8Q2aIF5TBDT2Xuf866TyXUk?= =?iso-8859-1?Q?apDf3CHDOKSVMj042nvn1K3AcU38WwYAdz/IfytTxdolySP50vXE0OIJuy?= =?iso-8859-1?Q?Iu3H1ZPopNlAeHvGKg+YoHcD/TqGamiczct2QhwxzY6CTg5JzLOilrAHEK?= =?iso-8859-1?Q?mdgaiYAjkEqiCL0DK32ZNcGR233xD8CPypms4HqBQaTI2mqgk2jI6hvzK3?= =?iso-8859-1?Q?aD666MpcbOCbJZM5Y42hy7C6jm2QWJ+zdlOrEWCFMA94ePDD3mZ5bmPror?= =?iso-8859-1?Q?BeQagr9rgmIgmJBLdSH78Us+84fErtmAgGTFdRS9+JCKCwM1rxXyGUyp+4?= =?iso-8859-1?Q?i924EUYVT0Kr+N27HunSOasgiVSPHCVdW5EmQWakrpFUMudEkriw+B9e3R?= =?iso-8859-1?Q?Ogliezr7wD+N6236VIYCXyWWLq1L1VJEEZME61a9hbbTGyvSWJzZIQ5nmM?= =?iso-8859-1?Q?pIOWbKPWdIAkwlTEHpbTtJ/TqC2XZNIGg4eKcN8cVcuuig6etZcTfpFRq2?= =?iso-8859-1?Q?dN7YNMyQ40YnN7Jb3iry62XkdZLe/qly83SZc65ef7WGrD2YuzQ0NC35K9?= =?iso-8859-1?Q?iyKm3bO3/VnFRR8UINrGLOkTNOvTiRDleHnXKfgvGKliHEH30cz9GeQSC1?= =?iso-8859-1?Q?zsCtxZcIjqcZPpl8JRnGiKo33pzyG9hpicokdSiGty5aOCjDIEIylsTEiE?= =?iso-8859-1?Q?KmTYRPu0PQc7+69VJRDf8+hz3UsjhigSkXHyP8YbDatw+5/BQL7AjMoxS7?= =?iso-8859-1?Q?B+y3w9EyBaCLaUr3TNtPlCMc/2/0iPCJIRJZcURJ3ISRbztNeX86hb9c38?= =?iso-8859-1?Q?OitJ/aStXRnGENWbbCmAoB9+LZbnKP3X4oe+bKItSbKp7JS+C4ZjU+CZi0?= =?iso-8859-1?Q?AhYr/lJvfCkwPl8o47o0L/e9sE/dZ5gAzm/BGS66vpoCcYsYK7VZahi2vy?= =?iso-8859-1?Q?keZkVV1DpxCtQzHsZVN8q9F9m1G4WLsQUTqNv8Ic9//KfzclGGAvwEdIY3?= =?iso-8859-1?Q?ba17X5DeegSBqGkFrxhlC4202bdINSrTvVLX4Mz3hpIanU96UuGNgIJITr?= =?iso-8859-1?Q?ZWwv4n2fxc5JAg2HuA3044NtzMszjM9l6TQ/V1cdX6B+gjlB+oJRhpmpsE?= =?iso-8859-1?Q?3v8YhLHBO9pcXowvCj5H6d/FziEGshT5YaulPiNJRnLpecpFhfB1lne5oo?= =?iso-8859-1?Q?ZECW5b+QpN6iD2yRNB6kgjhfjcnfmKcBZXSQvJ8a+Q8wTC08yX9a6PW6C1?= =?iso-8859-1?Q?dUG0JrphxwGwpIjUJmaYl+5GDLEky1mSsQt5hEn8IfNKZw9uBV+fR/RM/w?= =?iso-8859-1?Q?DJGiqWXNRio0gWgyNvOhRQrPvJHTSWEBVln/DcqZDSUh2u7F1WU8c7L1fU?= =?iso-8859-1?Q?73e7t3kWLAiuY=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8460.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8e107c38-57a7-4b39-5504-08dcf435b14c X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Oct 2024 14:11:02.4388 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 5oGbytXJ5HoJHxlAsJt8ExvyO14rCPm4yiBfo0BbKMLlURjuXczZ+R4V/rXC8TKZ0cyPkBH1kihHxL+K0yrAFg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7122 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi,=0A= =0A= From: Dariusz Sosnowski =0A= Sent: Tuesday, October 22, 2024 3:06 PM=0A= To: Slava Ovsiienko; Bing Zhao; Ori Kam; Suanming Mou; Matan Azrad=0A= Cc: dev@dpdk.org=0A= Subject: [PATCH v2 00/10] net/mlx5: improve MAC address and VLAN add latenc= y=0A= =0A= Whenever a new MAC address is added to the port, mlx5 PMD will:=0A= =0A= - Add this address to `dev->data->mac_addrs[]`.=0A= - Destroy all control flow rules.=0A= - Recreate all control flow rules.=0A= =0A= Similar logic is also implemented for VLAN filters.=0A= =0A= Because of such logic, the latency of adding the new MAC address=0A= (i.e., latency of `rte_eth_dev_mac_addr_add()` function call)=0A= is actually linear to number of MAC addresses already configured.=0A= Since each operation of creating/destroying a control flow rule,=0A= involves an `ioctl()` syscall, on some setups the latency of adding=0A= a single MAC address can reach ~100ms, when port is operating with >=3D 100= MAC addresses.=0A= The same problem exists for VLAN filters (and even compounded by it).=0A= =0A= This patchset aims to resolve these issues,=0A= by reworking how mlx5 PMD handles adding/removing MAC addresses and VLAN fi= lters.=0A= Instead of recreating all control flow rules,=0A= only necessary flow rules will be created/removed on each operation,=0A= thus minimizing number of syscalls triggered.=0A= =0A= Summary of patches:=0A= =0A= - Patch 1-2 - Extends existing `mlx5_hw_ctrl_flow_type` enum with special v= ariants,=0A= which will be used for tracking MAC and VLAN control flow rules.=0A= - Patch 3-4 - Refactors HWS code for control flow rule creation to allow=0A= creation of specific control flow rules with unicast MAC/VLAN match.=0A= Also functions are added for deletion of specific rules.=0A= - Patch 5-6 - Prepares the control flow rules list, used by HWS flow engine= ,=0A= to be used by other flow engine.=0A= Goal is to reuse the similar logic in Verbs and DV flow engines.=0A= - Patch 7-8 - Adjusts legacy flow engines, so that unicast DMAC/VLAN contro= l flow rules=0A= are added to the control flow rules list.=0A= Also exposes functions for creating/destroying specific ones.=0A= - Patch 9-10 - Extends `mlx5_traffic_*` interface with `mlx5_traffic_mac_ad= d/remove` and=0A= `mlx5_traffic_vlan_add/remove` functions.=0A= They are used in implementations of DPDK APIs for adding/removing MAC add= resses/VLAN filters=0A= and their goal is to update the set of control flow rules in a minimal nu= mber of steps possible,=0A= without recreating the rules.=0A= =0A= As a result of these patches the time to add 128th MAC address,=0A= after 127th was added drops **from ~72 ms to ~197 us** (at least on my setu= p).=0A= =0A= v2:=0A= - Rebase on top of 24.11-rc1.=0A= - Added Acked-by tags from v1.=0A= =0A= Dariusz Sosnowski (10):=0A= net/mlx5: track unicast DMAC control flow rules=0A= net/mlx5: add checking if unicast flow rule exists=0A= net/mlx5: rework creation of unicast flow rules=0A= net/mlx5: support destroying unicast flow rules=0A= net/mlx5: rename control flow rules types=0A= net/mlx5: shared init of control flow rules=0A= net/mlx5: add legacy unicast flow rules management=0A= net/mlx5: add legacy unicast flow rule registration=0A= net/mlx5: add dynamic unicast flow rule management=0A= net/mlx5: optimize MAC address and VLAN filter handling=0A= =0A= drivers/net/mlx5/linux/mlx5_os.c | 3 +=0A= drivers/net/mlx5/meson.build | 1 +=0A= drivers/net/mlx5/mlx5.h | 62 +++--=0A= drivers/net/mlx5/mlx5_flow.c | 149 ++++++++++-=0A= drivers/net/mlx5/mlx5_flow.h | 36 +++=0A= drivers/net/mlx5/mlx5_flow_hw.c | 349 ++++++++++++++++++++------=0A= drivers/net/mlx5/mlx5_flow_hw_stubs.c | 68 +++++=0A= drivers/net/mlx5/mlx5_mac.c | 41 ++-=0A= drivers/net/mlx5/mlx5_trigger.c | 262 ++++++++++++++++++-=0A= drivers/net/mlx5/mlx5_vlan.c | 9 +-=0A= drivers/net/mlx5/windows/mlx5_os.c | 3 +=0A= 11 files changed, 867 insertions(+), 116 deletions(-)=0A= create mode 100644 drivers/net/mlx5/mlx5_flow_hw_stubs.c=0A= =0A= --=0A= 2.39.5=0A= =0A= Series applied to next-net-mlx,=0A= =0A= Kindest regards,=0A= Raslan Darawsheh=0A= =0A=