From: Slava Ovsiienko <viacheslavo@nvidia.com>
To: Dariusz Sosnowski <dsosnowski@nvidia.com>,
Bing Zhao <bingz@nvidia.com>, Ori Kam <orika@nvidia.com>,
Suanming Mou <suanmingm@nvidia.com>,
Matan Azrad <matan@nvidia.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: RE: [PATCH 00/10] net/mlx5: improve MAC address and VLAN add latency
Date: Thu, 17 Oct 2024 08:01:49 +0000 [thread overview]
Message-ID: <DM4PR12MB754970B1D31D56B739B7F161DF472@DM4PR12MB7549.namprd12.prod.outlook.com> (raw)
In-Reply-To: <20241017075738.190064-1-dsosnowski@nvidia.com>
For the entire series:
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
> -----Original Message-----
> From: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Sent: Thursday, October 17, 2024 10:57 AM
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Bing Zhao
> <bingz@nvidia.com>; Ori Kam <orika@nvidia.com>; Suanming Mou
> <suanmingm@nvidia.com>; Matan Azrad <matan@nvidia.com>
> Cc: dev@dpdk.org
> Subject: [PATCH 00/10] net/mlx5: improve MAC address and VLAN add
> latency
>
> Whenever a new MAC address is added to the port, mlx5 PMD will:
>
> - Add this address to `dev->data->mac_addrs[]`.
> - Destroy all control flow rules.
> - Recreate all control flow rules.
>
> Similar logic is also implemented for VLAN filters.
>
> Because of such logic, the latency of adding the new MAC address (i.e.,
> latency of `rte_eth_dev_mac_addr_add()` function call) is actually linear to
> number of MAC addresses already configured.
> Since each operation of creating/destroying a control flow rule, involves an
> `ioctl()` syscall, on some setups the latency of adding a single MAC address
> can reach ~100ms, when port is operating with >= 100 MAC addresses.
> The same problem exists for VLAN filters (and even compounded by it).
>
> This patchset aims to resolve these issues, by reworking how mlx5 PMD
> handles adding/removing MAC addresses and VLAN filters.
> Instead of recreating all control flow rules, only necessary flow rules will be
> created/removed on each operation, thus minimizing number of syscalls
> triggered.
>
> Summary of patches:
>
> - Patch 1-2 - Extends existing `mlx5_hw_ctrl_flow_type` enum with special
> variants,
> which will be used for tracking MAC and VLAN control flow rules.
> - Patch 3-4 - Refactors HWS code for control flow rule creation to allow
> creation of specific control flow rules with unicast MAC/VLAN match.
> Also functions are added for deletion of specific rules.
> - Patch 5-6 - Prepares the control flow rules list, used by HWS flow engine,
> to be used by other flow engine.
> Goal is to reuse the similar logic in Verbs and DV flow engines.
> - Patch 7-8 - Adjusts legacy flow engines, so that unicast DMAC/VLAN control
> flow rules
> are added to the control flow rules list.
> Also exposes functions for creating/destroying specific ones.
> - Patch 9-10 - Extends `mlx5_traffic_*` interface with
> `mlx5_traffic_mac_add/remove` and
> `mlx5_traffic_vlan_add/remove` functions.
> They are used in implementations of DPDK APIs for adding/removing MAC
> addresses/VLAN filters
> and their goal is to update the set of control flow rules in a minimal number
> of steps possible,
> without recreating the rules.
>
> As a result of these patches the time to add 128th MAC address, after 127th
> was added drops **from ~72 ms to ~197 us** (at least on my setup).
>
> Dariusz Sosnowski (10):
> net/mlx5: track unicast DMAC control flow rules
> net/mlx5: add checking if unicast flow rule exists
> net/mlx5: rework creation of unicast flow rules
> net/mlx5: support destroying unicast flow rules
> net/mlx5: rename control flow rules types
> net/mlx5: shared init of control flow rules
> net/mlx5: add legacy unicast flow rules management
> net/mlx5: add legacy unicast flow rule registration
> net/mlx5: add dynamic unicast flow rule management
> net/mlx5: optimize MAC address and VLAN filter handling
>
> drivers/net/mlx5/linux/mlx5_os.c | 3 +
> drivers/net/mlx5/meson.build | 1 +
> drivers/net/mlx5/mlx5.h | 62 +++--
> drivers/net/mlx5/mlx5_flow.c | 149 ++++++++++-
> drivers/net/mlx5/mlx5_flow.h | 36 +++
> drivers/net/mlx5/mlx5_flow_hw.c | 349 ++++++++++++++++++++------
> drivers/net/mlx5/mlx5_flow_hw_stubs.c | 68 +++++
> drivers/net/mlx5/mlx5_mac.c | 41 ++-
> drivers/net/mlx5/mlx5_trigger.c | 262 ++++++++++++++++++-
> drivers/net/mlx5/mlx5_vlan.c | 9 +-
> drivers/net/mlx5/windows/mlx5_os.c | 3 +
> 11 files changed, 867 insertions(+), 116 deletions(-) create mode 100644
> drivers/net/mlx5/mlx5_flow_hw_stubs.c
>
> --
> 2.39.5
prev parent reply other threads:[~2024-10-17 8:01 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-10-17 7:57 Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 01/10] net/mlx5: track unicast DMAC control flow rules Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 02/10] net/mlx5: add checking if unicast flow rule exists Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 03/10] net/mlx5: rework creation of unicast flow rules Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 04/10] net/mlx5: support destroying " Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 05/10] net/mlx5: rename control flow rules types Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 06/10] net/mlx5: shared init of control flow rules Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 07/10] net/mlx5: add legacy unicast flow rules management Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 08/10] net/mlx5: add legacy unicast flow rule registration Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 09/10] net/mlx5: add dynamic unicast flow rule management Dariusz Sosnowski
2024-10-17 7:57 ` [PATCH 10/10] net/mlx5: optimize MAC address and VLAN filter handling Dariusz Sosnowski
2024-10-17 8:01 ` Slava Ovsiienko [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM4PR12MB754970B1D31D56B739B7F161DF472@DM4PR12MB7549.namprd12.prod.outlook.com \
--to=viacheslavo@nvidia.com \
--cc=bingz@nvidia.com \
--cc=dev@dpdk.org \
--cc=dsosnowski@nvidia.com \
--cc=matan@nvidia.com \
--cc=orika@nvidia.com \
--cc=suanmingm@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).