[PATCH 0/4] [25.11] net/mlx5: add driver event and steering toggle APIs

[Maayan Kashani <mkashani@nvidia.com>]

mlx5 PMD is a bifurcated driver, which means that instead of communicating with HW directly
using UIO or VFIO, the driver uses existing kernel and userspace APIs for that purpose.
mlx5 PMD requires that `mlx5_core`, `mlx5_ib` and `ib_uverbs` kernel modules are loaded,
and that NIC is bound to `mlx5_core` driver.

Kernel bypass is achieved through rdma-core library
which exposes required functionalities e.g.,
mapping memory to the device, FW access for setting up Rx and Tx queues
(through mlx5dv_devx_* interfaces) and flow rule offloading.

One benefit of such architecture is that it allows libraries and applications
to build additional features on top of DPDK, utilizing rdma-core directly.
For example it is possible to implement additional flow rule offloads.
This can be achieved by creating an IBV (IB Verbs) context and PD (protection domain)
through rdma-core and sharing it with mlx5 PMD through `cmd_fd` and `pd_handle` device arguments
provided during probing [1].
mlx5 PMD will then create all necessary resources for DPDK using imported context and PD.

What is missing for that approach to work, is that
external libraries and applications are not aware of HW identifiers of Rx and Tx queues
(returned from mlx5dv_devx_* APIs) created by DPDK through imported context and PD.
Knowledge of these identifiers is crucial for correct flow rule offload.

This patchset proposes the introduction of the following private PMD APIs
to address the above use case:

1. mlx5 PMD driver events
2. mlx5 PMD steering toggle

mlx5 PMD driver events, exposed through `rte_pmd_mlx5_driver_event_cb_register/unregister()` functions,
allow libraries and applications to register a custom callback.
This callback will be called whenever mlx5 PMD creates or destroys an Rx/Tx queue.
All necessary HW identifiers along with corresponding DPDK port and queue
indexes will be passed as parameters.

mlx5 PMD steering toggle, exposed through `rte_pmd_mlx5_driver_disable/enable_steering()` functions,
allow libraries and applications to disable/enable flow rule offloading provided by mlx5 PMD.
This is added to support use cases where library/application needs full control of flow rule offloading.
Library/application will use data provided through driver events API to configure flow rules by itself.
When flow rule offloading is disabled, DPDK flow API will return ENOSYS for mlx5 PMD.

Patches:

- Patch 1 - Add mlx5 PMD driver events private API
- Patches 2-3 - Minor clean ups in mlx5 PMD which make introduction of
  steering toggle private API easier.
- Patch 4 - Add mlx5 PMD steering toggle private API

[1]: https://doc.dpdk.org/guides/platform/mlx5.html#device-arguments

Dariusz Sosnowski (4):
  net/mlx5: add driver event callbacks
  net/mlx5: move eCPRI release function to flex
  net/mlx5: rework Rx queue mark flag functions
  net/mlx5: add steering toggle API

 drivers/net/mlx5/meson.build         |   1 +
 drivers/net/mlx5/mlx5.c              |  18 --
 drivers/net/mlx5/mlx5.h              |   1 +
 drivers/net/mlx5/mlx5_devx.c         |  17 ++
 drivers/net/mlx5/mlx5_driver_event.c | 300 +++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_driver_event.h |  18 ++
 drivers/net/mlx5/mlx5_flow.c         | 199 +++++++++++++++++-
 drivers/net/mlx5/mlx5_flow.h         |   7 +-
 drivers/net/mlx5/mlx5_flow_flex.c    |  18 ++
 drivers/net/mlx5/mlx5_trigger.c      |  30 +++
 drivers/net/mlx5/rte_pmd_mlx5.h      | 192 +++++++++++++++++
 11 files changed, 772 insertions(+), 29 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_driver_event.c
 create mode 100644 drivers/net/mlx5/mlx5_driver_event.h

-- 
2.21.0