[PATCH v5 00/13] Packet capture using port mirroring

[Stephen Hemminger <stephen@networkplumber.org>]

This is a rework of how packet capture is done in DPDK.
The existing mechanism using callbacks has a number of problems:
  - can't work when packets are sent and received in secondary process
    because callbacks only function in the primary process
  - requires "opt-in" from application

The new mechanism builds on the concept of port mirroring used
in Linux/FreeBSD and also router vendors (SPAN ports).
See: https://en.wikipedia.org/wiki/Port_mirroring for a general description.

The implementation uses two new ethdev API's:
one to add a port mirror, and one to remove it.

The internals of dumpcap are redone but the program command syntax is
unchanged (no doc change needed). Using the new mirror mechanism, the
dumpcap program creates a port using ring PMD and then directs a port
mirror to that new ring PMD instance.  Then the packets are extracted
from the ring.  If capturing on multiple devices, they all get
mirrored to the same ring PMD.

The dumpcap program needs to be able to start the ring PMD it created
but the existing ethdev API is broken when used from secondary process.
One of the patches fixes that, it was also reported previously in Bugzilla.

The mirror API has a restriction that the port being mirrored to must allow
lock free transmit. This is because both Rx and Tx as well as multiple
queues need to go over the same transmit queue. Although it might be possible
to have some complex mapping between multiple ports/queues being mirrored
to multiple transmit queues, that gets to be a lot of overhead in API's
and implementation. Fixing the ring and null PMD to allow lockless
transmit is good enough.

Added tests for the mirror features.
The performance has not been measured but the overhead of checking for
mirror port is no more that the overhead of looking at the callback list.

TODO items:
  - need release notes for new features (will wait till 25.11)
  - more through testing on real hardware

v5 - fix build warnings when experimental not enabled.

Stephen Hemminger (13):
  ethdev: allow start/stop from secondary process
  ethdev: make sure all necessary headers included
  test: add test for hotplug and secondary process operations
  net/ring: allow lockfree transmit if ring supports it
  net/ring: add new devarg to create vdev from existing ring
  mbuf: add dynamic flag for use by port mirroring
  ethdev: add port mirroring feature
  pcapng: split packet copy from header insertion
  pcapng: make queue optional
  test: add tests for ethdev mirror
  app/testpmd: support for port mirroring
  app/dumpcap: use port mirror instead of pdump
  pdump: mark API and application as deprecated

 app/dumpcap/main.c                            | 443 +++++++++++++++---
 app/dumpcap/meson.build                       |   2 +-
 app/pdump/main.c                              |   3 +
 app/pdump/meson.build                         |   1 +
 app/test-pmd/cmdline.c                        |   1 +
 app/test-pmd/cmdline_mirror.c                 | 129 +++++
 app/test-pmd/cmdline_mirror.h                 |  12 +
 app/test-pmd/meson.build                      |   2 +
 app/test/meson.build                          |   7 +-
 app/test/test_ethdev_mirror.c                 | 286 +++++++++++
 app/test/test_mp_secondary.c                  |  51 +-
 config/rte_config.h                           |   1 +
 doc/guides/howto/packet_capture_framework.rst |  15 +-
 doc/guides/nics/bnxt.rst                      |   2 +-
 doc/guides/prog_guide/ethdev/index.rst        |   1 +
 doc/guides/prog_guide/pdump_lib.rst           |   4 +
 doc/guides/rel_notes/deprecation.rst          |   3 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |  15 +
 doc/guides/tools/pdump.rst                    |   2 +
 drivers/net/ring/rte_eth_ring.c               |  47 ++
 lib/ethdev/ethdev_driver.c                    |  11 +-
 lib/ethdev/ethdev_driver.h                    |   6 +
 lib/ethdev/ethdev_private.c                   | 142 ++++++
 lib/ethdev/ethdev_private.h                   |  29 ++
 lib/ethdev/ethdev_trace.h                     |  17 +
 lib/ethdev/ethdev_trace_points.c              |   6 +
 lib/ethdev/meson.build                        |   1 +
 lib/ethdev/rte_ethdev.c                       | 101 ++--
 lib/ethdev/rte_ethdev.h                       |  23 +-
 lib/ethdev/rte_ethdev_core.h                  |  12 +-
 lib/ethdev/rte_mirror.c                       | 359 ++++++++++++++
 lib/ethdev/rte_mirror.h                       |  99 ++++
 lib/mbuf/rte_mbuf_dyn.h                       |   9 +
 lib/pcapng/rte_pcapng.c                       | 179 +++----
 lib/pcapng/rte_pcapng.h                       |  27 +-
 lib/pdump/rte_pdump.h                         |  12 +-
 36 files changed, 1843 insertions(+), 217 deletions(-)
 create mode 100644 app/test-pmd/cmdline_mirror.c
 create mode 100644 app/test-pmd/cmdline_mirror.h
 create mode 100644 app/test/test_ethdev_mirror.c
 create mode 100644 lib/ethdev/rte_mirror.c
 create mode 100644 lib/ethdev/rte_mirror.h

-- 
2.47.2