RE: [RFC 00/13] Packet capture using port mirroring

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Saturday, 12 April 2025 01.45
> 
> This is a rework of how packet capture is done in DPDK.
> The existing mechanism using callbacks has a number of problems;
> the main one is that any packets sent/received in a secondary process
> are not visible. The root cause is that callbacks can not function
> across process boundaries, they are specific to the role.
> 
> The new mechanism builds on the concept of port mirroring used
> in Linux/FreeBSD and also router vendors (SPAN ports). The
> infrastructure
> is built around a new ethdev call to mirror a port.
> 
> The internals of dumpcap are redone but the program command
> syntax is unchanged (no doc change needed). Usingthe new mirror
> mechanism,
> the dumpcap program creates a port using ring PMD and
> then directs mirror to that. Then the packets are extracted from the
> ring.
> If capturing on multiple devices, they all get mirrored to
> the same ring PMD.

Here's some general feedback...

I very much like the concept of using a shared ring for carrying the mirrored packets.
It allows other types of future consumers to process the mirrored packets, e.g. encapsulating and forwarding them into an L2 or L3 tunnel, or Wireshark remote capture.

Using a Ring PMD instead of setting up a dedicated ring also has some advantages, such as the ability to set up multiple separate mirror target instances.
For performance reasons, we should ensure that a lightweight Ring PMD is available for mirroring, in case the Ring PMD is extended with new features affecting its performance.
Or maybe create a new type of mirror/capture virtual PMD. This would allow applications to enqueue packets from non-ethdev interfaces into it.

I'm not convinced you should undo the VLAN offloads when enqueueing a mirror packet... If you do, you should also undo QinQ offloads. Undoing offloads is never going to end.
And if you create a dedicated PMD type for carrying mirrored packets, you can ensure that the offload fields remain intact on dequeue.

You should consider sampling and VLAN filtering as typical mirroring features.
It would improve the performance if such filtering is done before copying the packets.

PS: I agree with your choice of copying (rather than cloning by refcount) when mirroring the packets.

> 
> Doing this uncovered a bunch of additional missing features and bugs.
> The first is that test-pmd auto attach introduced in 24.11 breaks this
> (and it broke old pdump as well) so it is removed.
> 
> The dumpcap program needs to be able to start the ring PMD it created
> but the existing ethdev API is broken when used from secondary process.
> So fix that.
> 
> The mirror API has a restriction that the port being mirrored to must
> allow
> lock free transmit. This is because both rx and tx as well as multiple
> queues need to go over the same transmit queue. Although it might be
> possible
> to have some complex mapping between multiple ports/queues being
> mirrored
> to multiple transmit queues, that gets to be a lot of overhead in API's
> and implementation. Fixing the ring and null PMD to allow lockeless
> transmit is good enough.
> 
> Added tests for the new features.
> The performance has not been measured but the overhead of checking for
> mirror port is no more that the overhead of looking at the callback
> list.
> 
> TODO items:
>   - need release notes for new features
>   - more through testing on real hardware
>   - mark old pdump API and application for deprecation in 25.07 and
> removal in 25.11
>   - more cleanup of patch wording and docs needed.