[DPDK/ethdev Bug 1592] AF_PACKET PMD loops back packets on veth with tc

[DPDK/ethdev Bug 1592] AF_PACKET PMD loops back packets on veth with tc

[bugzilla]

[-- Attachment #1: Type: text/plain, Size: 4026 bytes --]

https://bugs.dpdk.org/show_bug.cgi?id=1592

            Bug ID: 1592
           Summary: AF_PACKET PMD loops back packets on veth with tc
           Product: DPDK
           Version: unspecified
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: dev@dpdk.org
          Reporter: thea.rossman@cs.stanford.edu
  Target Milestone: ---

Found using AF_PACKET PMD on veth with Linux TC via Mininet. Ubuntu 24.04.1
LTS.

High-level, I have a basic Mininet topology with `h1 <-> r1 <-> h2`. (Each link
represents a pair of veths.) `r1` is a transparent middlebox running the DPDK
`skeleton/basicfwd.c` example, which should forward packets through r1 (eth0 ->
eth1 and vice verse). I ping h1 <-> h2, expecting `r1` to perform this
bridging. 

The DPDK basicfwd program was started with the vdev configuration:
`--vdev=eth_af_packet0,iface=r1-eth0,qdisc_bypass=0
--vdev=eth_af_packet1,iface=r1-eth1,qdisc_bypass=0 &`.

(I turn off qdisc_bypass because I'd like to use TC.) 

When one of r1's veths have TC params (e.g., delay), this all works fine. 

However, when both of r1's veths (r1-eth0 and r1-eth1) have TC params set up,
traffic is looped forever. Here's an example output from h1: 

```
...
64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!)
64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!)
64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!)
64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!)
64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=161 ms (DUP!)
64 bytes from 10.0.1.9: icmp_seq=1 ttl=64 time=181 ms (DUP!)
... (continues perpetually until stopped)
```

Note that this issue only occurs if TC parameters (e.g., delay) are configured
on both egress veths. If qdisc_bypass is disabled on both, but no actual TC
parameters are set, the duplication does not happen. 

I took packet captures (tcpdump), and I also printed packets that the
application was actually seeing. It appears that when the middlebox writes a
packet to a socket, it immediately reads the same packet out from the same
socket. 

I also tried making small modifications in the DPDK driver to investigate
whether this could be a Linux or Mininet issue (e.g., packet mmap + TC?) vs. a
DPDK driver issue. I reduced the # of RX/TX queues to 1, and I also turned of
packet mmap (by removing the setsockopt calls in rte_eth_af_packet.c for
`PACKET_RX_RING` and `PACKET_TX_RING` -- not sure if there's anything else I
would need to do here?).  

A basic python script for repro is below. This works fine when only one of the
`r1.cmd("tc qdisc add dev r1-ethX root netem delay 10ms")` commands is present.
When both are added, the pings begin to loop. 

```python
from mininet.net import Mininet
from mininet.link import TCLink
from mininet.cli import CLI

net = Mininet(controller=None, link=TCLink)
# Add hosts in same subnet
h1 = net.addHost('h1', ip='10.0.1.10/24', mac='00:00:00:00:00:01')
h2 = net.addHost('h2', ip='10.0.1.9/24', mac='00:00:00:00:00:02')
r1 = net.addHost('r1')
net.addLink(r1, h1)
net.addLink(r1, h2)
net.build()

# Configure TC on egress 
# (could also be done by adding `delay` param to `addLink` above)
h2.cmd("tc qdisc add dev h2-eth0 root netem delay 10ms")
h1.cmd("tc qdisc add dev h1-eth0 root netem delay 10ms")
r1.cmd("tc qdisc add dev r1-eth0 root netem delay 10ms")
r1.cmd("tc qdisc add dev r1-eth1 root netem delay 10ms")

r1.cmd('sudo /path/to/dpdk-24.07/examples/skeleton/build/basicfwd 0 1
--vdev=eth_af_packet0,iface=r1-eth0,qdisc_bypass=0
--vdev=eth_af_packet1,iface=r1-eth1,qdisc_bypass=0 &')

CLI(net)
net.stop()
```

I think there is a bug here, though I haven't been able to figure out for sure
what might be going on or confirm 100% whether the issue is in the PMD.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #2: Type: text/html, Size: 5981 bytes --]

[DPDK/ethdev Bug 1592] AF_PACKET PMD loops back packets on veth with tc

[bugzilla]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

https://bugs.dpdk.org/show_bug.cgi?id=1592

Stephen Hemminger (stephen@networkplumber.org) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED
                 CC|                            |stephen@networkplumber.org

--- Comment #1 from Stephen Hemminger (stephen@networkplumber.org) ---
This is not a DPDK bug.
The Linux kernel will interpret the packets it receives.
By default Linux has a weak host model, and it will look at the L3 header.

If you want to forward you need to either use hardware DPDK network devices
(not packet, tun, ...); or put each eth device into a different network or
namespace.

The DPDK part of forwarding has nothing wrong here.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #2: Type: text/html, Size: 3123 bytes --]