[dpdk-dev] Trouble with Tx on some i40 ports in KVM VM

[Andrew Theurer <atheurer@redhat.com>]

Hello.  I am having some trouble with getting various dpdk applications to work when using in a KVM VM.  This happens with the 2nd port on a dual port adapter.  What I have configured:  

Haswell-ep host with KVM, 2 x i40e phys functions bound to vfio-pci, 2 functions assigned to KVM guest.  In the guest, the i40 functions bound to uio_pci_generic, running testpmd with these options: --nb-cores=2 --nb-ports=2 --portmask=3 --interactive --auto-start.  

On another host directly connected to these adapters, I run pktgen on the 2 i40e ports.  If I Tx on port1, I get packets back on port0.  If I Tx on port0, no packets come back on port0.  For some reason Tx simply does not happen on that second port.

I have tried this test where there is no KVM, just using host, and it works as expected.  I have also tried this test with ixgbe adapter in a KVM VM (I have both in same system), and it works as expected.  I have also tried just using a bridge in the VM with i40e ports, with i40e module in use, and that also works as expected.  So, I don't think this has anything to do with the adapters themselves, or cables, etc.  Something is not working when using the KVM VM, I just don't know what it is.

Here is xstats from testpmd:
###### NIC extended statistics for port 1
rx_packets: 134615456
tx_packets: 0
rx_bytes: 8615389184
tx_bytes: 0
tx_errors: 0
rx_missed_errors: 0
rx_crc_errors: 0
rx_bad_length_errors: 0
rx_errors: 0
alloc_rx_buff_failed: 0
fdir_match: 0
fdir_miss: 0
tx_flow_control_xon: 0
rx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_flow_control_xoff: 0
rx_queue_0_rx_packets: 0
rx_queue_0_rx_bytes: 0
tx_queue_0_tx_packets: 0
tx_queue_0_tx_bytes: 0
tx_queue_0_tx_errors: 0

I can't find any evidence of errors anywhere.  It's just not doing Tx at all on that port.  I have also tried using ptkgen in the VM, to manually send packets on that port, and again, no packets sent, but no errors either.

Has anyone else tried 2 i40e ports in a KVM VM?  Any ideas what could be going on here?

Thanks,

-Andrew