[dpdk-users] Generic Virtualization approach of DPDK based application using SR-IOV and PCI Passthrough to be compatible with all Hardware NICs

[dpdk-users] Generic Virtualization approach of DPDK based application using SR-IOV and PCI Passthrough to be compatible with all Hardware NICs

[Suchetha p]

Hi,

We are trying to bring up KVM based VMs on HP Gen10 Servers running on RHEL
7.6 and having Intel X710 NIC's.

The VMs are also running on RHEL 7.6 and the DPDK version used in the guest
application is 18.08.

Intel DPDK VF drivers (igb_uio) compiled and loaded currently in our VM are
not compatible with the Host PF drivers(vfio-pci) for Intel X710 Ethernet
NICs.

The igb_uio driver may be only compatible with Intel NICs like 82599 as it
worked in our lab for HP Gen9 servers running RHEL 7.6 having Intel 82599
NICs.

Issues observed with DPDK VF driver igb_uio for Intel X710 NICs:

The outgoing packets from DPDK application are leaving VM via VF towards
Host’s Physical ports and reaching remote Host’s Physical ports.

But the incoming packets from Host’s Physical port are not reaching VM via
VF.

Could you please confirm whether igb_uio driver is compatible with only
Intel NICs like 82599 ?

So we had to change PCI device binding to another driver vfio-pci available
in RHEL7.

But based on the debugging of our DPDK based application with VF driver
vfio-pci on X710 NICs we observed that the incoming packets are VLAN
stripped by HW when the OFFLOAD flags are not set for the same.

why  does X710 NIC VF driver remove the VLAN without RX offload VLAN strip
flags set?

Could you please check and confirm if the unexpected VLAN removal behaviour
of X710 NIC VF driver vfio-pci is a known bug?

We want to generalize DPDK implementation for all HW NICs.Could you please
clarify if this is achievable with igb_uio or vfio-pci driver ?

We would  appreciate if you could provide your feedback and suggestions
towards generic solution.

Thanks,

Suchetha

Re: [dpdk-users] Generic Virtualization approach of DPDK based application using SR-IOV and PCI Passthrough to be compatible with all Hardware NICs

[Hirok J. Bora]

Avoid SR-IOV and implement OVS on the host. Implement acceleration at host
level first then guest level, if needed.

SR-IOV does not provide L2 level switching flexibility. Do use overlay
between ovs and the guest, implement vxlan. This will give you 100% control
on the HW. All the best 👍

On Sat, Feb 1, 2020, 10:50 PM Suchetha p <suchetha2005@gmail.com> wrote:

> Hi,
>
> We are trying to bring up KVM based VMs on HP Gen10 Servers running on RHEL
> 7.6 and having Intel X710 NIC's.
>
> The VMs are also running on RHEL 7.6 and the DPDK version used in the guest
> application is 18.08.
>
> Intel DPDK VF drivers (igb_uio) compiled and loaded currently in our VM are
> not compatible with the Host PF drivers(vfio-pci) for Intel X710 Ethernet
> NICs.
>
> The igb_uio driver may be only compatible with Intel NICs like 82599 as it
> worked in our lab for HP Gen9 servers running RHEL 7.6 having Intel 82599
> NICs.
>
> Issues observed with DPDK VF driver igb_uio for Intel X710 NICs:
>
> The outgoing packets from DPDK application are leaving VM via VF towards
> Host’s Physical ports and reaching remote Host’s Physical ports.
>
> But the incoming packets from Host’s Physical port are not reaching VM via
> VF.
>
> Could you please confirm whether igb_uio driver is compatible with only
> Intel NICs like 82599 ?
>
> So we had to change PCI device binding to another driver vfio-pci available
> in RHEL7.
>
> But based on the debugging of our DPDK based application with VF driver
> vfio-pci on X710 NICs we observed that the incoming packets are VLAN
> stripped by HW when the OFFLOAD flags are not set for the same.
>
> why  does X710 NIC VF driver remove the VLAN without RX offload VLAN strip
> flags set?
>
> Could you please check and confirm if the unexpected VLAN removal behaviour
> of X710 NIC VF driver vfio-pci is a known bug?
>
> We want to generalize DPDK implementation for all HW NICs.Could you please
> clarify if this is achievable with igb_uio or vfio-pci driver ?
>
> We would  appreciate if you could provide your feedback and suggestions
> towards generic solution.
>
> Thanks,
>
> Suchetha
>

Re: [dpdk-users] Generic Virtualization approach of DPDK based application using SR-IOV and PCI Passthrough to be compatible with all Hardware NICs

[madhukar mythri]

It worked well with "igb_uio" on both X710 and 82599 NIC's with SR-IOV
enabled. I had tested with DPDK-18.11 version.
Have you enabled the "intel_iommu=on" in host Kernel boot parameters ?


On Sat, Feb 1, 2020 at 10:50 PM Suchetha p <suchetha2005@gmail.com> wrote:

> Hi,
>
> We are trying to bring up KVM based VMs on HP Gen10 Servers running on RHEL
> 7.6 and having Intel X710 NIC's.
>
> The VMs are also running on RHEL 7.6 and the DPDK version used in the guest
> application is 18.08.
>
> Intel DPDK VF drivers (igb_uio) compiled and loaded currently in our VM are
> not compatible with the Host PF drivers(vfio-pci) for Intel X710 Ethernet
> NICs.
>
> The igb_uio driver may be only compatible with Intel NICs like 82599 as it
> worked in our lab for HP Gen9 servers running RHEL 7.6 having Intel 82599
> NICs.
>
> Issues observed with DPDK VF driver igb_uio for Intel X710 NICs:
>
> The outgoing packets from DPDK application are leaving VM via VF towards
> Host’s Physical ports and reaching remote Host’s Physical ports.
>
> But the incoming packets from Host’s Physical port are not reaching VM via
> VF.
>
> Could you please confirm whether igb_uio driver is compatible with only
> Intel NICs like 82599 ?
>
> So we had to change PCI device binding to another driver vfio-pci available
> in RHEL7.
>
> But based on the debugging of our DPDK based application with VF driver
> vfio-pci on X710 NICs we observed that the incoming packets are VLAN
> stripped by HW when the OFFLOAD flags are not set for the same.
>
> why  does X710 NIC VF driver remove the VLAN without RX offload VLAN strip
> flags set?
>
> Could you please check and confirm if the unexpected VLAN removal behaviour
> of X710 NIC VF driver vfio-pci is a known bug?
>
> We want to generalize DPDK implementation for all HW NICs.Could you please
> clarify if this is achievable with igb_uio or vfio-pci driver ?
>
> We would  appreciate if you could provide your feedback and suggestions
> towards generic solution.
>
> Thanks,
>
> Suchetha
>