My question was theorical, is it possible to do the same load balance with the rss flow for vf as for pf for inner ip over gre, but then I tried and failed to initiate the testpmd application and also my own application. I sent a separate email about it, but still didn’t succeed to find the problem – with mail title “sriov net_mlx5 error - Unexpected error in DR drop action support detection” That was the content of the mail - 1. Dpdk version: 19.11 (dpdk-stable-19.11.11) 2. Version of rdma core is 26.0 (from config.h #define PACKAGE_VERSION "26.0") 3. “Unexpected error in DR drop action support detection” message is coming from rte_eal_init 4. “DPDK: Error starting or stopping port -12” is coming from rte_eth_dev_start Here is the code of the program I run ( I run it with one rx queue and one tx queue), same program run for real interface (pf not vf) I run it on vm (over esxi – vmware ) I don’t have access to host https://coliru.stacked-crooked.com/a/6e2b9cd6cf047048 Here is the output of the program I run /home/deployment/testReceiver --log-level='.*',8 -l 7,0-15 -w 0b:00.0 Testpmd also failed to initiate with same parmeters (I succeeded to run on another host with pf with dpdk19, and also +/- same code with dpdk17 with vf and pf.) Some information about the system: Network devices using kernel driver =================================== 0000:0b:00.0 'MT27800 Family [ConnectX-5 Virtual Function] 1018' if=ens192 drv=mlx5_core unused=igb_uio 0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens224 drv=vmxnet3 unused=igb_uio *Active* 0000:1b:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens256 drv=vmxnet3 unused=igb_uio *Active* ethtool -i ens192 driver: mlx5_core version: 5.0-0 firmware-version: 16.29.1016 (HPE0000000009) expansion-rom-version: bus-info: 0000:0b:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes Some output logs: EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Detected lcore 4 as core 0 on socket 0 EAL: Detected lcore 5 as core 0 on socket 0 EAL: Detected lcore 6 as core 0 on socket 0 EAL: Detected lcore 7 as core 0 on socket 0 EAL: Detected lcore 8 as core 0 on socket 0 EAL: Detected lcore 9 as core 0 on socket 0 EAL: Detected lcore 10 as core 0 on socket 0 EAL: Detected lcore 11 as core 0 on socket 0 EAL: Detected lcore 12 as core 0 on socket 0 EAL: Detected lcore 13 as core 0 on socket 0 EAL: Detected lcore 14 as core 0 on socket 0 EAL: Detected lcore 15 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 16 lcore(s) EAL: Detected 1 NUMA nodes EAL: Ask a virtual area of 0x5000 bytes EAL: Virtual area found at 0x100000000 (size = 0x5000) EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory) EAL: VFIO PCI modules not loaded EAL: Bus pci wants IOVA as 'DC' EAL: Buses did not request a specific IOVA mode. EAL: IOMMU is not available, selecting IOVA as PA mode. EAL: Selected IOVA mode 'PA' EAL: Probing VFIO support... EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) EAL: VFIO modules not loaded, skipping VFIO support... EAL: Ask a virtual area of 0x2e000 bytes EAL: Virtual area found at 0x100005000 (size = 0x2e000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 4096 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 EAL: Creating 4 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x1000 bytes EAL: Virtual area found at 0x100033000 (size = 0x1000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x140000000 (size = 0x800000000) EAL: Ask a virtual area of 0x1000 bytes EAL: Virtual area found at 0x940000000 (size = 0x1000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x980000000 (size = 0x800000000) EAL: Ask a virtual area of 0x1000 bytes EAL: Virtual area found at 0x1180000000 (size = 0x1000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x11c0000000 (size = 0x800000000) EAL: Ask a virtual area of 0x1000 bytes EAL: Virtual area found at 0x19c0000000 (size = 0x1000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x1a00000000 (size = 0x800000000) EAL: TSC frequency is ~2000000 KHz EAL: Master lcore 0 is ready (tid=7ffff7fe7900;cpuset=[0]) EAL: lcore 1 is ready (tid=7ffff4c81700;cpuset=[1]) EAL: lcore 11 is ready (tid=7fffefc77700;cpuset=[11]) EAL: lcore 10 is ready (tid=7ffff0478700;cpuset=[10]) EAL: lcore 14 is ready (tid=7fffee474700;cpuset=[14]) EAL: lcore 2 is ready (tid=7ffff4480700;cpuset=[2]) EAL: lcore 3 is ready (tid=7ffff3c7f700;cpuset=[3]) EAL: lcore 4 is ready (tid=7ffff347e700;cpuset=[4]) EAL: lcore 15 is ready (tid=7fffedc73700;cpuset=[15]) EAL: lcore 12 is ready (tid=7fffef476700;cpuset=[12]) EAL: lcore 9 is ready (tid=7ffff0c79700;cpuset=[9]) EAL: lcore 7 is ready (tid=7ffff1c7b700;cpuset=[7]) EAL: lcore 8 is ready (tid=7ffff147a700;cpuset=[8]) EAL: lcore 5 is ready (tid=7ffff2c7d700;cpuset=[5]) EAL: lcore 13 is ready (tid=7fffeec75700;cpuset=[13]) EAL: lcore 6 is ready (tid=7ffff247c700;cpuset=[6]) EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 1024MB EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: probe driver: 15b3:1018 net_mlx5 EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered net_mlx5: checking device "mlx5_0" net_mlx5: PCI information matches for device "mlx5_0" net_mlx5: no E-Switch support detected net_mlx5: naming Ethernet device "0b:00.0" net_mlx5: DevX is NOT supported net_mlx5: enhanced MPW is supported net_mlx5: SWP support: 0 net_mlx5: min_single_stride_log_num_of_bytes: 6 net_mlx5: max_single_stride_log_num_of_bytes: 13 net_mlx5: min_single_wqe_log_num_of_strides: 9 net_mlx5: max_single_wqe_log_num_of_strides: 16 net_mlx5: supported_qpts: 256 net_mlx5: device supports Multi-Packet RQ net_mlx5: tunnel offloading is supported net_mlx5: MPLS over GRE/UDP tunnel offloading is not supported net_mlx5: checksum offloading is supported net_mlx5: maximum Rx indirection table size is 512 net_mlx5: VLAN stripping is supported net_mlx5: FCS stripping configuration is supported net_mlx5: enhanced MPS is enabled net_mlx5: port 0 MAC address is 00:50:56:a9:a9:3e net_mlx5: port 0 MTU is 2040 net_mlx5: port 0 forcing Ethernet interface up net_mlx5: min tx inline configured: 0 net_mlx5: Hash list with mlx5_0_flow_table size 0x1000 is created. net_mlx5: Hash list with mlx5_0_tags size 0x2000 is created. net_mlx5: port 0 flow maximum priority: 3 net_mlx5: metadata mode 0 net_mlx5: metadata MARK mask 00FFFFFF net_mlx5: metadata META mask FFFFFFFF net_mlx5: metadata reg_c0 mask FFFFFFFF net_mlx5: port 0 extensive metadata register is not supported net_mlx5: Unexpected error in DR drop action support detection EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) Succeeded to init eal net_mlx5: port 0 Tx queues number update: 0 -> 1 net_mlx5: port 0 Rx queues number update: 0 -> 1 Succeeded to configure port net_mlx5: port 0 adapter MTU set to 2040 Succeeded to set mtu port net_mlx5: port 0 configuring Rx queue 0 for 512 descriptors net_mlx5: port 0 maximum number of segments per packet: 1 net_mlx5: port 0 CRC stripping is disabled, 4 bytes will be subtracted from incoming frames to hide it net_mlx5: port 0 adding Rx queue 0 to list Succeeded to rx queue setup queue 0 testReceiver.cpp 103 net_mlx5: port 0 configuring queue 0 for 512 descriptors net_mlx5: port 0 adding Tx queue 0 to list Succeeded to tx queue setup queue 0 net_mlx5: port 0 starting device net_mlx5: port 0 Rx queues number update: 1 -> 1 net_mlx5: port 0 Tx queue 0 allocated and configured 512 WRs net_mlx5: port 0: uar_mmap_offset 0x306000 net_mlx5: port 0 Rx queue 0 registering mp pool having 1 chunks net_mlx5: port 0 creating a MR using address (0x15a9fd6c0) net_mlx5: device mlx5_0 inserting MR(0x15a9f4ec0) to global cache net_mlx5: inserted B-tree(0x17ffdcb60)[1], [0x140000000, 0x180000000) lkey=0x7a0d0100 net_mlx5: inserted B-tree(0x15a9fbfec)[1], [0x140000000, 0x180000000) lkey=0x7a0d0100 net_mlx5: port 0 Rx queue 0 allocated and configured 512 segments (max 512 packets) net_mlx5: port 0 device_attr.max_qp_wr is 32768 net_mlx5: port 0 device_attr.max_sge is 30 net_mlx5: port 0 rxq 0 updated with 0x7fffffffd688 net_mlx5: port 0 failed to set defaults flows net_mlx5: port 0 Rx queue 0 freeing WRs DPDK: Error starting or stopping port -12 From: Asaf Penso Sent: Friday, April 1, 2022 9:39 AM To: Yaron Illouz ; users@dpdk.org; dev@dpdk.org Subject: Re: dpdk rte rss flow with sriov I understand your scenario. I want referring to action VF. Can you share the testpmd commands line you use in the VM? Regards, Asaf Penso ________________________________ From: Yaron Illouz > Sent: Tuesday, March 29, 2022 5:33:39 PM To: Asaf Penso >; users@dpdk.org >; dev@dpdk.org > Subject: RE: dpdk rte rss flow with sriov Perhaps I didn’t ask correctly My vm have a vf sriov, The dpdk application will create x rx queues on the vf as it should have done for a pf I expect traffic to be spreaded between the rx queues according to rte_flow rss defined (RTE_FLOW_ACTION_TYPE_QUEUE or RTE_FLOW_ACTION_TYPE_RSS) Asaf: I am not asking about the RTE_FLOW_ACTION_TYPE_VF – When you said Mellanox support where you referring to RTE_FLOW_ACTION_TYPE_VF? From: Asaf Penso > Sent: Tuesday, March 29, 2022 4:40 PM To: Yaron Illouz >; users@dpdk.org; dev@dpdk.org Subject: RE: dpdk rte rss flow with sriov EXTERNAL EMAIL: Do not click links or attachments unless you recognize the sender and know the content is safe Mellanox NICs support RSS for PF/VF/SF. Regards, Asaf Penso From: Yaron Illouz > Sent: Tuesday, March 29, 2022 4:35 PM To: users@dpdk.org; dev@dpdk.org Subject: dpdk rte rss flow with sriov Hi Is this possible to use dpdk rte rss flow with a vf sriov, or is it an option available only for real interface? I use dpdk 19.11 and a Mellanox card 100G with mlx5