My question was theorical, is it possible to do the same load balance with the rss flow for vf as for pf for inner ip over gre, but then I tried and failed to initiate the testpmd application
and also my own application.
I sent a separate email about it, but still didn’t succeed to find the problem – with mail title “sriov net_mlx5 error - Unexpected error in DR drop action support detection”
That was the content of the mail -
Here is the code of the program I run ( I run it with one rx queue and one tx queue), same program run for real interface (pf not vf)
I run it on vm (over esxi – vmware ) I don’t have access to host
https://coliru.stacked-crooked.com/a/6e2b9cd6cf047048
Here is the output of the program I run
/home/deployment/testReceiver --log-level='.*',8 -l 7,0-15 -w 0b:00.0
Testpmd also failed to initiate with same parmeters
(I succeeded to run on another host with pf with dpdk19, and also +/- same code with dpdk17 with vf and pf.)
Some information about the system:
Network devices using kernel driver
===================================
0000:0b:00.0 'MT27800 Family [ConnectX-5 Virtual Function] 1018' if=ens192 drv=mlx5_core unused=igb_uio
0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens224 drv=vmxnet3 unused=igb_uio *Active*
0000:1b:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens256 drv=vmxnet3 unused=igb_uio *Active*
ethtool -i ens192
driver: mlx5_core
version: 5.0-0
firmware-version: 16.29.1016 (HPE0000000009)
expansion-rom-version:
bus-info: 0000:0b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
Some output logs:
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Detected lcore 2 as core 0 on socket 0
EAL: Detected lcore 3 as core 0 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 0 on socket 0
EAL: Detected lcore 6 as core 0 on socket 0
EAL: Detected lcore 7 as core 0 on socket 0
EAL: Detected lcore 8 as core 0 on socket 0
EAL: Detected lcore 9 as core 0 on socket 0
EAL: Detected lcore 10 as core 0 on socket 0
EAL: Detected lcore 11 as core 0 on socket 0
EAL: Detected lcore 12 as core 0 on socket 0
EAL: Detected lcore 13 as core 0 on socket 0
EAL: Detected lcore 14 as core 0 on socket 0
EAL: Detected lcore 15 as core 0 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Ask a virtual area of 0x5000 bytes
EAL: Virtual area found at 0x100000000 (size = 0x5000)
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory)
EAL: VFIO PCI modules not loaded
EAL: Bus pci wants IOVA as 'DC'
EAL: Buses did not request a specific IOVA mode.
EAL: IOMMU is not available, selecting IOVA as PA mode.
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: Module /sys/module/vfio not found! error 2 (No such file or directory)
EAL: VFIO modules not loaded, skipping VFIO support...
EAL: Ask a virtual area of 0x2e000 bytes
EAL: Virtual area found at 0x100005000 (size = 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 4096
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Creating 4 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x100033000 (size = 0x1000)
EAL: Memseg list allocated: 0x100000kB at socket 0
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x940000000 (size = 0x1000)
EAL: Memseg list allocated: 0x100000kB at socket 0
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x980000000 (size = 0x800000000)
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x1180000000 (size = 0x1000)
EAL: Memseg list allocated: 0x100000kB at socket 0
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x11c0000000 (size = 0x800000000)
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x19c0000000 (size = 0x1000)
EAL: Memseg list allocated: 0x100000kB at socket 0
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x1a00000000 (size = 0x800000000)
EAL: TSC frequency is ~2000000 KHz
EAL: Master lcore 0 is ready (tid=7ffff7fe7900;cpuset=[0])
EAL: lcore 1 is ready (tid=7ffff4c81700;cpuset=[1])
EAL: lcore 11 is ready (tid=7fffefc77700;cpuset=[11])
EAL: lcore 10 is ready (tid=7ffff0478700;cpuset=[10])
EAL: lcore 14 is ready (tid=7fffee474700;cpuset=[14])
EAL: lcore 2 is ready (tid=7ffff4480700;cpuset=[2])
EAL: lcore 3 is ready (tid=7ffff3c7f700;cpuset=[3])
EAL: lcore 4 is ready (tid=7ffff347e700;cpuset=[4])
EAL: lcore 15 is ready (tid=7fffedc73700;cpuset=[15])
EAL: lcore 12 is ready (tid=7fffef476700;cpuset=[12])
EAL: lcore 9 is ready (tid=7ffff0c79700;cpuset=[9])
EAL: lcore 7 is ready (tid=7ffff1c7b700;cpuset=[7])
EAL: lcore 8 is ready (tid=7ffff147a700;cpuset=[8])
EAL: lcore 5 is ready (tid=7ffff2c7d700;cpuset=[5])
EAL: lcore 13 is ready (tid=7fffeec75700;cpuset=[13])
EAL: lcore 6 is ready (tid=7ffff247c700;cpuset=[6])
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0
EAL: request: mp_malloc_sync
EAL: Heap on socket 0 was expanded by 1024MB
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL: probe driver: 15b3:1018 net_mlx5
EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered
net_mlx5: checking device "mlx5_0"
net_mlx5: PCI information matches for device "mlx5_0"
net_mlx5: no E-Switch support detected
net_mlx5: naming Ethernet device "0b:00.0"
net_mlx5: DevX is NOT supported
net_mlx5: enhanced MPW is supported
net_mlx5: SWP support: 0
net_mlx5: min_single_stride_log_num_of_bytes: 6
net_mlx5: max_single_stride_log_num_of_bytes: 13
net_mlx5: min_single_wqe_log_num_of_strides: 9
net_mlx5: max_single_wqe_log_num_of_strides: 16
net_mlx5: supported_qpts: 256
net_mlx5: device supports Multi-Packet RQ
net_mlx5: tunnel offloading is supported
net_mlx5: MPLS over GRE/UDP tunnel offloading is not supported
net_mlx5: checksum offloading is supported
net_mlx5: maximum Rx indirection table size is 512
net_mlx5: VLAN stripping is supported
net_mlx5: FCS stripping configuration is supported
net_mlx5: enhanced MPS is enabled
net_mlx5: port 0 MAC address is 00:50:56:a9:a9:3e
net_mlx5: port 0 MTU is 2040
net_mlx5: port 0 forcing Ethernet interface up
net_mlx5: min tx inline configured: 0
net_mlx5: Hash list with mlx5_0_flow_table size 0x1000 is created.
net_mlx5: Hash list with mlx5_0_tags size 0x2000 is created.
net_mlx5: port 0 flow maximum priority: 3
net_mlx5: metadata mode 0
net_mlx5: metadata MARK mask 00FFFFFF
net_mlx5: metadata META mask FFFFFFFF
net_mlx5: metadata reg_c0 mask FFFFFFFF
net_mlx5: port 0 extensive metadata register is not supported
net_mlx5: Unexpected error in DR drop action support detection
EAL: Module /sys/module/vfio not found! error 2 (No such file or directory)
Succeeded to init eal
net_mlx5: port 0 Tx queues number update: 0 -> 1
net_mlx5: port 0 Rx queues number update: 0 -> 1
Succeeded to configure port
net_mlx5: port 0 adapter MTU set to 2040
Succeeded to set mtu port
net_mlx5: port 0 configuring Rx queue 0 for 512 descriptors
net_mlx5: port 0 maximum number of segments per packet: 1
net_mlx5: port 0 CRC stripping is disabled, 4 bytes will be subtracted from incoming frames to hide it
net_mlx5: port 0 adding Rx queue 0 to list
Succeeded to rx queue setup queue 0
testReceiver.cpp 103
net_mlx5: port 0 configuring queue 0 for 512 descriptors
net_mlx5: port 0 adding Tx queue 0 to list
Succeeded to tx queue setup queue 0
net_mlx5: port 0 starting device
net_mlx5: port 0 Rx queues number update: 1 -> 1
net_mlx5: port 0 Tx queue 0 allocated and configured 512 WRs
net_mlx5: port 0: uar_mmap_offset 0x306000
net_mlx5: port 0 Rx queue 0 registering mp pool having 1 chunks
net_mlx5: port 0 creating a MR using address (0x15a9fd6c0)
net_mlx5: device mlx5_0 inserting MR(0x15a9f4ec0) to global cache
net_mlx5: inserted B-tree(0x17ffdcb60)[1], [0x140000000, 0x180000000) lkey=0x7a0d0100
net_mlx5: inserted B-tree(0x15a9fbfec)[1], [0x140000000, 0x180000000) lkey=0x7a0d0100
net_mlx5: port 0 Rx queue 0 allocated and configured 512 segments (max 512 packets)
net_mlx5: port 0 device_attr.max_qp_wr is 32768
net_mlx5: port 0 device_attr.max_sge is 30
net_mlx5: port 0 rxq 0 updated with 0x7fffffffd688
net_mlx5: port 0 failed to set defaults flows
net_mlx5: port 0 Rx queue 0 freeing WRs
DPDK: Error starting or stopping port -12
From: Asaf Penso <asafp@nvidia.com>
Sent: Friday, April 1, 2022 9:39 AM
To: Yaron Illouz <yaroni@radcom.com>; users@dpdk.org; dev@dpdk.org
Subject: Re: dpdk rte rss flow with sriov
I understand your scenario.
I want referring to action VF.
Can you share the testpmd commands line you use in the VM?
Regards,
Asaf Penso
From: Yaron Illouz <yaroni@radcom.com>
Sent: Tuesday, March 29, 2022 5:33:39 PM
To: Asaf Penso <asafp@nvidia.com>;
users@dpdk.org <users@dpdk.org>;
dev@dpdk.org <dev@dpdk.org>
Subject: RE: dpdk rte rss flow with sriov
Perhaps I didn’t ask correctly
My vm have a
vf sriov,
The dpdk application will create x rx queues on the vf
as it should have done for a pf
I expect traffic
to be spreaded between the rx queues
according to rte_flow rss defined (RTE_FLOW_ACTION_TYPE_QUEUE or RTE_FLOW_ACTION_TYPE_RSS)
Asaf: I am not asking about the RTE_FLOW_ACTION_TYPE_VF – When you said Mellanox
support where you referring to RTE_FLOW_ACTION_TYPE_VF?
From: Asaf Penso <asafp@nvidia.com>
Sent: Tuesday, March 29, 2022 4:40 PM
To: Yaron Illouz <yaroni@radcom.com>;
users@dpdk.org;
dev@dpdk.org
Subject: RE: dpdk rte rss flow with sriov
|
EXTERNAL EMAIL: Do not click links or attachments
unless you recognize the sender and know the content is safe |
Mellanox NICs support RSS for PF/VF/SF.
Regards,
Asaf Penso
From: Yaron Illouz <yaroni@radcom.com>
Sent: Tuesday, March 29, 2022 4:35 PM
To: users@dpdk.org;
dev@dpdk.org
Subject: dpdk rte rss flow with sriov
Hi
Is this possible to use dpdk rte rss flow with a vf sriov, or is it an option available only for real interface?
I use dpdk 19.11 and a Mellanox card 100G with mlx5