From: fwefew 4t4tg <7532yahoo@gmail.com>
To: users@dpdk.org, dev@dpdk.org
Subject: RSS Hash suport in DPDK on Amazon ENA NICs broken?
Date: Tue, 9 Nov 2021 18:34:17 -0500 [thread overview]
Message-ID: <CA+Tq66WJzsSYKZ1QAuwzs-bwQGDbWPYqHvnBMO8++Lg+VRxs6Q@mail.gmail.com> (raw)
[-- Attachment #1: Type: text/plain, Size: 5077 bytes --]
On AWS EC2 I configure:
- c5n.metal instance
- Stock AWS AMI Ubuntu 20.04 LTS ($ cat /proc/version
Linux version 5.4.0-1045-aws)
- Apply the vfio pci driver from https://github.com/amzn/am
zn-drivers/tree/master/userspace/dpdk/enav2-vfio-patch
- Using DPDK code from head of main branch. I built using ninja/meson
- Add a 2nd ENA enabled NIC
- Setup 2Mb huge pages, and bound the 2nd NIC with dpdk-devbind
I run test-pmd with this command:
$ sudo ./dpdk-testpmd --huge-dir=/mnt/huge -c 0xf -n 4 -- -i
EAL: Detected CPU lcores: 72
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(3)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:7f:00.0 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
Interactive-mode selected
Error picking flow transfer proxy for port 0: Function not implemented -
ignore
testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Warning! port-topology=paired and odd forward ports number, the last port
will pair with itself.
Configuring Port 0 (socket 0)
Port 0: 16:9A:11:4A:5D:61
Checking link statuses...
Done
Error during enabling promiscuous mode for port 0: Operation not supported
- ignore
*Key point: I try to enable RSS hash offload for UDP or TCP. I get the
following error shown in bold:*
testpmd> port config all rss udp
*Port 0 modified RSS hash function based on hardware
support,requested:0x20820 configured:0x820*ena_rss_hash_set(): Setting RSS
hash fields is not supported
ena_rss_hash_update(): Failed to set RSS hash
*Configuration of RSS hash at ethernet port 0 failed with error (95):
Operation not supported.*
testpmd> port config all rss tcp
*Port 0 modified RSS hash function based on hardware
support,requested:0x10410 configured:0x410*ena_rss_hash_set(): Setting RSS
hash fields is not supported
ena_rss_hash_update(): Failed to set RSS hash
*Configuration of RSS hash at ethernet port 0 failed with error (95):
Operation not supported.*
If I run ethtool on the NIC before putting it down state and DPDK binding
to vfio-pci I see an indication that there is a hash key, and RSS Hash
works:
ubuntu$ ethtool -x ens6
RX flow hash indirection table for ens6 with 32 RX ring(s):
0: 0 1 2 3 4 5 6 7
8: 8 9 10 11 12 13 14 15
16: 16 17 18 19 20 21 22 23
24: 24 25 26 27 28 29 30 31
32: 0 1 2 3 4 5 6 7
40: 8 9 10 11 12 13 14 15
48: 16 17 18 19 20 21 22 23
56: 24 25 26 27 28 29 30 31
64: 0 1 2 3 4 5 6 7
72: 8 9 10 11 12 13 14 15
80: 16 17 18 19 20 21 22 23
88: 24 25 26 27 28 29 30 31
96: 0 1 2 3 4 5 6 7
104: 8 9 10 11 12 13 14 15
112: 16 17 18 19 20 21 22 23
120: 24 25 26 27 28 29 30 31
RSS hash key:
80:d3:0f:c4:1d:8a:48:9e:7f:1d:18:91:d3:12:cb:38:87:fc:33:49:18:30:e4:1d:19:8f:9d:fd:c3:94:fe:f3:b2:f0:7f:e9:e9:9d:2d:af
RSS hash function:
toeplitz: on
xor: off
crc32: off
What gives? Do I need to modify a constant somewhere in DPDK?
At the risk of expanding scope, I am really trying to run eRPC which runs
on top of DPDK. It fails in dpdk_init with a similar but slightly different
error shown in bold in which the putative valid value is different than
above:
$ sudo ./hello_server 172.31.77.76:31850
20:536566 INFOR: eRPC Nexus: Launching 0 background threads.
20:536582 INFOR: eRPC Nexus: Launching session management thread on core 53.
20:536776 INFOR: eRPC Nexus: Created with management UDP port 31850,
hostname 172.31.77.76.
20:536832 INFOR: DPDK transport for Rpc 0 initializing DPDK EAL.
EAL: Detected CPU lcores: 72
EAL: Detected NUMA nodes: 2
EAL: Auto-detected process type: PRIMARY
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(3)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:7f:00.0 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
21:428883 WARNG: Running as primary DPDK process. eRPC DPDK daemon is not
running.
21:428893 INFOR: DPDK transport for Rpc 0 got QP 0
21:428927 INFOR: Initializing port 0 with driver net_ena
*Ethdev port_id=0 invalid rss_hf: 0x20820, valid value: 0xc30*terminate
called after throwing an instance of 'std::runtime_error'
what(): Ethdev configuration error: Invalid argument
Aborted
[-- Attachment #2: Type: text/html, Size: 5999 bytes --]
next reply other threads:[~2021-11-09 23:34 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-09 23:34 fwefew 4t4tg [this message]
2021-11-10 7:32 ` [dpdk-dev] " Michał Krawczyk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+Tq66WJzsSYKZ1QAuwzs-bwQGDbWPYqHvnBMO8++Lg+VRxs6Q@mail.gmail.com \
--to=7532yahoo@gmail.com \
--cc=dev@dpdk.org \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).