On AWS EC2 I configure:

- c5n.metal instance
- Stock AWS AMI Ubuntu 20.04 LTS ($ cat /proc/version
Linux version 5.4.0-1045-aws)
- Apply the vfio pci driver from https://github.com/amzn/am
zn-drivers/tree/master/userspace/dpdk/enav2-vfio-patch
- Using DPDK code from head of main branch. I built using ninja/meson
- Add a 2nd ENA enabled NIC
- Setup 2Mb huge pages, and bound the 2nd NIC with dpdk-devbind

I run test-pmd with this command:

$ sudo ./dpdk-testpmd --huge-dir=/mnt/huge -c 0xf -n 4 -- -i
EAL: Detected CPU lcores: 72
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(3)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:7f:00.0 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
Interactive-mode selected
Error picking flow transfer proxy for port 0: Function not implemented -
ignore
testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Warning! port-topology=paired and odd forward ports number, the last port
will pair with itself.
Configuring Port 0 (socket 0)
Port 0: 16:9A:11:4A:5D:61
Checking link statuses...
Done
Error during enabling promiscuous mode for port 0: Operation not supported
- ignore

*Key point: I try to enable RSS hash offload for UDP or TCP. I get the
following error shown in bold:*

testpmd> port config all rss udp

*Port 0 modified RSS hash function based on hardware
support,requested:0x20820 configured:0x820*ena_rss_hash_set(): Setting RSS
hash fields is not supported
ena_rss_hash_update(): Failed to set RSS hash
*Configuration of RSS hash at ethernet port 0 failed with error (95):
Operation not supported.*
testpmd> port config all rss tcp

*Port 0 modified RSS hash function based on hardware
support,requested:0x10410 configured:0x410*ena_rss_hash_set(): Setting RSS
hash fields is not supported
ena_rss_hash_update(): Failed to set RSS hash

*Configuration of RSS hash at ethernet port 0 failed with error (95):
Operation not supported.*
If I run ethtool on the NIC before putting it down state and DPDK binding
to vfio-pci I see an indication that there is a hash key, and RSS Hash
works:

ubuntu$ ethtool -x ens6
RX flow hash indirection table for ens6 with 32 RX ring(s):
    0:      0     1     2     3     4     5     6     7
    8:      8     9    10    11    12    13    14    15
   16:     16    17    18    19    20    21    22    23
   24:     24    25    26    27    28    29    30    31
   32:      0     1     2     3     4     5     6     7
   40:      8     9    10    11    12    13    14    15
   48:     16    17    18    19    20    21    22    23
   56:     24    25    26    27    28    29    30    31
   64:      0     1     2     3     4     5     6     7
   72:      8     9    10    11    12    13    14    15
   80:     16    17    18    19    20    21    22    23
   88:     24    25    26    27    28    29    30    31
   96:      0     1     2     3     4     5     6     7
  104:      8     9    10    11    12    13    14    15
  112:     16    17    18    19    20    21    22    23
  120:     24    25    26    27    28    29    30    31
RSS hash key:
80:d3:0f:c4:1d:8a:48:9e:7f:1d:18:91:d3:12:cb:38:87:fc:33:49:18:30:e4:1d:19:8f:9d:fd:c3:94:fe:f3:b2:f0:7f:e9:e9:9d:2d:af
RSS hash function:
    toeplitz: on
    xor: off
    crc32: off

What gives? Do I need to modify a constant somewhere in DPDK?

At the risk of expanding scope, I am really trying to run eRPC which runs
on top of DPDK. It fails in dpdk_init with a similar but slightly different
error shown in bold in which the putative valid value is different than
above:

$ sudo ./hello_server 172.31.77.76:31850
20:536566 INFOR: eRPC Nexus: Launching 0 background threads.
20:536582 INFOR: eRPC Nexus: Launching session management thread on core 53.
20:536776 INFOR: eRPC Nexus: Created with management UDP port 31850,
hostname 172.31.77.76.
20:536832 INFOR: DPDK transport for Rpc 0 initializing DPDK EAL.
EAL: Detected CPU lcores: 72
EAL: Detected NUMA nodes: 2
EAL: Auto-detected process type: PRIMARY
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(3)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:7f:00.0 (socket 0)
TELEMETRY: No legacy callbacks, legacy socket not created
21:428883 WARNG: Running as primary DPDK process. eRPC DPDK daemon is not
running.
21:428893 INFOR: DPDK transport for Rpc 0 got QP 0
21:428927 INFOR: Initializing port 0 with driver net_ena

*Ethdev port_id=0 invalid rss_hf: 0x20820, valid value: 0xc30*terminate
called after throwing an instance of 'std::runtime_error'
  what():  Ethdev configuration error: Invalid argument
Aborted