DPDK Queues

DPDK Queues

[Dylan Baros]

[-- Attachment #1: Type: text/plain, Size: 1278 bytes --]

Good afternoon,

I am working on learning DPDK in an attempt to write a DPDK based
application and I have a few questions.

My setup:

   - 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
      - 28 cores per socket
      - Max 3.5 Ghz
      - Hyperthreading disabled
      - Ubuntu 22.04.1 LTS
      - Kernel 5.15.0-53-generic
      - Cores set to performance governor
      - 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
      - 128 GB DDR4 Memory
      - 10 1GB HugePages (Can change to what is required)
   - 1 x Mellanox ConnectX-5 100gbe NIC
      - 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
      [ConnectX-5]
      - Firmware-version: 16.35.1012
   - UDP Source:
      - 100 gbe NIC
      - 9000 MTU Packets
      - ipv4-udp packets


The UDP source provides packets with 4 different destination ports. I want
to direct each port to a queue and have writer threads pull data from the
queue and write the payloads to one file per queue.

My questions are:
1. How can I direct the data to different queues depending on port
destination? RSS or rte_flow? If so how do I direct it to do so?

2. Can you have one lcore per queue or does it have to be one lcore per
port?

3. How can I get to the payload in the packets? Any code examples of this?

Thanks,
DB

[-- Attachment #2: Type: text/html, Size: 10037 bytes --]

Re: DPDK Queues

[Stephen Hemminger]

On Thu, 8 Dec 2022 12:32:21 -0600
Dylan Baros <dcbaros@utexas.edu> wrote:

> Good afternoon,
> 
> I am working on learning DPDK in an attempt to write a DPDK based
> application and I have a few questions.
> 
> My setup:
> 
>    - 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
>       - 28 cores per socket
>       - Max 3.5 Ghz
>       - Hyperthreading disabled
>       - Ubuntu 22.04.1 LTS
>       - Kernel 5.15.0-53-generic
>       - Cores set to performance governor
>       - 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
>       - 128 GB DDR4 Memory
>       - 10 1GB HugePages (Can change to what is required)
>    - 1 x Mellanox ConnectX-5 100gbe NIC
>       - 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
>       [ConnectX-5]
>       - Firmware-version: 16.35.1012
>    - UDP Source:
>       - 100 gbe NIC
>       - 9000 MTU Packets
>       - ipv4-udp packets
> 
> 
> The UDP source provides packets with 4 different destination ports. I want
> to direct each port to a queue and have writer threads pull data from the
> queue and write the payloads to one file per queue.
> 
> My questions are:
> 1. How can I direct the data to different queues depending on port
> destination? RSS or rte_flow? If so how do I direct it to do so?

RSS randomizes the queue assignment, and is available in almost all hardware.
The randomization is based on 4-tuple of src/dst ip and port number.

Rte flow allows directing port to queue, but is NIC dependent and is more
difficult to setup.

> 2. Can you have one lcore per queue or does it have to be one lcore per
> port?

The mapping of how queues are handled by cores is up to the application.
DPDK uses a pull model so it is up to which threads request packets
from which queue. Your application decides how that is managed.
But don't have multiple threads polling the same queue.

> 
> 3. How can I get to the payload in the packets? Any code examples of this?

Look at l3fwd. The macro rte_pktmbuf_mtod() to see look at contents.

> 
> Thanks,
> DB