How do you calculate DPDK mempool requirements?

How do you calculate DPDK mempool requirements?

[Dylan Baros]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

How do you figure out what settings need to be used to correctly configure
a DPDK mempool for your application?

Specifically using rte_pktmbuf_pool_create():

- n, the number of elements in the mbuf pool
- cache_size
- priv_size
- data_room_size

EAL arguments:
- n number of memory channels
- r number of memory ranks
- m amount of memory to preallocate at startup
- in-memory no shared data structures
- IOVA mode
- huge-worker-stack

My setup:

   - 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
      - 28 cores per socket
      - Max 3.5 Ghz
      - Hyperthreading disabled
      - Ubuntu 22.04.1 LTS
      - Kernel 5.15.0-53-generic
      - Cores set to performance governor
      - 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
      - 128 GB DDR4 Memory
      - 10 1GB HugePages (Can change to what is required)
   - 1 x Mellanox ConnectX-5 100gbe NIC
      - 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
      [ConnectX-5]
      - Firmware-version: 16.35.1012
   - UDP Source:
      - 100 gbe NIC
      - 9000 MTU Packets
      - ipv4-udp packets

Will be receiving 10GB/s UDP packets over a 100gbe link. Right now trying
to get it working for 2GB/s to a single queue.

Reviewed the DPDK Programmers guide:
https://doc.dpdk.org/guides/prog_guide/mempool_lib.html Also searched
online but the resources seem limited. Would appreciate any help or a push
in the right direction.


Sincerely,

DB

[-- Attachment #2: Type: text/html, Size: 11543 bytes --]

Re: How do you calculate DPDK mempool requirements?

[Stephen Hemminger]

On Mon, 5 Dec 2022 11:32:40 -0600
Dylan Baros <dcbaros@utexas.edu> wrote:

> How do you figure out what settings need to be used to correctly configure
> a DPDK mempool for your application?
> 
> Specifically using rte_pktmbuf_pool_create():
> 
> - n, the number of elements in the mbuf pool
> - cache_size
> - priv_size
> - data_room_size
> 
> EAL arguments:
> - n number of memory channels
> - r number of memory ranks
> - m amount of memory to preallocate at startup
> - in-memory no shared data structures
> - IOVA mode
> - huge-worker-stack
> 
> My setup:
> 
>    - 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
>       - 28 cores per socket
>       - Max 3.5 Ghz
>       - Hyperthreading disabled
>       - Ubuntu 22.04.1 LTS
>       - Kernel 5.15.0-53-generic
>       - Cores set to performance governor
>       - 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
>       - 128 GB DDR4 Memory
>       - 10 1GB HugePages (Can change to what is required)
>    - 1 x Mellanox ConnectX-5 100gbe NIC
>       - 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
>       [ConnectX-5]
>       - Firmware-version: 16.35.1012
>    - UDP Source:
>       - 100 gbe NIC
>       - 9000 MTU Packets
>       - ipv4-udp packets
> 
> Will be receiving 10GB/s UDP packets over a 100gbe link. Right now trying
> to get it working for 2GB/s to a single queue.
> 
> Reviewed the DPDK Programmers guide:
> https://doc.dpdk.org/guides/prog_guide/mempool_lib.html Also searched
> online but the resources seem limited. Would appreciate any help or a push
> in the right direction.
> 
> 
> Sincerely,
> 
> DB

Compute the maximum number of live memory elements in the system.
This depends on the NIC card and the queue depths.
Add some additional overhead for packets in flight.

Size of mbuf depends on MTU, amount of reserved private area etc.

Re: How do you calculate DPDK mempool requirements?

[IraM]

Sent from my iPhone

> On Dec 5, 2022, at 12:39, Dylan Baros <dcbaros@utexas.edu> wrote:
> 
> 
> 
> 
> How do you figure out what settings need to be used to correctly configure a DPDK mempool for your application?

I went to rough a similar exercise when I was playing around with packet ordering example.

Here’s a rough formula I came up with.
Comments are welcomed

Total pool size = Sum of the following numbers 
-RX queue size  x number of ports
-TX queue size x number of ports
-Size of ring x number of rings
-MAX_pkt_burstsize x number of threads since each does a read of that size

Re: How do you calculate DPDK mempool requirements?

[Stephen Hemminger]

On Tue, 6 Dec 2022 07:16:13 -0500
IraM <iramica@yahoo.com> wrote:

> Sent from my iPhone
> 
> > On Dec 5, 2022, at 12:39, Dylan Baros <dcbaros@utexas.edu> wrote:
> > 
> > 
> > 
> > 
> > How do you figure out what settings need to be used to correctly configure a DPDK mempool for your application?  
> 
> I went to rough a similar exercise when I was playing around with packet ordering example.
> 
> Here’s a rough formula I came up with.
> Comments are welcomed
> 
> Total pool size = Sum of the following numbers 
> -RX queue size  x number of ports
> -TX queue size x number of ports
> -Size of ring x number of rings
> -MAX_pkt_burstsize x number of threads since each does a read of that size
> 
> 
> 

Some drivers like Broadcom bnxt require multiple mbufs per rx descriptor.
And if you use jumbo frames and scatter/gather it gets a little more complex.