DPDK usage discussions
 help / color / mirror / Atom feed
* How do you calculate DPDK mempool requirements?
@ 2022-12-05 17:32 Dylan Baros
  2022-12-05 18:28 ` Stephen Hemminger
  2022-12-06 12:16 ` IraM
  0 siblings, 2 replies; 4+ messages in thread
From: Dylan Baros @ 2022-12-05 17:32 UTC (permalink / raw)
  To: users

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

How do you figure out what settings need to be used to correctly configure
a DPDK mempool for your application?

Specifically using rte_pktmbuf_pool_create():

- n, the number of elements in the mbuf pool
- cache_size
- priv_size
- data_room_size

EAL arguments:
- n number of memory channels
- r number of memory ranks
- m amount of memory to preallocate at startup
- in-memory no shared data structures
- IOVA mode
- huge-worker-stack

My setup:

   - 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
      - 28 cores per socket
      - Max 3.5 Ghz
      - Hyperthreading disabled
      - Ubuntu 22.04.1 LTS
      - Kernel 5.15.0-53-generic
      - Cores set to performance governor
      - 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
      - 128 GB DDR4 Memory
      - 10 1GB HugePages (Can change to what is required)
   - 1 x Mellanox ConnectX-5 100gbe NIC
      - 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
      [ConnectX-5]
      - Firmware-version: 16.35.1012
   - UDP Source:
      - 100 gbe NIC
      - 9000 MTU Packets
      - ipv4-udp packets

Will be receiving 10GB/s UDP packets over a 100gbe link. Right now trying
to get it working for 2GB/s to a single queue.

Reviewed the DPDK Programmers guide:
https://doc.dpdk.org/guides/prog_guide/mempool_lib.html Also searched
online but the resources seem limited. Would appreciate any help or a push
in the right direction.


Sincerely,

DB

[-- Attachment #2: Type: text/html, Size: 11543 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: How do you calculate DPDK mempool requirements?
  2022-12-05 17:32 How do you calculate DPDK mempool requirements? Dylan Baros
@ 2022-12-05 18:28 ` Stephen Hemminger
  2022-12-06 12:16 ` IraM
  1 sibling, 0 replies; 4+ messages in thread
From: Stephen Hemminger @ 2022-12-05 18:28 UTC (permalink / raw)
  To: Dylan Baros; +Cc: users

On Mon, 5 Dec 2022 11:32:40 -0600
Dylan Baros <dcbaros@utexas.edu> wrote:

> How do you figure out what settings need to be used to correctly configure
> a DPDK mempool for your application?
> 
> Specifically using rte_pktmbuf_pool_create():
> 
> - n, the number of elements in the mbuf pool
> - cache_size
> - priv_size
> - data_room_size
> 
> EAL arguments:
> - n number of memory channels
> - r number of memory ranks
> - m amount of memory to preallocate at startup
> - in-memory no shared data structures
> - IOVA mode
> - huge-worker-stack
> 
> My setup:
> 
>    - 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
>       - 28 cores per socket
>       - Max 3.5 Ghz
>       - Hyperthreading disabled
>       - Ubuntu 22.04.1 LTS
>       - Kernel 5.15.0-53-generic
>       - Cores set to performance governor
>       - 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
>       - 128 GB DDR4 Memory
>       - 10 1GB HugePages (Can change to what is required)
>    - 1 x Mellanox ConnectX-5 100gbe NIC
>       - 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
>       [ConnectX-5]
>       - Firmware-version: 16.35.1012
>    - UDP Source:
>       - 100 gbe NIC
>       - 9000 MTU Packets
>       - ipv4-udp packets
> 
> Will be receiving 10GB/s UDP packets over a 100gbe link. Right now trying
> to get it working for 2GB/s to a single queue.
> 
> Reviewed the DPDK Programmers guide:
> https://doc.dpdk.org/guides/prog_guide/mempool_lib.html Also searched
> online but the resources seem limited. Would appreciate any help or a push
> in the right direction.
> 
> 
> Sincerely,
> 
> DB

Compute the maximum number of live memory elements in the system.
This depends on the NIC card and the queue depths.
Add some additional overhead for packets in flight.

Size of mbuf depends on MTU, amount of reserved private area etc.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: How do you calculate DPDK mempool requirements?
  2022-12-05 17:32 How do you calculate DPDK mempool requirements? Dylan Baros
  2022-12-05 18:28 ` Stephen Hemminger
@ 2022-12-06 12:16 ` IraM
  2022-12-06 16:24   ` Stephen Hemminger
  1 sibling, 1 reply; 4+ messages in thread
From: IraM @ 2022-12-06 12:16 UTC (permalink / raw)
  To: users



Sent from my iPhone

> On Dec 5, 2022, at 12:39, Dylan Baros <dcbaros@utexas.edu> wrote:
> 
> 
> 
> 
> How do you figure out what settings need to be used to correctly configure a DPDK mempool for your application?

I went to rough a similar exercise when I was playing around with packet ordering example.

Here’s a rough formula I came up with.
Comments are welcomed

Total pool size = Sum of the following numbers 
-RX queue size  x number of ports
-TX queue size x number of ports
-Size of ring x number of rings
-MAX_pkt_burstsize x number of threads since each does a read of that size




^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: How do you calculate DPDK mempool requirements?
  2022-12-06 12:16 ` IraM
@ 2022-12-06 16:24   ` Stephen Hemminger
  0 siblings, 0 replies; 4+ messages in thread
From: Stephen Hemminger @ 2022-12-06 16:24 UTC (permalink / raw)
  To: IraM; +Cc: users

On Tue, 6 Dec 2022 07:16:13 -0500
IraM <iramica@yahoo.com> wrote:

> Sent from my iPhone
> 
> > On Dec 5, 2022, at 12:39, Dylan Baros <dcbaros@utexas.edu> wrote:
> > 
> > 
> > 
> > 
> > How do you figure out what settings need to be used to correctly configure a DPDK mempool for your application?  
> 
> I went to rough a similar exercise when I was playing around with packet ordering example.
> 
> Here’s a rough formula I came up with.
> Comments are welcomed
> 
> Total pool size = Sum of the following numbers 
> -RX queue size  x number of ports
> -TX queue size x number of ports
> -Size of ring x number of rings
> -MAX_pkt_burstsize x number of threads since each does a read of that size
> 
> 
> 

Some drivers like Broadcom bnxt require multiple mbufs per rx descriptor.
And if you use jumbo frames and scatter/gather it gets a little more complex.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-12-06 16:25 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-05 17:32 How do you calculate DPDK mempool requirements? Dylan Baros
2022-12-05 18:28 ` Stephen Hemminger
2022-12-06 12:16 ` IraM
2022-12-06 16:24   ` Stephen Hemminger

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).