[dpdk-users] Memory Allocation

[dpdk-users] Memory Allocation

[Kumaraparameshwaran Rathnavel]

Hi All,

When I have a system with NUMA enabled and say my application is running one DPDK thread and I use that thread for the memory allocation will the memory be physically allocated during the access time or will the memory be allocated on the Node where the DPDK thread was initially running when the memory allocation functions was called. So threads accessing the memory locations from the other nodes remotely will suffer a degradation in performance. Please correct me if I am wrong. 

Thanking You,
Param.

Re: [dpdk-users] Memory Allocation

[Anupam Kapoor]

On Tue, May 16, 2017 at 9:44 PM, Kumaraparameshwaran Rathnavel <
krath@cloudsimple.com> wrote:

> will the memory be allocated on the Node where the DPDK thread was
> initially running when the memory allocation functions was called


​rte_malloc​(...) will return memory allocated on the NUMA socket of the
core which made the invokation. access to this from other locations will
incur the qpi penalty...

​--
thanks
anupam​


In the beginning was the lambda, and the lambda was with Emacs, and Emacs
was the lambda.

Re: [dpdk-users] Memory Allocation

[Stephen Hemminger]

On Tue, 16 May 2017 21:44:46 +0530
Kumaraparameshwaran Rathnavel <krath@cloudsimple.com> wrote:

> Hi All,
> 
> When I have a system with NUMA enabled and say my application is running one DPDK thread and I use that thread for the memory allocation will the memory be physically allocated during the access time or will the memory be allocated on the Node where the DPDK thread was initially running when the memory allocation functions was called. So threads accessing the memory locations from the other nodes remotely will suffer a degradation in performance. Please correct me if I am wrong. 
> 
> Thanking You,
> Param.
> 
> 

Most of the memory allocation mechansisms (malloc, mmap) in Linux are not NUMA aware.
If you read the source (which is always the definitive place to look), for DPDK you
will see that rte_malloc, rte_mempool, and other routines have option to take a NUMA
node as argument. It is up to the application to determine what the "right" node is.
Often it is the node where the hardware is located (for mbuf pools), or the node
where the thread is running. But unless you lock the thread to a node, the scheduler
is likely to move it around under load.