Re: [dpdk-dev] [PATCH] mempool: allow for user-owned mempool caches

["Venkatesan, Venky" <venky.venkatesan@intel.com>]

> -----Original Message-----
> From: Olivier Matz [mailto:olivier.matz@6wind.com]
> Sent: Friday, March 25, 2016 3:56 AM
> To: Venkatesan, Venky <venky.venkatesan@intel.com>; Lazaros Koromilas
> <l@nofutznetworks.com>; Wiles, Keith <keith.wiles@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] mempool: allow for user-owned mempool
> caches
> 
> Hi Venky,
> 
> >> The main benefit of having an external cache is to allow mempool
> >> users
> >> (threads) to maintain a local cache even though they don't have a
> >> valid lcore_id (non-EAL threads). The fact that cache access is done
> >> by indexing with the lcore_id is what makes it difficult...
> >
> > Hi Lazaros,
> >
> > Alternative suggestion: This could actually be very simply done via creating
> an EAL API to register and return an lcore_id for a thread wanting to use
> DPDK services. That way, you could simply create your pthread, call the
> eal_register_thread() function that assigns an lcore_id to the caller (and
> internally sets up the per_lcore variable.
> >
> > The advantage of doing it this way is that you could extend it to other
> things other than the mempool that may need an lcore_id setup.
> 
> From my opinion, externalize the cache structure as Lazaros suggests would
> make things simpler, especially in case of dynamic threads
> allocation/destruction.
> 
> If a lcore_id regristration API is added in EAL, we still need a max lcore value
> when the mempool is created so the cache can be allocated. Moreover, the
> API would not be as simple, especially if it needs to support secondary
> processes.
>
Not really - the secondary process is simply another series of threads. They have their own caches. Yes, we will need a max lcore value, but we can make the allocations dynamic as opposed to static. That way, we will have MAX_LCORE pointers to store per mempool. 

The approach that's suggested currently is workable (and if I were solving mempool alone, this is very likely what I would do too), but is limited to the mempool alone. Adding the API to the eal has a rather huge secondary advantage - you now no longer need to create DPDK threads explicitly any more - you can create pthreads, and manage them how you wish. Architecturally speaking, longer term for DPDK that would be bigger win. 
 
> 
> Regards,
> Olivier