[dpdk-users] Membership library thread safety

[dpdk-users] Membership library thread safety

[longtb5]

Hi, 

I'm using rte_member in my application. I have two questions.
1. Could a rte_member_setsum be initialized on one thread and used on
another thread?
2. Are the operations (lookup, add, reset,.) atomic, or do I need to use
some kind of locking (e.g. rte_spinlock)?

I couldn't find anything in the doc. My guess is that you can init and use a
setsum on different threads just fine, but sharing a setsum among different
threads does require locking.

Thanks and best regards,
-BL
 

Re: [dpdk-users] Membership library thread safety

[longtb5]

Hi,

My use case is pretty simple. I have two NICs and with separated lcores
(different threads) handling traffic for each. Incoming packets could hit
either of those separated datapath (I have not yet implemented any kind of
session persistence e.g., IP hash). I want to monitor the number of active
flows that are passing through my application, so for each datapath I use
one setsum to keep track of new flows. The problem is that a "new" packet on
one NIC may actually belong to an already existed flow that hit the other
NIC first. My initial thought is to share one setsum between the two paths
and that was the reason the my first email.

For completeness, I use vBF with only one setsum. Currently I just treat
each datapath separately, completely ignore the fact that they run in the
same app, which is good enough for me atm. But for future use cases a more
thorough solution is always welcomed. 

Best regards,
-BL
> -----Original Message-----
> From: yipeng1.wang@intel.com [mailto:yipeng1.wang@intel.com]
> Sent: Saturday, June 2, 2018 5:17 AM
> To: longtb5@viettel.com.vn; users@dpdk.org
> Cc: Gobriel, Sameh <sameh.gobriel@intel.com>; Tai, Charlie
> <charlie.tai@intel.com>; Wang, Ren <ren.wang@intel.com>
> Subject: RE: Membership library thread safety
> 
> Hi,
> 
> You are right that It is similar to other rte data structure that you can
initialize
> it in a master thread and do lookup in other threads.
> 
> w.r.t. concurrency,  the original plan is to have the fastest data
structure used
> mostly per-thread basis so any locking/atomicity guarantee is too costly.
> 
> So for non-cache mode HT based setsum, since deletion is supported and
> false-negative is not supposed to happen, I would say it only supports
single
> writer/delete thread.  For read-write concurrency, due to the "cuckoo
hash"
> algorithm, there might be a short window (very unlikely) that a key lookup
> could return an intermediate result while another thread writing to the
table.
> Depending on how your application could tolerate such case, you may or may
> not want to use it for read-write concurrency. One option is to turn off
the
> cuckoo algorithm if you would like to play with the code a little bit.
> 
> Vector BF also only supports single writer if you don't want false
negative,
> but read-write concurrency should be fine
> 
> For cache mode HT based setsum, things are less restrictive. Since it is
> supposed to be used as a cache and both false-negative and false-positive
> are allowed, it would be fine for multiple writer writing concurrently.
> 
> Note that you can always use multiple readers concurrently.
> 
> If possible, may I ask for the use case you have? The purpose is to make
the
> library more usable. Feel free to propose new patches as well. If
concurrency
> is definitely a requirement, we could collaborate on new concurrency APIs.
> We will improve the documentation as well to state things more clearly.
> 
> Thanks
> Yipeng
> 
> >-----Original Message-----
> >From: longtb5@viettel.com.vn [mailto:longtb5@viettel.com.vn]
> >Sent: Friday, June 1, 2018 12:10 AM
> >To: users@dpdk.org
> >Cc: Wang, Yipeng1 <yipeng1.wang@intel.com>; Gobriel, Sameh
> ><sameh.gobriel@intel.com>
> >Subject: Membership library thread safety
> >
> >Hi,
> >
> >I'm using rte_member in my application. I have two questions.
> >1. Could a rte_member_setsum be initialized on one thread and used on
> >another thread?
> >2. Are the operations (lookup, add, reset,.) atomic, or do I need to
> >use some kind of locking (e.g. rte_spinlock)?
> >
> >I couldn't find anything in the doc. My guess is that you can init and
> >use a setsum on different threads just fine, but sharing a setsum among
> >different threads does require locking.
> >
> >Thanks and best regards,
> >-BL
> >