Re: [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Wednesday, June 15, 2016 9:49 AM
> To: Ananyev, Konstantin
> Cc: Pattan, Reshma; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v9 1/8] ethdev: use locks to protect Rx/Tx callback lists
> 
> 2016-06-15 08:37, Ananyev, Konstantin:
> > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > 2016-06-15 05:30, Pattan, Reshma:
> > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > > > 2016-06-14 10:38, Reshma Pattan:
> > > > > > Added spinlocks around add/remove logic of Rx and Tx callbacks to
> > > > > > avoid corruption of callback lists in multithreaded context.
> > > > > >
> > > > > > Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
> > > > >
> > > > > Why cb->next is not locked in burst functions?
> > > > It is safe to do "read access" here and doesn't require any locking as rx/tx burst is initiated  by only local user(control plane)
> thread.
> > > >
> > > > > Just protecting add/remove but not its usage seems useless.
> > > > Here locks were required  around add/remove to protect "write access"  because write to callback list is now done from 2
> threads
> > > > i.e. one from local user thread(control plane) and another from pdump control thread(initiated by remote pdump request).
> > >
> > > So read and write can be done by different threads.
> >
> > Yes, and this is possible even in current DPDK version (16.04).
> > What is added by Reshma's patch - now it is possible to have concurrent write
> > from 2 different thread to that list.
> >
> > > I think the read access would need locking but we do not want it
> > > in fast path.
> >
> > I don't think it would be needed.
> > As I said - read/write interaction didn't change from what we have right now.
> > But if you have some particular scenario in mind that you believe would cause
> > a race condition - please speak up.
> 
> If we add/remove a callback during a burst? Is it possible that the next
> pointer would have a wrong value leading to a crash?
> Maybe we need a comment to state that we should not alter burst
> callbacks while running burst functions.

Current status (16.04):
It is safe to add/remove RX/TX callbacks while 
another thread is doing simultaneously RX/TX burst over same queue.
I.E: it is supposed to be safe to invoke
rte_eth_add(/remove)_rx(/tx)_callback() and rte_eth_rx_burst()/rte_eth_tx_burst()
from different threads simultaneously.
Though it is not safe to free/modify that rte_eth_rxtx_callback while current
rte_eth_rx_burst()/rte_eth_tx_burst() are still active.
That exactly what comments for rte_eth_remove_rx_callback() say:

* Note: the callback is removed from the callback list but it isn't freed
 * since the it may still be in use. The memory for the callback can be
 * subsequently freed back by the application by calling rte_free():
 *
 * - Immediately - if the port is stopped, or the user knows that no
 *   callbacks are in flight e.g. if called from the thread doing RX/TX
 *   on that queue.
 *
 * - After a short delay - where the delay is sufficient to allow any
 *   in-flight callbacks to complete.

In other words, right now there only way to know for sure that it is safe
to free the removed callback - is to stop the port.

Does it need to be changed, so when rte_eth_remove_rx_callback() returns
user can safely free the callback (or even better rte_eth_remove_rx_callback free the callback for us)?
In my opinion - yes.
Though, I think, it has nothing to do with pdump patches, and I think should be a matter
for separate a patch/discussion.

Now with pdump library introduction - there is possibility that 2 different threads
can try to  add/remove callbacks for the same queue simultaneously.
First one - thread executing control requests from local user,
second  one - pdump control thread executing pdump requests from pdump client.
That lock is introduced to avoid race condition between such 2 threads:
i.e. to prevent multiple threads to modify same list simultaneously.   
It is not intended to synchronise read/write accesses to the list, see above. 

Konstantin