Re: [dpdk-dev] [PATCH v3 2/2] ethdev: make rte_flow API thread safe

[Suanming Mou <suanmingm@nvidia.com>]

> -----Original Message-----
> From: Matan Azrad <matan@nvidia.com>
> Sent: Thursday, October 8, 2020 4:11 AM
> To: Suanming Mou <suanmingm@nvidia.com>; Ori Kam <orika@nvidia.com>;
> NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
> <ferruh.yigit@intel.com>; Andrew Rybchenko <arybchenko@solarflare.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v3 2/2] ethdev: make rte_flow API thread safe
> 
> 
> 
> From: Suanming Mou
> > Currently, the rte_flow functions are not defined as thread safe.
> > DPDK applications either call the functions in single thread or add
> > locks around the functions for the critical section.
> >
> 
> Better to write here "or protect any concurrent calling for the rte_flow
> operations using a lock."
> 
> > For PMDs support the flow operations thread safe natively, the
> > redundant protection in application hurts the performance of the
> > rte_flow operation functions.
> >
> > And the restriction of thread safe not guaranteed for the rte_flow
> > functions
> 
> not => is not
> 
> > also limits the applications' expectation.
> >
> > This feature is going to change the rte_flow functions to be thread
> > safe. As different PMDs have different flow operations, some may
> > support thread safe already and others may not. For PMDs don't support
> > flow thread safe operation, a new lock is defined in ethdev in order
> > to protects thread unsafe PMDs from rte_flow level.
> >
> > A new RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE device flag is added to
> > determine whether the PMD supports thread safe flow operation or not.
> > For PMDs support thread safe flow operations, set the
> > RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE flag, rte_flow level functions will
> > skip the thread safe helper lock for these PMDs. Again the rte_flow
> > level thread safe lock only works when PMD operation functions are not
> > thread safe.
> >
> > For the PMDs which don't want the default mutex lock, just set the
> > flag in the PMD, and add the prefer type of lock in the PMD. Then the
> > default mutex lock is easily replaced by the PMD level lock.
> >
> > The change has no effect on the current DPDK applications. No change
> > is required for the current DPDK applications. For the standard posix
> > pthread_mutex, if no lock contention with the added rte_flow level
> > mutex, the mutex only does the atomic increasing in
> > pthread_mutex_lock() and decreasing in pthread_mutex_unlock(). No
> > futex() syscall will be involved.
> >
> > Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
> > ---
> >
> > v3:
> >  - update flow_lock/unlock -> fts_enter/exit
> >
> > v2:
> >  - Update commit info and description doc.
> >  - Add inline for the flow lock and unlock functions.
> >  - Remove the PMD sample part flag configuration.
> >
> > ---
> >
> >  doc/guides/prog_guide/rte_flow.rst  |  9 ++--
> >  lib/librte_ethdev/rte_ethdev.c      |  2 +
> >  lib/librte_ethdev/rte_ethdev.h      |  2 +
> >  lib/librte_ethdev/rte_ethdev_core.h |  4 ++
> >  lib/librte_ethdev/rte_flow.c        | 84 ++++++++++++++++++++++++++++---
> > ------
> >  5 files changed, 78 insertions(+), 23 deletions(-)
> >
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index 119b128..ae2ddb3 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -3046,10 +3046,6 @@ Caveats
> >  - API operations are synchronous and blocking (``EAGAIN`` cannot be
> >    returned).
> >
> > -- There is no provision for re-entrancy/multi-thread safety, although
> > nothing
> > -  should prevent different devices from being configured at the same
> > -  time. PMDs may protect their control path functions accordingly.
> > -
> >  - Stopping the data path (TX/RX) should not be necessary when
> > managing flow
> >    rules. If this cannot be achieved naturally or with workarounds (such as
> >    temporarily replacing the burst function pointers), an appropriate
> > error @@
> > -3101,6 +3097,11 @@ This interface additionally defines the following
> > helper
> > function:
> >  - ``rte_flow_ops_get()``: get generic flow operations structure from a
> >    port.
> >
> > +If PMD interfaces do not support re-entrancy/multi-thread safety,
> > +rte_flow level functions will do it by mutex. The application can
> > +test
> 
> Good to mention mutex per port.
> 
> > +the dev_flags with RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE in struct
> > +rte_eth_dev_data to know if the rte_flow thread-safe works under
> > rte_flow level or PMD level.
> > +
> 
> If think it will be helpful to add here a note saying that it is unsafe to call other
> control commands In parallel to rte_flow operations.
> 
> >  More will be added over time.
> >
> >  Device compatibility

[snip]

> > 1.8.3.1
> 
> Besides the above (not must) suggestion, looks good,
> Acked-by: Matan Azrad <matan@nvidia.com>

Thanks for the suggestions, I will update that in the next version.