Re: [dpdk-dev] [PATCH] doc: announce ring ABI and API changes

["Eads, Gage" <gage.eads@intel.com>]

> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Tuesday, January 15, 2019 6:34 PM
> To: Eads, Gage <gage.eads@intel.com>
> Cc: dev@dpdk.org; olivier.matz@6wind.com; arybchenko@solarflare.com;
> Richardson, Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Subject: Re: [PATCH] doc: announce ring ABI and API changes
> 
> On Tue, 15 Jan 2019 17:59:34 -0600
> Gage Eads <gage.eads@intel.com> wrote:
> 
> > In order to support the non-blocking ring[1], one ABI change and one
> > API change are required in librte_ring. This commit updates the
> > deprecation notice to pave the way for their inclusion in 19.05.
> >
> > [1] http://mails.dpdk.org/archives/dev/2019-January/123475.html
> >
> > Signed-off-by: Gage Eads <gage.eads@intel.com>
> > ---
> >  doc/guides/rel_notes/deprecation.rst | 11 +++++++++++
> >  1 file changed, 11 insertions(+)
> >
> > diff --git a/doc/guides/rel_notes/deprecation.rst
> > b/doc/guides/rel_notes/deprecation.rst
> > index d4aea4b46..d74cff467 100644
> > --- a/doc/guides/rel_notes/deprecation.rst
> > +++ b/doc/guides/rel_notes/deprecation.rst
> > @@ -83,3 +83,14 @@ Deprecation Notices
> >    - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of
> >      ``rte_cryptodev_queue_pair_setup`` will change to to allow to use
> >      two different mempools for crypto and device private sessions.
> > +
> > +* ring: two changes are planned for rte_ring in v19.05:
> > +
> > +  - The ring head and tail values are planned to be changed from ``uint32_t``
> > +    to ``size_t``. This reduces the likelihood of wrap-around to effectively
> > +    zero for 64-bit builds, which is important in avoiding the ABA problem in
> > +    the upcoming non-blocking ring implementation. (32-bit builds are
> > +    unaffected by this change.)
> > +  - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can
> > +    calculate the memory required for rings that require more than 8B per
> entry
> > +    (such as the upcoming non-blocking ring).
> 
> 
> Would it be possible to support new and old ring types, either through naming
> tricks and/or new ring flag?  Changing things like ring buffer and mbuf are
> basically a flag day for all users.
> 
> I admit to having a personal interest in this since the API/ABI churn is this project
> causes vendors to stay on older code. And older code does not correctly support
> newer networks.

Fair enough -- I appreciate the additional context wrt avoiding churn.

This might be doable with the following change: 

"
@@ -70,6 +70,15 @@ struct rte_ring_headtail {
        uint32_t single;         /**< True if single prod/cons */
 };
 
+/* 64-bit version of rte_ring_headtail, for use by rings that need to avoid
+ * head/tail wrap-around.
+ */
+struct rte_ring_headtail_64 {
+       volatile uint64_t head;  /**< Prod/consumer head. */
+       volatile uint64_t tail;  /**< Prod/consumer tail. */
+       uint32_t single;       /**< True if single prod/cons */
+};
+
 /**
  * An RTE ring structure.
  *
@@ -97,11 +106,19 @@ struct rte_ring {
        char pad0 __rte_cache_aligned; /**< empty cache line */
 
        /** Ring producer status. */
-       struct rte_ring_headtail prod __rte_cache_aligned;
+       RTE_STD_C11
+       union {
+               struct rte_ring_headtail prod __rte_cache_aligned;
+               struct rte_ring_headtail_64 prod_64 __rte_cache_aligned;
+       };
        char pad1 __rte_cache_aligned; /**< empty cache line */
 
        /** Ring consumer status. */
-       struct rte_ring_headtail cons __rte_cache_aligned;
+       RTE_STD_C11
+       union {
+               struct rte_ring_headtail cons __rte_cache_aligned;
+               struct rte_ring_headtail_64 cons_64 __rte_cache_aligned;
+       };
        char pad2 __rte_cache_aligned; /**< empty cache line */
 };
"

The ABI compatibility hinges on the fact that today's prod and cons are both padded out to a full cache line, and the 64-bit version fits within a single cache line. (Confirmed with pahole.)

abi-compliance-checker reports two issues, but both appear to be false positives:
1. "Field cons has been removed from this type"
2. "Field prod has been removed from this type"

I need to do more work to see whether/how the ring functions are affected by such a change, but I first want to check if the community agrees with this approach. Note that I don't see any way to avoid the API change to rte_ring_get_memsize, but I doubt that would have near the impact of a ring data structure change.

Thanks,
Gage