RE: [PATCH] eal: introduce atomics abstraction

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Tyler Retzlaff [mailto:roretzla@linux.microsoft.com]
> Sent: Thursday, 2 February 2023 20.00
> 
> On Thu, Feb 02, 2023 at 09:43:58AM +0100, Morten Brørup wrote:
> > > From: Tyler Retzlaff [mailto:roretzla@linux.microsoft.com]
> > > Sent: Wednesday, 1 February 2023 22.41
> > >
> > > On Wed, Feb 01, 2023 at 01:07:59AM +0000, Honnappa Nagarahalli
> wrote:
> > > >
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > Sent: Tuesday, January 31, 2023 4:42 PM
> > > > >
> > > > > Honnappa, please could you give your view on the future of
> atomics
> > > in DPDK?
> > > > Thanks Thomas, apologies it has taken me a while to get to this
> > > discussion.
> > > >
> > > > IMO, we do not need DPDK's own abstractions. APIs from
> stdatomic.h
> > > (stdatomics as is called here) already serve the purpose. These
> APIs
> > > are well understood and documented.
> > >
> > > i agree that whatever atomics APIs we advocate for should align
> with
> > > the
> > > standard C atomics for the reasons you state including implied
> > > semantics.
> > >
> > > >
> > > > For environments where stdatomics are not supported, we could
> have a
> > > stdatomic.h in DPDK implementing the same APIs (we have to support
> only
> > > _explicit APIs). This allows the code to use stdatomics APIs and
> when
> > > we move to minimum supported standard C11, we just need to get rid
> of
> > > the file in DPDK repo.
> >
> > Perhaps we can use something already existing, such as this:
> > https://android.googlesource.com/platform/bionic/+/lollipop-
> release/libc/include/stdatomic.h
> >
> > >
> > > my concern with this is that if we provide a stdatomic.h or
> introduce
> > > names
> > > from stdatomic.h it's a violation of the C standard.
> > >
> > > references:
> > >  * ISO/IEC 9899:2011 sections 7.1.2, 7.1.3.
> > >  * GNU libc manual
> > >    https://www.gnu.org/software/libc/manual/html_node/Reserved-
> > > Names.html
> > >
> > > in effect the header, the names and in some instances namespaces
> > > introduced
> > > are reserved by the implementation. there are several reasons in
> the
> > > GNU libc
> > > manual that explain the justification for these reservations and if
> > > if we think about ODR and ABI compatibility we can conceive of
> others.
> >
> > I we are going to move to C11 soon, I consider the shim interim, and
> am inclined to ignore these warning factors.
> >
> > If we are not moving to C11 soon, I would consider these
> disadvantages more seriously.
> 
> I think it's reasonable to assume that we are talking years here.
> 
> We've had a few discussions about minimum C standard. I think my first
> mailing list exchanges about C99 was almost 2 years ago. Given that we
> still aren't on C99 now (though i know Bruce has a series up) indicates
> that progression to C11 isn't going to happen any time soon and even if
> it was the baseline we still can't just use it (reasons described
> later).
> 
> Also, i'll point out that we seem to have accepted moving to C99 with
> one of the holdback compilers technically being non-conformant but it
> isn't blocking us because it provides the subset of C99 features
> without
> being conforming that we happen to be using.
> 
> >
> > >
> > > i'll also remark that the inter-mingling of names from the POSIX
> > > standard implicitly exposed as a part of the EAL public API has
> been
> > > problematic for portability.
> >
> > This is a very important remark, which should be considered
> carefully! Tyler has firsthand experience with DPDK portability. If he
> thinks porting to Windows is going to be a headache if we expose the
> stdatomic.h API, we must listen! So, what is your gut feeling here,
> Tyler?
> 
> I think this is even more of a concern with language standard than it
> is
> with a platform standard. Because the language standard is used across
> platforms.
> 
> On the surface it looks appealing to just go through all the dpdk code
> one last time and #include <stdatomic.h> and directly depend on names
> that "look" standard. In practice though we aren't depending on the
> toolchain / libc surface we are binding ourselves to the shim and the
> implementation it provides.
> 
> This is aside from the mechanics of making it work in the different
> contexts we now have to care about. Here is a story of how things
> become tricky.
> 
> When i #include <stdatomic.h> which one gets used if the implementation
> provides one? Let's force our stdatomic.h
> 
> Now i need to force the build system to prefer my shim header? Keeping
> in mind that the presence of a libc stdatomic.h does not mean that the
> toolchain in fact supports standard atomics. Okay, that's under our
> control by editing some meson.build files maybe it isn't so bad but...
> 
> It seems my application also has to do the same in their build system
> now because...
> 
> The type definitions (size, alignment) and code generated from the
> body of inline functions as seen by the application built translation
> units may differ from those in the dpdk translation units if they don't
> use our header. The potential for ABI compat problems is increasing but
> maybe it is managable? it can be worse...
> 
> We can't limit our scope to thinking that there is just an
> application (a single binary) and dpdk. Complex applications will
> invariably depend on other libraries and if the application needs to
> interface with those compatibily at the ABI level using standard
> atomics
> then we've made it very difficult since the application has to choose
> to
> use our conflicting named atomic types which may not be compatible or
> the real standard atomics.  They can of course produce horrible shims
> of their own to interoperate.
> 
> We need consistency across the entire binary at runtime and i don't
> think it's practical to say that anyone who uses dpdk has to compile
> their whole world with our shim. So dealing with all this complexity
> for the sake of asthetics "looking" like the standard api seems kind
> of not worth it. Sure it saves having to deprecate later and one last
> session of shotgun surgery but that's kind of all we get.
> 
> Don't think i'm being biased in favor of windows/msvc here. From the
> perspective of the windows/msvc combination i intend to use only the
> standard C ABI provided by the implementation. I have no intention of
> trying to introduce support for the current ABI that doesn't use the
> standard atomic types. my discouraging of this approach is about
> avoiding
> subtle to detect but very painful problems on
> {linux,unix}/compiler<version>
> combinations that already have a shipped/stable ABI.
> 
> > >
> > > let's discuss this from here. if there's still overwhelming desire
> to
> > > go
> > > this route then we'll just do our best.
> > >
> > > ty
> >
> > I have a preference for exposing the stdatomic.h API. Tyler listed
> the disadvantages above. (I also have a preference for moving to C11
> soon.)
> 
> I am eager to see this happen, but as explained in my original proposal
> it doesn't eliminate the need for an abstraction. Unless we are willing
> to break our compatibility promises and potentially take a performance
> hit on some platform/compiler combinations which as i understand is not
> acceptable.
> 
> >
> > Exposing a 1:1 similar API with RTE prefixes would also be acceptable
> for me. The disadvantage is that the names are different than the C11
> names, which might lead to some confusion. And from an ABI stability
> perspective, such an important API should not be marked experimental.
> This means that years will pass before we can get rid of it again, due
> to ABI stability policies.
> 
> I think the key to success with rte_ prefixed names is making
> absolutely
> sure we mirror the semantics and types in the standard.
> 
> I will point out one bit of fine print here is that we will not support
> atomic operations on struct/union types (something the standard
> supports).
> With the rte_ namespace i think this becomes less ambiguous, if we
> present
> standard C names though what's to avoid the confusion? Aside from it
> fails
> to compile with one compiler vs another.
> 
> I agree that this may be around for years. But how many years depends a
> lot on how long we have to maintain compatibility for the existing
> platform/compiler combinations that can't (and aren't enabled) to use
> the standard.
> 
> Even if we introduced standard names we still have to undergo some kind
> of mutant deprecation process to get the world to recompile everything
> against the actual standard, so it doesn't give us forward
> compatibility.
> 
> Let me know what folks would like to do, i guess i'm firmly leaned
> toward no-shim and just rte_ explicit. But as a community i'll pursue
> whatever you decide.
> 
> Thanks!

Tyler is making a very strong case here.

I have changed my mind, and now support Tyler's approach.

-Morten