RFC abstracting atomics

RFC abstracting atomics

[Tyler Retzlaff]

hi folks,

i would like to introduce a layer of abstraction that would allow
optional use of standard C11 atomics when the platform / toolchain
combination has them available.

making the option usable would be a phased approach intended to focus
review and minimize dealing with churn on such a broad change.

1. provide an initial series to add the abstraction and the ability
   control enablement with a meson option enable_stdatomics=false will
   be the default.

   for all existing platform / toolchain combinations the default would
   remain false. i.e. i have no plans to enable it for existing platforms
   toolchain combinations but leaves a change of default open to the
   community as a future discussion if it is desired.

2. once the initial abstraction is integrated a series will be introduced to
   port the tree to the abstraction with enable_stdatomics=false. the goal
   being low or no change to the current use of gcc builtin C++11 memory
   model atomics.

3. once the tree is ported a final series will be introduced to introduce
   the remaining change to allow the use of enable_stdatomics=true.

would appreciate any assistance / suggestions you can provide to
introduce the abstraction smoothly.

thanks

Re: RFC abstracting atomics

[Bruce Richardson]

On Mon, Jan 09, 2023 at 02:56:04PM -0800, Tyler Retzlaff wrote:
> hi folks,
> 
> i would like to introduce a layer of abstraction that would allow
> optional use of standard C11 atomics when the platform / toolchain
> combination has them available.
> 
> making the option usable would be a phased approach intended to focus
> review and minimize dealing with churn on such a broad change.
> 
> 1. provide an initial series to add the abstraction and the ability
>    control enablement with a meson option enable_stdatomics=false will
>    be the default.
> 
>    for all existing platform / toolchain combinations the default would
>    remain false. i.e. i have no plans to enable it for existing platforms
>    toolchain combinations but leaves a change of default open to the
>    community as a future discussion if it is desired.
> 
> 2. once the initial abstraction is integrated a series will be introduced to
>    port the tree to the abstraction with enable_stdatomics=false. the goal
>    being low or no change to the current use of gcc builtin C++11 memory
>    model atomics.
> 
> 3. once the tree is ported a final series will be introduced to introduce
>    the remaining change to allow the use of enable_stdatomics=true.
> 
> would appreciate any assistance / suggestions you can provide to
> introduce the abstraction smoothly.
> 

Plan generally sounds ok. However, beyond point #3, would there then be
plans to remove the option and always use stdatomics in future?

To slightly expand the scope of the discussion - would it be worthwhile
putting these abstractions in a new library in DPDK other than EAL, to
start the process of splitting out some of the lower-level material from
that library?

/Bruce

RE: RFC abstracting atomics

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Tuesday, 10 January 2023 10.17
> 
> On Mon, Jan 09, 2023 at 02:56:04PM -0800, Tyler Retzlaff wrote:
> > hi folks,
> >
> > i would like to introduce a layer of abstraction that would allow
> > optional use of standard C11 atomics when the platform / toolchain
> > combination has them available.

It would be great if that abstraction layer could expose the exact same (or very similar) interface as standard C11 atomics, so the API is given in advance, and developers don't have to learn the API of yet another shim. It would also make it easier getting rid of the abstraction layer at a later time.

> >
> > making the option usable would be a phased approach intended to focus
> > review and minimize dealing with churn on such a broad change.
> >
> > 1. provide an initial series to add the abstraction and the ability
> >    control enablement with a meson option enable_stdatomics=false
> will
> >    be the default.
> >
> >    for all existing platform / toolchain combinations the default
> would
> >    remain false. i.e. i have no plans to enable it for existing
> platforms
> >    toolchain combinations but leaves a change of default open to the
> >    community as a future discussion if it is desired.
> >
> > 2. once the initial abstraction is integrated a series will be
> introduced to
> >    port the tree to the abstraction with enable_stdatomics=false. the
> goal
> >    being low or no change to the current use of gcc builtin C++11
> memory
> >    model atomics.
> >
> > 3. once the tree is ported a final series will be introduced to
> introduce
> >    the remaining change to allow the use of enable_stdatomics=true.
> >
> > would appreciate any assistance / suggestions you can provide to
> > introduce the abstraction smoothly.
> >
> 
> Plan generally sounds ok. However, beyond point #3, would there then be
> plans to remove the option and always use stdatomics in future?

We are already trying to migrate atomics in DPDK from using __sync built-ins (used in rte_atomic) to using __atomic built-ins; e.g. new contributions are recommended to use the latter (and not rte_atomic). When DPDK migrates to C11, I assume stdatomics will become the new standard for atomics in DPDK, and the process for migrating to stdatomics will resemble the currently ongoing process for migrating from rte_atomic to __atomic built-ins, which seems to be relatively uncomplicated (but slow).

Another thing: The proposed layer of abstraction should only live until DPDK supports C11, which is hopefully relatively soon. So is such a (hopefully) short-lived abstraction layer worth the effort?

> 
> To slightly expand the scope of the discussion - would it be worthwhile
> putting these abstractions in a new library in DPDK other than EAL, to
> start the process of splitting out some of the lower-level material
> from
> that library?

While I applaud this idea, I think we first need to discuss in detail how to split up EAL, and especially what new structure we want to end up with instead. E.g. we already have architecture specific code in various libraries and drivers, which proves that architecture specific stuff does not necessarily need to go into the EAL. We also need to discuss challenges/barriers to separating stuff from the EAL, and how we can solve those challenges; e.g. some parts are inside the EAL only due to startup sequence requirements, which might be solvable by offering better startup sequencing support for libraries in the EAL. Perhaps the EAL has even outlived itself, and could be replaced by set of DPDK Core libraries instead - some could be optional (at build time). But let's make this a separate discussion.

Re: RFC abstracting atomics

[Tyler Retzlaff]

On Tue, Jan 10, 2023 at 09:16:48AM +0000, Bruce Richardson wrote:
> On Mon, Jan 09, 2023 at 02:56:04PM -0800, Tyler Retzlaff wrote:
> > hi folks,
> > 
> > i would like to introduce a layer of abstraction that would allow
> > optional use of standard C11 atomics when the platform / toolchain
> > combination has them available.
> > 
> > making the option usable would be a phased approach intended to focus
> > review and minimize dealing with churn on such a broad change.
> > 
> > 1. provide an initial series to add the abstraction and the ability
> >    control enablement with a meson option enable_stdatomics=false will
> >    be the default.
> > 
> >    for all existing platform / toolchain combinations the default would
> >    remain false. i.e. i have no plans to enable it for existing platforms
> >    toolchain combinations but leaves a change of default open to the
> >    community as a future discussion if it is desired.
> > 
> > 2. once the initial abstraction is integrated a series will be introduced to
> >    port the tree to the abstraction with enable_stdatomics=false. the goal
> >    being low or no change to the current use of gcc builtin C++11 memory
> >    model atomics.
> > 
> > 3. once the tree is ported a final series will be introduced to introduce
> >    the remaining change to allow the use of enable_stdatomics=true.
> > 
> > would appreciate any assistance / suggestions you can provide to
> > introduce the abstraction smoothly.
> > 
> 
> Plan generally sounds ok. However, beyond point #3, would there then be
> plans to remove the option and always use stdatomics in future?

that is a discussion for the community i think on a per-platform /
per-toolchain basis. there is likely to be resistance which is why i'm
favoring the opt-in if you want model.

some potential arguments against switching.

* it's an abi break there is no way around it.
* old compiler x stdatomics implementation is less optimal than
  old compiler x __atomics (potential argument).
* there's some "mixed" use of variables in the tree where sometimes we
  operate on them as if they are atomic and sometimes not. the std
  atomics apis doesn't support this sometimes atomic codegen you either
  get atomic or you don't.
* direct use of atomics default to seq_cst ordering if the strengthening
  of the ordering is not desired each variable needs to be hunted down
  and explicitly returned to relaxed ordering access.

> To slightly expand the scope of the discussion - would it be worthwhile
> putting these abstractions in a new library in DPDK other than EAL, to
> start the process of splitting out some of the lower-level material from
> that library?

the abstraction as i have prototyped it is a set of conditionally
compiled macros where the macros mirror the standard c atomics for
naming and have been placed in include/generic/rte_atomics.h

i've no problem splitting it out into a separate library and then have
EAL depend on it, but it is currently header only so i'm not sure if it
is worth doing for that.

we can chew on that more in a couple days when i submit the base series
if you like.

> 
> /Bruce

Re: RFC abstracting atomics

[Tyler Retzlaff]

On Tue, Jan 10, 2023 at 12:45:05PM +0100, Morten Brørup wrote:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Tuesday, 10 January 2023 10.17
> > 
> > On Mon, Jan 09, 2023 at 02:56:04PM -0800, Tyler Retzlaff wrote:
> > > hi folks,
> > >
> > > i would like to introduce a layer of abstraction that would allow
> > > optional use of standard C11 atomics when the platform / toolchain
> > > combination has them available.
> 
> It would be great if that abstraction layer could expose the exact same (or very similar) interface as standard C11 atomics, so the API is given in advance, and developers don't have to learn the API of yet another shim. It would also make it easier getting rid of the abstraction layer at a later time.

the patch series i have prototype exactly mirrors the stdandard atomics
from C11. it requires a few fixups to fill convenience builtins that the
standard doesn't provide e.g. `__atomic_add_fetch(&o, v, order)` becomes
`__atomic_fetch_add(&o, v, order) + v;` sort of stuff, mostly minor.

> 
> > >
> > > making the option usable would be a phased approach intended to focus
> > > review and minimize dealing with churn on such a broad change.
> > >
> > > 1. provide an initial series to add the abstraction and the ability
> > >    control enablement with a meson option enable_stdatomics=false
> > will
> > >    be the default.
> > >
> > >    for all existing platform / toolchain combinations the default
> > would
> > >    remain false. i.e. i have no plans to enable it for existing
> > platforms
> > >    toolchain combinations but leaves a change of default open to the
> > >    community as a future discussion if it is desired.
> > >
> > > 2. once the initial abstraction is integrated a series will be
> > introduced to
> > >    port the tree to the abstraction with enable_stdatomics=false. the
> > goal
> > >    being low or no change to the current use of gcc builtin C++11
> > memory
> > >    model atomics.
> > >
> > > 3. once the tree is ported a final series will be introduced to
> > introduce
> > >    the remaining change to allow the use of enable_stdatomics=true.
> > >
> > > would appreciate any assistance / suggestions you can provide to
> > > introduce the abstraction smoothly.
> > >
> > 
> > Plan generally sounds ok. However, beyond point #3, would there then be
> > plans to remove the option and always use stdatomics in future?
> 
> We are already trying to migrate atomics in DPDK from using __sync built-ins (used in rte_atomic) to using __atomic built-ins; e.g. new contributions are recommended to use the latter (and not rte_atomic). When DPDK migrates to C11, I assume stdatomics will become the new standard for atomics in DPDK, and the process for migrating to stdatomics will resemble the currently ongoing process for migrating from rte_atomic to __atomic built-ins, which seems to be relatively uncomplicated (but slow).

the few __sync left in use are easily converted. there might be some
argument against converting __sync -> __atomic because of sub-optimal
codegen on older gcc for the semantically equivalent replacements.

the only comment i have on current/existing rte_atomic is they're
annoyingly stealing a little bit of the rte_ namespace that would allow
convenient alignment with the standard names. also some of them provide
only signed or unsigned variations which forces kind of non-standardy
things to be done in the code but are perfectly acceptable when using
the gcc implementation.

with the set of macros i'm introducing migrating to direct use of C11
atomics would likely be just a big search and replace (or that is my
goal).

> Another thing: The proposed layer of abstraction should only live until DPDK supports C11, which is hopefully relatively soon. So is such a (hopefully) short-lived abstraction layer worth the effort?

i kept hoping we would just baseline on C11 but after prototyping the
abstraction i realize there may be some valid arguments for not moving
to stdatomics now or possibly ever for some platform/toolchain
combinations (so long as they are declared supported). given the drawn
out discussion about RHEL-7 has shown that the migration path may be
long term. more possible headwinds were listed in my reply to Bruce.

> 
> > 
> > To slightly expand the scope of the discussion - would it be worthwhile
> > putting these abstractions in a new library in DPDK other than EAL, to
> > start the process of splitting out some of the lower-level material
> > from
> > that library?
> 
> While I applaud this idea, I think we first need to discuss in detail how to split up EAL, and especially what new structure we want to end up with instead. E.g. we already have architecture specific code in various libraries and drivers, which proves that architecture specific stuff does not necessarily need to go into the EAL. We also need to discuss challenges/barriers to separating stuff from the EAL, and how we can solve those challenges; e.g. some parts are inside the EAL only due to startup sequence requirements, which might be solvable by offering better startup sequencing support for libraries in the EAL. Perhaps the EAL has even outlived itself, and could be replaced by set of DPDK Core libraries instead - some could be optional (at build time). But let's make this a separate discussion.

i gave some detail about this to my reply with Bruce. i think when i
post the series we can discuss it more. but for atomics i expect it
isn't worth separating out.

RE: RFC abstracting atomics

[Morten Brørup]

> From: Tyler Retzlaff [mailto:roretzla@linux.microsoft.com]
> Sent: Tuesday, 10 January 2023 21.31
> 
> On Tue, Jan 10, 2023 at 12:45:05PM +0100, Morten Brørup wrote:
> > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > Sent: Tuesday, 10 January 2023 10.17
> > >
> > > On Mon, Jan 09, 2023 at 02:56:04PM -0800, Tyler Retzlaff wrote:
> > > > hi folks,
> > > >
> > > > i would like to introduce a layer of abstraction that would allow
> > > > optional use of standard C11 atomics when the platform /
> toolchain
> > > > combination has them available.
> >
> > It would be great if that abstraction layer could expose the exact
> same (or very similar) interface as standard C11 atomics, so the API is
> given in advance, and developers don't have to learn the API of yet
> another shim. It would also make it easier getting rid of the
> abstraction layer at a later time.
> 
> the patch series i have prototype exactly mirrors the stdandard atomics
> from C11. it requires a few fixups to fill convenience builtins that
> the
> standard doesn't provide e.g. `__atomic_add_fetch(&o, v, order)`
> becomes
> `__atomic_fetch_add(&o, v, order) + v;` sort of stuff, mostly minor.

Excellent!

> 
> >
> > > >
> > > > making the option usable would be a phased approach intended to
> focus
> > > > review and minimize dealing with churn on such a broad change.
> > > >
> > > > 1. provide an initial series to add the abstraction and the
> ability
> > > >    control enablement with a meson option enable_stdatomics=false
> > > will
> > > >    be the default.
> > > >
> > > >    for all existing platform / toolchain combinations the default
> > > would
> > > >    remain false. i.e. i have no plans to enable it for existing
> > > platforms
> > > >    toolchain combinations but leaves a change of default open to
> the
> > > >    community as a future discussion if it is desired.
> > > >
> > > > 2. once the initial abstraction is integrated a series will be
> > > introduced to
> > > >    port the tree to the abstraction with enable_stdatomics=false.
> the
> > > goal
> > > >    being low or no change to the current use of gcc builtin C++11
> > > memory
> > > >    model atomics.
> > > >
> > > > 3. once the tree is ported a final series will be introduced to
> > > introduce
> > > >    the remaining change to allow the use of
> enable_stdatomics=true.
> > > >
> > > > would appreciate any assistance / suggestions you can provide to
> > > > introduce the abstraction smoothly.
> > > >
> > >
> > > Plan generally sounds ok. However, beyond point #3, would there
> then be
> > > plans to remove the option and always use stdatomics in future?
> >
> > We are already trying to migrate atomics in DPDK from using __sync
> built-ins (used in rte_atomic) to using __atomic built-ins; e.g. new
> contributions are recommended to use the latter (and not rte_atomic).
> When DPDK migrates to C11, I assume stdatomics will become the new
> standard for atomics in DPDK, and the process for migrating to
> stdatomics will resemble the currently ongoing process for migrating
> from rte_atomic to __atomic built-ins, which seems to be relatively
> uncomplicated (but slow).
> 
> the few __sync left in use are easily converted. there might be some
> argument against converting __sync -> __atomic because of sub-optimal
> codegen on older gcc for the semantically equivalent replacements.

It would be easy to simply replace __sync with similar __atomic functions in the rte_atomic functions, but that is not the challenge.
The challenge is that the use of rte_atomic assumes the very strict order imposed by __sync (a full barrier), but the performance of some use cases could be improved by using the specific order required for those use cases (e.g. acquire/release semantics).

> 
> the only comment i have on current/existing rte_atomic is they're
> annoyingly stealing a little bit of the rte_ namespace that would allow
> convenient alignment with the standard names. also some of them provide
> only signed or unsigned variations which forces kind of non-standardy
> things to be done in the code but are perfectly acceptable when using
> the gcc implementation.

Yes, their signedness is slightly annoying, and makes some code look weird.

> 
> with the set of macros i'm introducing migrating to direct use of C11
> atomics would likely be just a big search and replace (or that is my
> goal).

Again, excellent!

> 
> > Another thing: The proposed layer of abstraction should only live
> until DPDK supports C11, which is hopefully relatively soon. So is such
> a (hopefully) short-lived abstraction layer worth the effort?
> 
> i kept hoping we would just baseline on C11 but after prototyping the
> abstraction i realize there may be some valid arguments for not moving
> to stdatomics now or possibly ever for some platform/toolchain
> combinations (so long as they are declared supported). given the drawn
> out discussion about RHEL-7 has shown that the migration path may be
> long term. more possible headwinds were listed in my reply to Bruce.

No objections from me - especially with the stdatomics roadmap you described.

> 
> >
> > >
> > > To slightly expand the scope of the discussion - would it be
> worthwhile
> > > putting these abstractions in a new library in DPDK other than EAL,
> to
> > > start the process of splitting out some of the lower-level material
> > > from
> > > that library?
> >
> > While I applaud this idea, I think we first need to discuss in detail
> how to split up EAL, and especially what new structure we want to end
> up with instead. E.g. we already have architecture specific code in
> various libraries and drivers, which proves that architecture specific
> stuff does not necessarily need to go into the EAL. We also need to
> discuss challenges/barriers to separating stuff from the EAL, and how
> we can solve those challenges; e.g. some parts are inside the EAL only
> due to startup sequence requirements, which might be solvable by
> offering better startup sequencing support for libraries in the EAL.
> Perhaps the EAL has even outlived itself, and could be replaced by set
> of DPDK Core libraries instead - some could be optional (at build
> time). But let's make this a separate discussion.
> 
> i gave some detail about this to my reply with Bruce. i think when i
> post the series we can discuss it more. but for atomics i expect it
> isn't worth separating out.

I think we're all in agreement at this point.
But as Bruce just did, we should think twice every time something is being proposed to be added to the EAL.

Re: RFC abstracting atomics

[Bruce Richardson]

On Tue, Jan 10, 2023 at 12:10:33PM -0800, Tyler Retzlaff wrote:
> On Tue, Jan 10, 2023 at 09:16:48AM +0000, Bruce Richardson wrote:
> > On Mon, Jan 09, 2023 at 02:56:04PM -0800, Tyler Retzlaff wrote:
> > > hi folks,
> > > 
> > > i would like to introduce a layer of abstraction that would allow
> > > optional use of standard C11 atomics when the platform / toolchain
> > > combination has them available.
> > > 
> > > making the option usable would be a phased approach intended to focus
> > > review and minimize dealing with churn on such a broad change.
> > > 
> > > 1. provide an initial series to add the abstraction and the ability
> > >    control enablement with a meson option enable_stdatomics=false will
> > >    be the default.
> > > 
> > >    for all existing platform / toolchain combinations the default would
> > >    remain false. i.e. i have no plans to enable it for existing platforms
> > >    toolchain combinations but leaves a change of default open to the
> > >    community as a future discussion if it is desired.
> > > 
> > > 2. once the initial abstraction is integrated a series will be introduced to
> > >    port the tree to the abstraction with enable_stdatomics=false. the goal
> > >    being low or no change to the current use of gcc builtin C++11 memory
> > >    model atomics.
> > > 
> > > 3. once the tree is ported a final series will be introduced to introduce
> > >    the remaining change to allow the use of enable_stdatomics=true.
> > > 
> > > would appreciate any assistance / suggestions you can provide to
> > > introduce the abstraction smoothly.
> > > 
> > 
> > Plan generally sounds ok. However, beyond point #3, would there then be
> > plans to remove the option and always use stdatomics in future?
> 
> that is a discussion for the community i think on a per-platform /
> per-toolchain basis. there is likely to be resistance which is why i'm
> favoring the opt-in if you want model.
> 
> some potential arguments against switching.
> 
> * it's an abi break there is no way around it.
> * old compiler x stdatomics implementation is less optimal than
>   old compiler x __atomics (potential argument).
> * there's some "mixed" use of variables in the tree where sometimes we
>   operate on them as if they are atomic and sometimes not. the std
>   atomics apis doesn't support this sometimes atomic codegen you either
>   get atomic or you don't.
> * direct use of atomics default to seq_cst ordering if the strengthening
>   of the ordering is not desired each variable needs to be hunted down
>   and explicitly returned to relaxed ordering access.
> 
> > To slightly expand the scope of the discussion - would it be worthwhile
> > putting these abstractions in a new library in DPDK other than EAL, to
> > start the process of splitting out some of the lower-level material from
> > that library?
> 
> the abstraction as i have prototyped it is a set of conditionally
> compiled macros where the macros mirror the standard c atomics for
> naming and have been placed in include/generic/rte_atomics.h
> 
> i've no problem splitting it out into a separate library and then have
> EAL depend on it, but it is currently header only so i'm not sure if it
> is worth doing for that.
> 
> we can chew on that more in a couple days when i submit the base series
> if you like.
> 
Thanks.

One additional point that just became clear to me when I started thinking
about upping our DPDK C-standard-baseline. We need to be careful what we
are considering when we up our C baseline. We can mandate a specific
compiler minimum and C version for compiling up DPDK itself, but I think we
should not mandate that for the end applications.

That means that our header files, such as atomics, should not require C99
or C11 even if the build of DPDK itself does. More specifically, even if we
bump DPDK minimum to C11, we should still allow apps to build using older
compiler settings.

Therefore, we probably need to maintain non-C11 atomics code paths in
headers beyond the point at which DPDK itself uses C11 as a code baseline.

/Bruce

RE: RFC abstracting atomics

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 11 January 2023 11.10
> 
> One additional point that just became clear to me when I started
> thinking
> about upping our DPDK C-standard-baseline. We need to be careful what
> we
> are considering when we up our C baseline. We can mandate a specific
> compiler minimum and C version for compiling up DPDK itself, but I
> think we
> should not mandate that for the end applications.

Why not?

And do you consider this backwards compatibility a build time or run time requirement?

> 
> That means that our header files, such as atomics, should not require
> C99
> or C11 even if the build of DPDK itself does. More specifically, even
> if we
> bump DPDK minimum to C11, we should still allow apps to build using
> older
> compiler settings.
> 
> Therefore, we probably need to maintain non-C11 atomics code paths in
> headers beyond the point at which DPDK itself uses C11 as a code
> baseline.

Am I misunderstanding your suggestion here: Code can be C11, but all APIs and header files must be C89?

Wouldn't that also prevent DPDK inline functions from being C11?

> 
> /Bruce

Re: RFC abstracting atomics

[Bruce Richardson]

On Wed, Jan 11, 2023 at 11:23:07AM +0100, Morten Brørup wrote:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Wednesday, 11 January 2023 11.10
> > 
> > One additional point that just became clear to me when I started
> > thinking
> > about upping our DPDK C-standard-baseline. We need to be careful what
> > we
> > are considering when we up our C baseline. We can mandate a specific
> > compiler minimum and C version for compiling up DPDK itself, but I
> > think we
> > should not mandate that for the end applications.
> 
> Why not?
> 
> And do you consider this backwards compatibility a build time or run time requirement?
> 
> > 
> > That means that our header files, such as atomics, should not require
> > C99
> > or C11 even if the build of DPDK itself does. More specifically, even
> > if we
> > bump DPDK minimum to C11, we should still allow apps to build using
> > older
> > compiler settings.
> > 
> > Therefore, we probably need to maintain non-C11 atomics code paths in
> > headers beyond the point at which DPDK itself uses C11 as a code
> > baseline.
> 
> Am I misunderstanding your suggestion here: Code can be C11, but all APIs and header files must be C89?
> 
> Wouldn't that also prevent DPDK inline functions from being C11?
>
Yes, it would.

Now, perhaps we don't need to ensure that our headers have strict C89
compatibility, but I think we need to be very careful about mandating that
end-user apps use particular c standard settings when compiling their own
code.

RE: RFC abstracting atomics

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 11 January 2023 12.57
> 
> On Wed, Jan 11, 2023 at 11:23:07AM +0100, Morten Brørup wrote:
> > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > Sent: Wednesday, 11 January 2023 11.10
> > >
> > > One additional point that just became clear to me when I started
> > > thinking
> > > about upping our DPDK C-standard-baseline. We need to be careful
> what
> > > we
> > > are considering when we up our C baseline. We can mandate a
> specific
> > > compiler minimum and C version for compiling up DPDK itself, but I
> > > think we
> > > should not mandate that for the end applications.
> >
> > Why not?
> >
> > And do you consider this backwards compatibility a build time or run
> time requirement?
> >
> > >
> > > That means that our header files, such as atomics, should not
> require
> > > C99
> > > or C11 even if the build of DPDK itself does. More specifically,
> even
> > > if we
> > > bump DPDK minimum to C11, we should still allow apps to build using
> > > older
> > > compiler settings.
> > >
> > > Therefore, we probably need to maintain non-C11 atomics code paths
> in
> > > headers beyond the point at which DPDK itself uses C11 as a code
> > > baseline.
> >
> > Am I misunderstanding your suggestion here: Code can be C11, but all
> APIs and header files must be C89?
> >
> > Wouldn't that also prevent DPDK inline functions from being C11?
> >
> Yes, it would.
> 
> Now, perhaps we don't need to ensure that our headers have strict C89
> compatibility, but I think we need to be very careful about mandating
> that
> end-user apps use particular c standard settings when compiling their
> own
> code.

I get your point, Bruce, but I disagree.

There should be a limit for how backwards compatible we want DPDK to be, and the limit should certainly not be C89. It might be C99 for a while, but it should soon be C11.

If someone is stuck with a very old C compiler, and already rely on (extended) LTS for their compiler and runtime environment, why would they expect bleeding edge DPDK to cater for them? They can use some old DPDK version and rely on DPDK LTS.

If you want to use an old compiler, you often have to use old libraries too, as new libraries often require newer compilers. This also applies to the Linux kernel. I don't see why DPDK should be any different.

But... DPDK LTS is only two years!?! My point is: What you are describing is not a DPDK problem, it is a DPDK LTS policy problem.

Re: RFC abstracting atomics

[Bruce Richardson]

On Wed, Jan 11, 2023 at 01:46:02PM +0100, Morten Brørup wrote:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Wednesday, 11 January 2023 12.57
> > 
> > On Wed, Jan 11, 2023 at 11:23:07AM +0100, Morten Brørup wrote:
> > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > > Sent: Wednesday, 11 January 2023 11.10
> > > >
> > > > One additional point that just became clear to me when I started
> > > > thinking
> > > > about upping our DPDK C-standard-baseline. We need to be careful
> > what
> > > > we
> > > > are considering when we up our C baseline. We can mandate a
> > specific
> > > > compiler minimum and C version for compiling up DPDK itself, but I
> > > > think we
> > > > should not mandate that for the end applications.
> > >
> > > Why not?
> > >
> > > And do you consider this backwards compatibility a build time or run
> > time requirement?
> > >
> > > >
> > > > That means that our header files, such as atomics, should not
> > require
> > > > C99
> > > > or C11 even if the build of DPDK itself does. More specifically,
> > even
> > > > if we
> > > > bump DPDK minimum to C11, we should still allow apps to build using
> > > > older
> > > > compiler settings.
> > > >
> > > > Therefore, we probably need to maintain non-C11 atomics code paths
> > in
> > > > headers beyond the point at which DPDK itself uses C11 as a code
> > > > baseline.
> > >
> > > Am I misunderstanding your suggestion here: Code can be C11, but all
> > APIs and header files must be C89?
> > >
> > > Wouldn't that also prevent DPDK inline functions from being C11?
> > >
> > Yes, it would.
> > 
> > Now, perhaps we don't need to ensure that our headers have strict C89
> > compatibility, but I think we need to be very careful about mandating
> > that
> > end-user apps use particular c standard settings when compiling their
> > own
> > code.
> 
> I get your point, Bruce, but I disagree.
> 
> There should be a limit for how backwards compatible we want DPDK to be, and the limit should certainly not be C89. It might be C99 for a while, but it should soon be C11.
> 
> If someone is stuck with a very old C compiler, and already rely on (extended) LTS for their compiler and runtime environment, why would they expect bleeding edge DPDK to cater for them? They can use some old DPDK version and rely on DPDK LTS.
> 
> If you want to use an old compiler, you often have to use old libraries too, as new libraries often require newer compilers. This also applies to the Linux kernel. I don't see why DPDK should be any different.
> 
> But... DPDK LTS is only two years!?! My point is: What you are describing is not a DPDK problem, it is a DPDK LTS policy problem.
> 

I don't see it as a compiler problem, but as a codebase one. It doesn't
matter if your compiler supports C11 if your codebase is using legacy
features from C89 that are no longer supported by later versions. Changing
compilers can be tricky, but updating a large legacy code-base is a much
more challenging proposition. There is a lot of old code out there in the
world!

That said, I would hope that there are few large codebases out there that
won't compile with a C99 or C11 standard language level, and there aren't
that many things that should cause problems. However, I don't really know for
sure, so urge caution.

/Bruce

RE: RFC abstracting atomics

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 11 January 2023 15.18
> 
> On Wed, Jan 11, 2023 at 01:46:02PM +0100, Morten Brørup wrote:
> > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > Sent: Wednesday, 11 January 2023 12.57
> > >
> > > On Wed, Jan 11, 2023 at 11:23:07AM +0100, Morten Brørup wrote:
> > > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > > > Sent: Wednesday, 11 January 2023 11.10
> > > > >
> > > > > One additional point that just became clear to me when I
> started
> > > > > thinking
> > > > > about upping our DPDK C-standard-baseline. We need to be
> careful
> > > what
> > > > > we
> > > > > are considering when we up our C baseline. We can mandate a
> > > specific
> > > > > compiler minimum and C version for compiling up DPDK itself,
> but I
> > > > > think we
> > > > > should not mandate that for the end applications.
> > > >
> > > > Why not?
> > > >
> > > > And do you consider this backwards compatibility a build time or
> run
> > > time requirement?
> > > >
> > > > >
> > > > > That means that our header files, such as atomics, should not
> > > require
> > > > > C99
> > > > > or C11 even if the build of DPDK itself does. More
> specifically,
> > > even
> > > > > if we
> > > > > bump DPDK minimum to C11, we should still allow apps to build
> using
> > > > > older
> > > > > compiler settings.
> > > > >
> > > > > Therefore, we probably need to maintain non-C11 atomics code
> paths
> > > in
> > > > > headers beyond the point at which DPDK itself uses C11 as a
> code
> > > > > baseline.
> > > >
> > > > Am I misunderstanding your suggestion here: Code can be C11, but
> all
> > > APIs and header files must be C89?
> > > >
> > > > Wouldn't that also prevent DPDK inline functions from being C11?
> > > >
> > > Yes, it would.
> > >
> > > Now, perhaps we don't need to ensure that our headers have strict
> C89
> > > compatibility, but I think we need to be very careful about
> mandating
> > > that
> > > end-user apps use particular c standard settings when compiling
> their
> > > own
> > > code.
> >
> > I get your point, Bruce, but I disagree.
> >
> > There should be a limit for how backwards compatible we want DPDK to
> be, and the limit should certainly not be C89. It might be C99 for a
> while, but it should soon be C11.
> >
> > If someone is stuck with a very old C compiler, and already rely on
> (extended) LTS for their compiler and runtime environment, why would
> they expect bleeding edge DPDK to cater for them? They can use some old
> DPDK version and rely on DPDK LTS.
> >
> > If you want to use an old compiler, you often have to use old
> libraries too, as new libraries often require newer compilers. This
> also applies to the Linux kernel. I don't see why DPDK should be any
> different.
> >
> > But... DPDK LTS is only two years!?! My point is: What you are
> describing is not a DPDK problem, it is a DPDK LTS policy problem.
> >
> 
> I don't see it as a compiler problem, but as a codebase one. It doesn't
> matter if your compiler supports C11 if your codebase is using legacy
> features from C89 that are no longer supported by later versions.
> Changing
> compilers can be tricky, but updating a large legacy code-base is a
> much
> more challenging proposition. There is a lot of old code out there in
> the
> world!

OK. But my same argument applies: Why would you need to use a brand new DPDK release in your project when the rest of your code base is ancient? In that case, you should rely on DPDK LTS.

> 
> That said, I would hope that there are few large codebases out there
> that
> won't compile with a C99 or C11 standard language level, and there
> aren't
> that many things that should cause problems. However, I don't really
> know for
> sure, so urge caution.
> 
> /Bruce

Re: RFC abstracting atomics

[Ben Magistro]

[-- Attachment #1: Type: text/plain, Size: 5020 bytes --]

As a user/developer I'll put a vote on Morten's side here.  There are other
libraries we utilize that have stated x.y.z is the last version that will
support w, beginning on version l.m.n it will be standard o.  I personally
don't think a project asking for C11 support at a minimum would be
unreasonable or overly burdensome.

In that vein I thought there was a supported operating systems page (can't
find it for 22.11 but did find it for an older version, 17.05).  On more
recent versions, there is the tested platforms page.  Going back to the
oldest LTS, 20.11 (and current 22.11 which includes some older OS not on
the 20.11 list), I would be shocked if any of the listed operating systems
didn't support C11 out of the box.

Just my $0.01

On Wed, Jan 11, 2023 at 10:07 AM Morten Brørup <mb@smartsharesystems.com>
wrote:

> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Wednesday, 11 January 2023 15.18
> >
> > On Wed, Jan 11, 2023 at 01:46:02PM +0100, Morten Brørup wrote:
> > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > > Sent: Wednesday, 11 January 2023 12.57
> > > >
> > > > On Wed, Jan 11, 2023 at 11:23:07AM +0100, Morten Brørup wrote:
> > > > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > > > > > Sent: Wednesday, 11 January 2023 11.10
> > > > > >
> > > > > > One additional point that just became clear to me when I
> > started
> > > > > > thinking
> > > > > > about upping our DPDK C-standard-baseline. We need to be
> > careful
> > > > what
> > > > > > we
> > > > > > are considering when we up our C baseline. We can mandate a
> > > > specific
> > > > > > compiler minimum and C version for compiling up DPDK itself,
> > but I
> > > > > > think we
> > > > > > should not mandate that for the end applications.
> > > > >
> > > > > Why not?
> > > > >
> > > > > And do you consider this backwards compatibility a build time or
> > run
> > > > time requirement?
> > > > >
> > > > > >
> > > > > > That means that our header files, such as atomics, should not
> > > > require
> > > > > > C99
> > > > > > or C11 even if the build of DPDK itself does. More
> > specifically,
> > > > even
> > > > > > if we
> > > > > > bump DPDK minimum to C11, we should still allow apps to build
> > using
> > > > > > older
> > > > > > compiler settings.
> > > > > >
> > > > > > Therefore, we probably need to maintain non-C11 atomics code
> > paths
> > > > in
> > > > > > headers beyond the point at which DPDK itself uses C11 as a
> > code
> > > > > > baseline.
> > > > >
> > > > > Am I misunderstanding your suggestion here: Code can be C11, but
> > all
> > > > APIs and header files must be C89?
> > > > >
> > > > > Wouldn't that also prevent DPDK inline functions from being C11?
> > > > >
> > > > Yes, it would.
> > > >
> > > > Now, perhaps we don't need to ensure that our headers have strict
> > C89
> > > > compatibility, but I think we need to be very careful about
> > mandating
> > > > that
> > > > end-user apps use particular c standard settings when compiling
> > their
> > > > own
> > > > code.
> > >
> > > I get your point, Bruce, but I disagree.
> > >
> > > There should be a limit for how backwards compatible we want DPDK to
> > be, and the limit should certainly not be C89. It might be C99 for a
> > while, but it should soon be C11.
> > >
> > > If someone is stuck with a very old C compiler, and already rely on
> > (extended) LTS for their compiler and runtime environment, why would
> > they expect bleeding edge DPDK to cater for them? They can use some old
> > DPDK version and rely on DPDK LTS.
> > >
> > > If you want to use an old compiler, you often have to use old
> > libraries too, as new libraries often require newer compilers. This
> > also applies to the Linux kernel. I don't see why DPDK should be any
> > different.
> > >
> > > But... DPDK LTS is only two years!?! My point is: What you are
> > describing is not a DPDK problem, it is a DPDK LTS policy problem.
> > >
> >
> > I don't see it as a compiler problem, but as a codebase one. It doesn't
> > matter if your compiler supports C11 if your codebase is using legacy
> > features from C89 that are no longer supported by later versions.
> > Changing
> > compilers can be tricky, but updating a large legacy code-base is a
> > much
> > more challenging proposition. There is a lot of old code out there in
> > the
> > world!
>
> OK. But my same argument applies: Why would you need to use a brand new
> DPDK release in your project when the rest of your code base is ancient? In
> that case, you should rely on DPDK LTS.
>
> >
> > That said, I would hope that there are few large codebases out there
> > that
> > won't compile with a C99 or C11 standard language level, and there
> > aren't
> > that many things that should cause problems. However, I don't really
> > know for
> > sure, so urge caution.
> >
> > /Bruce
>
>

[-- Attachment #2: Type: text/html, Size: 6642 bytes --]

Re: RFC abstracting atomics

[Jerin Jacob]

On Fri, Jan 13, 2023 at 7:49 PM Ben Magistro <koncept1@gmail.com> wrote:
>
> As a user/developer I'll put a vote on Morten's side here.  There are other libraries we utilize that have stated x.y.z is the last version that will support w, beginning on version l.m.n it will be standard o.  I personally don't think a project asking for C11 support at a minimum would be unreasonable or overly burdensome.

+1


Instead of polluting new DPDK code for legacy applications(If some
reason they want absolutely want to move latest and greatest DPDK), I
think it should be possible for legacy application selectivity turning
on/of like "#pragma GCC diagnostic warning "-std=c++11"
 or worst case  move DPDK function in wrapper(which is already case in
most of the applications) in their app and compile the wrapper only
with C11


>
> In that vein I thought there was a supported operating systems page (can't find it for 22.11 but did find it for an older version, 17.05).  On more recent versions, there is the tested platforms page.  Going back to the oldest LTS, 20.11 (and current 22.11 which includes some older OS not on the 20.11 list), I would be shocked if any of the listed operating systems didn't support C11 out of the box.
>
> Just my $0.01
>
> On Wed, Jan 11, 2023 at 10:07 AM Morten Brørup <mb@smartsharesystems.com> wrote:
>>
>> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
>> > Sent: Wednesday, 11 January 2023 15.18
>> >
>> > On Wed, Jan 11, 2023 at 01:46:02PM +0100, Morten Brørup wrote:
>> > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
>> > > > Sent: Wednesday, 11 January 2023 12.57
>> > > >
>> > > > On Wed, Jan 11, 2023 at 11:23:07AM +0100, Morten Brørup wrote:
>> > > > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
>> > > > > > Sent: Wednesday, 11 January 2023 11.10
>> > > > > >
>> > > > > > One additional point that just became clear to me when I
>> > started
>> > > > > > thinking
>> > > > > > about upping our DPDK C-standard-baseline. We need to be
>> > careful
>> > > > what
>> > > > > > we
>> > > > > > are considering when we up our C baseline. We can mandate a
>> > > > specific
>> > > > > > compiler minimum and C version for compiling up DPDK itself,
>> > but I
>> > > > > > think we
>> > > > > > should not mandate that for the end applications.
>> > > > >
>> > > > > Why not?
>> > > > >
>> > > > > And do you consider this backwards compatibility a build time or
>> > run
>> > > > time requirement?
>> > > > >
>> > > > > >
>> > > > > > That means that our header files, such as atomics, should not
>> > > > require
>> > > > > > C99
>> > > > > > or C11 even if the build of DPDK itself does. More
>> > specifically,
>> > > > even
>> > > > > > if we
>> > > > > > bump DPDK minimum to C11, we should still allow apps to build
>> > using
>> > > > > > older
>> > > > > > compiler settings.
>> > > > > >
>> > > > > > Therefore, we probably need to maintain non-C11 atomics code
>> > paths
>> > > > in
>> > > > > > headers beyond the point at which DPDK itself uses C11 as a
>> > code
>> > > > > > baseline.
>> > > > >
>> > > > > Am I misunderstanding your suggestion here: Code can be C11, but
>> > all
>> > > > APIs and header files must be C89?
>> > > > >
>> > > > > Wouldn't that also prevent DPDK inline functions from being C11?
>> > > > >
>> > > > Yes, it would.
>> > > >
>> > > > Now, perhaps we don't need to ensure that our headers have strict
>> > C89
>> > > > compatibility, but I think we need to be very careful about
>> > mandating
>> > > > that
>> > > > end-user apps use particular c standard settings when compiling
>> > their
>> > > > own
>> > > > code.
>> > >
>> > > I get your point, Bruce, but I disagree.
>> > >
>> > > There should be a limit for how backwards compatible we want DPDK to
>> > be, and the limit should certainly not be C89. It might be C99 for a
>> > while, but it should soon be C11.
>> > >
>> > > If someone is stuck with a very old C compiler, and already rely on
>> > (extended) LTS for their compiler and runtime environment, why would
>> > they expect bleeding edge DPDK to cater for them? They can use some old
>> > DPDK version and rely on DPDK LTS.
>> > >
>> > > If you want to use an old compiler, you often have to use old
>> > libraries too, as new libraries often require newer compilers. This
>> > also applies to the Linux kernel. I don't see why DPDK should be any
>> > different.
>> > >
>> > > But... DPDK LTS is only two years!?! My point is: What you are
>> > describing is not a DPDK problem, it is a DPDK LTS policy problem.
>> > >
>> >
>> > I don't see it as a compiler problem, but as a codebase one. It doesn't
>> > matter if your compiler supports C11 if your codebase is using legacy
>> > features from C89 that are no longer supported by later versions.
>> > Changing
>> > compilers can be tricky, but updating a large legacy code-base is a
>> > much
>> > more challenging proposition. There is a lot of old code out there in
>> > the
>> > world!
>>
>> OK. But my same argument applies: Why would you need to use a brand new DPDK release in your project when the rest of your code base is ancient? In that case, you should rely on DPDK LTS.
>>
>> >
>> > That said, I would hope that there are few large codebases out there
>> > that
>> > won't compile with a C99 or C11 standard language level, and there
>> > aren't
>> > that many things that should cause problems. However, I don't really
>> > know for
>> > sure, so urge caution.
>> >
>> > /Bruce
>>

Re: RFC abstracting atomics

[Tyler Retzlaff]

On Fri, Jan 13, 2023 at 09:40:20PM +0530, Jerin Jacob wrote:
> On Fri, Jan 13, 2023 at 7:49 PM Ben Magistro <koncept1@gmail.com> wrote:
> >
> > As a user/developer I'll put a vote on Morten's side here.  There are other libraries we utilize that have stated x.y.z is the last version that will support w, beginning on version l.m.n it will be standard o.  I personally don't think a project asking for C11 support at a minimum would be unreasonable or overly burdensome.
> 
> +1
> 
> 
> Instead of polluting new DPDK code for legacy applications(If some
> reason they want absolutely want to move latest and greatest DPDK), I
> think it should be possible for legacy application selectivity turning
> on/of like "#pragma GCC diagnostic warning "-std=c++11"
>  or worst case  move DPDK function in wrapper(which is already case in
> most of the applications) in their app and compile the wrapper only
> with C11

so just a caution that this mail thread isn't proposing any bump in C
standard requirement, it's about introducing an atomics abstraction
though it's really easy to start talking about standard C i understand.

let's move discussion about dpdk minimum standard C to the thread Bruce
posted yesterday to avoid distraction about atomics abstraction
integration.

Bruce's thread addressing setting the minimum standard is here.
http://mails.dpdk.org/archives/dev/2023-January/258925.html

thanks!