From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0422C41BB2; Thu, 2 Feb 2023 20:00:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 92EA140EDC; Thu, 2 Feb 2023 20:00:26 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 8216640693 for ; Thu, 2 Feb 2023 20:00:24 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id AE79D20B7102; Thu, 2 Feb 2023 11:00:23 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AE79D20B7102 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1675364423; bh=21GkfSLKkhoXvD6BQklmiUxWiFj5vlXk6yeTfBSQ3iQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=YX2RdBQWFfpD5x6x+hkbRWbcKb0tSQ7agilIYnhUnOX+p0alWfl4iFrYzSWtKWOUb QQmKRDoxk2p+59oYs4H6psv7FW6mSANVBpPwMatr55VyKtVS70Nglun6946+JnwOS0 hdRgsgXMsf85M+6wFKejMbrWMSIIZHdRxHBsdqLo= Date: Thu, 2 Feb 2023 11:00:23 -0800 From: Tyler Retzlaff To: Morten =?iso-8859-1?Q?Br=F8rup?= Cc: Honnappa Nagarahalli , thomas@monjalon.net, dev@dpdk.org, bruce.richardson@intel.com, david.marchand@redhat.com, jerinj@marvell.com, konstantin.ananyev@huawei.com, ferruh.yigit@amd.com, nd Subject: Re: [PATCH] eal: introduce atomics abstraction Message-ID: <20230202190023.GA32597@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: <1673558785-24992-1-git-send-email-roretzla@linux.microsoft.com> <1673558785-24992-2-git-send-email-roretzla@linux.microsoft.com> <1844463.CQOukoFCf9@thomas> <20230201214111.GA30564@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> <98CBD80474FA8B44BF855DF32C47DC35D876EE@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D876EE@smartserver.smartshare.dk> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Feb 02, 2023 at 09:43:58AM +0100, Morten Brørup wrote: > > From: Tyler Retzlaff [mailto:roretzla@linux.microsoft.com] > > Sent: Wednesday, 1 February 2023 22.41 > > > > On Wed, Feb 01, 2023 at 01:07:59AM +0000, Honnappa Nagarahalli wrote: > > > > > > > From: Thomas Monjalon > > > > Sent: Tuesday, January 31, 2023 4:42 PM > > > > > > > > Honnappa, please could you give your view on the future of atomics > > in DPDK? > > > Thanks Thomas, apologies it has taken me a while to get to this > > discussion. > > > > > > IMO, we do not need DPDK's own abstractions. APIs from stdatomic.h > > (stdatomics as is called here) already serve the purpose. These APIs > > are well understood and documented. > > > > i agree that whatever atomics APIs we advocate for should align with > > the > > standard C atomics for the reasons you state including implied > > semantics. > > > > > > > > For environments where stdatomics are not supported, we could have a > > stdatomic.h in DPDK implementing the same APIs (we have to support only > > _explicit APIs). This allows the code to use stdatomics APIs and when > > we move to minimum supported standard C11, we just need to get rid of > > the file in DPDK repo. > > Perhaps we can use something already existing, such as this: > https://android.googlesource.com/platform/bionic/+/lollipop-release/libc/include/stdatomic.h > > > > > my concern with this is that if we provide a stdatomic.h or introduce > > names > > from stdatomic.h it's a violation of the C standard. > > > > references: > > * ISO/IEC 9899:2011 sections 7.1.2, 7.1.3. > > * GNU libc manual > > https://www.gnu.org/software/libc/manual/html_node/Reserved- > > Names.html > > > > in effect the header, the names and in some instances namespaces > > introduced > > are reserved by the implementation. there are several reasons in the > > GNU libc > > manual that explain the justification for these reservations and if > > if we think about ODR and ABI compatibility we can conceive of others. > > I we are going to move to C11 soon, I consider the shim interim, and am inclined to ignore these warning factors. > > If we are not moving to C11 soon, I would consider these disadvantages more seriously. I think it's reasonable to assume that we are talking years here. We've had a few discussions about minimum C standard. I think my first mailing list exchanges about C99 was almost 2 years ago. Given that we still aren't on C99 now (though i know Bruce has a series up) indicates that progression to C11 isn't going to happen any time soon and even if it was the baseline we still can't just use it (reasons described later). Also, i'll point out that we seem to have accepted moving to C99 with one of the holdback compilers technically being non-conformant but it isn't blocking us because it provides the subset of C99 features without being conforming that we happen to be using. > > > > > i'll also remark that the inter-mingling of names from the POSIX > > standard implicitly exposed as a part of the EAL public API has been > > problematic for portability. > > This is a very important remark, which should be considered carefully! Tyler has firsthand experience with DPDK portability. If he thinks porting to Windows is going to be a headache if we expose the stdatomic.h API, we must listen! So, what is your gut feeling here, Tyler? I think this is even more of a concern with language standard than it is with a platform standard. Because the language standard is used across platforms. On the surface it looks appealing to just go through all the dpdk code one last time and #include and directly depend on names that "look" standard. In practice though we aren't depending on the toolchain / libc surface we are binding ourselves to the shim and the implementation it provides. This is aside from the mechanics of making it work in the different contexts we now have to care about. Here is a story of how things become tricky. When i #include which one gets used if the implementation provides one? Let's force our stdatomic.h Now i need to force the build system to prefer my shim header? Keeping in mind that the presence of a libc stdatomic.h does not mean that the toolchain in fact supports standard atomics. Okay, that's under our control by editing some meson.build files maybe it isn't so bad but... It seems my application also has to do the same in their build system now because... The type definitions (size, alignment) and code generated from the body of inline functions as seen by the application built translation units may differ from those in the dpdk translation units if they don't use our header. The potential for ABI compat problems is increasing but maybe it is managable? it can be worse... We can't limit our scope to thinking that there is just an application (a single binary) and dpdk. Complex applications will invariably depend on other libraries and if the application needs to interface with those compatibily at the ABI level using standard atomics then we've made it very difficult since the application has to choose to use our conflicting named atomic types which may not be compatible or the real standard atomics. They can of course produce horrible shims of their own to interoperate. We need consistency across the entire binary at runtime and i don't think it's practical to say that anyone who uses dpdk has to compile their whole world with our shim. So dealing with all this complexity for the sake of asthetics "looking" like the standard api seems kind of not worth it. Sure it saves having to deprecate later and one last session of shotgun surgery but that's kind of all we get. Don't think i'm being biased in favor of windows/msvc here. From the perspective of the windows/msvc combination i intend to use only the standard C ABI provided by the implementation. I have no intention of trying to introduce support for the current ABI that doesn't use the standard atomic types. my discouraging of this approach is about avoiding subtle to detect but very painful problems on {linux,unix}/compiler combinations that already have a shipped/stable ABI. > > > > let's discuss this from here. if there's still overwhelming desire to > > go > > this route then we'll just do our best. > > > > ty > > I have a preference for exposing the stdatomic.h API. Tyler listed the disadvantages above. (I also have a preference for moving to C11 soon.) I am eager to see this happen, but as explained in my original proposal it doesn't eliminate the need for an abstraction. Unless we are willing to break our compatibility promises and potentially take a performance hit on some platform/compiler combinations which as i understand is not acceptable. > > Exposing a 1:1 similar API with RTE prefixes would also be acceptable for me. The disadvantage is that the names are different than the C11 names, which might lead to some confusion. And from an ABI stability perspective, such an important API should not be marked experimental. This means that years will pass before we can get rid of it again, due to ABI stability policies. I think the key to success with rte_ prefixed names is making absolutely sure we mirror the semantics and types in the standard. I will point out one bit of fine print here is that we will not support atomic operations on struct/union types (something the standard supports). With the rte_ namespace i think this becomes less ambiguous, if we present standard C names though what's to avoid the confusion? Aside from it fails to compile with one compiler vs another. I agree that this may be around for years. But how many years depends a lot on how long we have to maintain compatibility for the existing platform/compiler combinations that can't (and aren't enabled) to use the standard. Even if we introduced standard names we still have to undergo some kind of mutant deprecation process to get the world to recompile everything against the actual standard, so it doesn't give us forward compatibility. Let me know what folks would like to do, i guess i'm firmly leaned toward no-shim and just rte_ explicit. But as a community i'll pursue whatever you decide. Thanks!