From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 126ABA00C2; Tue, 9 Aug 2022 13:56:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A88144069C; Tue, 9 Aug 2022 13:56:52 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id F15CB40143 for ; Tue, 9 Aug 2022 13:56:50 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id A3C5E75B0 for ; Tue, 9 Aug 2022 13:56:50 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 74A237266; Tue, 9 Aug 2022 13:56:50 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=ALL_TRUSTED, AWL, NICE_REPLY_A, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.6 X-Spam-Score: -1.7 Received: from [192.168.1.59] (unknown [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 4036D7338; Tue, 9 Aug 2022 13:56:49 +0200 (CEST) Message-ID: Date: Tue, 9 Aug 2022 13:56:48 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [RFC v2] non-temporal memcpy Content-Language: en-US To: =?UTF-8?Q?Morten_Br=c3=b8rup?= , Konstantin Ananyev , Konstantin Ananyev , dev@dpdk.org, Bruce Richardson , Honnappa Nagarahalli Cc: Jan Viktorin , Ruifeng Wang , David Christensen , Stanislaw Kardach References: <98CBD80474FA8B44BF855DF32C47DC35D871D4@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D871DB@smartserver.smartshare.dk> <262c214b-7870-a221-2621-6684dce42823@yandex.ru> <98CBD80474FA8B44BF855DF32C47DC35D871E6@smartserver.smartshare.dk> <2c646d01-14d0-e5cb-2d7c-50c8456fc3e5@yandex.ru> <98CBD80474FA8B44BF855DF32C47DC35D8720C@smartserver.smartshare.dk> <5e1567fb744841a0915348397a81b99d@huawei.com> <98CBD80474FA8B44BF855DF32C47DC35D87211@smartserver.smartshare.dk> <66b6efccde5b4d68bffcc47713cdf983@huawei.com> <98CBD80474FA8B44BF855DF32C47DC35D87213@smartserver.smartshare.dk> <681d5810-bfe6-3361-b3c3-30a5fb344b7e@lysator.liu.se> <98CBD80474FA8B44BF855DF32C47DC35D87246@smartserver.smartshare.dk> From: =?UTF-8?Q?Mattias_R=c3=b6nnblom?= In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87246@smartserver.smartshare.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2022-08-09 11:34, Morten Brørup wrote: >> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se] >> Sent: Sunday, 7 August 2022 22.20 >> >> On 2022-07-29 22:26, Morten Brørup wrote: >>> +TO: @Honnappa, we need input from ARM >>> >>>> From: Konstantin Ananyev [mailto:konstantin.ananyev@huawei.com] >>>> Sent: Friday, 29 July 2022 21.49 >>>>> >>>>>> From: Konstantin Ananyev [mailto:konstantin.ananyev@huawei.com] >>>>>> Sent: Friday, 29 July 2022 14.14 >>>>>> >>>>>> >>>>>> Sorry, missed that part. >>>>>> >>>>>>> >>>>>>>> Another question - who will do 'sfence' after the copying? >>>>>>>> Would it be inside memcpy_nt (seems quite costly), or would >>>>>>>> it be another API function for that: memcpy_nt_flush() or so? >>>>>>> >>>>>>> Outside. Only the developer knows when it is required, so it >>>> wouldn't >>>>>> make any sense to add the cost inside memcpy_nt(). >>>>>>> >>>>>>> I don't think we should add a flush function; it would just be >>>>>> another name for an already existing function. Referring to the >>>>>> required >>>>>>> operation in the memcpy_nt() function documentation should >>>> suffice. >>>>>>> >>>>>> >>>>>> Ok, but again wouldn't it be arch specific? >>>>>> AFAIK for x86 it needs to boil down to sfence, for other >>>> architectures >>>>>> - I don't know. >>>>>> If you think there already is some generic one (rte_wmb?) that >>>> would >>>>>> always produce >>>>>> correct instructions - sure let's use it. >>>>>> >>>>> >>>>> DPDK has generic functions to wrap architecture specific stuff like >>>> memory barriers. >>>>> >>>>> Because they are non-temporal stores, I suspect that rte_mb() is >>>> required before reading the data from the location it was copied to. >>>>> Ensuring that STORE operations are ordered (rte_wmb) might not >>>> suffice. However, I'm not a CPU expert, so I will seek advice from >>>>> more qualified people in the community on this. >>>> >>>> I think for IA sfence is enough, see citation below, >>>> for other architectures - no idea. >>>> What I am trying to say - it needs to be the *same* function on all >>>> archs we support. >>> >>> Now I get it: rte_wmb() might be appropriate on x86, but if any other >> architecture requires something else, we should add a new common >> function for flushing, e.g. rte_memcpy_nt_flush(). >>> >> >> rte_wmb() not being enough also my understanding. NT stores are weakly >> ordered on x86, and requires a sfence to be ordered with non-NT stores. >> >> Unfortunately, this per-memcpy sfence instruction make even 1500-byte >> sized copy operations much slower - at least in micro benchmarks. > > I agree that calling rte_mb() from each NT memcpy would be counterproductive on small/medium copy operations. > There's no need for a full barrier after the copy operation. sfence (e.g., rte_wmb()) is enough. I guess you will want a rte_rmb() before you start copying as well, if you are using NT loads. > Which is why rte_mb() is not called by the NT memcpy function itself, but by the application. This requirement will be part of the function's documentation. > rte_memcpy_surprise() :) > The application must call rte_mb() before it accesses the copy. > > Alternatively, the application can call rte_mb() after a burst of NT memcopies. > >> >>>> >>>> IA SW optimization manual: >>>> 9.4.2 Streaming Store Usage Models >>>> The two primary usage domains for streaming store are coherent >> requests >>>> and non-coherent requests. >>>> 9.4.2.1 Coherent Requests >>>> Coherent requests are normal loads and stores to system memory, >> which >>>> may also hit cache lines >>>> present in another processor in a multiprocessor environment. With >>>> coherent requests, a streaming store >>>> can be used in the same way as a regular store that has been mapped >>>> with a WC memory type (PAT or >>>> MTRR). An SFENCE instruction must be used within a producer-consumer >>>> usage model in order to ensure >>>> coherency and visibility of data between processors. >>>> Within a single-processor system, the CPU can also re-read the same >>>> memory location and be assured of >>>> coherence (that is, a single, consistent view of this memory >> location). >>>> The same is true for a multiprocessor >>>> (MP) system, assuming an accepted MP software producer-consumer >>>> synchronization policy is >>>> employed. >>>> >>> >>> With this reference, I am convinced that you are right about the >> SFENCE. This puts a checkmark on this item on my TODO list for the >> patch. Thank you, Konstantin! >>> >>> Any ARM CPU experts on the mailing list seeing this, not on vacation? >> @Honnappa, I'm looking at you. :-) >>> >>> Summing up, the question is: >>> >>> After a bunch of *non-temporal* stores (STNP instruction) on ARM >> architecture, does calling rte_wmb() suffice to ensure the data is >> visible across the system? >>> >