RE: [RFC v2] non-temporal memcpy

[Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>]

<snip>

> 
> +TO: @Honnappa, we need input from ARM
> 
> > From: Konstantin Ananyev [mailto:konstantin.ananyev@huawei.com]
> > Sent: Friday, 29 July 2022 21.49
> > >
> > > > From: Konstantin Ananyev [mailto:konstantin.ananyev@huawei.com]
> > > > Sent: Friday, 29 July 2022 14.14
> > > >
> > > >
> > > > Sorry, missed that part.
> > > >
> > > > >
> > > > > > Another question - who will do 'sfence' after the copying?
> > > > > > Would it be inside memcpy_nt (seems quite costly), or would it
> > > > > > be another API function for that: memcpy_nt_flush() or so?
> > > > >
> > > > > Outside. Only the developer knows when it is required, so it
> > wouldn't
> > > > make any sense to add the cost inside memcpy_nt().
> > > > >
> > > > > I don't think we should add a flush function; it would just be
> > > > another name for an already existing function. Referring to the
> > > > required
> > > > > operation in the memcpy_nt() function documentation should
> > suffice.
> > > > >
> > > >
> > > > Ok, but again wouldn't it be arch specific?
> > > > AFAIK for x86 it needs to boil down to sfence, for other
> > architectures
> > > > - I don't know.
> > > > If you think there already is some generic one (rte_wmb?) that
> > would
> > > > always produce
> > > > correct instructions - sure let's use it.
> > > >
> > >
> > > DPDK has generic functions to wrap architecture specific stuff like
> > memory barriers.
> > >
> > > Because they are non-temporal stores, I suspect that rte_mb() is
> > required before reading the data from the location it was copied to.
> > > Ensuring that STORE operations are ordered (rte_wmb) might not
> > suffice. However, I'm not a CPU expert, so I will seek advice from
> > > more qualified people in the community on this.
> >
> > I think for IA sfence is enough, see citation below, for other
> > architectures - no idea.
> > What I am trying to say - it needs to be the *same* function on all
> > archs we support.
> 
> Now I get it: rte_wmb() might be appropriate on x86, but if any other
> architecture requires something else, we should add a new common function
> for flushing, e.g. rte_memcpy_nt_flush().
> 
> >
> > IA SW optimization manual:
> > 9.4.2 Streaming Store Usage Models
> > The two primary usage domains for streaming store are coherent
> > requests and non-coherent requests.
> > 9.4.2.1 Coherent Requests
> > Coherent requests are normal loads and stores to system memory, which
> > may also hit cache lines present in another processor in a
> > multiprocessor environment. With coherent requests, a streaming store
> > can be used in the same way as a regular store that has been mapped
> > with a WC memory type (PAT or MTRR). An SFENCE instruction must be
> > used within a producer-consumer usage model in order to ensure
> > coherency and visibility of data between processors.
> > Within a single-processor system, the CPU can also re-read the same
> > memory location and be assured of coherence (that is, a single,
> > consistent view of this memory location).
> > The same is true for a multiprocessor
> > (MP) system, assuming an accepted MP software producer-consumer
> > synchronization policy is employed.
> >
> 
> With this reference, I am convinced that you are right about the SFENCE. This
> puts a checkmark on this item on my TODO list for the patch. Thank you,
> Konstantin!
> 
> Any ARM CPU experts on the mailing list seeing this, not on vacation?
> @Honnappa, I'm looking at you. :-)
> 
> Summing up, the question is:
> 
> After a bunch of *non-temporal* stores (STNP instruction) on ARM
> architecture, does calling rte_wmb() suffice to ensure the data is visible across
> the system?
Apologies for the late response, the docs did not have enough information. The internal dialogue is still going on, but I have some information now. There is some information in ArmV8 programmer's guide [1], though it is not complete.
In summary, rte_wmb()/rte_mb() would not suffice, we need new APIs.

From my perspective, I see several scenarios:
1)	Need for ordering before the memcpy_nt. Here there are several cases:
	a.	LD – LDNP/STNP – DMB NSHLD
	b.	ST – LDNP/STNP – DMB NSH
2)	Need for ordering after the memcpy. Again, we have the similar use cases:
	a.	LDNP/STNP – LD – DMB NSH
	b.	LDNP/STNP – ST – DMB NSH

The 'ST - STNP' and 'STNP - ST' do not apply here, but good to add an API for completion.

So, may be we could have rte_[r|w]mb_nt() APIs.

[1] https://developer.arm.com/documentation/den0024/a/The-A64-instruction-set/Memory-access-instructions/Non-temporal-load-and-store-pair