RE: [RFC v2] non-temporal memcpy

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se]
> Sent: Wednesday, 10 August 2022 14.00
> 
> On 2022-08-09 19:24, Morten Brørup wrote:
> >> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> >> Sent: Tuesday, 9 August 2022 17.26
> >>
> >> On Tue, 9 Aug 2022 11:46:19 +0200
> >> Morten Brørup <mb@smartsharesystems.com> wrote:
> >>
> >>>>
> >>>> I don't think memcpy() functions should have alignment
> >> requirements.
> >>>> That's not very practical, and violates the principle of least
> >>>> surprise.
> >>>
> >>> I didn't make the CPUs with these alignment requirements.
> >>>
> >>> However, I will offer optimized performance in a generic NT
> memcpy()
> >> function in the cases where the individual alignment requirements of
> >> various CPUs happen to be met.
> >>
> >> Rather than making a generic equivalent memcpy function, why not
> have
> >> something which only takes aligned data.
> >
> > Our application is copying data not meeting x86 NT load alignment
> requirements (16 byte), so the function must support that.
> Specifically, our application is copying complete or truncated IP
> packets excl. the Ethernet and VLAN headers, i.e. offset by 14, 18 or
> 22 byte from the cache line aligned packet buffer.
> >
> 
> Sure, but you can use regular loads for the non-aligned parts, and the
> you continue to use NT load for the rest of the data. I suspect there
> is
> no point in doing NT loads for data on the same cache line that you've
> done regular loads for, so you might as well treat the alignment
> requirements as 64 byte, not 16.

I'm NT loading from the aligned address preceding the source address, and when NT storing these data, I'm skipping past the initial few bytes that were too many.

In some scenarios, the lcore capturing the packets has not touched the packet data at all, so not even the Ethernet header is in its data cache. Remember, not all applications are run-to-completion, so the capturing lcore might not access the packet header, but only the mbuf structure.

> 
> >> And to avoid user confusion
> >> change the name to be something not suggestive of memcpy.
> >>
> >> Maybe rte_non_cache_copy()?
> >>
> >> Want to avoid the naive user just doing s/memcpy/rte_memcpy_nt/ and
> >> expect
> >> everything to work.
> >
> > I see the risk you point out here... But it's not advertised in
> presentations, whitepapers and elsewhere like rte_memcpy() having much
> better performance than classic memcpy(), which might lead to that
> misconception. So the probability should be low.
> >