RE: [RFC v2] non-temporal memcpy

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Konstantin Ananyev [mailto:konstantin.ananyev@huawei.com]
> Sent: Saturday, 30 July 2022 00.00
> 
> > > > > Actually, one question I have for such small data-transfer
> > > > > (16B per packet) - do you still see some noticable perfomance
> > > > > improvement for such scenario?
> > > >
> > > > Copying 16 byte from each packet in a burst of 32 packets would
> > > otherwise pollute 64 cache lines = 4 KB cache. With typically 64 KB
> L1
> > > > cache, I think it makes a difference.
> > >
> > > I understand the intention behind, my question was - it is really
> > > measurable?
> > > Something like: using pktmbuf_copy_nt(len=16) over using
> > > pktmbuf_copy(len=16)
> > > on workload X gives Y% thoughtput improvement?
> >
> > If the application is complex enough, and needs some of those 4 KB
> cache otherwise wasted, there will be a significant throughput
> > improvement; otherwise probably not.
> >
> > I have a general problem with this type of question: I hate that
> throughput is the only KPI (Key Performance Indicator) getting any
> > attention on the mailing list! Other KPIs, such as latency and
> resource conservation, are just as important in many real life use
> cases.
> 
> Well, I suppose that sort of expected question for the patch that
> introduces performance optimization:
> what is the benefit we expect to get and is it worth the effort?
> Throughput or latency improvement seems like an obvious choice here.
> About resource conservation - if the patch aims to improve cache
> consumption, then on some cache-bound
> workloads it should result in throughput improvement, correct?

The benefit is cache conservation.

Copying a burst of 32 1518 byte packets - using memcpy() - pollutes the entire L1 cache. Not trashing the entire L1 cache - using memcpy_nt() - should provide derived benefits for latency and/or throughput for most applications copying entire packets.

> 
> >
> > Here's a number for you: 6.25 % reduction in L1 data cache
> consumption. (Assuming 64 KB L1 cache with 64 byte cache lines and
> > application burst length of 32 packets.)
> 
> I understand that it should reduce cache eviction rate.
> The thing is that non-temporal stores are not free also: they consume
> WC buffers and some memory-bus bandwidth.
> AFAIK, for 16B non-consecutive NT stores, it means that only 25% of WC
> buffers capacity will be used,
> and in theory it might lead to extra memory pressure and worse
> performance in general.

I'm not a CPU expert, so I wonder if it makes any difference if the 16B non-consecutive store is non-temporal or normal... intuitively, the need to use a WC buffer and memory-bus bandwidth seems similar to me.

Also, my 16B example might be a bit silly... I used it to argue for the execution performance cost of omitting the alignment hints (added compares and branches). I suppose most NT copies will be packets, so mostly 64 or 1518 byte copies.

And in our application also 16 byte metadata being copied to the front of each packet. The copied packet follows immediately after the 16B metadata, so perhaps I should try to find a way to make these stores consecutive. Feature creep? ;-)

> In fact, IA manuals explicitly recommend to avoid partial cach-line
> writes whenever possible.
> Now, I don't know what would be more expensive in that case: re-fill
> extra cache-lines,
> or extra partial write memory transactions.

Good input, Konstantin. I will take this into consideration when optimizing the copy loops.

> That's why I asked for some performance numbers here.
> 

Got it.

The benefit of the patch is to avoid data cache pollution, and we agree about this.

So, to consider the other side of the coin, i.e. the potentially degraded memory copy throughput, I will measure both the NT copy and the normal copy using our application's packet capture feature, and provide both performance numbers.