Re: [PATCH] mbuf: add mbuf physical address field to dynamic field

[Bruce Richardson <bruce.richardson@intel.com>]

On Sun, Jul 03, 2022 at 09:31:01AM +0200, Morten Brørup wrote:
> > From: Shijith Thotton [mailto:sthotton@marvell.com]
> > Sent: Friday, 1 July 2022 14.25
> > 
> > >> If all devices are configured to run in IOVA mode as VA, physical
> > >> address field of mbuf (buf_iova) won't be used.
> 
> Will some of the hardware vendors please comment on this: Has IOVA VA mode become common over time, or is it still an exotic bleeding edge feature?
> 
> If it has become common, we should let DPDK evolve accordingly, and consider PA (non-VA) mode legacy, treating it as such. Don't get stuck in the past.
> 

IOVA as VA mode is indeed common and we are constantly encouraging users to
switch to using vfio to try and take advantage of this.

However, in my experience IOVA as PA is still very, very common too. We
cannot drop support for this mode just yet, unfortunately.

> > >> In such cases,
> > buf_iova
> > >> space is free to use as a dynamic field. So a new dynamic field
> > member
> > >> (dynfield2) is added in mbuf structure to make use of that space.
> > >>
> > >> A new mbuf flag RTE_MBUF_F_DYNFIELD2 is introduced to help identify
> > the
> > >> mbuf that can use dynfield2.
> > >>
> > >> Signed-off-by: Shijith Thotton <sthotton@marvell.com>
> > >
> > > This seems like a complex and potentially error prone way to do this.
> 
> Perhaps this optimization should be a compile time option instead?
>

It could indeed be, and probably could be done very safely, in that we could
mark as disabled all drivers when the mode is enabled for a build.
Thereafter, drivers could be marked as VA-only safe as they are updated as
necessary, i.e. use the build system to enforce that only drivers known to
work with the mode are built when the mode is enabled.

That said, verifying all drivers to work with this mode is a decent effort.
Do we have indications of the perf benefit we would get from doing this for
some real-world app?

/Bruce