Re: [dpdk-dev] [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers

[Thomas Monjalon <thomas@monjalon.net>]

15/07/2019 16:26, Jerin Jacob Kollanukkaran:
> > > > +
> > > > +IOVA Mode is selected by considering what the current usable
> > > > +Devices on the system requires and/or supports.
> > > > +
> > > > +Below is the 2-step heuristic for this choice.
> > > > +
> > > > +For the first step, EAL asks each bus its requirement in terms of
> > > > +IOVA mode and decides on a preferred IOVA mode.
> > > > +
> > > > +- if all buses report RTE_IOVA_PA, then the preferred IOVA mode is
> > > > +RTE_IOVA_PA,
> > > > +- if all buses report RTE_IOVA_VA, then the preferred IOVA mode is
> > > > +RTE_IOVA_VA,
> > > > +- if all buses report RTE_IOVA_DC, no bus expressed a preferrence,
> > > > +then the
> > > > +  preferred mode is RTE_IOVA_DC,
> > > > +- if the buses disagree (at least one wants RTE_IOVA_PA and at
> > > > +least one wants
> > > > +  RTE_IOVA_VA), then the preferred IOVA mode is RTE_IOVA_DC (see
> > > > +below with the
> > > > +  check on Physical Addresses availability),
> > > > +
> > > > +The second step is checking if the preferred mode complies with the
> > > > +Physical Addresses availability since those are only available to
> > > > +root user in recent kernels.
> > > > +
> > > > +- if the preferred mode is RTE_IOVA_PA but there is no access to
> > > > +Physical
> > > > +  Addresses, then EAL init will fail early, since later probing of
> > > > +the devices
> > > > +  would fail anyway,
> > > > +- if the preferred mode is RTE_IOVA_DC then based on the Physical
> > > > +Addresses
> > > > +  availability, the preferred mode is adjusted to RTE_IOVA_PA or
> > RTE_IOVA_VA.
> > > > +  In the case when the buses had disagreed on the IOVA Mode at the
> > > > +first step,
> > > > +  part of the buses won't work because of this decision.
> > >
> > > Is there any specific reason why we always prefer PA if physical
> > > addresses are available? Since we're already assuming that all devices
> > > support PA and VA anyway, what's the harm in enabling VA by default?
> > 
> > If PA is available, it means we are running as root.
> > We can assume that using root is a choice, probably related to a preference
> > for PA.
> 
> # Even if we are running as root, Why to choose PA in case of DC?
> ie. Following logic is not need
>                 if (iova_mode == RTE_IOVA_DC) {
>                         iova_mode = phys_addrs ? RTE_IOVA_PA : RTE_IOVA_VA;
>                         RTE_LOG(DEBUG, EAL,
>                                 "Buses did not request a specific IOVA mode, using '%s' based on physical addresses availability.\n",
>                                 phys_addrs ? "PA" : "VA");
>                 }

Why running as root if using VA anyway?
We can assume the user knows what he is doing, so it is a user choice.
We want to allow the user choosing, right?

> # When DPDK running on guest, Anyway it can not access the real PA, It will be IPA.

What is IPA? Isn't it a beer?

> So I don't understand logic behind choose PA when DC.
> To me, it make sense to choose PA when DC.

You probably mean "choose VA".

> # To align with RTE_PCI_DRV_NEED_MAPPING flag and reflect it "need" rather
> than support, I think, flag can be changed to RTE_PCI_DRV_NEED_IOVA_AS_VA

I think the most important is to have a good documentation of this flag
(it was not done properly when Cavium introduced it initially).
If you want to rename the flag, you can do it in a separate patch.
If renaming, I really would like to get an answer to an old question:
Why IO adress is called IOVA? The name "IOVA_AS_VA" looks strange.
For reference, one description of addressing:
https://lists.linuxfoundation.org/pipermail/iommu/2018-May/027686.html

About the naming, do you remember how I insisted to have a correct naming
of all related stuff in DPDK? It was hard to get it accepted,
the discussion was not nice and I stopped insisting to get all details fine
because I just got bored. It was a really bad experience.
You can ask why I remind this now? Because we must take care of all
details, make sure our messages are well understood, and be cooperative.

> Other than above points,
> Reviewed this patch and tested on octeontx2, It looks good to me.