Re: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection

[Thomas Monjalon <thomas@monjalon.net>]

23/07/2019 07:27, Jerin Jacob Kollanukkaran:
> From: Stojaczyk, Dariusz <dariusz.stojaczyk@intel.com>
> > From: Jerin Jacob Kollanukkaran [mailto:jerinj@marvell.com]
> > > From: Stojaczyk, Dariusz <dariusz.stojaczyk@intel.com>
> > > >
> > > > This introduces a regression where uio-bound devies are attached to
> > > > a DPDK app at runtime.

Yes it is a regression on purpose.
We can also name it a behaviour change (more below).

> > >
[...]
> There reason to choose VA incase if bus detects DC is following:
> 
> - All drivers are expected to work in RTE_IOVA_VA mode, irrespective of
>       physical address availability.
> - By default, the mempool, first asks for IOVA-contiguous memory using
>       ``RTE_MEMZONE_IOVA_CONTIG``. This is slow in RTE_IOVA_PA mode and it may
>       affect the application boot time.
> - It is easy to enable large amount of IOVA-contiguous memory use-cases
>       with IOVA in VA mode.
> 
[...]
> > The PCI device could have been simply hotplugged to the system after DPDK
> > app start. DPDK didn't know about it at initialization, so it picked
> > RTE_IOVA_VA and then would fail to attach any UIO-bound device ever
> > after:
> > 
> > EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not initializing
> 
> We have RTE_PCI_DRV_NEED_IOVA_AS_VA devices in DPDK, Which can work
> Only on VA. If we default 'PA' incase of DC, then what do with hotplugging on those devices?
[...]
> > > > When there are no devices attached at initialization, the only safe
> > > > default should be RTE_IOVA_PA. With RTE_IOVA_VA we just won't be
> > > > able to do any DMA to uio-bound PCI devices.

As Jerin explained, there is no safe default.
There are two cases which cannot work together:
	1/ no IOMMU
	2/ driver supporting only IOMMU address (named IOVA_AS_VA)

In the past we were defaulting to physical addressing,
it was in favor of case 1.
Now we decided to switch to IOMMU address by default,
which is in favor of case 2.
As explained above by Jerin, this is considered as an improvement.
We should explain this change in the known issues of the release notes.

The only real fix would be to allow both addresses at the same time,
with separate memory allocators.