Re: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection

["Stojaczyk, Dariusz" <dariusz.stojaczyk@intel.com>]

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran [mailto:jerinj@marvell.com]
> Sent: Tuesday, July 23, 2019 6:19 AM
> 
> > -----Original Message-----
> > From: Stojaczyk, Dariusz <dariusz.stojaczyk@intel.com>
> > Sent: Tuesday, July 23, 2019 9:06 AM
> > To: Thomas Monjalon <thomas@monjalon.net>; David Marchand
> > <david.marchand@redhat.com>; Burakov, Anatoly
> > <anatoly.burakov@intel.com>; Jerin Jacob Kollanukkaran
> > <jerinj@marvell.com>
> > Cc: dev@dpdk.org
> > Subject: [EXT] RE: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection
> >
> > This introduces a regression where uio-bound devies are attached to a
> DPDK
> > app at runtime.
> 
> Just to understand the requirements;
> # Is this requirement for SPDK?
> # Is brand new PCI device scanned and attached to DPDK at runtime?
> # Any specific reason for using uio vs vfio?

Jerin,

It came up in SPDK tests, but it's certainly nothing SPDK-specific, I can't
give you the steps but it should be reproducible even with testpmd.

The PCI device could have been simply hotplugged to the system after
DPDK app start. DPDK didn't know about it at initialization, so it picked
RTE_IOVA_VA and then would fail to attach any UIO-bound device
ever after:

EAL:   Expecting 'PA' IOVA mode but current mode is 'VA', not initializing
EAL: Driver cannot attach the device (0000:00:09.0)
EAL: Failed to attach device on primary process

UIO is commonly used on systems without IOMMU- including VMs.

> 
> If it is for SPDK,
> # How about introducing rte_eal_init_with_mode(enum rte_iova_mode)?
> # How about adding dummy bus which returns RTE_IOVA_PA in the
> bus_get_iommus_class() in SPDK code base?

There's already an --iova=mode option in DPDK that forces the iova mode.
I'm not concerned about configurability, but the regression in the
default behavior.

I can add workarounds to SPDK, sure, but that wouldn't be a very healthy
approach.

D.

> 
> >
> > When there are no devices attached at initialization, the only safe default
> > should be RTE_IOVA_PA. With RTE_IOVA_VA we just won't be able to do
> > any DMA to uio-bound PCI devices.
> >
> > Can we revert this patch?
> >
> > D.
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas
> Monjalon
> > > Sent: Monday, July 22, 2019 5:53 PM
> > > To: David Marchand <david.marchand@redhat.com>; Burakov, Anatoly
> > > <anatoly.burakov@intel.com>; jerinj@marvell.com
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection
> > >
> > > 22/07/2019 14:56, David Marchand:
> > > > Following the issues reported by Jerin and the discussion that
> > > > emerged from it, here are fixes to restore and document the behavior
> > > > of the EAL and the pci bus driver.
> > > >
> > > > I pondered all the arguments and tried to have the less changes
> > > > possible.
> > > > I can't find a need for a flag to just announce support of physical
> > > > addresses from the pmd point of view.
> > > > So it ended up with something really close to what Jerin had suggested.
> > > >
> > > > But the problem is that this is still unfinished wrt the documentation.
> > > > I will be offline for 10 days and we need this to move forward, so
> > > > sending anyway.
> > > >
> > > > Changelog since v3:
> > > > - fixed typos in patch 2,
> > > > - updated patch 3 title,
> > > > - moved and reworded comments in the note section in patch 4,
> > > >
> > > > Changelog since v2 (Jerin):
> > > > - Patch 2/4 - Remove personal appeals in log messages(Anatoly)
> > > > - Patch 4/4 - Added documentation (Anatoly)
> > > >
> > > > Changelog since v1 (Jerin):
> > > > - Changed RTE_PCI_DRV_IOVA_AS_VA flag name as
> > > RTE_PCI_DRV_NEED_IOVA_AS_VA
> > > >   (patch 3/4)
> > > > - Changed IOVA mode as VA for default case(patch 4/4) with
> > > documentation
> > > > - Tested the patch series on octeontx2 platform
> > >
> > > Applied, thanks Jerin, Anatoly and David for converging on a
> > > documented solution together.
> > >
> > >