Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default

["Tan, Jianfeng" <jianfeng.tan@intel.com>]

> -----Original Message-----
> From: Yigit, Ferruh
> Sent: Wednesday, November 1, 2017 3:29 PM
> To: Tan, Jianfeng; Thomas Monjalon; Richardson, Bruce; Gonzalez Monroy,
> Sergio
> Cc: dev@dpdk.org; Santosh Shukla
> Subject: Re: [PATCH] eal: disable IOVA mode detection by default
> 
> On 10/31/2017 11:37 PM, Tan, Jianfeng wrote:
> >
> >
> >> -----Original Message-----
> >> From: Yigit, Ferruh
> >> Sent: Wednesday, November 1, 2017 2:33 PM
> >> To: Tan, Jianfeng; Thomas Monjalon; Richardson, Bruce; Gonzalez Monroy,
> >> Sergio
> >> Cc: dev@dpdk.org; Santosh Shukla
> >> Subject: Re: [PATCH] eal: disable IOVA mode detection by default
> >>
> >> On 10/31/2017 7:17 PM, Tan, Jianfeng wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Yigit, Ferruh
> >>>> Sent: Wednesday, November 1, 2017 9:07 AM
> >>>> To: Thomas Monjalon; Richardson, Bruce; Gonzalez Monroy, Sergio
> >>>> Cc: dev@dpdk.org; Yigit, Ferruh; Tan, Jianfeng; Santosh Shukla
> >>>> Subject: [PATCH] eal: disable IOVA mode detection by default
> >>>>
> >>>> Fix kernel crash with KNI because KNI requires physical addresses.
> >>>>
> >>>> A config option introduced to disable IOVA mode detection and to set it
> >>>> to physical address by default. Disabling config option will enable IOVA
> >>>> mode detection.
> >>>>
> >>>> When there is no intension to use KNI, it is safe to enable detection.
> >>>>
> >>>> Config option disable IOVA mode detection by default to be sure only
> >> who
> >>>> is aware of result enable it.
> >>>>
> >>>> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy")
> >>>>
> >>>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> >>>> ---
> >>>> Cc: Jianfeng Tan <jianfeng.tan@intel.com>
> >>>> Cc: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> >>>> Cc: Thomas Monjalon <thomas@monjalon.net>
> >>>
> >>> Refer to how vhost-kernel works, we may leverage a memory region
> table
> >> to do the translation. The bad side is it's less efficient than current
> >> phys_to_virt.
> >>
> >> Hi Jianfeng,
> >>
> >> Can you please elaborate?
> >
> > It is very similar to what Jacob proposed (KNI IOMMU).
> > 1. Share memsegs (memory regions) with KNI, some pairs of
> (userspace_addr, length).
> > 2. KNI maintains an address translation table, (userspace_addr, length,
> kernel_addr).
> > 3. Every time we need a kernel va, we search the table to obtain the addr.
> 
> I see, this looks like can work.
> But I concern about cost of this lookup, which needs to be done per packet.
> Current address translations are basic arithmetic operations.

The cost depends on how memory distribution of DPDK is. It's LOG(N) complexity.

> 
> What do you think doing va -> pa translation in userspace via eal APIs, and
> keep
> KNI kernel code intact?

That works, like a tx callback or tx_prepare for KNI pmd.
But it modifies mbuf, when we switch back this mbuf to use on IOVA_VA PMD, we change back to va again.

> Overall translation cost should be same, but the code complexity kept in eal
> layer which already knows about memsegs.

This is more complex as I mentioned above.

Thanks,
Jianfeng

> 
> >
> > Thanks,
> > Jianfeng
> >
> >
> >>
> >> Thanks,
> >> ferruh
> >>
> >>> Another side, we did not check the result of phys_to_virt, that's why
> >> kernel crashes.
> >>>
> >>> Thanks,
> >>> Jianfeng
> >>>
> >