DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] eal: force IOVA mode to physical
@ 2018-08-29 15:58 eric zhang
  2018-08-30  6:13 ` Hemant
  2018-08-30  9:09 ` Burakov, Anatoly
  0 siblings, 2 replies; 15+ messages in thread
From: eric zhang @ 2018-08-29 15:58 UTC (permalink / raw)
  To: anatoly.burakov, bruce.richardson; +Cc: dev, Allain.Legacy, Matt.Peters

This patch adds a configuration option to force the IOVA mode to
physical address (PA). There exists virtual devices that are not
directly attached to the PCI bus, and therefore the auto detection
of the IOVA mode based on probing the PCI bus and IOMMU configuration
may not report the required addressing mode. Having the configuration
option permits the mode to be explicitly configured in this scenario.

Signed-off-by: eric zhang <eric.zhang@windriver.com>
---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
index e0b5ae1..bee4aed 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -805,6 +805,7 @@ static void rte_eal_init_alert(const char *msg)
 		return -1;
 	}
 
+#ifndef RTE_EAL_IOVA_MODE_PA
 	/* autodetect the iova mapping mode (default is iova_pa) */
 	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
 
@@ -816,6 +817,12 @@ static void rte_eal_init_alert(const char *msg)
 			"Some devices want IOVA as VA but PA will be used because.. "
 			"KNI module inserted\n");
 	}
+#else
+	/* Force iova mapping mode to be physical address */
+	rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
+	RTE_LOG(WARNING, EAL,
+		"Force the iova mapping mode to be physical address\n");
+#endif
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-29 15:58 [dpdk-dev] [PATCH] eal: force IOVA mode to physical eric zhang
@ 2018-08-30  6:13 ` Hemant
  2018-08-30  9:09 ` Burakov, Anatoly
  1 sibling, 0 replies; 15+ messages in thread
From: Hemant @ 2018-08-30  6:13 UTC (permalink / raw)
  To: eric zhang, anatoly.burakov, bruce.richardson
  Cc: dev, Allain.Legacy, Matt.Peters

On 8/29/2018 9:28 PM, eric zhang wrote:
> This patch adds a configuration option to force the IOVA mode to
> physical address (PA). There exists virtual devices that are not
> directly attached to the PCI bus, and therefore the auto detection
> of the IOVA mode based on probing the PCI bus and IOMMU configuration
> may not report the required addressing mode. Having the configuration
> option permits the mode to be explicitly configured in this scenario.
>
> Signed-off-by: eric zhang <eric.zhang@windriver.com>
> ---
>   lib/librte_eal/linuxapp/eal/eal.c | 7 +++++++
you will also need the same in bsdapp/eal

>   1 file changed, 7 insertions(+)
>
> diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
> index e0b5ae1..bee4aed 100644
> --- a/lib/librte_eal/linuxapp/eal/eal.c
> +++ b/lib/librte_eal/linuxapp/eal/eal.c
> @@ -805,6 +805,7 @@ static void rte_eal_init_alert(const char *msg)
>   		return -1;
>   	}
>   
> +#ifndef RTE_EAL_IOVA_MODE_PA
Add the default option as 'n'  in config/common_base
>   	/* autodetect the iova mapping mode (default is iova_pa) */
>   	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
>   
> @@ -816,6 +817,12 @@ static void rte_eal_init_alert(const char *msg)
>   			"Some devices want IOVA as VA but PA will be used because.. "
>   			"KNI module inserted\n");
>   	}
> +#else
> +	/* Force iova mapping mode to be physical address */
> +	rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
> +	RTE_LOG(WARNING, EAL,
> +		"Force the iova mapping mode to be physical address\n");
> +#endif
>   
>   	if (internal_config.no_hugetlbfs == 0 &&
>   			internal_config.process_type != RTE_PROC_SECONDARY &&

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-29 15:58 [dpdk-dev] [PATCH] eal: force IOVA mode to physical eric zhang
  2018-08-30  6:13 ` Hemant
@ 2018-08-30  9:09 ` Burakov, Anatoly
  2018-08-30  9:43   ` Gaëtan Rivet
  1 sibling, 1 reply; 15+ messages in thread
From: Burakov, Anatoly @ 2018-08-30  9:09 UTC (permalink / raw)
  To: eric zhang, bruce.richardson; +Cc: dev, Allain.Legacy, Matt.Peters

On 29-Aug-18 4:58 PM, eric zhang wrote:
> This patch adds a configuration option to force the IOVA mode to
> physical address (PA). There exists virtual devices that are not
> directly attached to the PCI bus, and therefore the auto detection
> of the IOVA mode based on probing the PCI bus and IOMMU configuration
> may not report the required addressing mode. Having the configuration
> option permits the mode to be explicitly configured in this scenario.
> 
> Signed-off-by: eric zhang <eric.zhang@windriver.com>
> ---

Defining this at compile-time seems like an overkill. Wouldn't it be 
better to just add an EAL command-line option to force IOVA mode to a 
particular value?

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-30  9:09 ` Burakov, Anatoly
@ 2018-08-30  9:43   ` Gaëtan Rivet
  2018-08-30 12:13     ` Hemant
  0 siblings, 1 reply; 15+ messages in thread
From: Gaëtan Rivet @ 2018-08-30  9:43 UTC (permalink / raw)
  To: Burakov, Anatoly
  Cc: eric zhang, bruce.richardson, dev, Allain.Legacy, Matt.Peters

Hi,

On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
> On 29-Aug-18 4:58 PM, eric zhang wrote:
> > This patch adds a configuration option to force the IOVA mode to
> > physical address (PA). There exists virtual devices that are not
> > directly attached to the PCI bus, and therefore the auto detection
> > of the IOVA mode based on probing the PCI bus and IOMMU configuration
> > may not report the required addressing mode. Having the configuration
> > option permits the mode to be explicitly configured in this scenario.
> > 
> > Signed-off-by: eric zhang <eric.zhang@windriver.com>
> > ---
> 
> Defining this at compile-time seems like an overkill. Wouldn't it be better
> to just add an EAL command-line option to force IOVA mode to a particular
> value?
> 
> -- 
> Thanks,
> Anatoly

What is the bus of these devices and why not implement get_iommu_class
in it?

-- 
Gaëtan Rivet
6WIND

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-30  9:43   ` Gaëtan Rivet
@ 2018-08-30 12:13     ` Hemant
  2018-08-30 12:59       ` santosh
  0 siblings, 1 reply; 15+ messages in thread
From: Hemant @ 2018-08-30 12:13 UTC (permalink / raw)
  To: Gaëtan Rivet, Burakov, Anatoly
  Cc: eric zhang, bruce.richardson, dev, Allain.Legacy, Matt.Peters

Hi,

On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
> Hi,
>
> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>> This patch adds a configuration option to force the IOVA mode to
>>> physical address (PA). There exists virtual devices that are not
>>> directly attached to the PCI bus, and therefore the auto detection
>>> of the IOVA mode based on probing the PCI bus and IOMMU configuration
>>> may not report the required addressing mode. Having the configuration
>>> option permits the mode to be explicitly configured in this scenario.
>>>
>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>> ---
>> Defining this at compile-time seems like an overkill. Wouldn't it be better
>> to just add an EAL command-line option to force IOVA mode to a particular
>> value?
That is a good suggestion.
>> -- 
>> Thanks,
>> Anatoly
> What is the bus of these devices and why not implement get_iommu_class
> in it?
There are cases, where you are using dpdk libraries with external 
libraries and you need to change the default behavior DPDK lib to use 
physical address instead of virtual address.
Providing an option to user will help.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-30 12:13     ` Hemant
@ 2018-08-30 12:59       ` santosh
  2018-08-30 13:56         ` Legacy, Allain
  2018-09-05  3:40         ` Eric Zhang
  0 siblings, 2 replies; 15+ messages in thread
From: santosh @ 2018-08-30 12:59 UTC (permalink / raw)
  To: hemant.agrawal, Gaëtan Rivet, Burakov, Anatoly
  Cc: eric zhang, bruce.richardson, dev, Allain.Legacy, Matt.Peters


On Thursday 30 August 2018 05:43 PM, Hemant wrote:
> External Email
>
> Hi,
>
> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>> Hi,
>>
>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>> This patch adds a configuration option to force the IOVA mode to
>>>> physical address (PA). There exists virtual devices that are not
>>>> directly attached to the PCI bus, and therefore the auto detection
>>>> of the IOVA mode based on probing the PCI bus and IOMMU configuration
>>>> may not report the required addressing mode. Having the configuration
>>>> option permits the mode to be explicitly configured in this scenario.
>>>>
>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>>> ---
>>> Defining this at compile-time seems like an overkill. Wouldn't it be better
>>> to just add an EAL command-line option to force IOVA mode to a particular
>>> value?
> That is a good suggestion.
>>> -- 
>>> Thanks,
>>> Anatoly
>> What is the bus of these devices and why not implement get_iommu_class
>> in it?
> There are cases, where you are using dpdk libraries with external
> libraries and you need to change the default behavior DPDK lib to use
> physical address instead of virtual address.
> Providing an option to user will help.
>
>
More appropriate solution could be:
* Either fix it at bus layer .. i.e.. get_iommu_class()..
* Or introduce something like [1] --iova-mode=<pa/va> param.

Former is better solution than latter if autodetection is a key criteria.
Thanks.

[1] http://patchwork.dpdk.org/patch/25192/

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-30 12:59       ` santosh
@ 2018-08-30 13:56         ` Legacy, Allain
  2018-08-30 13:58           ` santosh
  2018-09-05  3:40         ` Eric Zhang
  1 sibling, 1 reply; 15+ messages in thread
From: Legacy, Allain @ 2018-08-30 13:56 UTC (permalink / raw)
  To: santosh, hemant.agrawal, Gaëtan Rivet, Burakov, Anatoly
  Cc: Zhang, Qing Long (Eric), bruce.richardson, dev, Peters, Matt

> -----Original Message-----
> From: santosh [mailto:santosh.shukla@caviumnetworks.com]
> Sent: Thursday, August 30, 2018 8:59 AM
> To: hemant.agrawal@nxp.com; Gaëtan Rivet; Burakov, Anatoly
> Cc: Zhang, Qing Long (Eric); bruce.richardson@intel.com; dev@dpdk.org;
> Legacy, Allain; Peters, Matt
> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
> 
> 
> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
> > External Email
<...>
> 
> [1] http://patchwork.dpdk.org/patch/25192/

Is there a reason this was abandoned? 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-30 13:56         ` Legacy, Allain
@ 2018-08-30 13:58           ` santosh
  0 siblings, 0 replies; 15+ messages in thread
From: santosh @ 2018-08-30 13:58 UTC (permalink / raw)
  To: Legacy, Allain, hemant.agrawal, Gaëtan Rivet, Burakov, Anatoly
  Cc: Zhang, Qing Long (Eric), bruce.richardson, dev, Peters, Matt


On Thursday 30 August 2018 07:26 PM, Legacy, Allain wrote:
> External Email
>
>> -----Original Message-----
>> From: santosh [mailto:santosh.shukla@caviumnetworks.com]
>> Sent: Thursday, August 30, 2018 8:59 AM
>> To: hemant.agrawal@nxp.com; Gaëtan Rivet; Burakov, Anatoly
>> Cc: Zhang, Qing Long (Eric); bruce.richardson@intel.com; dev@dpdk.org;
>> Legacy, Allain; Peters, Matt
>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>
>>
>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>> External Email
> <...>
>> [1] http://patchwork.dpdk.org/patch/25192/
> Is there a reason this was abandoned?

Wasn't abandoned exactly, in-fact we had better solution which is to introduce autodetection notion at bus layer.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-08-30 12:59       ` santosh
  2018-08-30 13:56         ` Legacy, Allain
@ 2018-09-05  3:40         ` Eric Zhang
  2018-09-06  7:34           ` Jerin Jacob
  1 sibling, 1 reply; 15+ messages in thread
From: Eric Zhang @ 2018-09-05  3:40 UTC (permalink / raw)
  To: santosh, hemant.agrawal, Gaëtan Rivet, Burakov, Anatoly
  Cc: bruce.richardson, dev, Allain.Legacy, Matt.Peters



On 08/30/2018 08:59 AM, santosh wrote:
> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>> External Email
>>
>> Hi,
>>
>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>> Hi,
>>>
>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>> This patch adds a configuration option to force the IOVA mode to
>>>>> physical address (PA). There exists virtual devices that are not
>>>>> directly attached to the PCI bus, and therefore the auto detection
>>>>> of the IOVA mode based on probing the PCI bus and IOMMU configuration
>>>>> may not report the required addressing mode. Having the configuration
>>>>> option permits the mode to be explicitly configured in this scenario.
>>>>>
>>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>>>> ---
>>>> Defining this at compile-time seems like an overkill. Wouldn't it be better
>>>> to just add an EAL command-line option to force IOVA mode to a particular
>>>> value?
>> That is a good suggestion.
>>>> -- 
>>>> Thanks,
>>>> Anatoly
>>> What is the bus of these devices and why not implement get_iommu_class
>>> in it?
>> There are cases, where you are using dpdk libraries with external
>> libraries and you need to change the default behavior DPDK lib to use
>> physical address instead of virtual address.
>> Providing an option to user will help.
>>
>>
> More appropriate solution could be:
> * Either fix it at bus layer .. i.e.. get_iommu_class()..
> * Or introduce something like [1] --iova-mode=<pa/va> param.
>
> Former is better solution than latter if autodetection is a key criteria.
> Thanks.
>
> [1] http://patchwork.dpdk.org/patch/25192/
>
It's not generic which couldn't be fixed at bus layer.
So what's the preference of EAL option or compile time solution?
Adding --iova-mode as patch [1] will overrivde auto-detection 
rte_bus_get_iommu_class()
make it no use; compile time solution will align with upstream and keep 
new atuodetection
solution in #ifndef.

Thanks
Eric

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-09-05  3:40         ` Eric Zhang
@ 2018-09-06  7:34           ` Jerin Jacob
  2018-09-07  9:26             ` Burakov, Anatoly
  0 siblings, 1 reply; 15+ messages in thread
From: Jerin Jacob @ 2018-09-06  7:34 UTC (permalink / raw)
  To: Eric Zhang
  Cc: santosh, hemant.agrawal, Gaëtan Rivet, Burakov, Anatoly,
	bruce.richardson, dev, Allain.Legacy, Matt.Peters

-----Original Message-----
> Date: Tue, 4 Sep 2018 23:40:36 -0400
> From: Eric Zhang <eric.zhang@windriver.com>
> To: santosh <santosh.shukla@caviumnetworks.com>, hemant.agrawal@nxp.com,
>  Gaëtan Rivet <gaetan.rivet@6wind.com>, "Burakov, Anatoly"
>  <anatoly.burakov@intel.com>
> CC: bruce.richardson@intel.com, dev@dpdk.org, Allain.Legacy@windriver.com,
>  Matt.Peters@windriver.com
> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>  Thunderbird/52.9.1
> 
> On 08/30/2018 08:59 AM, santosh wrote:
> > On Thursday 30 August 2018 05:43 PM, Hemant wrote:
> > > External Email
> > > 
> > > Hi,
> > > 
> > > On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
> > > > Hi,
> > > > 
> > > > On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
> > > > > On 29-Aug-18 4:58 PM, eric zhang wrote:
> > > > > > This patch adds a configuration option to force the IOVA mode to
> > > > > > physical address (PA). There exists virtual devices that are not
> > > > > > directly attached to the PCI bus, and therefore the auto detection
> > > > > > of the IOVA mode based on probing the PCI bus and IOMMU configuration
> > > > > > may not report the required addressing mode. Having the configuration
> > > > > > option permits the mode to be explicitly configured in this scenario.
> > > > > > 
> > > > > > Signed-off-by: eric zhang <eric.zhang@windriver.com>
> > > > > > ---
> > > > > Defining this at compile-time seems like an overkill. Wouldn't it be better
> > > > > to just add an EAL command-line option to force IOVA mode to a particular
> > > > > value?
> > > That is a good suggestion.
> > > > > --
> > > > > Thanks,
> > > > > Anatoly
> > > > What is the bus of these devices and why not implement get_iommu_class
> > > > in it?
> > > There are cases, where you are using dpdk libraries with external
> > > libraries and you need to change the default behavior DPDK lib to use
> > > physical address instead of virtual address.
> > > Providing an option to user will help.
> > > 
> > > 
> > More appropriate solution could be:
> > * Either fix it at bus layer .. i.e.. get_iommu_class()..
> > * Or introduce something like [1] --iova-mode=<pa/va> param.
> > 
> > Former is better solution than latter if autodetection is a key criteria.
> > Thanks.
> > 
> > [1] http://patchwork.dpdk.org/patch/25192/
> > 
> It's not generic which couldn't be fixed at bus layer.
> So what's the preference of EAL option or compile time solution?
> Adding --iova-mode as patch [1] will overrivde auto-detection
> rte_bus_get_iommu_class()
> make it no use; compile time solution will align with upstream and keep
> new atuodetection
> solution in #ifndef.

If it is for vdev devices, why not introduce something like
RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class()
of vdev can decide the mode just like PCI bus.


> 
> Thanks
> Eric
> 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-09-06  7:34           ` Jerin Jacob
@ 2018-09-07  9:26             ` Burakov, Anatoly
  2018-09-07 20:13               ` Eric Zhang
  0 siblings, 1 reply; 15+ messages in thread
From: Burakov, Anatoly @ 2018-09-07  9:26 UTC (permalink / raw)
  To: Jerin Jacob, Eric Zhang
  Cc: santosh, hemant.agrawal, Gaëtan Rivet, bruce.richardson,
	dev, Allain.Legacy, Matt.Peters

On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
> -----Original Message-----
>> Date: Tue, 4 Sep 2018 23:40:36 -0400
>> From: Eric Zhang <eric.zhang@windriver.com>
>> To: santosh <santosh.shukla@caviumnetworks.com>, hemant.agrawal@nxp.com,
>>   Gaëtan Rivet <gaetan.rivet@6wind.com>, "Burakov, Anatoly"
>>   <anatoly.burakov@intel.com>
>> CC: bruce.richardson@intel.com, dev@dpdk.org, Allain.Legacy@windriver.com,
>>   Matt.Peters@windriver.com
>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>>   Thunderbird/52.9.1
>>
>> On 08/30/2018 08:59 AM, santosh wrote:
>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>>> External Email
>>>>
>>>> Hi,
>>>>
>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>>>> Hi,
>>>>>
>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>>>> This patch adds a configuration option to force the IOVA mode to
>>>>>>> physical address (PA). There exists virtual devices that are not
>>>>>>> directly attached to the PCI bus, and therefore the auto detection
>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU configuration
>>>>>>> may not report the required addressing mode. Having the configuration
>>>>>>> option permits the mode to be explicitly configured in this scenario.
>>>>>>>
>>>>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>>>>>> ---
>>>>>> Defining this at compile-time seems like an overkill. Wouldn't it be better
>>>>>> to just add an EAL command-line option to force IOVA mode to a particular
>>>>>> value?
>>>> That is a good suggestion.
>>>>>> --
>>>>>> Thanks,
>>>>>> Anatoly
>>>>> What is the bus of these devices and why not implement get_iommu_class
>>>>> in it?
>>>> There are cases, where you are using dpdk libraries with external
>>>> libraries and you need to change the default behavior DPDK lib to use
>>>> physical address instead of virtual address.
>>>> Providing an option to user will help.
>>>>
>>>>
>>> More appropriate solution could be:
>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
>>>
>>> Former is better solution than latter if autodetection is a key criteria.
>>> Thanks.
>>>
>>> [1] http://patchwork.dpdk.org/patch/25192/
>>>
>> It's not generic which couldn't be fixed at bus layer.
>> So what's the preference of EAL option or compile time solution?
>> Adding --iova-mode as patch [1] will overrivde auto-detection
>> rte_bus_get_iommu_class()
>> make it no use; compile time solution will align with upstream and keep
>> new atuodetection
>> solution in #ifndef.
> 
> If it is for vdev devices, why not introduce something like
> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class()
> of vdev can decide the mode just like PCI bus.
> 

That seems like a better option to me, +1. As far as i know, at the 
moment if there are no devices attached at all, or if there are only 
vdev devices attached, DPDK will default to IOVA as PA mode for no good 
reason; such a change would certainly fix this.

> 
>>
>> Thanks
>> Eric
>>
> 


-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-09-07  9:26             ` Burakov, Anatoly
@ 2018-09-07 20:13               ` Eric Zhang
  2018-09-11 17:21                 ` Eric Zhang
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Zhang @ 2018-09-07 20:13 UTC (permalink / raw)
  To: Burakov, Anatoly, Jerin Jacob
  Cc: santosh, hemant.agrawal, Gaëtan Rivet, bruce.richardson,
	dev, Allain.Legacy, Matt.Peters



On 09/07/2018 05:26 AM, Burakov, Anatoly wrote:
> On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
>> -----Original Message-----
>>> Date: Tue, 4 Sep 2018 23:40:36 -0400
>>> From: Eric Zhang <eric.zhang@windriver.com>
>>> To: santosh <santosh.shukla@caviumnetworks.com>, 
>>> hemant.agrawal@nxp.com,
>>>   Gaëtan Rivet <gaetan.rivet@6wind.com>, "Burakov, Anatoly"
>>>   <anatoly.burakov@intel.com>
>>> CC: bruce.richardson@intel.com, dev@dpdk.org, 
>>> Allain.Legacy@windriver.com,
>>>   Matt.Peters@windriver.com
>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>>>   Thunderbird/52.9.1
>>>
>>> On 08/30/2018 08:59 AM, santosh wrote:
>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>>>> External Email
>>>>>
>>>>> Hi,
>>>>>
>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>>>>> This patch adds a configuration option to force the IOVA mode to
>>>>>>>> physical address (PA). There exists virtual devices that are not
>>>>>>>> directly attached to the PCI bus, and therefore the auto detection
>>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU 
>>>>>>>> configuration
>>>>>>>> may not report the required addressing mode. Having the 
>>>>>>>> configuration
>>>>>>>> option permits the mode to be explicitly configured in this 
>>>>>>>> scenario.
>>>>>>>>
>>>>>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>>>>>>> ---
>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't 
>>>>>>> it be better
>>>>>>> to just add an EAL command-line option to force IOVA mode to a 
>>>>>>> particular
>>>>>>> value?
>>>>> That is a good suggestion.
>>>>>>> -- 
>>>>>>> Thanks,
>>>>>>> Anatoly
>>>>>> What is the bus of these devices and why not implement 
>>>>>> get_iommu_class
>>>>>> in it?
>>>>> There are cases, where you are using dpdk libraries with external
>>>>> libraries and you need to change the default behavior DPDK lib to use
>>>>> physical address instead of virtual address.
>>>>> Providing an option to user will help.
>>>>>
>>>>>
>>>> More appropriate solution could be:
>>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
>>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
>>>>
>>>> Former is better solution than latter if autodetection is a key 
>>>> criteria.
>>>> Thanks.
>>>>
>>>> [1] http://patchwork.dpdk.org/patch/25192/
>>>>
>>> It's not generic which couldn't be fixed at bus layer.
>>> So what's the preference of EAL option or compile time solution?
>>> Adding --iova-mode as patch [1] will overrivde auto-detection
>>> rte_bus_get_iommu_class()
>>> make it no use; compile time solution will align with upstream and keep
>>> new atuodetection
>>> solution in #ifndef.
>>
>> If it is for vdev devices, why not introduce something like
>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
>> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class()
>> of vdev can decide the mode just like PCI bus.
>>
>
> That seems like a better option to me, +1. As far as i know, at the 
> moment if there are no devices attached at all, or if there are only 
> vdev devices attached, DPDK will default to IOVA as PA mode for no 
> good reason; such a change would certainly fix this.
Thanks for the suggestions however our virtual device doesn't run dpdk 
vdev code so we can't use the flag.
Notice that in eal.c there is one workaround that force iova to be PA 
per virtual device is not directly attached to pci. That case is 
checking kni module. Ours is a similar case that virtual device not 
attach pci directly.
So we have to turn to force iova to PA either 1. compilation option 2. 
eal option.  Which one should be the preference by taking into 
consideration that align with upstream?

Thanks

>
>>
>>>
>>> Thanks
>>> Eric
>>>
>>
>
>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-09-07 20:13               ` Eric Zhang
@ 2018-09-11 17:21                 ` Eric Zhang
  2018-09-17  8:32                   ` Stojaczyk, Dariusz
  0 siblings, 1 reply; 15+ messages in thread
From: Eric Zhang @ 2018-09-11 17:21 UTC (permalink / raw)
  To: Burakov, Anatoly, Jerin Jacob
  Cc: santosh, hemant.agrawal, Gaëtan Rivet, bruce.richardson,
	dev, Allain.Legacy, Matt.Peters



On 09/07/2018 04:13 PM, Eric Zhang wrote:
>
>
> On 09/07/2018 05:26 AM, Burakov, Anatoly wrote:
>> On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
>>> -----Original Message-----
>>>> Date: Tue, 4 Sep 2018 23:40:36 -0400
>>>> From: Eric Zhang <eric.zhang@windriver.com>
>>>> To: santosh <santosh.shukla@caviumnetworks.com>, 
>>>> hemant.agrawal@nxp.com,
>>>>   Gaëtan Rivet <gaetan.rivet@6wind.com>, "Burakov, Anatoly"
>>>>   <anatoly.burakov@intel.com>
>>>> CC: bruce.richardson@intel.com, dev@dpdk.org, 
>>>> Allain.Legacy@windriver.com,
>>>>   Matt.Peters@windriver.com
>>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>>>>   Thunderbird/52.9.1
>>>>
>>>> On 08/30/2018 08:59 AM, santosh wrote:
>>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>>>>> External Email
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>>>>>> This patch adds a configuration option to force the IOVA mode to
>>>>>>>>> physical address (PA). There exists virtual devices that are not
>>>>>>>>> directly attached to the PCI bus, and therefore the auto 
>>>>>>>>> detection
>>>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU 
>>>>>>>>> configuration
>>>>>>>>> may not report the required addressing mode. Having the 
>>>>>>>>> configuration
>>>>>>>>> option permits the mode to be explicitly configured in this 
>>>>>>>>> scenario.
>>>>>>>>>
>>>>>>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>>>>>>>> ---
>>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't 
>>>>>>>> it be better
>>>>>>>> to just add an EAL command-line option to force IOVA mode to a 
>>>>>>>> particular
>>>>>>>> value?
>>>>>> That is a good suggestion.
>>>>>>>> -- 
>>>>>>>> Thanks,
>>>>>>>> Anatoly
>>>>>>> What is the bus of these devices and why not implement 
>>>>>>> get_iommu_class
>>>>>>> in it?
>>>>>> There are cases, where you are using dpdk libraries with external
>>>>>> libraries and you need to change the default behavior DPDK lib to 
>>>>>> use
>>>>>> physical address instead of virtual address.
>>>>>> Providing an option to user will help.
>>>>>>
>>>>>>
>>>>> More appropriate solution could be:
>>>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
>>>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
>>>>>
>>>>> Former is better solution than latter if autodetection is a key 
>>>>> criteria.
>>>>> Thanks.
>>>>>
>>>>> [1] http://patchwork.dpdk.org/patch/25192/
>>>>>
>>>> It's not generic which couldn't be fixed at bus layer.
>>>> So what's the preference of EAL option or compile time solution?
>>>> Adding --iova-mode as patch [1] will overrivde auto-detection
>>>> rte_bus_get_iommu_class()
>>>> make it no use; compile time solution will align with upstream and 
>>>> keep
>>>> new atuodetection
>>>> solution in #ifndef.
>>>
>>> If it is for vdev devices, why not introduce something like
>>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
>>> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class()
>>> of vdev can decide the mode just like PCI bus.
>>>
>>
>> That seems like a better option to me, +1. As far as i know, at the 
>> moment if there are no devices attached at all, or if there are only 
>> vdev devices attached, DPDK will default to IOVA as PA mode for no 
>> good reason; such a change would certainly fix this.
> Thanks for the suggestions however our virtual device doesn't run dpdk 
> vdev code so we can't use the flag.
> Notice that in eal.c there is one workaround that force iova to be PA 
> per virtual device is not directly attached to pci. That case is 
> checking kni module. Ours is a similar case that virtual device not 
> attach pci directly.
> So we have to turn to force iova to PA either 1. compilation option 2. 
> eal option.  Which one should be the preference by taking into 
> consideration that align with upstream?
>
> Thanks
>
Any comments?
>>
>>>
>>>>
>>>> Thanks
>>>> Eric
>>>>
>>>
>>
>>
>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-09-11 17:21                 ` Eric Zhang
@ 2018-09-17  8:32                   ` Stojaczyk, Dariusz
  2018-09-17 19:04                     ` Eric Zhang
  0 siblings, 1 reply; 15+ messages in thread
From: Stojaczyk, Dariusz @ 2018-09-17  8:32 UTC (permalink / raw)
  To: Eric Zhang, Burakov, Anatoly, Jerin Jacob
  Cc: santosh, hemant.agrawal, Gaëtan Rivet, Richardson, Bruce,
	dev, Allain.Legacy, Matt.Peters

Hi,

A little bit of self-advertising:
I recently pushed patches that will make DPDK default to RTE_IOVA_VA when physical addresses were not explicitly requested and are not available, e.g. when running as a non-privileged user. It shouldn't cause any conflicts with the changes you're proposing here, but any review is welcome.

pci/linux: use RTE_IOVA_VA whenever possible http://patches.dpdk.org/patch/44392/ 
[v2] eal/bus: use RTE_IOVA_PA only if phys addresses are available http://patches.dpdk.org/patch/44420/

As for the --iova-mode=<pa/va>, I agree it could help DPDK use cases where most drivers or devices are hotplugged/hotattached at runtime - e.g. SPDK, it could certainly make use of such param.

D.

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Eric Zhang
> Sent: Tuesday, September 11, 2018 7:22 PM
> To: Burakov, Anatoly <anatoly.burakov@intel.com>; Jerin Jacob
> <jerin.jacob@caviumnetworks.com>
> Cc: santosh <santosh.shukla@caviumnetworks.com>;
> hemant.agrawal@nxp.com; Gaëtan Rivet <gaetan.rivet@6wind.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; dev@dpdk.org;
> Allain.Legacy@windriver.com; Matt.Peters@windriver.com
> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
> 
> 
> 
> On 09/07/2018 04:13 PM, Eric Zhang wrote:
> >
> >
> > On 09/07/2018 05:26 AM, Burakov, Anatoly wrote:
> >> On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
> >>> -----Original Message-----
> >>>> Date: Tue, 4 Sep 2018 23:40:36 -0400
> >>>> From: Eric Zhang <eric.zhang@windriver.com>
> >>>> To: santosh <santosh.shukla@caviumnetworks.com>,
> >>>> hemant.agrawal@nxp.com,
> >>>>   Gaëtan Rivet <gaetan.rivet@6wind.com>, "Burakov, Anatoly"
> >>>>   <anatoly.burakov@intel.com>
> >>>> CC: bruce.richardson@intel.com, dev@dpdk.org,
> >>>> Allain.Legacy@windriver.com,
> >>>>   Matt.Peters@windriver.com
> >>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
> >>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
> >>>>   Thunderbird/52.9.1
> >>>>
> >>>> On 08/30/2018 08:59 AM, santosh wrote:
> >>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
> >>>>>> External Email
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
> >>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
> >>>>>>>>> This patch adds a configuration option to force the IOVA mode
> >>>>>>>>> to physical address (PA). There exists virtual devices that
> >>>>>>>>> are not directly attached to the PCI bus, and therefore the
> >>>>>>>>> auto detection of the IOVA mode based on probing the PCI bus
> >>>>>>>>> and IOMMU configuration may not report the required
> addressing
> >>>>>>>>> mode. Having the configuration option permits the mode to be
> >>>>>>>>> explicitly configured in this scenario.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
> >>>>>>>>> ---
> >>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't
> >>>>>>>> it be better to just add an EAL command-line option to force
> >>>>>>>> IOVA mode to a particular value?
> >>>>>> That is a good suggestion.
> >>>>>>>> --
> >>>>>>>> Thanks,
> >>>>>>>> Anatoly
> >>>>>>> What is the bus of these devices and why not implement
> >>>>>>> get_iommu_class in it?
> >>>>>> There are cases, where you are using dpdk libraries with external
> >>>>>> libraries and you need to change the default behavior DPDK lib to
> >>>>>> use physical address instead of virtual address.
> >>>>>> Providing an option to user will help.
> >>>>>>
> >>>>>>
> >>>>> More appropriate solution could be:
> >>>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
> >>>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
> >>>>>
> >>>>> Former is better solution than latter if autodetection is a key
> >>>>> criteria.
> >>>>> Thanks.
> >>>>>
> >>>>> [1] http://patchwork.dpdk.org/patch/25192/
> >>>>>
> >>>> It's not generic which couldn't be fixed at bus layer.
> >>>> So what's the preference of EAL option or compile time solution?
> >>>> Adding --iova-mode as patch [1] will overrivde auto-detection
> >>>> rte_bus_get_iommu_class()
> >>>> make it no use; compile time solution will align with upstream and
> >>>> keep new atuodetection solution in #ifndef.
> >>>
> >>> If it is for vdev devices, why not introduce something like
> >>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
> >>> And based on the devices(flags) on vdev bus,
> >>> rte_bus_get_iommu_class() of vdev can decide the mode just like PCI
> bus.
> >>>
> >>
> >> That seems like a better option to me, +1. As far as i know, at the
> >> moment if there are no devices attached at all, or if there are only
> >> vdev devices attached, DPDK will default to IOVA as PA mode for no
> >> good reason; such a change would certainly fix this.
> > Thanks for the suggestions however our virtual device doesn't run dpdk
> > vdev code so we can't use the flag.
> > Notice that in eal.c there is one workaround that force iova to be PA
> > per virtual device is not directly attached to pci. That case is
> > checking kni module. Ours is a similar case that virtual device not
> > attach pci directly.
> > So we have to turn to force iova to PA either 1. compilation option 2.
> > eal option.  Which one should be the preference by taking into
> > consideration that align with upstream?
> >
> > Thanks
> >
> Any comments?
> >>
> >>>
> >>>>
> >>>> Thanks
> >>>> Eric
> >>>>
> >>>
> >>
> >>
> >


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
  2018-09-17  8:32                   ` Stojaczyk, Dariusz
@ 2018-09-17 19:04                     ` Eric Zhang
  0 siblings, 0 replies; 15+ messages in thread
From: Eric Zhang @ 2018-09-17 19:04 UTC (permalink / raw)
  To: Stojaczyk, Dariusz, Burakov, Anatoly, Jerin Jacob
  Cc: santosh, hemant.agrawal, Gaëtan Rivet, Richardson, Bruce,
	dev, Allain.Legacy, Matt.Peters

Hi All,
Considered that most people commented that eal option "--iova-mode" is a 
better solution that gives user a chance not to use bus iova auto 
detection scheme and want to override for some reason, I will generate 
2nd version patch which will use eal option instead of compilation 
configuration.

Eric

On 09/17/2018 04:32 AM, Stojaczyk, Dariusz wrote:
> Hi,
>
> A little bit of self-advertising:
> I recently pushed patches that will make DPDK default to RTE_IOVA_VA when physical addresses were not explicitly requested and are not available, e.g. when running as a non-privileged user. It shouldn't cause any conflicts with the changes you're proposing here, but any review is welcome.
>
> pci/linux: use RTE_IOVA_VA whenever possible http://patches.dpdk.org/patch/44392/
> [v2] eal/bus: use RTE_IOVA_PA only if phys addresses are available http://patches.dpdk.org/patch/44420/
>
> As for the --iova-mode=<pa/va>, I agree it could help DPDK use cases where most drivers or devices are hotplugged/hotattached at runtime - e.g. SPDK, it could certainly make use of such param.
>
> D.
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Eric Zhang
>> Sent: Tuesday, September 11, 2018 7:22 PM
>> To: Burakov, Anatoly <anatoly.burakov@intel.com>; Jerin Jacob
>> <jerin.jacob@caviumnetworks.com>
>> Cc: santosh <santosh.shukla@caviumnetworks.com>;
>> hemant.agrawal@nxp.com; Gaëtan Rivet <gaetan.rivet@6wind.com>;
>> Richardson, Bruce <bruce.richardson@intel.com>; dev@dpdk.org;
>> Allain.Legacy@windriver.com; Matt.Peters@windriver.com
>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>
>>
>>
>> On 09/07/2018 04:13 PM, Eric Zhang wrote:
>>>
>>> On 09/07/2018 05:26 AM, Burakov, Anatoly wrote:
>>>> On 06-Sep-18 8:34 AM, Jerin Jacob wrote:
>>>>> -----Original Message-----
>>>>>> Date: Tue, 4 Sep 2018 23:40:36 -0400
>>>>>> From: Eric Zhang <eric.zhang@windriver.com>
>>>>>> To: santosh <santosh.shukla@caviumnetworks.com>,
>>>>>> hemant.agrawal@nxp.com,
>>>>>>    Gaëtan Rivet <gaetan.rivet@6wind.com>, "Burakov, Anatoly"
>>>>>>    <anatoly.burakov@intel.com>
>>>>>> CC: bruce.richardson@intel.com, dev@dpdk.org,
>>>>>> Allain.Legacy@windriver.com,
>>>>>>    Matt.Peters@windriver.com
>>>>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical
>>>>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
>>>>>>    Thunderbird/52.9.1
>>>>>>
>>>>>> On 08/30/2018 08:59 AM, santosh wrote:
>>>>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote:
>>>>>>>> External Email
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote:
>>>>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote:
>>>>>>>>>>> This patch adds a configuration option to force the IOVA mode
>>>>>>>>>>> to physical address (PA). There exists virtual devices that
>>>>>>>>>>> are not directly attached to the PCI bus, and therefore the
>>>>>>>>>>> auto detection of the IOVA mode based on probing the PCI bus
>>>>>>>>>>> and IOMMU configuration may not report the required
>> addressing
>>>>>>>>>>> mode. Having the configuration option permits the mode to be
>>>>>>>>>>> explicitly configured in this scenario.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: eric zhang <eric.zhang@windriver.com>
>>>>>>>>>>> ---
>>>>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't
>>>>>>>>>> it be better to just add an EAL command-line option to force
>>>>>>>>>> IOVA mode to a particular value?
>>>>>>>> That is a good suggestion.
>>>>>>>>>> --
>>>>>>>>>> Thanks,
>>>>>>>>>> Anatoly
>>>>>>>>> What is the bus of these devices and why not implement
>>>>>>>>> get_iommu_class in it?
>>>>>>>> There are cases, where you are using dpdk libraries with external
>>>>>>>> libraries and you need to change the default behavior DPDK lib to
>>>>>>>> use physical address instead of virtual address.
>>>>>>>> Providing an option to user will help.
>>>>>>>>
>>>>>>>>
>>>>>>> More appropriate solution could be:
>>>>>>> * Either fix it at bus layer .. i.e.. get_iommu_class()..
>>>>>>> * Or introduce something like [1] --iova-mode=<pa/va> param.
>>>>>>>
>>>>>>> Former is better solution than latter if autodetection is a key
>>>>>>> criteria.
>>>>>>> Thanks.
>>>>>>>
>>>>>>> [1] http://patchwork.dpdk.org/patch/25192/
>>>>>>>
>>>>>> It's not generic which couldn't be fixed at bus layer.
>>>>>> So what's the preference of EAL option or compile time solution?
>>>>>> Adding --iova-mode as patch [1] will overrivde auto-detection
>>>>>> rte_bus_get_iommu_class()
>>>>>> make it no use; compile time solution will align with upstream and
>>>>>> keep new atuodetection solution in #ifndef.
>>>>> If it is for vdev devices, why not introduce something like
>>>>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality.
>>>>> And based on the devices(flags) on vdev bus,
>>>>> rte_bus_get_iommu_class() of vdev can decide the mode just like PCI
>> bus.
>>>> That seems like a better option to me, +1. As far as i know, at the
>>>> moment if there are no devices attached at all, or if there are only
>>>> vdev devices attached, DPDK will default to IOVA as PA mode for no
>>>> good reason; such a change would certainly fix this.
>>> Thanks for the suggestions however our virtual device doesn't run dpdk
>>> vdev code so we can't use the flag.
>>> Notice that in eal.c there is one workaround that force iova to be PA
>>> per virtual device is not directly attached to pci. That case is
>>> checking kni module. Ours is a similar case that virtual device not
>>> attach pci directly.
>>> So we have to turn to force iova to PA either 1. compilation option 2.
>>> eal option.  Which one should be the preference by taking into
>>> consideration that align with upstream?
>>>
>>> Thanks
>>>
>> Any comments?
>>>>>> Thanks
>>>>>> Eric
>>>>>>
>>>>

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2018-09-17 19:05 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-29 15:58 [dpdk-dev] [PATCH] eal: force IOVA mode to physical eric zhang
2018-08-30  6:13 ` Hemant
2018-08-30  9:09 ` Burakov, Anatoly
2018-08-30  9:43   ` Gaëtan Rivet
2018-08-30 12:13     ` Hemant
2018-08-30 12:59       ` santosh
2018-08-30 13:56         ` Legacy, Allain
2018-08-30 13:58           ` santosh
2018-09-05  3:40         ` Eric Zhang
2018-09-06  7:34           ` Jerin Jacob
2018-09-07  9:26             ` Burakov, Anatoly
2018-09-07 20:13               ` Eric Zhang
2018-09-11 17:21                 ` Eric Zhang
2018-09-17  8:32                   ` Stojaczyk, Dariusz
2018-09-17 19:04                     ` Eric Zhang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).