[dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[tone.zhang]

When binding the devices used by DPDK to the "uio-pci" kernel module,
the IOMMU should be disabled in order not to break the IO transmission
because of the virtual / physical address mapping.

The patch clarifies the IOMMU configuration on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..8f9ec8f 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices for used DPDK bound to the ``uio-pci`` kernel module, please make
+   sure that the IOMMU is disabled. We can add ``intel_iommu=off`` or ``amd_iommu=off``
+   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add ``iommu.passthrough=1``
+   on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Gavin Hu (Arm Technology China)]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of tone.zhang
> Sent: Tuesday, September 4, 2018 4:59 PM
> To: dev@dpdk.org
> Cc: nd <nd@arm.com>
> Subject: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel
> module
> 
> When binding the devices used by DPDK to the "uio-pci" kernel module, the
> IOMMU should be disabled in order not to break the IO transmission
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
Acked-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
> 
> +.. note::
> +
> +   If the devices for used DPDK bound to the ``uio-pci`` kernel module, please
> make
> +   sure that the IOMMU is disabled. We can add ``intel_iommu=off`` or
> ``amd_iommu=off``
> +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add
> ``iommu.passthrough=1``
> +   on arm64 system.
> +
>  Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
> for platforms that support using VFIO.
> 
> --
> 2.7.4

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Bruce Richardson]

On Tue, Sep 04, 2018 at 04:59:07PM +0800, tone.zhang wrote:
> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the IO transmission
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
>  
> +.. note::
> +
> +   If the devices for used DPDK bound to the ``uio-pci`` kernel module, please make
> +   sure that the IOMMU is disabled. We can add ``intel_iommu=off`` or ``amd_iommu=off``
> +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add ``iommu.passthrough=1``
> +   on arm64 system.
> +

I think passthrough mode should work on x86 too. I remember running with
iommu=pt setting in the kernel in the past.

/Bruce

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Luca Boccassi]

On Tue, 2018-09-04 at 11:06 +0100, Bruce Richardson wrote:
> On Tue, Sep 04, 2018 at 04:59:07PM +0800, tone.zhang wrote:
> > When binding the devices used by DPDK to the "uio-pci" kernel
> > module,
> > the IOMMU should be disabled in order not to break the IO
> > transmission
> > because of the virtual / physical address mapping.
> > 
> > The patch clarifies the IOMMU configuration on both x86_64 and
> > arm64
> > systems.
> > 
> > Signed-off-by: tone.zhang <tone.zhang@arm.com>
> > ---
> >  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> > b/doc/guides/linux_gsg/linux_drivers.rst
> > index 371a817..8f9ec8f 100644
> > --- a/doc/guides/linux_gsg/linux_drivers.rst
> > +++ b/doc/guides/linux_gsg/linux_drivers.rst
> > @@ -48,6 +48,13 @@ be loaded as shown below:
> >     ``vfio-pci`` kernel module rather than ``igb_uio`` or
> > ``uio_pci_generic``.
> >     For more details see :ref:`linux_gsg_binding_kernel` below.
> >  
> > +.. note::
> > +
> > +   If the devices for used DPDK bound to the ``uio-pci`` kernel
> > module, please make
> > +   sure that the IOMMU is disabled. We can add ``intel_iommu=off``
> > or ``amd_iommu=off``
> > +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add
> > ``iommu.passthrough=1``
> > +   on arm64 system.
> > +
> 
> I think passthrough mode should work on x86 too. I remember running
> with
> iommu=pt setting in the kernel in the past.
> 
> /Bruce

It does, can confirm.

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Tone Zhang (Arm Technology China)]

-----Original Message-----
From: Luca Boccassi <bluca@debian.org> 
Sent: Tuesday, September 4, 2018 6:15 PM
To: Bruce Richardson <bruce.richardson@intel.com>; Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: dev@dpdk.org; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

On Tue, 2018-09-04 at 11:06 +0100, Bruce Richardson wrote:
> On Tue, Sep 04, 2018 at 04:59:07PM +0800, tone.zhang wrote:
> > When binding the devices used by DPDK to the "uio-pci" kernel 
> > module, the IOMMU should be disabled in order not to break the IO 
> > transmission because of the virtual / physical address mapping.
> > 
> > The patch clarifies the IOMMU configuration on both x86_64 and
> > arm64
> > systems.
> > 
> > Signed-off-by: tone.zhang <tone.zhang@arm.com>
> > ---
> >  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> > b/doc/guides/linux_gsg/linux_drivers.rst
> > index 371a817..8f9ec8f 100644
> > --- a/doc/guides/linux_gsg/linux_drivers.rst
> > +++ b/doc/guides/linux_gsg/linux_drivers.rst
> > @@ -48,6 +48,13 @@ be loaded as shown below:
> >     ``vfio-pci`` kernel module rather than ``igb_uio`` or 
> > ``uio_pci_generic``.
> >     For more details see :ref:`linux_gsg_binding_kernel` below.
> >  
> > +.. note::
> > +
> > +   If the devices for used DPDK bound to the ``uio-pci`` kernel
> > module, please make
> > +   sure that the IOMMU is disabled. We can add ``intel_iommu=off``
> > or ``amd_iommu=off``
> > +   in ``GRUB_CMDLINE_LINUX`` in grub on x86_64 systems, or add
> > ``iommu.passthrough=1``
> > +   on arm64 system.
> > +
> 
> I think passthrough mode should work on x86 too. I remember running 
> with iommu=pt setting in the kernel in the past.
> 
> /Bruce

It does, can confirm.

--
Kind regards,
Luca Boccassi


@Luca, @Bruce, 
Thanks  for the comments. I will update the change and push V2. Thanks!

Br,
Tone

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Rami Rosen]

Hi Tone,
>if the devices for used DPDK bound to the ``uio-pci`` kernel module,
please make

The three kernel modules which can be used for DPDK binding are vfio-pci,
uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?

Regards,
Rami Rosen


בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z

> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the
> because of the virtual / physical address mapping.
>
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
>
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci``
> 2.7.4
>
>

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Tone Zhang (Arm Technology China)]

Hi Rami,

Yes, I mean “uio_pci_generic” kernel module.

I will update the change accordingly.

Thanks.

Br,
Tone

From: Rami Rosen <roszenrami@gmail.com>
Sent: Wednesday, September 5, 2018 3:55 AM
To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: dev@dpdk.org; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

Hi Tone,
>if the devices for used DPDK bound to the ``uio-pci`` kernel module, please make

The three kernel modules which can be used for DPDK binding are vfio-pci, uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?

Regards,
Rami Rosen


בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z
When binding the devices used by DPDK to the "uio-pci" kernel module,
the IOMMU should be disabled in order not to break the
because of the virtual / physical address mapping.

The patch clarifies the IOMMU configuration on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com<mailto:tone.zhang@arm.com>>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..8f9ec8f 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci``
2.7.4

[dpdk-dev] [PATCH v2] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[tone.zhang]

If the devices used for DPDK are bound to the "uio_pci_generic" kernel
module, the IOMMU should be disabled in order not to break the IO transmission
because of the virtual / physical address mapping.

The patch clarifies the IOMMU configurations on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..b46fc6a 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
+   please make sure that the IOMMU is disabled. One can add ``intel_iommu=off`` or
+   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on x86_64 systems,
+   or add ``iommu.passthrough=1`` on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.
 
-- 
2.7.4

[dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[tone.zhang]

If the devices used for DPDK are bound to the "uio_pci_generic" kernel
module, the IOMMU should be disabled in order not to break the IO
transmission because of the virtual / physical address mapping.

The patch clarifies the IOMMU configurations on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..b46fc6a 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
+   please make sure that the IOMMU is disabled. One can add ``intel_iommu=off`` or
+   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on x86_64 systems,
+   or add ``iommu.passthrough=1`` on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Gavin Hu (Arm Technology China)]

> -----Original Message-----
> From: tone.zhang <tone.zhang@arm.com>
> Sent: Wednesday, September 5, 2018 1:18 PM
> To: dev@dpdk.org
> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> bruce.richardson@intel.com; bluca@debian.org; roszenrami@gmail.com; nd
> <nd@arm.com>
> Subject: [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel
> module
> 
> If the devices used for DPDK are bound to the "uio_pci_generic" kernel
> module, the IOMMU should be disabled in order not to break the IO
> transmission because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configurations on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
Acked-by: Gavin Hu <gavin.hu@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..b46fc6a 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
> 
> +.. note::
> +
> +   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel
> module,
> +   please make sure that the IOMMU is disabled. One can add
> ``intel_iommu=off`` or
> +   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on
> x86_64 systems,
> +   or add ``iommu.passthrough=1`` on arm64 system.
> +
>  Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
> for platforms that support using VFIO.
> 
> --
> 2.7.4

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Stephen Hemminger]

On Wed, 5 Sep 2018 00:20:29 +0000
"Tone Zhang (Arm Technology China)" <Tone.Zhang@arm.com> wrote:

> Hi Rami,
> 
> Yes, I mean “uio_pci_generic” kernel module.
> 
> I will update the change accordingly.
> 
> Thanks.
> 
> Br,
> Tone
> 
> From: Rami Rosen <roszenrami@gmail.com>
> Sent: Wednesday, September 5, 2018 3:55 AM
> To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
> Cc: dev@dpdk.org; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module
> 
> Hi Tone,
> >if the devices for used DPDK bound to the ``uio-pci`` kernel module, please make  
> 
> The three kernel modules which can be used for DPDK binding are vfio-pci, uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?
> 
> Regards,
> Rami Rosen
> 
> 
> בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z
> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com<mailto:tone.zhang@arm.com>>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci``
> 2.7.4

Also, it would be good if documentation was clear in recommending using VFIO
instead if IOMMU is present.

Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

[Tone Zhang (Arm Technology China)]

-----Original Message-----
From: Stephen Hemminger <stephen@networkplumber.org> 
Sent: Wednesday, September 5, 2018 5:22 PM
To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: Rami Rosen <roszenrami@gmail.com>; dev@dpdk.org; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module

On Wed, 5 Sep 2018 00:20:29 +0000
"Tone Zhang (Arm Technology China)" <Tone.Zhang@arm.com> wrote:

> Hi Rami,
> 
> Yes, I mean “uio_pci_generic” kernel module.
> 
> I will update the change accordingly.
> 
> Thanks.
> 
> Br,
> Tone
> 
> From: Rami Rosen <roszenrami@gmail.com>
> Sent: Wednesday, September 5, 2018 3:55 AM
> To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
> Cc: dev@dpdk.org; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH] doc: Clarify IOMMU usage with "uio-pci" kernel module
> 
> Hi Tone,
> >if the devices for used DPDK bound to the ``uio-pci`` kernel module, please make  
> 
> The three kernel modules which can be used for DPDK binding are vfio-pci, uio_pci_generic and igb_uio. Don't you mean here uio_pci_generic ?
> 
> Regards,
> Rami Rosen
> 
> 
> בתאריך יום ג׳, 4 בספט׳ 2018, 11:59, מאת tone.z
> When binding the devices used by DPDK to the "uio-pci" kernel module,
> the IOMMU should be disabled in order not to break the
> because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configuration on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com<mailto:tone.zhang@arm.com>>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..8f9ec8f 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci``
> 2.7.4

Also, it would be good if documentation was clear in recommending using VFIO
instead if IOMMU is present.

Hi Stephen,

Yes, according to the description of the "UIO" and "VFIO" in the document, I believe that VFIO is recommended.

BTW, I have update the change to V3 (https://patches.dpdk.org/patch/44295/). Please have a review.

Thanks a lot!

Br,
Tone

Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Tone Zhang (Arm Technology China)]

Hello Bruce, Luca, Rami and Stephen,

Could you please help to review the change? 

Any comments are welcome.

Thanks a lot!

Br,
Tone

-----Original Message-----
From: dev <dev-bounces@dpdk.org> On Behalf Of tone.zhang
Sent: Wednesday, September 5, 2018 1:18 PM
To: dev@dpdk.org
Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; bruce.richardson@intel.com; bluca@debian.org; roszenrami@gmail.com; nd <nd@arm.com>
Subject: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

If the devices used for DPDK are bound to the "uio_pci_generic" kernel module, the IOMMU should be disabled in order not to break the IO transmission because of the virtual / physical address mapping.

The patch clarifies the IOMMU configurations on both x86_64 and arm64 systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..b46fc6a 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
+   please make sure that the IOMMU is disabled. One can add ``intel_iommu=off`` or
+   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on x86_64 systems,
+   or add ``iommu.passthrough=1`` on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional  for platforms that support using VFIO.
 
--
2.7.4

Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Luca Boccassi]

On Wed, 2018-09-05 at 13:17 +0800, tone.zhang wrote:
> If the devices used for DPDK are bound to the "uio_pci_generic"
> kernel
> module, the IOMMU should be disabled in order not to break the IO
> transmission because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configurations on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..b46fc6a 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or
> ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
>  
> +.. note::
> +
> +   If the devices used for DPDK are bound to the ``uio_pci_generic``
> kernel module,
> +   please make sure that the IOMMU is disabled. One can add
> ``intel_iommu=off`` or
> +   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on
> x86_64 systems,
> +   or add ``iommu.passthrough=1`` on arm64 system.
> +
>  Since DPDK release 1.7 onward provides VFIO support, use of UIO is
> optional
>  for platforms that support using VFIO.
>  

Acked-by: Luca Boccassi <bluca@debian.org>

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Bruce Richardson]

On Wed, Sep 19, 2018 at 07:44:36AM +0000, Tone Zhang (Arm Technology China) wrote:
> Hello Bruce, Luca, Rami and Stephen,
> 
> Could you please help to review the change? 
> 
> Any comments are welcome.
> 
> Thanks a lot!
> 
> Br,
> Tone
> 
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of tone.zhang
> Sent: Wednesday, September 5, 2018 1:18 PM
> To: dev@dpdk.org
> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; bruce.richardson@intel.com; bluca@debian.org; roszenrami@gmail.com; nd <nd@arm.com>
> Subject: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module
> 
> If the devices used for DPDK are bound to the "uio_pci_generic" kernel module, the IOMMU should be disabled in order not to break the IO transmission because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configurations on both x86_64 and arm64 systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..b46fc6a 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
>  
> +.. note::
> +
> +   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
> +   please make sure that the IOMMU is disabled. One can add ``intel_iommu=off`` or
> +   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on x86_64 systems,
> +   or add ``iommu.passthrough=1`` on arm64 system.
> +

I'm not really an expert in these IOMMU commands, but for the passthrough
option on my systems, I've always used "intel_iommu=on iommu=pt", rather
than "intel_iommu=pt". Can someone confirm that the latter works ok, as
documented here?

/Bruce

Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Luca Boccassi]

On Wed, 2018-09-19 at 12:33 +0100, Bruce Richardson wrote:
> On Wed, Sep 19, 2018 at 07:44:36AM +0000, Tone Zhang (Arm Technology
> China) wrote:
> > Hello Bruce, Luca, Rami and Stephen,
> > 
> > Could you please help to review the change? 
> > 
> > Any comments are welcome.
> > 
> > Thanks a lot!
> > 
> > Br,
> > Tone
> > 
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of tone.zhang
> > Sent: Wednesday, September 5, 2018 1:18 PM
> > To: dev@dpdk.org
> > Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; bruce.richa
> > rdson@intel.com; bluca@debian.org; roszenrami@gmail.com; nd <nd@arm
> > .com>
> > Subject: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with
> > "uio_pci_generic" kernel module
> > 
> > If the devices used for DPDK are bound to the "uio_pci_generic"
> > kernel module, the IOMMU should be disabled in order not to break
> > the IO transmission because of the virtual / physical address
> > mapping.
> > 
> > The patch clarifies the IOMMU configurations on both x86_64 and
> > arm64 systems.
> > 
> > Signed-off-by: tone.zhang <tone.zhang@arm.com>
> > ---
> >  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> > b/doc/guides/linux_gsg/linux_drivers.rst
> > index 371a817..b46fc6a 100644
> > --- a/doc/guides/linux_gsg/linux_drivers.rst
> > +++ b/doc/guides/linux_gsg/linux_drivers.rst
> > @@ -48,6 +48,13 @@ be loaded as shown below:
> >     ``vfio-pci`` kernel module rather than ``igb_uio`` or
> > ``uio_pci_generic``.
> >     For more details see :ref:`linux_gsg_binding_kernel` below.
> >  
> > +.. note::
> > +
> > +   If the devices used for DPDK are bound to the
> > ``uio_pci_generic`` kernel module,
> > +   please make sure that the IOMMU is disabled. One can add
> > ``intel_iommu=off`` or
> > +   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on
> > x86_64 systems,
> > +   or add ``iommu.passthrough=1`` on arm64 system.
> > +
> 
> I'm not really an expert in these IOMMU commands, but for the
> passthrough
> option on my systems, I've always used "intel_iommu=on iommu=pt",
> rather
> than "intel_iommu=pt". Can someone confirm that the latter works ok,
> as
> documented here?
> 
> /Bruce

Whops you are right, just checked and we use iommu=pt

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Tone Zhang (Arm Technology China)]

Hi Luca and Bruce,

I appreciate for the comments. I ran more test cases in my test bed, and I will update the patch and push v4 soon.

Thanks a lot for your help!

Br,
Tone

-----Original Message-----
From: Luca Boccassi <bluca@debian.org> 
Sent: Wednesday, September 19, 2018 7:54 PM
To: Bruce Richardson <bruce.richardson@intel.com>; Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; roszenrami@gmail.com; Stephen Hemminger <stephen@networkplumber.org>; nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

On Wed, 2018-09-19 at 12:33 +0100, Bruce Richardson wrote:
> On Wed, Sep 19, 2018 at 07:44:36AM +0000, Tone Zhang (Arm Technology
> China) wrote:
> > Hello Bruce, Luca, Rami and Stephen,
> > 
> > Could you please help to review the change?
> > 
> > Any comments are welcome.
> > 
> > Thanks a lot!
> > 
> > Br,
> > Tone
> > 
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of tone.zhang
> > Sent: Wednesday, September 5, 2018 1:18 PM
> > To: dev@dpdk.org
> > Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; bruce.richa 
> > rdson@intel.com; bluca@debian.org; roszenrami@gmail.com; nd <nd@arm 
> > .com>
> > Subject: [dpdk-dev] [PATCH v3] doc: Clarify IOMMU usage with 
> > "uio_pci_generic" kernel module
> > 
> > If the devices used for DPDK are bound to the "uio_pci_generic"
> > kernel module, the IOMMU should be disabled in order not to break 
> > the IO transmission because of the virtual / physical address 
> > mapping.
> > 
> > The patch clarifies the IOMMU configurations on both x86_64 and
> > arm64 systems.
> > 
> > Signed-off-by: tone.zhang <tone.zhang@arm.com>
> > ---
> >  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/doc/guides/linux_gsg/linux_drivers.rst
> > b/doc/guides/linux_gsg/linux_drivers.rst
> > index 371a817..b46fc6a 100644
> > --- a/doc/guides/linux_gsg/linux_drivers.rst
> > +++ b/doc/guides/linux_gsg/linux_drivers.rst
> > @@ -48,6 +48,13 @@ be loaded as shown below:
> >     ``vfio-pci`` kernel module rather than ``igb_uio`` or 
> > ``uio_pci_generic``.
> >     For more details see :ref:`linux_gsg_binding_kernel` below.
> >  
> > +.. note::
> > +
> > +   If the devices used for DPDK are bound to the
> > ``uio_pci_generic`` kernel module,
> > +   please make sure that the IOMMU is disabled. One can add
> > ``intel_iommu=off`` or
> > +   ``intel_iommu=pt`` or ``amd_iommu=off`` in GRUB command line on
> > x86_64 systems,
> > +   or add ``iommu.passthrough=1`` on arm64 system.
> > +
> 
> I'm not really an expert in these IOMMU commands, but for the 
> passthrough option on my systems, I've always used "intel_iommu=on 
> iommu=pt", rather than "intel_iommu=pt". Can someone confirm that the 
> latter works ok, as documented here?
> 
> /Bruce

Whops you are right, just checked and we use iommu=pt

--
Kind regards,
Luca Boccassi

[dpdk-dev] [PATCH v4] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[tone.zhang]

If the devices used for DPDK are bound to the "uio_pci_generic" kernel
module, the IOMMU should be disabled in order not to break the IO
transmission because of the virtual / physical address mapping.

The patch clarifies the IOMMU configurations on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..3304800 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
+   please make sure that the IOMMU is disabledi or passthrough. One can add
+   ``intel_iommu=off`` or ``amd_iommu=off`` or ``intel_iommu=on iommu=pt``in GRUB
+   command line on x86_64 systems, or add ``iommu.passthrough=1`` on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v4] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Bruce Richardson]

On Fri, Sep 21, 2018 at 11:07:42AM +0800, tone.zhang wrote:
> If the devices used for DPDK are bound to the "uio_pci_generic" kernel
> module, the IOMMU should be disabled in order not to break the IO
> transmission because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configurations on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..3304800 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
>  
> +.. note::
> +
> +   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
> +   please make sure that the IOMMU is disabledi or passthrough. One can add
typo ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

> +   ``intel_iommu=off`` or ``amd_iommu=off`` or ``intel_iommu=on iommu=pt``in GRUB
> +   command line on x86_64 systems, or add ``iommu.passthrough=1`` on arm64 system.
> +
>  Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
>  for platforms that support using VFIO.
>  
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [dpdk-dev] [PATCH v4] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Tone Zhang (Arm Technology China)]

Hello Bruce,

Thanks for the comments.

I will update the change and issue v5 soon.

Thanks!

Br,
Tone

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Friday, September 21, 2018 7:20 PM
To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>
Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; bluca@debian.org; roszenrami@gmail.com; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
Subject: Re: [PATCH v4] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

On Fri, Sep 21, 2018 at 11:07:42AM +0800, tone.zhang wrote:
> If the devices used for DPDK are bound to the "uio_pci_generic" kernel 
> module, the IOMMU should be disabled in order not to break the IO 
> transmission because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configurations on both x86_64 and arm64 
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> ---
>  doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/doc/guides/linux_gsg/linux_drivers.rst 
> b/doc/guides/linux_gsg/linux_drivers.rst
> index 371a817..3304800 100644
> --- a/doc/guides/linux_gsg/linux_drivers.rst
> +++ b/doc/guides/linux_gsg/linux_drivers.rst
> @@ -48,6 +48,13 @@ be loaded as shown below:
>     ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
>     For more details see :ref:`linux_gsg_binding_kernel` below.
>  
> +.. note::
> +
> +   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
> +   please make sure that the IOMMU is disabledi or passthrough. One 
> + can add
typo ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

> +   ``intel_iommu=off`` or ``amd_iommu=off`` or ``intel_iommu=on iommu=pt``in GRUB
> +   command line on x86_64 systems, or add ``iommu.passthrough=1`` on arm64 system.
> +
>  Since DPDK release 1.7 onward provides VFIO support, use of UIO is 
> optional  for platforms that support using VFIO.
>  
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

[dpdk-dev] [PATCH v5] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[tone.zhang]

If the devices used for DPDK are bound to the "uio_pci_generic" kernel
module, the IOMMU should be disabled in order not to break the IO
transmission because of the virtual / physical address mapping.

The patch clarifies the IOMMU configurations on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..8da6a31 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
+   please make sure that the IOMMU is disabled or passthrough. One can add
+   ``intel_iommu=off`` or ``amd_iommu=off`` or ``intel_iommu=on iommu=pt``in GRUB
+   command line on x86_64 systems, or add ``iommu.passthrough=1`` on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.
 
-- 
2.7.4

[dpdk-dev] [PATCH v6] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[tone.zhang]

If the devices used for DPDK are bound to the "uio_pci_generic" kernel
module, the IOMMU should be disabled in order not to break the IO
transmission because of the virtual / physical address mapping.

The patch clarifies the IOMMU configurations on both x86_64 and arm64
systems.

Signed-off-by: tone.zhang <tone.zhang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Rami Rosen <roszenrami@gmail.com>
Acked-by: Gavin Hu <Gavin.Hu@arm.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
---
 doc/guides/linux_gsg/linux_drivers.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst
index 371a817..8da6a31 100644
--- a/doc/guides/linux_gsg/linux_drivers.rst
+++ b/doc/guides/linux_gsg/linux_drivers.rst
@@ -48,6 +48,13 @@ be loaded as shown below:
    ``vfio-pci`` kernel module rather than ``igb_uio`` or ``uio_pci_generic``.
    For more details see :ref:`linux_gsg_binding_kernel` below.
 
+.. note::
+
+   If the devices used for DPDK are bound to the ``uio_pci_generic`` kernel module,
+   please make sure that the IOMMU is disabled or passthrough. One can add
+   ``intel_iommu=off`` or ``amd_iommu=off`` or ``intel_iommu=on iommu=pt``in GRUB
+   command line on x86_64 systems, or add ``iommu.passthrough=1`` on arm64 system.
+
 Since DPDK release 1.7 onward provides VFIO support, use of UIO is optional
 for platforms that support using VFIO.
 
-- 
2.7.4

Re: [dpdk-dev] [PATCH v6] doc: Clarify IOMMU usage with "uio_pci_generic" kernel module

[Thomas Monjalon]

26/09/2018 07:08, tone.zhang:
> If the devices used for DPDK are bound to the "uio_pci_generic" kernel
> module, the IOMMU should be disabled in order not to break the IO
> transmission because of the virtual / physical address mapping.
> 
> The patch clarifies the IOMMU configurations on both x86_64 and arm64
> systems.
> 
> Signed-off-by: tone.zhang <tone.zhang@arm.com>
> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
> Reviewed-by: Rami Rosen <roszenrami@gmail.com>
> Acked-by: Gavin Hu <Gavin.Hu@arm.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> Acked-by: Luca Boccassi <bluca@debian.org>

Applied, thanks