From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 4E3CC42ECA
for <public@inbox.dpdk.org>; Thu, 20 Jul 2023 14:44:53 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 1CB9F40E2D;
Thu, 20 Jul 2023 14:44:53 +0200 (CEST)
Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com
[209.85.221.175])
by mails.dpdk.org (Postfix) with ESMTP id 8252D40DF5
for <users@dpdk.org>; Thu, 20 Jul 2023 14:44:51 +0200 (CEST)
Received: by mail-vk1-f175.google.com with SMTP id
71dfb90a1353d-481646425fcso300922e0c.3
for <users@dpdk.org>; Thu, 20 Jul 2023 05:44:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20221208; t=1689857091; x=1690461891;
h=cc:to:subject:message-id:date:from:in-reply-to:references
:mime-version:from:to:cc:subject:date:message-id:reply-to;
bh=RhCOp4XXx2bsrkuSnkg8kl245y/gE23SuGxIdUd312I=;
b=WpyXIyWKkKSW71KEPc7JmvGMNvmWEU2I5/64l2uyeluEZG1YUPnTpobTGLjWGawXQe
05j1bkkceBBMPFRVo275wppqBLyPMHGcimpFz/G40ILFK5AKBqFOH2UKaNF/cbf4WcCf
XVHc0Bzp63cMPQRAI2GtOqKanUTU0NL77clGW+nGAXIXetMNJmemcJD+1cSUkjaH18wz
Hk69ud6Rh1mnh+GJV/EpzsEWM+5cPuLkOTFyOMPOd6Iif6FDptw+R9cfGPEQuitbBQ7N
IEHYBi3y7Q4h/Z+9QF5VEfi0iUXGwfP5AIurdj5G/Q5iWsDoOG6aFlH0CtTAI15Qu5x5
Wp2g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20221208; t=1689857091; x=1690461891;
h=cc:to:subject:message-id:date:from:in-reply-to:references
:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=RhCOp4XXx2bsrkuSnkg8kl245y/gE23SuGxIdUd312I=;
b=VeKvxA2Vef9K4rQR9LAL5KO67mIY73U5XayxU2QEUQYoxl8YbvkfJwR0DYg0grSqvy
BjKIVVNnKI3Zs+lTevG3UdBL2n4noJE0wByc2MlOas/59iu8mGK0iqGfcNcbQIndtBX4
x9lWsG7CVdZPIKlwerWmjACQy/TUxR8++lU0IUcaTCSC3SEkel2aDDEsBKdPTT1o3ryY
aWPaOJxrJOCDeoGUbAeGniQz7x2EUe/VJUDB9APVFmnKTbD57GkjCCszgEoQHc9iSnCh
Aou/pn1COqOwZPXge0OE2ZIWcOx+yIBiq/faI4/BOrM8pi6N/bGx8ihNKVXPS8dAouY9
Cxig==
X-Gm-Message-State: ABy/qLaHOMQvfcXcKzqNTzkk2YaTwg2EPFJc7QoWsCzqtZ7KvAxl3jBn
JbHg9M/g5hqpmSsIsTB7g//wEE3DcfTIkVzFqe0=
X-Google-Smtp-Source: APBJJlHCeEmKMUzzjXv9SdYaHe7oTWqUA8Y7TlOa0d/ucBV4Mx5/XEOkLOvxNOuhmsH4Xsqj9Bu9Hi+xnsr9+spJqtw=
X-Received: by 2002:a1f:e7c4:0:b0:47e:9bbb:103b with SMTP id
e187-20020a1fe7c4000000b0047e9bbb103bmr1720227vkh.6.1689857090746; Thu, 20
Jul 2023 05:44:50 -0700 (PDT)
MIME-Version: 1.0
References: <CALDr01SrSDZXeuVyBC3b0RTvPhN-c7TNJsis1Qq9rMxC+jV3cg@mail.gmail.com>
<8740856.JRmrKFJ9eK@thomas>
<CALDr01TK5rb0F6HGdhrVSTAV-ZCE1MSFaT4DucNnq9JHeO64vQ@mail.gmail.com>
<2663502.2IRrRt1zHL@thomas>
In-Reply-To: <2663502.2IRrRt1zHL@thomas>
From: Igor de Paula <igordptx@gmail.com>
Date: Thu, 20 Jul 2023 13:44:39 +0100
Message-ID: <CALDr01SAVpp34HFkF2sGD+Jt--cUYtUoQ1KEALnVT-rY_71MQg@mail.gmail.com>
Subject: Re: help
To: Thomas Monjalon <thomas@monjalon.net>
Cc: users@dpdk.org, Jochen Behrens <jbehrens@vmware.com>
Content-Type: multipart/alternative; boundary="0000000000009d7ceb0600ea8305"
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
<mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
<mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org
--0000000000009d7ceb0600ea8305
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
I have enabled it in the host and in the BIOS for AMD...
In the Bios I changed to amd_iommu=3Don and in the host it's the same for
either.
On Thu, Jul 20, 2023 at 1:31=E2=80=AFPM Thomas Monjalon <thomas@monjalon.ne=
t> wrote:
> 20/07/2023 11:35, Igor de Paula:
> > The weird thing is that it only happens when I am using a host with an
> AMD
> > processor. It doesn't happen when I use a host with an Intel processor.
>
> So it's probably a matter of BIOS settings for the IOMMU?
>
>
> > On Thu, Jul 20, 2023 at 10:32=E2=80=AFAM Thomas Monjalon <thomas@monjal=
on.net>
> > wrote:
> >
> > > +Cc the vmxnet3 maintainer.
> > >
> > > Please Jochen, do you have an idea what's wrong below?
> > >
> > >
> > > 20/07/2023 11:25, Igor de Paula:
> > > > This is because it can't negotiate the IOMMU type with any port.
> > > >
> > > > On Thu, Jul 20, 2023 at 5:08=E2=80=AFAM Thomas Monjalon <thomas@mon=
jalon.net
> >
> > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > The first error is "Cause: Error: number of ports must be even"
> > > > >
> > > > >
> > > > > 03/05/2023 18:13, Igor de Paula:
> > > > > > I am running a VM inside a VMWARE server (vSphere).
> > > > > > My goal it to set up DPDK with two HW ports, and set up a
> > > virtio_user to
> > > > > > interact with the kernel stack.
> > > > > > In another app I have it working but instead of virtio_user I a=
m
> > > running
> > > > > > KNI, it works in IOVA-PA mode.
> > > > > > I am looking to replace the KNI.
> > > > > >
> > > > > > When I try to set up virtio_user port as in the doc:
> > > > > >
> > > > >
> > >
> https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virt=
io-user-as-exception-path
> > > > > > I get a error it can't run in PA mode.
> > > > > >
> > > > > >
> > > > > > When I try to run as VA mode from a parameter, I get the
> following
> > > > > errors:
> > > > > > EAL: lib.eal log level changed from info to debug
> > > > > > EAL: Detected lcore 0 as core 0 on socket 0
> > > > > > EAL: Detected lcore 1 as core 0 on socket 0
> > > > > > EAL: Support maximum 128 logical core(s) by configuration.
> > > > > > EAL: Detected 2 lcore(s)
> > > > > > EAL: Detected 1 NUMA nodes
> > > > > > EAL: Checking presence of .so 'librte_eal.so.21.3'
> > > > > > EAL: Checking presence of .so 'librte_eal.so.21'
> > > > > > EAL: Checking presence of .so 'librte_eal.so'
> > > > > > EAL: Detected static linkage of DPDK
> > > > > > EAL: Ask a virtual area of 0x7000 bytes
> > > > > > EAL: Virtual area found at 0x100000000 (size =3D 0x7000)
> > > > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > > > > EAL: DPAA Bus not present. Skipping.
> > > > > > EAL: VFIO PCI modules not loaded
> > > > > > EAL: Selected IOVA mode 'VA'
> > > > > > EAL: Probing VFIO support...
> > > > > > EAL: IOMMU type 1 (Type 1) is supported
> > > > > > EAL: IOMMU type 7 (sPAPR) is not supported
> > > > > > EAL: IOMMU type 8 (No-IOMMU) is supported
> > > > > > EAL: VFIO support initialized
> > > > > > EAL: Ask a virtual area of 0x5b000 bytes
> > > > > > EAL: Virtual area found at 0x100007000 (size =3D 0x5b000)
> > > > > > EAL: Setting up physically contiguous memory...
> > > > > > EAL: Setting maximum number of open files to 1048576
> > > > > > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > > > > > EAL: Creating 2 segment lists: n_segs:128 socket_id:0
> > > > > hugepage_sz:1073741824
> > > > > > EAL: Ask a virtual area of 0x2000 bytes
> > > > > > EAL: Virtual area found at 0x100062000 (size =3D 0x2000)
> > > > > > EAL: Memseg list allocated at socket 0, page size 0x100000kB
> > > > > > EAL: Ask a virtual area of 0x2000000000 bytes
> > > > > > EAL: Virtual area found at 0x140000000 (size =3D 0x2000000000)
> > > > > > EAL: VA reserved for memseg list at 0x140000000, size 200000000=
0
> > > > > > EAL: Ask a virtual area of 0x2000 bytes
> > > > > > EAL: Virtual area found at 0x2140000000 (size =3D 0x2000)
> > > > > > EAL: Memseg list allocated at socket 0, page size 0x100000kB
> > > > > > EAL: Ask a virtual area of 0x2000000000 bytes
> > > > > > EAL: Virtual area found at 0x2180000000 (size =3D 0x2000000000)
> > > > > > EAL: VA reserved for memseg list at 0x2180000000, size 20000000=
00
> > > > > > EAL: TSC frequency is ~2350000 KHz
> > > > > > EAL: Main lcore 0 is ready (tid=3D7f8ad790ec00;cpuset=3D[0])
> > > > > > EAL: lcore 1 is ready (tid=3D7f8ad6907400;cpuset=3D[1])
> > > > > > EAL: Trying to obtain current memory policy.
> > > > > > EAL: Setting policy MPOL_PREFERRED for socket 0
> > > > > > EAL: Restoring previous memory policy: 0
> > > > > > EAL: request: mp_malloc_sync
> > > > > > EAL: Heap on socket 0 was expanded by 1024MB
> > > > > > EAL: PCI device 0000:0b:00.0 on NUMA socket -1
> > > > > > EAL: probe driver: 15ad:7b0 net_vmxnet3
> > > > > > EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not
> > > > > initializing
> > > > > > EAL: Requested device 0000:0b:00.0 cannot be used
> > > > > > EAL: PCI device 0000:13:00.0 on NUMA socket -1
> > > > > > EAL: probe driver: 15ad:7b0 net_vmxnet3
> > > > > > EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not
> > > > > initializing
> > > > > > EAL: Requested device 0000:13:00.0 cannot be used
> > > > > > EAL: Bus (pci) probe failed.
> > > > > > EAL: lib.telemetry log level changed from disabled to warning
> > > > > > EAL: Error - exiting with code: 1
> > > > > > Cause: Error: number of ports must be even
> > > > > > EAL: request: mp_malloc_sync
> > > > > > EAL: Heap on socket 0 was shrunk by 1024MB
> > > > > >
> > > > > >
> > > > > >
> > > > > > For some reason the HW ports won't setup. From what I understan=
d
> > > > > > net_vmxnet3 should work with VA mode.
> > > > > > I enabled I/OMUU for the VM.
> > > > > > The weird thing even when enabled, I still have the
> > > > > > enable_unsafe_noiommu_mode flag on.
> > > > > > And because it's on the this:
> > > > > >
> > > > > > dev_iova_mode =3D pci_device_iova_mode(dr, dev);
> > > > > >
> > > > > > return PA mode, and it fails.
> > > > > >
> > > > > > When I disable it by modifying
> > > > > > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode, I get
> another
> > > > > error.
> > > > > > The error is that it doesn't find a suitable IOMMU type:
> > > > > > Just putting the relevant message:
> > > > > >
> > > > > >
> > > > > > EAL: Heap on socket 0 was expanded by 1024MB
> > > > > > EAL: PCI device 0000:0b:00.0 on NUMA socket -1
> > > > > > EAL: probe driver: 15ad:7b0 net_vmxnet3
> > > > > > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device=
)
> > > > > > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device)
> > > > > > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such
> device)
> > > > > > EAL: 0000:0b:00.0 failed to select IOMMU type
> > > > > > EAL: Requested device 0000:0b:00.0 cannot be used
> > > > > > EAL: PCI device 0000:13:00.0 on NUMA socket -1
> > > > > > EAL: probe driver: 15ad:7b0 net_vmxnet3
> > > > > > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device=
)
> > > > > > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device)
> > > > > > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such
> device)
> > > > > > EAL: 0000:13:00.0 failed to select IOMMU type
> > > > > > EAL: Requested device 0000:13:00.0 cannot be used
> > > > > >
> > > > > >
> > > > > > I can see where it fails:
> > > > > >
> > > > > >
> > > > > > const struct vfio_iommu_type *
> > > > > > vfio_set_iommu_type(int vfio_container_fd)
> > > > > > {
> > > > > > unsigned idx;
> > > > > > for (idx =3D 0; idx < RTE_DIM(iommu_types); idx++) {
> > > > > > const struct vfio_iommu_type *t =3D &iommu_types[idx];
> > > > > >
> > > > > > int ret =3D ioctl(vfio_container_fd, VFIO_SET_IOMMU,
> > > > > > t->type_id);
> > > > > > if (!ret) {
> > > > > > RTE_LOG(INFO, EAL, "Using IOMMU type %d (%s)\n",
> > > > > > t->type_id, t->name);
> > > > > > return t;
> > > > > > }
> > > > > > /* not an error, there may be more supported IOMMU types */
> > > > > > RTE_LOG(DEBUG, EAL, "Set IOMMU type %d (%s) failed, error "
> > > > > > "%i (%s)\n", t->type_id, t->name, errno,
> > > > > > strerror(errno));
> > > > > > }
> > > > > > /* if we didn't find a suitable IOMMU type, fail */
> > > > > > return NULL;
> > > > > > }
> > > > > >
> > > > > >
> > > > > > The ioctl returns -1.
> > > > > >
> > > > > >
> > > > > >
> > > > > > I tried many things.
> > > > > > Enabling/disabling IOMMU.
> > > > > > Modifying the enable_unsafe_noiommu_mode flag.
> > > > > > Running in PA mode.
> > > > > > Any help will be much appreciated.
>
>
>
>
>
>
--0000000000009d7ceb0600ea8305
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">I have enabled it in the host and in the BIOS for AMD...<b=
r>In the Bios I changed to amd_iommu=3Don and in the host it's the same=
for either.</div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"=
gmail_attr">On Thu, Jul 20, 2023 at 1:31=E2=80=AFPM Thomas Monjalon <<a =
href=3D"mailto:thomas@monjalon.net">thomas@monjalon.net</a>> wrote:<br><=
/div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left:1px solid rgb(204,204,204);padding-left:1ex">20/07/2023 11:35, Ig=
or de Paula:<br>
> The weird thing is that it only happens when I am using a host with an=
AMD<br>
> processor. It doesn't happen when I use a host with an Intel proce=
ssor.<br>
<br>
So it's probably a matter of BIOS settings for the IOMMU?<br>
<br>
<br>
> On Thu, Jul 20, 2023 at 10:32=E2=80=AFAM Thomas Monjalon <<a href=
=3D"mailto:thomas@monjalon.net" target=3D"_blank">thomas@monjalon.net</a>&g=
t;<br>
> wrote:<br>
> <br>
> > +Cc the vmxnet3 maintainer.<br>
> ><br>
> > Please Jochen, do you have an idea what's wrong below?<br>
> ><br>
> ><br>
> > 20/07/2023 11:25, Igor de Paula:<br>
> > > This is because it can't negotiate the IOMMU type with a=
ny port.<br>
> > ><br>
> > > On Thu, Jul 20, 2023 at 5:08=E2=80=AFAM Thomas Monjalon <=
<a href=3D"mailto:thomas@monjalon.net" target=3D"_blank">thomas@monjalon.ne=
t</a>><br>
> > wrote:<br>
> > ><br>
> > > > Hello,<br>
> > > ><br>
> > > > The first error is "Cause: Error: number of ports =
must be even"<br>
> > > ><br>
> > > ><br>
> > > > 03/05/2023 18:13, Igor de Paula:<br>
> > > > > I am running a VM inside a VMWARE server (vSphere)=
.<br>
> > > > > My goal it to set up DPDK with two HW ports, and s=
et up a<br>
> > virtio_user to<br>
> > > > > interact with the kernel stack.<br>
> > > > > In another app I have it working but instead of vi=
rtio_user I am<br>
> > running<br>
> > > > > KNI, it works in IOVA-PA mode.<br>
> > > > > I am looking to replace the KNI.<br>
> > > > ><br>
> > > > > When I try to set up virtio_user port as in the do=
c:<br>
> > > > ><br>
> > > ><br>
> > <a href=3D"https://doc.dpdk.org/guides/howto/virtio_user_as_excep=
tion_path.html#virtio-user-as-exception-path" rel=3D"noreferrer" target=3D"=
_blank">https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.htm=
l#virtio-user-as-exception-path</a><br>
> > > > > I get a error it can't run in PA mode.<br>
> > > > ><br>
> > > > ><br>
> > > > > When I try to run as VA mode from a parameter, I g=
et the following<br>
> > > > errors:<br>
> > > > > EAL: lib.eal log level changed from info to debug<=
br>
> > > > > EAL: Detected lcore 0 as core 0 on socket 0<br>
> > > > > EAL: Detected lcore 1 as core 0 on socket 0<br>
> > > > > EAL: Support maximum 128 logical core(s) by config=
uration.<br>
> > > > > EAL: Detected 2 lcore(s)<br>
> > > > > EAL: Detected 1 NUMA nodes<br>
> > > > > EAL: Checking presence of .so 'librte_eal.so.2=
1.3'<br>
> > > > > EAL: Checking presence of .so 'librte_eal.so.2=
1'<br>
> > > > > EAL: Checking presence of .so 'librte_eal.so&#=
39;<br>
> > > > > EAL: Detected static linkage of DPDK<br>
> > > > > EAL: Ask a virtual area of 0x7000 bytes<br>
> > > > > EAL: Virtual area found at 0x100000000 (size =3D 0=
x7000)<br>
> > > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_soc=
ket<br>
> > > > > EAL: DPAA Bus not present. Skipping.<br>
> > > > > EAL: VFIO PCI modules not loaded<br>
> > > > > EAL: Selected IOVA mode 'VA'<br>
> > > > > EAL: Probing VFIO support...<br>
> > > > > EAL: IOMMU type 1 (Type 1) is supported<br>
> > > > > EAL: IOMMU type 7 (sPAPR) is not supported<br>
> > > > > EAL: IOMMU type 8 (No-IOMMU) is supported<br>
> > > > > EAL: VFIO support initialized<br>
> > > > > EAL: Ask a virtual area of 0x5b000 bytes<br>
> > > > > EAL: Virtual area found at 0x100007000 (size =3D 0=
x5b000)<br>
> > > > > EAL: Setting up physically contiguous memory...<br=
>
> > > > > EAL: Setting maximum number of open files to 10485=
76<br>
> > > > > EAL: Detected memory type: socket_id:0 hugepage_sz=
:1073741824<br>
> > > > > EAL: Creating 2 segment lists: n_segs:128 socket_i=
d:0<br>
> > > > hugepage_sz:1073741824<br>
> > > > > EAL: Ask a virtual area of 0x2000 bytes<br>
> > > > > EAL: Virtual area found at 0x100062000 (size =3D 0=
x2000)<br>
> > > > > EAL: Memseg list allocated at socket 0, page size =
0x100000kB<br>
> > > > > EAL: Ask a virtual area of 0x2000000000 bytes<br>
> > > > > EAL: Virtual area found at 0x140000000 (size =3D 0=
x2000000000)<br>
> > > > > EAL: VA reserved for memseg list at 0x140000000, s=
ize 2000000000<br>
> > > > > EAL: Ask a virtual area of 0x2000 bytes<br>
> > > > > EAL: Virtual area found at 0x2140000000 (size =3D =
0x2000)<br>
> > > > > EAL: Memseg list allocated at socket 0, page size =
0x100000kB<br>
> > > > > EAL: Ask a virtual area of 0x2000000000 bytes<br>
> > > > > EAL: Virtual area found at 0x2180000000 (size =3D =
0x2000000000)<br>
> > > > > EAL: VA reserved for memseg list at 0x2180000000, =
size 2000000000<br>
> > > > > EAL: TSC frequency is ~2350000 KHz<br>
> > > > > EAL: Main lcore 0 is ready (tid=3D7f8ad790ec00;cpu=
set=3D[0])<br>
> > > > > EAL: lcore 1 is ready (tid=3D7f8ad6907400;cpuset=
=3D[1])<br>
> > > > > EAL: Trying to obtain current memory policy.<br>
> > > > > EAL: Setting policy MPOL_PREFERRED for socket 0<br=
>
> > > > > EAL: Restoring previous memory policy: 0<br>
> > > > > EAL: request: mp_malloc_sync<br>
> > > > > EAL: Heap on socket 0 was expanded by 1024MB<br>
> > > > > EAL: PCI device 0000:0b:00.0 on NUMA socket -1<br>
> > > > > EAL:=C2=A0 =C2=A0probe driver: 15ad:7b0 net_vmxnet=
3<br>
> > > > > EAL:=C2=A0 =C2=A0Expecting 'PA' IOVA mode =
but current mode is 'VA', not<br>
> > > > initializing<br>
> > > > > EAL: Requested device 0000:0b:00.0 cannot be used<=
br>
> > > > > EAL: PCI device 0000:13:00.0 on NUMA socket -1<br>
> > > > > EAL:=C2=A0 =C2=A0probe driver: 15ad:7b0 net_vmxnet=
3<br>
> > > > > EAL:=C2=A0 =C2=A0Expecting 'PA' IOVA mode =
but current mode is 'VA', not<br>
> > > > initializing<br>
> > > > > EAL: Requested device 0000:13:00.0 cannot be used<=
br>
> > > > > EAL: Bus (pci) probe failed.<br>
> > > > > EAL: lib.telemetry log level changed from disabled=
to warning<br>
> > > > > EAL: Error - exiting with code: 1<br>
> > > > >=C2=A0 =C2=A0Cause: Error: number of ports must be =
even<br>
> > > > > EAL: request: mp_malloc_sync<br>
> > > > > EAL: Heap on socket 0 was shrunk by 1024MB<br>
> > > > ><br>
> > > > ><br>
> > > > ><br>
> > > > > For some reason the HW ports won't setup. From=
what I understand<br>
> > > > > net_vmxnet3 should work with VA mode.<br>
> > > > > I enabled I/OMUU for the VM.<br>
> > > > > The weird thing even when enabled, I still have th=
e<br>
> > > > > enable_unsafe_noiommu_mode flag on.<br>
> > > > > And because it's on the this:<br>
> > > > ><br>
> > > > > dev_iova_mode =3D pci_device_iova_mode(dr, dev);<b=
r>
> > > > ><br>
> > > > > return PA mode, and it fails.<br>
> > > > ><br>
> > > > > When I disable it by modifying<br>
> > > > > /sys/module/vfio/parameters/enable_unsafe_noiommu_=
mode, I get another<br>
> > > > error.<br>
> > > > > The error is that it doesn't find a suitable I=
OMMU type:<br>
> > > > > Just putting the relevant message:<br>
> > > > ><br>
> > > > ><br>
> > > > > EAL: Heap on socket 0 was expanded by 1024MB<br>
> > > > > EAL: PCI device 0000:0b:00.0 on NUMA socket -1<br>
> > > > > EAL:=C2=A0 =C2=A0probe driver: 15ad:7b0 net_vmxnet=
3<br>
> > > > > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (N=
o such device)<br>
> > > > > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No=
such device)<br>
> > > > > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 =
(No such device)<br>
> > > > > EAL: 0000:0b:00.0 failed to select IOMMU type<br>
> > > > > EAL: Requested device 0000:0b:00.0 cannot be used<=
br>
> > > > > EAL: PCI device 0000:13:00.0 on NUMA socket -1<br>
> > > > > EAL:=C2=A0 =C2=A0probe driver: 15ad:7b0 net_vmxnet=
3<br>
> > > > > EAL: Set IOMMU type 1 (Type 1) failed, error 19 (N=
o such device)<br>
> > > > > EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No=
such device)<br>
> > > > > EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 =
(No such device)<br>
> > > > > EAL: 0000:13:00.0 failed to select IOMMU type<br>
> > > > > EAL: Requested device 0000:13:00.0 cannot be used<=
br>
> > > > ><br>
> > > > ><br>
> > > > > I can see where it fails:<br>
> > > > ><br>
> > > > ><br>
> > > > > const struct vfio_iommu_type *<br>
> > > > > vfio_set_iommu_type(int vfio_container_fd)<br>
> > > > > {<br>
> > > > > unsigned idx;<br>
> > > > > for (idx =3D 0; idx < RTE_DIM(iommu_types); idx=
++) {<br>
> > > > > const struct vfio_iommu_type *t =3D &iommu_typ=
es[idx];<br>
> > > > ><br>
> > > > > int ret =3D ioctl(vfio_container_fd, VFIO_SET_IOMM=
U,<br>
> > > > > t->type_id);<br>
> > > > > if (!ret) {<br>
> > > > > RTE_LOG(INFO, EAL, "Using IOMMU type %d (%s)\=
n",<br>
> > > > > t->type_id, t->name);<br>
> > > > > return t;<br>
> > > > > }<br>
> > > > > /* not an error, there may be more supported IOMMU=
types */<br>
> > > > > RTE_LOG(DEBUG, EAL, "Set IOMMU type %d (%s) f=
ailed, error "<br>
> > > > > "%i (%s)\n", t->type_id, t->name, =
errno,<br>
> > > > > strerror(errno));<br>
> > > > > }<br>
> > > > > /* if we didn't find a suitable IOMMU type, fa=
il */<br>
> > > > > return NULL;<br>
> > > > > }<br>
> > > > ><br>
> > > > ><br>
> > > > > The ioctl returns -1.<br>
> > > > ><br>
> > > > ><br>
> > > > ><br>
> > > > > I tried many things.<br>
> > > > > Enabling/disabling IOMMU.<br>
> > > > > Modifying the enable_unsafe_noiommu_mode flag.<br>
> > > > > Running in PA mode.<br>
> > > > > Any help will be much appreciated.<br>
<br>
<br>
<br>
<br>
<br>
</blockquote></div>
--0000000000009d7ceb0600ea8305--