From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64FDD4298D for ; Thu, 20 Apr 2023 01:31:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CC93A40A79; Thu, 20 Apr 2023 01:31:50 +0200 (CEST) Received: from mail-ua1-f48.google.com (mail-ua1-f48.google.com [209.85.222.48]) by mails.dpdk.org (Postfix) with ESMTP id 9E69940A4B for ; Thu, 20 Apr 2023 01:31:49 +0200 (CEST) Received: by mail-ua1-f48.google.com with SMTP id r10so1031641uat.6 for ; Wed, 19 Apr 2023 16:31:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681947109; x=1684539109; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=vK6zeRN58Kx1MSOAIIymrmJ8yzkgpZ/6skTd1LEFGCM=; b=XnrhBa8a651BK9lBsHeoFxd8baC2nWkcco8q85aKVvP9MnBKY1HHLmhTk39FVsjHo8 BSeKth/hk3UzAdnqgMc+LiSXQYsifw1EncqsknGwJGsAHtWF1cZVpP1YCJ6N3bmKq4Vf eQQHx6GbW9NxH1j0WJfTvjKqmd2CPtcl/aOSCV5q3bduUZ6VcCq0We0CtD6nN79WbMua PoBgx6VNvXqdmpjNLV568zQy3deNeIka/Fpi/aFlgr4KW892gNCVwq+d8pm3NgPZyNZe MXPAOeec1acsMhcjwCenEEyaaRZoiVQYgEd4c0Rqa2GIM5XK8gCL6vdjsWcTdgcBRGQ8 pXxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681947109; x=1684539109; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vK6zeRN58Kx1MSOAIIymrmJ8yzkgpZ/6skTd1LEFGCM=; b=NNxYGLKr2Jgyv8+RFTXOCP0gHIsuc1Pe21OsQqbyA/xbrzucf6DD/iByRGf8tFkccj TsgDMTFi171jfHM9yYaHhEehRWQOwSRRq9ERbInbLHn387IpaJN7CIoTs7EYFkjQuCwy FtpjlBnmRdegSzsiEJAhGuiI6C10lE/y5ZLNTkf5H6JXN2qIZi3wm2IsjNLPg7OufCRj iWbydyQpMB7GBV/oaWC+yZE7ZrqZk5zYl8D3RnvNEnk/G9CgPj6C0yhFNTUedmVwuTwZ P9a7K3GNyR0rywJa47UAs0k3V8Bl8Dei8DznpuD5HoALs/l2PtN6xLP+oaf06JMVL76D 71XQ== X-Gm-Message-State: AAQBX9fRca/dJABzqC35wryEck146M6TwZsNhKukL/49ANNa9miJXm61 ONQwU+EPtEMcALbLm5Jh6Fuixppdwfi8sQffmxIqRga1nBw= X-Google-Smtp-Source: AKy350bBNs5qV42rBqpwTuigz0ofS/1CFwY5NJQZg30UodNSC2jkx8bL2GmzkaSrKAXmAF/3lNT/zky3EVMJkoam3ZE= X-Received: by 2002:a1f:4883:0:b0:43f:fc32:15f0 with SMTP id v125-20020a1f4883000000b0043ffc3215f0mr699546vka.14.1681947108903; Wed, 19 Apr 2023 16:31:48 -0700 (PDT) MIME-Version: 1.0 References: <20230418184719.4a6a1d01@hermes.local> In-Reply-To: <20230418184719.4a6a1d01@hermes.local> From: Verghis Koshi Date: Wed, 19 Apr 2023 16:31:37 -0700 Message-ID: Subject: Re: Issues with basicfwd To: Stephen Hemminger Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000f50e1505f9b8d3fd" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000f50e1505f9b8d3fd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Stephen, Thanks for the response. I've tried with two virtual NICs, each bound to VFIO, as you can see below. verghis@verghis-VirtualBox:~/dpdk-stable-22.11.1/build$ ../usertools/dpdk-devbind.py --status Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:00:08.0 '79c970 [PCnet32 LANCE] 2000' drv=3Dvfio-pci unused=3Dpcnet32 0000:00:09.0 '79c970 [PCnet32 LANCE] 2000' drv=3Dvfio-pci unused=3Dpcnet32 Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:00:03.0 '82540EM Gigabit Ethernet Controller 100e' if=3Denp0s3 drv=3De= 1000 unused=3Dvfio-pci *Active* I'm not sure I understand what you mean by 'is the virtual NIC in VirtualBox the same as virtio. The NICs are PCnet32 LANCE, a standard NIC, bound to VFIO. In Linux Mint 21 VFIO is built into the kernel, not a loadable module. And yet the basicfwd code cannot find even 1 VFIO port. Very weird. Any ideas? Thanks. On Tue, Apr 18, 2023 at 6:47=E2=80=AFPM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Tue, 18 Apr 2023 17:14:21 -0700 > Verghis Koshi wrote: > > > It appears that the vdev_device_list is empty - isn't this where th= e > > probe function for VFIO lives? > > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, vbus->name vdev > > vdev_probe, file ../drivers/bus/vdev/vdev.c, PROBE, &vdev_device_list > > 0x56337bb30d30 > > vdev_probe, file ../drivers/bus/vdev/vdev.c, dev is (nil) > > > > This is how I call basicfwd, am I missing something? > > > > sudo examples/dpdk-skeleton -l 1 -n 4 > > > > Thanks,. > > > > Verghis > > > > > > > > > > m > > > > On Tue, Apr 18, 2023 at 10:11=E2=80=AFAM Verghis Koshi wrote: > > > > > I'm having trouble running the basicfwd example and would > appreciate > > > any help. > > > I'm running Linux Mint 21.1 inside VirtualBox, and I've created t= wo > > > NICs: > > > > > > verghis@verghis-VirtualBox:~/dpdk-stable-22.11.1/build$ > > > ../usertools/dpdk-devbind.py --status > > > > > > Network devices using DPDK-compatible driver > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > 0000:00:03.0 '79c970 [PCnet32 LANCE] 2000' drv=3Dvfio-pci unused=3Dpc= net32 > > > > > > Network devices using kernel driver > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > 0000:00:08.0 '82540EM Gigabit Ethernet Controller 100e' if=3Denp0s8 > > > drv=3De1000 unused=3Dvfio-pci *Active* > > > > > > The first is bound to vfio-pci, to be used by basicfwd, and the > second > > > uses the normal e1000 driver. > > > But when I run the code, it doesn't seem to see the VFIO driver a= t > > > all; further, it seems to think that 00:08.0 > > > is using a non-kernel driver - why? > > > Here's the debug output; it should pick up the single VFIO port. = I > > > don't care about the 'even number of ports', that's > > > easy to fix. > > > My apologies if I'm overlooking something simple. > > > > > > verghis@verghis-VirtualBox:~/dpdk-stable-22.11.1/build$ sudo > > > examples/dpdk-skeleton -l 1 -n 4 > > > EAL: Detected CPU lcores: 2 > > > EAL: Detected NUMA nodes: 1 > > > EAL: Detected static linkage of DPDK > > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > > > EAL: Selected IOVA mode 'VA' > > > EAL: VFIO support initialized > > > rte_vfio_enable, file ../lib/eal/linux/eal_vfio.c, VFIO support > initialized > > > 0: examples/dpdk-skeleton (rte_dump_stack+0x42) [55bed7d42d62] > > > 1: examples/dpdk-skeleton (55bed6cbb000+0x23b39f) [55bed6ef639f] > > > 2: examples/dpdk-skeleton (55bed6cbb000+0x239211) [55bed6ef4211] > > > 3: examples/dpdk-skeleton (main+0xf) [55bed70ac51f] > > > 4: /lib/x86_64-linux-gnu/libc.so.6 (7fbde5fb2000+0x29d90) > [7fbde5fdbd90] > > > 5: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0x80) > [7fbde5fdbe40] > > > 6: examples/dpdk-skeleton (_start+0x25) [55bed7b86055] > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name > auxiliary > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name > dpaa_bus > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name fsl= mc > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name ifp= ga > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name pci > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:01= .1 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:02= .0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:03= .0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:04= .0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:05= .0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:06= .0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:07= .0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:08= .0 > > > rte_pci_map_device, file ../drivers/bus/pci/linux/pci.c, > > > rte_pci_device->name 0000:00:08.0, dev->kdrv 0 > > > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 0000:00:0d= .0 > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name vmb= us > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->name dsa > > > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, vbus->name vd= ev > > > vdev_probe, file ../drivers/bus/vdev/vdev.c, PROBE, &vdev_device_list > > > 0x55bed8764d30 > > > vdev_probe, file ../drivers/bus/vdev/vdev.c, dev is (nil) > > > TELEMETRY: No legacy callbacks, legacy socket not created > > > main, file ../examples/skeleton/basicfwd.c, nb_ports 0 > > > EAL: Error - exiting with code: 1 > > > Cause: Error: number of ports must be even > > > > > > Verghis > > > > > You need to create two virtual nic's and bind them to VFIO which > will remove from the kernel driver. Not familiar with VirtualBox config. > Is the virtual NIC in Virtual Box is not the same as virtio. > If it depends on the proprietary kernel driver, then > you are unlikely to get DPDK to work in virtual box environment. > --000000000000f50e1505f9b8d3fd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Stephen,
=C2=A0=C2=A0=C2=A0 Thanks for t= he response.
=C2=A0=C2=A0=C2=A0 I've tried with two virtual N= ICs, each bound to VFIO, as you can see below.=C2=A0

verghis@verghis-VirtualBox:~/dpdk-stable-22.11.1/build$ ../usertool= s/dpdk-devbind.py --status

Network devices using DPDK-compatible dri= ver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
0000:= 00:08.0 '79c970 [PCnet32 LANCE] 2000' drv=3Dvfio-pci unused=3Dpcnet= 32
0000:00:09.0 '79c970 [PCnet32 LANCE] 2000' drv=3Dvfio-pci unu= sed=3Dpcnet32

Network devices using kernel driver
=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D
0000:00:03.0 '82540EM Gigabit Ethernet Controller 10= 0e' if=3Denp0s3 drv=3De1000 unused=3Dvfio-pci *Active*

=C2=A0=C2=A0=C2=A0 I'm not sure I understand what you mean by = 'is the virtual NIC in VirtualBox the same as virtio.=C2=A0 The
NICs are PCnet32 LANCE, a standard NIC, bound to VFIO.=C2=A0 In Linux Mi= nt 21 VFIO is built into the kernel,
not a loadable module.
=
=C2=A0=C2=A0=C2=A0 And yet the basicfwd code cannot find even 1 VFIO p= ort.=C2=A0 Very weird.=C2=A0 Any ideas?
=C2=A0=C2=A0=C2=A0 Thanks= .

On Tue, Apr 18, 2023 at 6:47=E2=80=AFPM Stephen Hemminger <stephen@networkplumber.org&= gt; wrote:
On Tu= e, 18 Apr 2023 17:14:21 -0700
Verghis Koshi <vko= shi@gmail.com> wrote:

>=C2=A0 =C2=A0 =C2=A0It appears that the vdev_device_list is empty - isn= 't this where the
> probe function for VFIO lives?
>
> rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, vbus->name = vdev
> vdev_probe, file ../drivers/bus/vdev/vdev.c, PROBE, &vdev_device_l= ist
> 0x56337bb30d30
> vdev_probe, file ../drivers/bus/vdev/vdev.c, dev is (nil)
>
>=C2=A0 =C2=A0 =C2=A0This is how I call basicfwd, am I missing something= ?
>
> sudo examples/dpdk-skeleton -l 1 -n 4
>
>=C2=A0 =C2=A0 =C2=A0Thanks,.
>
> Verghis
>
>
>
>
> m
>
> On Tue, Apr 18, 2023 at 10:11=E2=80=AFAM Verghis Koshi <vkoshi@gmail.com> wrote:<= br> >
> >=C2=A0 =C2=A0 =C2=A0I'm having trouble running the basicfwd ex= ample and would appreciate
> > any help.
> >=C2=A0 =C2=A0 =C2=A0I'm running Linux Mint 21.1 inside Virtual= Box, and I've created two
> > NICs:
> >
> > verghis@verghis-VirtualBox:~/dpdk-stable-22.11.1/build$
> > ../usertools/dpdk-devbind.py --status
> >
> > Network devices using DPDK-compatible driver
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > 0000:00:03.0 '79c970 [PCnet32 LANCE] 2000' drv=3Dvfio-pci= unused=3Dpcnet32
> >
> > Network devices using kernel driver
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > 0000:00:08.0 '82540EM Gigabit Ethernet Controller 100e' i= f=3Denp0s8
> > drv=3De1000 unused=3Dvfio-pci *Active*
> >
> >=C2=A0 =C2=A0 =C2=A0The first is bound to vfio-pci, to be used by = basicfwd, and the second
> > uses the normal e1000 driver.
> >=C2=A0 =C2=A0 =C2=A0But when I run the code, it doesn't seem t= o see the VFIO driver at
> > all; further, it seems to think that 00:08.0
> > is using a non-kernel driver - why?
> >=C2=A0 =C2=A0 =C2=A0Here's the debug output; it should pick up= the single VFIO port.=C2=A0 I
> > don't care about the 'even number of ports', that'= ;s
> > easy to fix.
> >=C2=A0 =C2=A0 =C2=A0My apologies if I'm overlooking something = simple.
> >
> > verghis@verghis-VirtualBox:~/dpdk-stable-22.11.1/build$ sudo
> > examples/dpdk-skeleton -l 1 -n 4
> > EAL: Detected CPU lcores: 2
> > EAL: Detected NUMA nodes: 1
> > EAL: Detected static linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: VFIO support initialized
> > rte_vfio_enable, file ../lib/eal/linux/eal_vfio.c, VFIO support i= nitialized
> > 0: examples/dpdk-skeleton (rte_dump_stack+0x42) [55bed7d42d62] > > 1: examples/dpdk-skeleton (55bed6cbb000+0x23b39f) [55bed6ef639f]<= br> > > 2: examples/dpdk-skeleton (55bed6cbb000+0x239211) [55bed6ef4211]<= br> > > 3: examples/dpdk-skeleton (main+0xf) [55bed70ac51f]
> > 4: /lib/x86_64-linux-gnu/libc.so.6 (7fbde5fb2000+0x29d90) [7fbde5= fdbd90]
> > 5: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0x80) [7fbd= e5fdbe40]
> > 6: examples/dpdk-skeleton (_start+0x25) [55bed7b86055]
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame auxiliary
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame dpaa_bus
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame fslmc
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame ifpga
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame pci
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:01.1
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:02.0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:03.0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:04.0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:05.0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:06.0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:07.0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:08.0
> > rte_pci_map_device, file ../drivers/bus/pci/linux/pci.c,
> > rte_pci_device->name 0000:00:08.0, dev->kdrv 0
> > pci_probe, file ../drivers/bus/pci/pci_common.c, dev->name 000= 0:00:0d.0
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame vmbus
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, bus->n= ame dsa
> > rte_bus_probe, file ../lib/eal/common/eal_common_bus.c, vbus->= name vdev
> > vdev_probe, file ../drivers/bus/vdev/vdev.c, PROBE, &vdev_dev= ice_list
> > 0x55bed8764d30
> > vdev_probe, file ../drivers/bus/vdev/vdev.c, dev is (nil)
> > TELEMETRY: No legacy callbacks, legacy socket not created
> > main, file ../examples/skeleton/basicfwd.c, nb_ports 0
> > EAL: Error - exiting with code: 1
> >=C2=A0 =C2=A0Cause: Error: number of ports must be even
> >
> > Verghis
> >=C2=A0

You need to create two virtual nic's and bind them to VFIO which
will remove from the kernel driver. Not familiar with VirtualBox config. Is the virtual NIC in Virtual Box is not the same as virtio.
If it depends on the proprietary kernel driver, then
you are unlikely to get DPDK to work in virtual box environment.
--000000000000f50e1505f9b8d3fd--