Re: skeleton code failing (Lokesh Chakka)

Re: skeleton code failing (Lokesh Chakka)

[Varghese, Vipin]

[AMD Official Use Only - General]

Based on the compilation command shared it looks like you are using the build in shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk --libs libdpdk`
Hence in EAL PCIe probe, the BNXT PMD is not triggered to identify the NIC. 

Solutions for these can be
1. Build with static library ` gcc main.c -g `pkg-config --cflags libdpdk --libs --static libdpdk`
2. pass the bxnt_en PMD shared library to eal args by ` sudo ./a.out -l 1 -d librte_net_bnxt.so`

Can you try any of the above ?

> -----Original Message-----
> From: users-request@dpdk.org <users-request@dpdk.org>
> Sent: Wednesday, July 13, 2022 3:30 PM
> To: users@dpdk.org
> Subject: users Digest, Vol 347, Issue 6
> 
> [CAUTION: External Email]
> 
> Send users mailing list submissions to
>         users@dpdk.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp
> dk.org%2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40amd.co
> m%7C55465d0d4faa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d9
> 94e183d%7C0%7C0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8
> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> %7C3000%7C%7C%7C&amp;sdata=6JHjf2Fu%2BkQ56CF9xTZvBS%2Bh8Uomlqv
> E9dq9M7VefiU%3D&amp;reserved=0
> or, via email, send a message with subject or body 'help' to
>         users-request@dpdk.org
> 
> You can reach the person managing the list at
>         users-owner@dpdk.org
> 
> When replying, please edit your Subject line so it is more specific than "Re:
> Contents of users digest..."
> 
> 
> Today's Topics:
> 
>    1. Re: skeleton code failing (Lokesh Chakka)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 13 Jul 2022 12:56:37 +0530
> From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> To: David Marchand <david.marchand@redhat.com>
> Cc: users <users@dpdk.org>
> Subject: Re: skeleton code failing
> Message-ID:
>         <CACh--sgKJ9B2=Nd-EUUx4D7Kz87-
> Nm_pwq59fY9QVNH+GBgwRg@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Dear David,
> 
> following is some more stuff i did
> ==================================================
> $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b vfio-pci
> 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0 83:00.1
> Error: Driver 'uio_pci_generic' is not loaded.
> $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> Error: Driver 'igb_uio' is not loaded.
> $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> ==================================================
> ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags libdpdk
> --libs libdpdk` lokesh@lokesh-ProLiant-DL385-
> Gen10:~/Desktop/dpdk_examples/skeleton$ sudo ./a.out
> EAL: Detected CPU lcores: 64
> EAL: Detected NUMA nodes: 4
> EAL: Detected shared linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available 1048576 kB hugepages reported
> EAL: VFIO support initialized
> EAL: Using IOMMU type 1 (Type 1)
> EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0 (socket 2)
> EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1 (socket 2)
> TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC: bc 97
> e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
> 
> WARNING: Too many lcores enabled. Only 1 used.
> WARNING, port 0 is on remote NUMA node to polling thread.
> Performance will not be optimal.
> WARNING, port 1 is on remote NUMA node to polling thread.
> Performance will not be optimal.
> 
> Core 0 forwarding packets. [Ctrl+C to quit] ^C
> ==================================================
> After a few seconds, I presses ctrl+c
> 
> surprisingly cards are not showing up even in ifconfig.
> 
> 
> 
> Thanks & Regards
> --
> Lokesh Chakka.
> 
> 
> On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> lvenkatakumarchakka@gmail.com> wrote:
> 
> > Dear David,
> >
> > =================================================
> > $ lspci | grep -i broadcom
> > 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> > 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $ lspci
> > -n -s 83:00.0
> > 83:00.0 0200: 14e4:1750 (rev 11)
> > =================================================
> >
> > I am compiling my code like this :
> > =================================================
> > gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> > =================================================
> >
> > Hence it is statically linked code.
> > If I try
> > $ dpdk-pmdinfo.py ./a.out
> >
> > But I am not seeing any output
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:22 PM David Marchand
> > <david.marchand@redhat.com>
> > wrote:
> >
> >> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> >> <lvenkatakumarchakka@gmail.com> wrote:
> >> > Would like to understand if I am missing something. I am new to
> >> > this
> >> platform.
> >> > rte_eth_dev_count_avail is returning zero.
> >> > OS is Ubuntu 22.04. DPDK is latest version.
> >> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> >> LED is also glowing.
> >>
> >> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> >> (which is net/bnxt).
> >> This userspace driver does not rely on the bnxt Linux kernel driver.
> >> IOW, this card being detecting and working with the Linux kernel does
> >> not automatically mean that this nic can work with DPDK.
> >>
> >> We need more info on your nic, first.
> >>
> >> Can you share the pci id of this nic (like running lspci -n -s
> >> $pci_address)?
> >> It should be a 14e4:XXXX.
> >>
> >> Then you can check this XXXX against what your dpdk application supports.
> >>
> >> If it is a statically linked application, you can run:
> >> $ dpdk-pmdinfo.py /path/to/your/application
> >>
> >> Else, if your application is dynamically linked against DPDK driver,
> >> you can run this command against the net/bnxt driver .so.22 (for
> >> 21.11 and later releases):
> >> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> >>
> >> You should get a list of supported NetXtreme nics, like:
> >>
> >> [snipped some other drivers compiled in my application] PMD NAME:
> >> net_bnxt PMD HW SUPPORT:
> >>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> >> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> >> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> >> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> >> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614) (All
> >> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E RDMA
> >> Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> >> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> >> (All Subdevices) [snipped the rest]
> >>
> >> I hope you can find a (XXXX) corresponding to your NIC.
> >>
> >>
> >> --
> >> David Marchand
> >>
> >>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp
> dk.org%2Farchives%2Fusers%2Fattachments%2F20220713%2Fa362f7fe%2Fatta
> chment-
> 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com%7C55465d0d4f
> aa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C
> 0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> %7C&amp;sdata=jXnaf1qrCA5VSCMyrPPc5S5WYsHveR1tNBL4lczkGVE%3D&am
> p;reserved=0>
> 
> End of users Digest, Vol 347, Issue 6
> *************************************

Re: skeleton code failing (Lokesh Chakka)

[Lokesh Chakka]

[-- Attachment #1: Type: text/plain, Size: 8867 bytes --]

I have one more observation here.
After "$ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1"

Cards are shutting down. ifconfig is not showing the cards. reinsert of sfp
is also not bringing the cards up. I had to restart the server. Not sure
why the cards are going down.

Thanks & Regards
--
Lokesh Chakka.


On Thu, Jul 14, 2022 at 8:50 AM Varghese, Vipin <Vipin.Varghese@amd.com>
wrote:

> [AMD Official Use Only - General]
>
> Based on the compilation command shared it looks like you are using the
> build in shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk
> --libs libdpdk`
> Hence in EAL PCIe probe, the BNXT PMD is not triggered to identify the
> NIC.
>
> Solutions for these can be
> 1. Build with static library ` gcc main.c -g `pkg-config --cflags libdpdk
> --libs --static libdpdk`
> 2. pass the bxnt_en PMD shared library to eal args by ` sudo ./a.out -l 1
> -d librte_net_bnxt.so`
>
> Can you try any of the above ?
>
> > -----Original Message-----
> > From: users-request@dpdk.org <users-request@dpdk.org>
> > Sent: Wednesday, July 13, 2022 3:30 PM
> > To: users@dpdk.org
> > Subject: users Digest, Vol 347, Issue 6
> >
> > [CAUTION: External Email]
> >
> > Send users mailing list submissions to
> >         users@dpdk.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp
> > dk.org%2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40amd.co
> > m%7C55465d0d4faa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d9
> > 94e183d%7C0%7C0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8
> > eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > %7C3000%7C%7C%7C&amp;sdata=6JHjf2Fu%2BkQ56CF9xTZvBS%2Bh8Uomlqv
> > E9dq9M7VefiU%3D&amp;reserved=0
> > or, via email, send a message with subject or body 'help' to
> >         users-request@dpdk.org
> >
> > You can reach the person managing the list at
> >         users-owner@dpdk.org
> >
> > When replying, please edit your Subject line so it is more specific than
> "Re:
> > Contents of users digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Re: skeleton code failing (Lokesh Chakka)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 13 Jul 2022 12:56:37 +0530
> > From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> > To: David Marchand <david.marchand@redhat.com>
> > Cc: users <users@dpdk.org>
> > Subject: Re: skeleton code failing
> > Message-ID:
> >         <CACh--sgKJ9B2=Nd-EUUx4D7Kz87-
> > Nm_pwq59fY9QVNH+GBgwRg@mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Dear David,
> >
> > following is some more stuff i did
> > ==================================================
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b
> vfio-pci
> > 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0 83:00.1
> > Error: Driver 'uio_pci_generic' is not loaded.
> > $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> > Error: Driver 'igb_uio' is not loaded.
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> > Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> > Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> > ==================================================
> > ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags
> libdpdk
> > --libs libdpdk` lokesh@lokesh-ProLiant-DL385-
> > Gen10:~/Desktop/dpdk_examples/skeleton$ sudo ./a.out
> > EAL: Detected CPU lcores: 64
> > EAL: Detected NUMA nodes: 4
> > EAL: Detected shared linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: No available 1048576 kB hugepages reported
> > EAL: VFIO support initialized
> > EAL: Using IOMMU type 1 (Type 1)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0 (socket
> 2)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1 (socket
> 2)
> > TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC: bc
> 97
> > e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
> >
> > WARNING: Too many lcores enabled. Only 1 used.
> > WARNING, port 0 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> > WARNING, port 1 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> >
> > Core 0 forwarding packets. [Ctrl+C to quit] ^C
> > ==================================================
> > After a few seconds, I presses ctrl+c
> >
> > surprisingly cards are not showing up even in ifconfig.
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> > lvenkatakumarchakka@gmail.com> wrote:
> >
> > > Dear David,
> > >
> > > =================================================
> > > $ lspci | grep -i broadcom
> > > 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> > > 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $ lspci
> > > -n -s 83:00.0
> > > 83:00.0 0200: 14e4:1750 (rev 11)
> > > =================================================
> > >
> > > I am compiling my code like this :
> > > =================================================
> > > gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> > > =================================================
> > >
> > > Hence it is statically linked code.
> > > If I try
> > > $ dpdk-pmdinfo.py ./a.out
> > >
> > > But I am not seeing any output
> > >
> > >
> > >
> > > Thanks & Regards
> > > --
> > > Lokesh Chakka.
> > >
> > >
> > > On Wed, Jul 13, 2022 at 12:22 PM David Marchand
> > > <david.marchand@redhat.com>
> > > wrote:
> > >
> > >> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> > >> <lvenkatakumarchakka@gmail.com> wrote:
> > >> > Would like to understand if I am missing something. I am new to
> > >> > this
> > >> platform.
> > >> > rte_eth_dev_count_avail is returning zero.
> > >> > OS is Ubuntu 22.04. DPDK is latest version.
> > >> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> > >> LED is also glowing.
> > >>
> > >> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> > >> (which is net/bnxt).
> > >> This userspace driver does not rely on the bnxt Linux kernel driver.
> > >> IOW, this card being detecting and working with the Linux kernel does
> > >> not automatically mean that this nic can work with DPDK.
> > >>
> > >> We need more info on your nic, first.
> > >>
> > >> Can you share the pci id of this nic (like running lspci -n -s
> > >> $pci_address)?
> > >> It should be a 14e4:XXXX.
> > >>
> > >> Then you can check this XXXX against what your dpdk application
> supports.
> > >>
> > >> If it is a statically linked application, you can run:
> > >> $ dpdk-pmdinfo.py /path/to/your/application
> > >>
> > >> Else, if your application is dynamically linked against DPDK driver,
> > >> you can run this command against the net/bnxt driver .so.22 (for
> > >> 21.11 and later releases):
> > >> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> > >>
> > >> You should get a list of supported NetXtreme nics, like:
> > >>
> > >> [snipped some other drivers compiled in my application] PMD NAME:
> > >> net_bnxt PMD HW SUPPORT:
> > >>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> > >> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> > >> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> > >> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> > >> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614) (All
> > >> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E RDMA
> > >> Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> > >> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> > >> (All Subdevices) [snipped the rest]
> > >>
> > >> I hope you can find a (XXXX) corresponding to your NIC.
> > >>
> > >>
> > >> --
> > >> David Marchand
> > >>
> > >>
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > <
> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp
> > dk.org%2Farchives%2Fusers%2Fattachments%2F20220713%2Fa362f7fe%2Fatta
> > chment-
> > 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com%7C55465d0d4f
> > aa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C
> > 0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> > wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> > %7C&amp;sdata=jXnaf1qrCA5VSCMyrPPc5S5WYsHveR1tNBL4lczkGVE%3D&am
> > p;reserved=0>
> >
> > End of users Digest, Vol 347, Issue 6
> > *************************************
>

[-- Attachment #2: Type: text/html, Size: 12614 bytes --]

RE: skeleton code failing (Lokesh Chakka)

[Varghese, Vipin]

[-- Attachment #1: Type: text/plain, Size: 12266 bytes --]

[AMD Official Use Only - General]

Is this not Broadcom extreme net card? Please refer http://doc.dpdk.org/guides/nics/bnxt.html, once you bind with `igb_uio, vfio_pci or uio_pci_generic` all linux netdevice instance will be no longer available. This is because it is not a `port representator`

Observation: due some hardware or firmware bug auto-neg take more time with the current DPDK BXNT PMD. Hence my advice is try to wait longer for link state up in DPDK.
You can verify the same with testpmd.

From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
Sent: Thursday, July 14, 2022 11:11 AM
To: Varghese, Vipin <Vipin.Varghese@amd.com>
Cc: users@dpdk.org; Yigit, Ferruh <Ferruh.Yigit@amd.com>; Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>
Subject: Re: skeleton code failing (Lokesh Chakka)

[CAUTION: External Email]
I have one more observation here.
After "$ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1"

Cards are shutting down. ifconfig is not showing the cards. reinsert of sfp is also not bringing the cards up. I had to restart the server. Not sure why the cards are going down.

Thanks & Regards
--
Lokesh Chakka.


On Thu, Jul 14, 2022 at 8:50 AM Varghese, Vipin <Vipin.Varghese@amd.com<mailto:Vipin.Varghese@amd.com>> wrote:
[AMD Official Use Only - General]

Based on the compilation command shared it looks like you are using the build in shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk --libs libdpdk`
Hence in EAL PCIe probe, the BNXT PMD is not triggered to identify the NIC.

Solutions for these can be
1. Build with static library ` gcc main.c -g `pkg-config --cflags libdpdk --libs --static libdpdk`
2. pass the bxnt_en PMD shared library to eal args by ` sudo ./a.out -l 1 -d librte_net_bnxt.so`

Can you try any of the above ?

> -----Original Message-----
> From: users-request@dpdk.org<mailto:users-request@dpdk.org> <users-request@dpdk.org<mailto:users-request@dpdk.org>>
> Sent: Wednesday, July 13, 2022 3:30 PM
> To: users@dpdk.org<mailto:users@dpdk.org>
> Subject: users Digest, Vol 347, Issue 6
>
> [CAUTION: External Email]
>
> Send users mailing list submissions to
>         users@dpdk.org<mailto:users@dpdk.org>
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kEjFmF8PaxwPyIXKcpB1B9VJjuLzzFqB5zl3Ahmd%2BAs%3D&reserved=0>
> dk.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0zrwvV24mjHlR9qLiLsL9cUHePu%2B4l2mr9iTfFYQRyI%3D&reserved=0>%2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40amd.co<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.co%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rlZbwJ5OkgTUEgS2pPROwbYV9VhtuQDxSgrOrRIP9FQ%3D&reserved=0>
> m%7C55465d0d4faa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d9
> 94e183d%7C0%7C0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8
> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> %7C3000%7C%7C%7C&amp;sdata=6JHjf2Fu%2BkQ56CF9xTZvBS%2Bh8Uomlqv
> E9dq9M7VefiU%3D&amp;reserved=0
> or, via email, send a message with subject or body 'help' to
>         users-request@dpdk.org<mailto:users-request@dpdk.org>
>
> You can reach the person managing the list at
>         users-owner@dpdk.org<mailto:users-owner@dpdk.org>
>
> When replying, please edit your Subject line so it is more specific than "Re:
> Contents of users digest..."
>
>
> Today's Topics:
>
>    1. Re: skeleton code failing (Lokesh Chakka)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 13 Jul 2022 12:56:37 +0530
> From: Lokesh Chakka <lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>>
> To: David Marchand <david.marchand@redhat.com<mailto:david.marchand@redhat.com>>
> Cc: users <users@dpdk.org<mailto:users@dpdk.org>>
> Subject: Re: skeleton code failing
> Message-ID:
>         <CACh--sgKJ9B2=Nd-EUUx4D7Kz87-
> Nm_pwq59fY9QVNH+GBgwRg@mail.gmail.com<mailto:Nm_pwq59fY9QVNH%2BGBgwRg@mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Dear David,
>
> following is some more stuff i did
> ==================================================
> $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b vfio-pci
> 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0 83:00.1
> Error: Driver 'uio_pci_generic' is not loaded.
> $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> Error: Driver 'igb_uio' is not loaded.
> $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> ==================================================
> ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags libdpdk
> --libs libdpdk` lokesh@lokesh-ProLiant-DL385-
> Gen10:~/Desktop/dpdk_examples/skeleton$ sudo ./a.out
> EAL: Detected CPU lcores: 64
> EAL: Detected NUMA nodes: 4
> EAL: Detected shared linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available 1048576 kB hugepages reported
> EAL: VFIO support initialized
> EAL: Using IOMMU type 1 (Type 1)
> EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0 (socket 2)
> EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1 (socket 2)
> TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC: bc 97
> e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
>
> WARNING: Too many lcores enabled. Only 1 used.
> WARNING, port 0 is on remote NUMA node to polling thread.
> Performance will not be optimal.
> WARNING, port 1 is on remote NUMA node to polling thread.
> Performance will not be optimal.
>
> Core 0 forwarding packets. [Ctrl+C to quit] ^C
> ==================================================
> After a few seconds, I presses ctrl+c
>
> surprisingly cards are not showing up even in ifconfig.
>
>
>
> Thanks & Regards
> --
> Lokesh Chakka.
>
>
> On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>> wrote:
>
> > Dear David,
> >
> > =================================================
> > $ lspci | grep -i broadcom
> > 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> > 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $ lspci
> > -n -s 83:00.0
> > 83:00.0 0200: 14e4:1750 (rev 11)
> > =================================================
> >
> > I am compiling my code like this :
> > =================================================
> > gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> > =================================================
> >
> > Hence it is statically linked code.
> > If I try
> > $ dpdk-pmdinfo.py ./a.out
> >
> > But I am not seeing any output
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:22 PM David Marchand
> > <david.marchand@redhat.com<mailto:david.marchand@redhat.com>>
> > wrote:
> >
> >> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> >> <lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>> wrote:
> >> > Would like to understand if I am missing something. I am new to
> >> > this
> >> platform.
> >> > rte_eth_dev_count_avail is returning zero.
> >> > OS is Ubuntu 22.04. DPDK is latest version.
> >> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> >> LED is also glowing.
> >>
> >> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> >> (which is net/bnxt).
> >> This userspace driver does not rely on the bnxt Linux kernel driver.
> >> IOW, this card being detecting and working with the Linux kernel does
> >> not automatically mean that this nic can work with DPDK.
> >>
> >> We need more info on your nic, first.
> >>
> >> Can you share the pci id of this nic (like running lspci -n -s
> >> $pci_address)?
> >> It should be a 14e4:XXXX.
> >>
> >> Then you can check this XXXX against what your dpdk application supports.
> >>
> >> If it is a statically linked application, you can run:
> >> $ dpdk-pmdinfo.py /path/to/your/application
> >>
> >> Else, if your application is dynamically linked against DPDK driver,
> >> you can run this command against the net/bnxt driver .so.22 (for
> >> 21.11 and later releases):
> >> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> >>
> >> You should get a list of supported NetXtreme nics, like:
> >>
> >> [snipped some other drivers compiled in my application] PMD NAME:
> >> net_bnxt PMD HW SUPPORT:
> >>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> >> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> >> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> >> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> >> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614) (All
> >> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E RDMA
> >> Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> >> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> >> (All Subdevices) [snipped the rest]
> >>
> >> I hope you can find a (XXXX) corresponding to your NIC.
> >>
> >>
> >> --
> >> David Marchand
> >>
> >>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HG9oukk%2Ffe%2BhaU9PtfMd8CVeKo0iWtwCFtHkDPDZmvg%3D&reserved=0>
> dk.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0zrwvV24mjHlR9qLiLsL9cUHePu%2B4l2mr9iTfFYQRyI%3D&reserved=0>%2Farchives%2Fusers%2Fattachments%2F20220713%2Fa362f7fe%2Fatta
> chment-
> 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.com%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZJMzN1nKLSMSfRM2KMLnoj2s%2BFLS8wjvePPESwkpO5c%3D&reserved=0>%7C55465d0d4f
> aa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C
> 0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> %7C&amp;sdata=jXnaf1qrCA5VSCMyrPPc5S5WYsHveR1tNBL4lczkGVE%3D&am
> p;reserved=0>
>
> End of users Digest, Vol 347, Issue 6
> *************************************

[-- Attachment #2: Type: text/html, Size: 18540 bytes --]

Re: skeleton code failing (Lokesh Chakka)

[Lokesh Chakka]

[-- Attachment #1: Type: text/plain, Size: 13068 bytes --]

The card I am having is
https://www.broadcom.com/products/ethernet-connectivity/network-adapters/p2100g
However, cards are up after waiting for around 10 minutes.

Thanks for the valuable input. skeleton code is running till I press ctrl+c.
Now I have a big concern.
If netdevice is not available, how can I capture the packets using
wireshark?
Interfaces are not visible to the wireshark. Very basic requirement is to
send the packets, capture them and see the contents.


Thanks & Regards
--
Lokesh Chakka.


On Thu, Jul 14, 2022 at 11:46 AM Varghese, Vipin <Vipin.Varghese@amd.com>
wrote:

> [AMD Official Use Only - General]
>
>
>
> Is this not Broadcom extreme net card? Please refer
> http://doc.dpdk.org/guides/nics/bnxt.html, once you bind with `igb_uio,
> vfio_pci or uio_pci_generic` all linux netdevice instance will be no longer
> available. This is because it is not a `port representator`
>
>
>
> Observation: due some hardware or firmware bug auto-neg take more time
> with the current DPDK BXNT PMD. Hence my advice is try to wait longer for
> link state up in DPDK.
>
> You can verify the same with testpmd.
>
>
>
> *From:* Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> *Sent:* Thursday, July 14, 2022 11:11 AM
> *To:* Varghese, Vipin <Vipin.Varghese@amd.com>
> *Cc:* users@dpdk.org; Yigit, Ferruh <Ferruh.Yigit@amd.com>; Tummala,
> Sivaprasad <Sivaprasad.Tummala@amd.com>
> *Subject:* Re: skeleton code failing (Lokesh Chakka)
>
>
>
> [CAUTION: External Email]
>
> I have one more observation here.
>
> After "$ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1"
>
>
>
> Cards are shutting down. ifconfig is not showing the cards. reinsert of
> sfp is also not bringing the cards up. I had to restart the server. Not
> sure why the cards are going down.
>
>
>
> Thanks & Regards
> --
> Lokesh Chakka.
>
>
>
>
>
> On Thu, Jul 14, 2022 at 8:50 AM Varghese, Vipin <Vipin.Varghese@amd.com>
> wrote:
>
> [AMD Official Use Only - General]
>
> Based on the compilation command shared it looks like you are using the
> build in shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk
> --libs libdpdk`
> Hence in EAL PCIe probe, the BNXT PMD is not triggered to identify the
> NIC.
>
> Solutions for these can be
> 1. Build with static library ` gcc main.c -g `pkg-config --cflags libdpdk
> --libs --static libdpdk`
> 2. pass the bxnt_en PMD shared library to eal args by ` sudo ./a.out -l 1
> -d librte_net_bnxt.so`
>
> Can you try any of the above ?
>
> > -----Original Message-----
> > From: users-request@dpdk.org <users-request@dpdk.org>
> > Sent: Wednesday, July 13, 2022 3:30 PM
> > To: users@dpdk.org
> > Subject: users Digest, Vol 347, Issue 6
> >
> > [CAUTION: External Email]
> >
> > Send users mailing list submissions to
> >         users@dpdk.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kEjFmF8PaxwPyIXKcpB1B9VJjuLzzFqB5zl3Ahmd%2BAs%3D&reserved=0>
> > dk.org
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0zrwvV24mjHlR9qLiLsL9cUHePu%2B4l2mr9iTfFYQRyI%3D&reserved=0>
> %2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40amd.co
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.co%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rlZbwJ5OkgTUEgS2pPROwbYV9VhtuQDxSgrOrRIP9FQ%3D&reserved=0>
> > m%7C55465d0d4faa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d9
> > 94e183d%7C0%7C0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8
> > eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > %7C3000%7C%7C%7C&amp;sdata=6JHjf2Fu%2BkQ56CF9xTZvBS%2Bh8Uomlqv
> > E9dq9M7VefiU%3D&amp;reserved=0
> > or, via email, send a message with subject or body 'help' to
> >         users-request@dpdk.org
> >
> > You can reach the person managing the list at
> >         users-owner@dpdk.org
> >
> > When replying, please edit your Subject line so it is more specific than
> "Re:
> > Contents of users digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Re: skeleton code failing (Lokesh Chakka)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 13 Jul 2022 12:56:37 +0530
> > From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> > To: David Marchand <david.marchand@redhat.com>
> > Cc: users <users@dpdk.org>
> > Subject: Re: skeleton code failing
> > Message-ID:
> >         <CACh--sgKJ9B2=Nd-EUUx4D7Kz87-
> > Nm_pwq59fY9QVNH+GBgwRg@mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Dear David,
> >
> > following is some more stuff i did
> > ==================================================
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b
> vfio-pci
> > 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0 83:00.1
> > Error: Driver 'uio_pci_generic' is not loaded.
> > $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> > Error: Driver 'igb_uio' is not loaded.
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> > Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> > Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> > ==================================================
> > ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags
> libdpdk
> > --libs libdpdk` lokesh@lokesh-ProLiant-DL385-
> > Gen10:~/Desktop/dpdk_examples/skeleton$ sudo ./a.out
> > EAL: Detected CPU lcores: 64
> > EAL: Detected NUMA nodes: 4
> > EAL: Detected shared linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: No available 1048576 kB hugepages reported
> > EAL: VFIO support initialized
> > EAL: Using IOMMU type 1 (Type 1)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0 (socket
> 2)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1 (socket
> 2)
> > TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC: bc
> 97
> > e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
> >
> > WARNING: Too many lcores enabled. Only 1 used.
> > WARNING, port 0 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> > WARNING, port 1 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> >
> > Core 0 forwarding packets. [Ctrl+C to quit] ^C
> > ==================================================
> > After a few seconds, I presses ctrl+c
> >
> > surprisingly cards are not showing up even in ifconfig.
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> > lvenkatakumarchakka@gmail.com> wrote:
> >
> > > Dear David,
> > >
> > > =================================================
> > > $ lspci | grep -i broadcom
> > > 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> > > 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $ lspci
> > > -n -s 83:00.0
> > > 83:00.0 0200: 14e4:1750 (rev 11)
> > > =================================================
> > >
> > > I am compiling my code like this :
> > > =================================================
> > > gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> > > =================================================
> > >
> > > Hence it is statically linked code.
> > > If I try
> > > $ dpdk-pmdinfo.py ./a.out
> > >
> > > But I am not seeing any output
> > >
> > >
> > >
> > > Thanks & Regards
> > > --
> > > Lokesh Chakka.
> > >
> > >
> > > On Wed, Jul 13, 2022 at 12:22 PM David Marchand
> > > <david.marchand@redhat.com>
> > > wrote:
> > >
> > >> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> > >> <lvenkatakumarchakka@gmail.com> wrote:
> > >> > Would like to understand if I am missing something. I am new to
> > >> > this
> > >> platform.
> > >> > rte_eth_dev_count_avail is returning zero.
> > >> > OS is Ubuntu 22.04. DPDK is latest version.
> > >> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> > >> LED is also glowing.
> > >>
> > >> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> > >> (which is net/bnxt).
> > >> This userspace driver does not rely on the bnxt Linux kernel driver.
> > >> IOW, this card being detecting and working with the Linux kernel does
> > >> not automatically mean that this nic can work with DPDK.
> > >>
> > >> We need more info on your nic, first.
> > >>
> > >> Can you share the pci id of this nic (like running lspci -n -s
> > >> $pci_address)?
> > >> It should be a 14e4:XXXX.
> > >>
> > >> Then you can check this XXXX against what your dpdk application
> supports.
> > >>
> > >> If it is a statically linked application, you can run:
> > >> $ dpdk-pmdinfo.py /path/to/your/application
> > >>
> > >> Else, if your application is dynamically linked against DPDK driver,
> > >> you can run this command against the net/bnxt driver .so.22 (for
> > >> 21.11 and later releases):
> > >> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> > >>
> > >> You should get a list of supported NetXtreme nics, like:
> > >>
> > >> [snipped some other drivers compiled in my application] PMD NAME:
> > >> net_bnxt PMD HW SUPPORT:
> > >>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> > >> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> > >> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> > >> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> > >> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614) (All
> > >> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E RDMA
> > >> Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> > >> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> > >> (All Subdevices) [snipped the rest]
> > >>
> > >> I hope you can find a (XXXX) corresponding to your NIC.
> > >>
> > >>
> > >> --
> > >> David Marchand
> > >>
> > >>
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > <
> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HG9oukk%2Ffe%2BhaU9PtfMd8CVeKo0iWtwCFtHkDPDZmvg%3D&reserved=0>
> > dk.org
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0zrwvV24mjHlR9qLiLsL9cUHePu%2B4l2mr9iTfFYQRyI%3D&reserved=0>
> %2Farchives%2Fusers%2Fattachments%2F20220713%2Fa362f7fe%2Fatta
> > chment-
> > 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.com%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C4b323bb32c214497ef5e08da655b7693%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933740761659125%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZJMzN1nKLSMSfRM2KMLnoj2s%2BFLS8wjvePPESwkpO5c%3D&reserved=0>
> %7C55465d0d4f
> > aa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C
> > 0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> > wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> > %7C&amp;sdata=jXnaf1qrCA5VSCMyrPPc5S5WYsHveR1tNBL4lczkGVE%3D&am
> > p;reserved=0>
> >
> > End of users Digest, Vol 347, Issue 6
> > *************************************
>
>

[-- Attachment #2: Type: text/html, Size: 18821 bytes --]

RE: skeleton code failing (Lokesh Chakka)

[Varghese, Vipin]

[-- Attachment #1: Type: text/plain, Size: 15723 bytes --]

[Public]

[LC] I am having is https://www.broadcom.com/products/ethernet-connectivity/network-adapters/p2100g<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.broadcom.com%2Fproducts%2Fethernet-connectivity%2Fnetwork-adapters%2Fp2100g&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GInrRYkg8qTsJ02CbjPSl%2FLT6C4cJ4Yt2fMkNfGmFho%3D&reserved=0> However, cards are up after waiting for around 10 minutes.

[VV] I do face similar issue, hence I have recommended ` Observation: due some hardware or firmware bug auto-neg take more time with the current DPDK BXNT PMD. Hence my advice is try to wait longer for link state up in DPDK.`. Please try reaching out to Broadcom PMD maintainer `ajit.khaparde@broadcom.com`. Once the solution is identified please share too


[LC] If netdevice is not available, how can I capture the packets using wireshark?
[VV] Please explore DPDK tool PDUMP. https://doc.dpdk.org/guides/tools/pdump.html

[LC] Interfaces are not visible to the wireshark. Very basic requirement is to send the packets, capture them and see the contents.
[VV] If you want the kernel netdev visible and use it under DPDK, I recommend using LIBPCAP PMD by sacrificing performance and higher functionality.

From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
Sent: Thursday, July 14, 2022 12:56 PM
To: Varghese, Vipin <Vipin.Varghese@amd.com>
Cc: users@dpdk.org; Yigit, Ferruh <Ferruh.Yigit@amd.com>; Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>
Subject: Re: skeleton code failing (Lokesh Chakka)

[CAUTION: External Email]

The card I am having is https://www.broadcom.com/products/ethernet-connectivity/network-adapters/p2100g<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.broadcom.com%2Fproducts%2Fethernet-connectivity%2Fnetwork-adapters%2Fp2100g&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GInrRYkg8qTsJ02CbjPSl%2FLT6C4cJ4Yt2fMkNfGmFho%3D&reserved=0>
However, cards are up after waiting for around 10 minutes.

Thanks for the valuable input. skeleton code is running till I press ctrl+c.
Now I have a big concern.
If netdevice is not available, how can I capture the packets using wireshark?
Interfaces are not visible to the wireshark. Very basic requirement is to send the packets, capture them and see the contents.


Thanks & Regards
--
Lokesh Chakka.


On Thu, Jul 14, 2022 at 11:46 AM Varghese, Vipin <Vipin.Varghese@amd.com<mailto:Vipin.Varghese@amd.com>> wrote:

[AMD Official Use Only - General]

Is this not Broadcom extreme net card? Please refer http://doc.dpdk.org/guides/nics/bnxt.html<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fbnxt.html&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qsqUSFyPvnm%2Frqqy5RkOfMu2WREnXXNPAoQNrBb09ig%3D&reserved=0>, once you bind with `igb_uio, vfio_pci or uio_pci_generic` all linux netdevice instance will be no longer available. This is because it is not a `port representator`

Observation: due some hardware or firmware bug auto-neg take more time with the current DPDK BXNT PMD. Hence my advice is try to wait longer for link state up in DPDK.
You can verify the same with testpmd.

From: Lokesh Chakka <lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>>
Sent: Thursday, July 14, 2022 11:11 AM
To: Varghese, Vipin <Vipin.Varghese@amd.com<mailto:Vipin.Varghese@amd.com>>
Cc: users@dpdk.org<mailto:users@dpdk.org>; Yigit, Ferruh <Ferruh.Yigit@amd.com<mailto:Ferruh.Yigit@amd.com>>; Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com<mailto:Sivaprasad.Tummala@amd.com>>
Subject: Re: skeleton code failing (Lokesh Chakka)

[CAUTION: External Email]
I have one more observation here.
After "$ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1"

Cards are shutting down. ifconfig is not showing the cards. reinsert of sfp is also not bringing the cards up. I had to restart the server. Not sure why the cards are going down.

Thanks & Regards
--
Lokesh Chakka.


On Thu, Jul 14, 2022 at 8:50 AM Varghese, Vipin <Vipin.Varghese@amd.com<mailto:Vipin.Varghese@amd.com>> wrote:
[AMD Official Use Only - General]

Based on the compilation command shared it looks like you are using the build in shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk --libs libdpdk`
Hence in EAL PCIe probe, the BNXT PMD is not triggered to identify the NIC.

Solutions for these can be
1. Build with static library ` gcc main.c -g `pkg-config --cflags libdpdk --libs --static libdpdk`
2. pass the bxnt_en PMD shared library to eal args by ` sudo ./a.out -l 1 -d librte_net_bnxt.so`

Can you try any of the above ?

> -----Original Message-----
> From: users-request@dpdk.org<mailto:users-request@dpdk.org> <users-request@dpdk.org<mailto:users-request@dpdk.org>>
> Sent: Wednesday, July 13, 2022 3:30 PM
> To: users@dpdk.org<mailto:users@dpdk.org>
> Subject: users Digest, Vol 347, Issue 6
>
> [CAUTION: External Email]
>
> Send users mailing list submissions to
>         users@dpdk.org<mailto:users@dpdk.org>
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sRwPMJBmusuk53CQJeL5Ul%2BKaJq1XC3PjqtOCkSC5vY%3D&reserved=0>
> dk.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=p7tLWYu7vQDf%2Bpi%2FYxUPM1pu7HfWunI14n2XQqPAmAQ%3D&reserved=0>%2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40amd.co<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.co%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J8sevycMs8Nd1uTlw0adu5Pt5R3ikhOitLYWGfT9kEQ%3D&reserved=0>
> m%7C55465d0d4faa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d9
> 94e183d%7C0%7C0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8
> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> %7C3000%7C%7C%7C&amp;sdata=6JHjf2Fu%2BkQ56CF9xTZvBS%2Bh8Uomlqv
> E9dq9M7VefiU%3D&amp;reserved=0
> or, via email, send a message with subject or body 'help' to
>         users-request@dpdk.org<mailto:users-request@dpdk.org>
>
> You can reach the person managing the list at
>         users-owner@dpdk.org<mailto:users-owner@dpdk.org>
>
> When replying, please edit your Subject line so it is more specific than "Re:
> Contents of users digest..."
>
>
> Today's Topics:
>
>    1. Re: skeleton code failing (Lokesh Chakka)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 13 Jul 2022 12:56:37 +0530
> From: Lokesh Chakka <lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>>
> To: David Marchand <david.marchand@redhat.com<mailto:david.marchand@redhat.com>>
> Cc: users <users@dpdk.org<mailto:users@dpdk.org>>
> Subject: Re: skeleton code failing
> Message-ID:
>         <CACh--sgKJ9B2=Nd-EUUx4D7Kz87-
> Nm_pwq59fY9QVNH+GBgwRg@mail.gmail.com<mailto:Nm_pwq59fY9QVNH%2BGBgwRg@mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Dear David,
>
> following is some more stuff i did
> ==================================================
> $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b vfio-pci
> 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0 83:00.1
> Error: Driver 'uio_pci_generic' is not loaded.
> $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> Error: Driver 'igb_uio' is not loaded.
> $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> ==================================================
> ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags libdpdk
> --libs libdpdk` lokesh@lokesh-ProLiant-DL385-
> Gen10:~/Desktop/dpdk_examples/skeleton$ sudo ./a.out
> EAL: Detected CPU lcores: 64
> EAL: Detected NUMA nodes: 4
> EAL: Detected shared linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available 1048576 kB hugepages reported
> EAL: VFIO support initialized
> EAL: Using IOMMU type 1 (Type 1)
> EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0 (socket 2)
> EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1 (socket 2)
> TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC: bc 97
> e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
>
> WARNING: Too many lcores enabled. Only 1 used.
> WARNING, port 0 is on remote NUMA node to polling thread.
> Performance will not be optimal.
> WARNING, port 1 is on remote NUMA node to polling thread.
> Performance will not be optimal.
>
> Core 0 forwarding packets. [Ctrl+C to quit] ^C
> ==================================================
> After a few seconds, I presses ctrl+c
>
> surprisingly cards are not showing up even in ifconfig.
>
>
>
> Thanks & Regards
> --
> Lokesh Chakka.
>
>
> On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>> wrote:
>
> > Dear David,
> >
> > =================================================
> > $ lspci | grep -i broadcom
> > 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> > 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $ lspci
> > -n -s 83:00.0
> > 83:00.0 0200: 14e4:1750 (rev 11)
> > =================================================
> >
> > I am compiling my code like this :
> > =================================================
> > gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> > =================================================
> >
> > Hence it is statically linked code.
> > If I try
> > $ dpdk-pmdinfo.py ./a.out
> >
> > But I am not seeing any output
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:22 PM David Marchand
> > <david.marchand@redhat.com<mailto:david.marchand@redhat.com>>
> > wrote:
> >
> >> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> >> <lvenkatakumarchakka@gmail.com<mailto:lvenkatakumarchakka@gmail.com>> wrote:
> >> > Would like to understand if I am missing something. I am new to
> >> > this
> >> platform.
> >> > rte_eth_dev_count_avail is returning zero.
> >> > OS is Ubuntu 22.04. DPDK is latest version.
> >> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> >> LED is also glowing.
> >>
> >> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> >> (which is net/bnxt).
> >> This userspace driver does not rely on the bnxt Linux kernel driver.
> >> IOW, this card being detecting and working with the Linux kernel does
> >> not automatically mean that this nic can work with DPDK.
> >>
> >> We need more info on your nic, first.
> >>
> >> Can you share the pci id of this nic (like running lspci -n -s
> >> $pci_address)?
> >> It should be a 14e4:XXXX.
> >>
> >> Then you can check this XXXX against what your dpdk application supports.
> >>
> >> If it is a statically linked application, you can run:
> >> $ dpdk-pmdinfo.py /path/to/your/application
> >>
> >> Else, if your application is dynamically linked against DPDK driver,
> >> you can run this command against the net/bnxt driver .so.22 (for
> >> 21.11 and later releases):
> >> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> >>
> >> You should get a list of supported NetXtreme nics, like:
> >>
> >> [snipped some other drivers compiled in my application] PMD NAME:
> >> net_bnxt PMD HW SUPPORT:
> >>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> >> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> >> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> >> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> >> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614) (All
> >> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E RDMA
> >> Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> >> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> >> (All Subdevices) [snipped the rest]
> >>
> >> I hope you can find a (XXXX) corresponding to your NIC.
> >>
> >>
> >> --
> >> David Marchand
> >>
> >>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GiimqiLieVwujxLIiPTyBAxVLa3WL24fKBSnVccmeUo%3D&reserved=0>
> dk.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=p7tLWYu7vQDf%2Bpi%2FYxUPM1pu7HfWunI14n2XQqPAmAQ%3D&reserved=0>%2Farchives%2Fusers%2Fattachments%2F20220713%2Fa362f7fe%2Fatta
> chment-
> 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.com%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ddHB8XShDZ32JF2bDLRQKWc1ZGSYo%2BVFFj%2FS8fZI%2Fdw%3D&reserved=0>%7C55465d0d4f
> aa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C
> 0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> %7C&amp;sdata=jXnaf1qrCA5VSCMyrPPc5S5WYsHveR1tNBL4lczkGVE%3D&am
> p;reserved=0>
>
> End of users Digest, Vol 347, Issue 6
> *************************************

[-- Attachment #2: Type: text/html, Size: 26472 bytes --]

Re: skeleton code failing (Lokesh Chakka)

[Lokesh Chakka]

[-- Attachment #1: Type: text/plain, Size: 16178 bytes --]

Dear vipin,

Thanks a lot for your valuable assistance and observation. I can't
sacrifice performance.I will look into pdump.

Best Regards
--
Lokesh Chakka.


On Thu, Jul 14, 2022 at 1:03 PM Varghese, Vipin <Vipin.Varghese@amd.com>
wrote:

> [Public]
>
>
>
> [LC] I am having is
> https://www.broadcom.com/products/ethernet-connectivity/network-adapters/p2100g
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.broadcom.com%2Fproducts%2Fethernet-connectivity%2Fnetwork-adapters%2Fp2100g&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GInrRYkg8qTsJ02CbjPSl%2FLT6C4cJ4Yt2fMkNfGmFho%3D&reserved=0>
> However, cards are up after waiting for around 10 minutes.
>
> [VV] I do face similar issue, hence I have recommended ` Observation: due some hardware or firmware bug auto-neg take more time with the current DPDK BXNT PMD. Hence my advice is try to wait longer for link state up in DPDK.`. Please try reaching out to Broadcom PMD maintainer `ajit.khaparde@broadcom.com`. Once the solution is identified please share too
>
>
>
>
>
> [LC] If netdevice is not available, how can I capture the packets using
> wireshark?
>
> [VV] Please explore DPDK tool PDUMP.
> https://doc.dpdk.org/guides/tools/pdump.html
>
>
>
> [LC] Interfaces are not visible to the wireshark. Very basic requirement
> is to send the packets, capture them and see the contents.
>
> [VV] If you want the kernel netdev visible and use it under DPDK, I
> recommend using LIBPCAP PMD by sacrificing performance and higher
> functionality.
>
>
>
> *From:* Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> *Sent:* Thursday, July 14, 2022 12:56 PM
> *To:* Varghese, Vipin <Vipin.Varghese@amd.com>
> *Cc:* users@dpdk.org; Yigit, Ferruh <Ferruh.Yigit@amd.com>; Tummala,
> Sivaprasad <Sivaprasad.Tummala@amd.com>
> *Subject:* Re: skeleton code failing (Lokesh Chakka)
>
>
>
> [CAUTION: External Email]
>
>
> The card I am having is
> https://www.broadcom.com/products/ethernet-connectivity/network-adapters/p2100g
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.broadcom.com%2Fproducts%2Fethernet-connectivity%2Fnetwork-adapters%2Fp2100g&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GInrRYkg8qTsJ02CbjPSl%2FLT6C4cJ4Yt2fMkNfGmFho%3D&reserved=0>
>
> However, cards are up after waiting for around 10 minutes.
>
>
>
> Thanks for the valuable input. skeleton code is running till I press
> ctrl+c.
>
> Now I have a big concern.
>
> If netdevice is not available, how can I capture the packets using
> wireshark?
>
> Interfaces are not visible to the wireshark. Very basic requirement is to
> send the packets, capture them and see the contents.
>
>
>
>
> Thanks & Regards
> --
> Lokesh Chakka.
>
>
>
>
>
> On Thu, Jul 14, 2022 at 11:46 AM Varghese, Vipin <Vipin.Varghese@amd.com>
> wrote:
>
> [AMD Official Use Only - General]
>
>
>
> Is this not Broadcom extreme net card? Please refer
> http://doc.dpdk.org/guides/nics/bnxt.html
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fbnxt.html&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=qsqUSFyPvnm%2Frqqy5RkOfMu2WREnXXNPAoQNrBb09ig%3D&reserved=0>,
> once you bind with `igb_uio, vfio_pci or uio_pci_generic` all linux
> netdevice instance will be no longer available. This is because it is not a
> `port representator`
>
>
>
> Observation: due some hardware or firmware bug auto-neg take more time
> with the current DPDK BXNT PMD. Hence my advice is try to wait longer for
> link state up in DPDK.
>
> You can verify the same with testpmd.
>
>
>
> *From:* Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> *Sent:* Thursday, July 14, 2022 11:11 AM
> *To:* Varghese, Vipin <Vipin.Varghese@amd.com>
> *Cc:* users@dpdk.org; Yigit, Ferruh <Ferruh.Yigit@amd.com>; Tummala,
> Sivaprasad <Sivaprasad.Tummala@amd.com>
> *Subject:* Re: skeleton code failing (Lokesh Chakka)
>
>
>
> [CAUTION: External Email]
>
> I have one more observation here.
>
> After "$ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1"
>
>
>
> Cards are shutting down. ifconfig is not showing the cards. reinsert of
> sfp is also not bringing the cards up. I had to restart the server. Not
> sure why the cards are going down.
>
>
>
> Thanks & Regards
> --
> Lokesh Chakka.
>
>
>
>
>
> On Thu, Jul 14, 2022 at 8:50 AM Varghese, Vipin <Vipin.Varghese@amd.com>
> wrote:
>
> [AMD Official Use Only - General]
>
> Based on the compilation command shared it looks like you are using the
> build in shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk
> --libs libdpdk`
> Hence in EAL PCIe probe, the BNXT PMD is not triggered to identify the
> NIC.
>
> Solutions for these can be
> 1. Build with static library ` gcc main.c -g `pkg-config --cflags libdpdk
> --libs --static libdpdk`
> 2. pass the bxnt_en PMD shared library to eal args by ` sudo ./a.out -l 1
> -d librte_net_bnxt.so`
>
> Can you try any of the above ?
>
> > -----Original Message-----
> > From: users-request@dpdk.org <users-request@dpdk.org>
> > Sent: Wednesday, July 13, 2022 3:30 PM
> > To: users@dpdk.org
> > Subject: users Digest, Vol 347, Issue 6
> >
> > [CAUTION: External Email]
> >
> > Send users mailing list submissions to
> >         users@dpdk.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sRwPMJBmusuk53CQJeL5Ul%2BKaJq1XC3PjqtOCkSC5vY%3D&reserved=0>
> > dk.org
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=p7tLWYu7vQDf%2Bpi%2FYxUPM1pu7HfWunI14n2XQqPAmAQ%3D&reserved=0>
> %2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40amd.co
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.co%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J8sevycMs8Nd1uTlw0adu5Pt5R3ikhOitLYWGfT9kEQ%3D&reserved=0>
> > m%7C55465d0d4faa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d9
> > 94e183d%7C0%7C0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8
> > eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > %7C3000%7C%7C%7C&amp;sdata=6JHjf2Fu%2BkQ56CF9xTZvBS%2Bh8Uomlqv
> > E9dq9M7VefiU%3D&amp;reserved=0
> > or, via email, send a message with subject or body 'help' to
> >         users-request@dpdk.org
> >
> > You can reach the person managing the list at
> >         users-owner@dpdk.org
> >
> > When replying, please edit your Subject line so it is more specific than
> "Re:
> > Contents of users digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Re: skeleton code failing (Lokesh Chakka)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 13 Jul 2022 12:56:37 +0530
> > From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> > To: David Marchand <david.marchand@redhat.com>
> > Cc: users <users@dpdk.org>
> > Subject: Re: skeleton code failing
> > Message-ID:
> >         <CACh--sgKJ9B2=Nd-EUUx4D7Kz87-
> > Nm_pwq59fY9QVNH+GBgwRg@mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Dear David,
> >
> > following is some more stuff i did
> > ==================================================
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b
> vfio-pci
> > 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0 83:00.1
> > Error: Driver 'uio_pci_generic' is not loaded.
> > $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> > Error: Driver 'igb_uio' is not loaded.
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> > Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> > Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> > ==================================================
> > ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags
> libdpdk
> > --libs libdpdk` lokesh@lokesh-ProLiant-DL385-
> > Gen10:~/Desktop/dpdk_examples/skeleton$ sudo ./a.out
> > EAL: Detected CPU lcores: 64
> > EAL: Detected NUMA nodes: 4
> > EAL: Detected shared linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: No available 1048576 kB hugepages reported
> > EAL: VFIO support initialized
> > EAL: Using IOMMU type 1 (Type 1)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0 (socket
> 2)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1 (socket
> 2)
> > TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC: bc
> 97
> > e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
> >
> > WARNING: Too many lcores enabled. Only 1 used.
> > WARNING, port 0 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> > WARNING, port 1 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> >
> > Core 0 forwarding packets. [Ctrl+C to quit] ^C
> > ==================================================
> > After a few seconds, I presses ctrl+c
> >
> > surprisingly cards are not showing up even in ifconfig.
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> > lvenkatakumarchakka@gmail.com> wrote:
> >
> > > Dear David,
> > >
> > > =================================================
> > > $ lspci | grep -i broadcom
> > > 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> > > 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> > > NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $ lspci
> > > -n -s 83:00.0
> > > 83:00.0 0200: 14e4:1750 (rev 11)
> > > =================================================
> > >
> > > I am compiling my code like this :
> > > =================================================
> > > gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> > > =================================================
> > >
> > > Hence it is statically linked code.
> > > If I try
> > > $ dpdk-pmdinfo.py ./a.out
> > >
> > > But I am not seeing any output
> > >
> > >
> > >
> > > Thanks & Regards
> > > --
> > > Lokesh Chakka.
> > >
> > >
> > > On Wed, Jul 13, 2022 at 12:22 PM David Marchand
> > > <david.marchand@redhat.com>
> > > wrote:
> > >
> > >> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> > >> <lvenkatakumarchakka@gmail.com> wrote:
> > >> > Would like to understand if I am missing something. I am new to
> > >> > this
> > >> platform.
> > >> > rte_eth_dev_count_avail is returning zero.
> > >> > OS is Ubuntu 22.04. DPDK is latest version.
> > >> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> > >> LED is also glowing.
> > >>
> > >> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> > >> (which is net/bnxt).
> > >> This userspace driver does not rely on the bnxt Linux kernel driver.
> > >> IOW, this card being detecting and working with the Linux kernel does
> > >> not automatically mean that this nic can work with DPDK.
> > >>
> > >> We need more info on your nic, first.
> > >>
> > >> Can you share the pci id of this nic (like running lspci -n -s
> > >> $pci_address)?
> > >> It should be a 14e4:XXXX.
> > >>
> > >> Then you can check this XXXX against what your dpdk application
> supports.
> > >>
> > >> If it is a statically linked application, you can run:
> > >> $ dpdk-pmdinfo.py /path/to/your/application
> > >>
> > >> Else, if your application is dynamically linked against DPDK driver,
> > >> you can run this command against the net/bnxt driver .so.22 (for
> > >> 21.11 and later releases):
> > >> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> > >>
> > >> You should get a list of supported NetXtreme nics, like:
> > >>
> > >> [snipped some other drivers compiled in my application] PMD NAME:
> > >> net_bnxt PMD HW SUPPORT:
> > >>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> > >> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> > >> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> > >> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> > >> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614) (All
> > >> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E RDMA
> > >> Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> > >> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> > >> (All Subdevices) [snipped the rest]
> > >>
> > >> I hope you can find a (XXXX) corresponding to your NIC.
> > >>
> > >>
> > >> --
> > >> David Marchand
> > >>
> > >>
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > <
> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dp%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GiimqiLieVwujxLIiPTyBAxVLa3WL24fKBSnVccmeUo%3D&reserved=0>
> > dk.org
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdk.org%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=p7tLWYu7vQDf%2Bpi%2FYxUPM1pu7HfWunI14n2XQqPAmAQ%3D&reserved=0>
> %2Farchives%2Fusers%2Fattachments%2F20220713%2Fa362f7fe%2Fatta
> > chment-
> > 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2F40amd.com%2F&data=05%7C01%7CVipin.Varghese%40amd.com%7C255e189a10194e84dab008da656a11ed%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637933803507082842%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ddHB8XShDZ32JF2bDLRQKWc1ZGSYo%2BVFFj%2FS8fZI%2Fdw%3D&reserved=0>
> %7C55465d0d4f
> > aa4cd8496a08da64b674f4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C
> > 0%7C637933032067208770%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> > wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> > %7C&amp;sdata=jXnaf1qrCA5VSCMyrPPc5S5WYsHveR1tNBL4lczkGVE%3D&am
> > p;reserved=0>
> >
> > End of users Digest, Vol 347, Issue 6
> > *************************************
>
>

[-- Attachment #2: Type: text/html, Size: 24576 bytes --]

Re: skeleton code failing (Lokesh Chakka)

[Varghese, Vipin]

Hi Lokesh,

Based on the DPDK logs, following are some of the observations

1. application is built with shared library mode ` gcc main.c -g `pkg-config --cflags libdpdk --libs libdpdk` which may require to pass shared dpdk PMD library with option `-d` in eal args
2. EAL PCIe probe identified `0000:83:00.0` and `0000:83:00.1`
3. there is warning from application on core NUMA-Socket vs NIC NUMA-Socket for the DPDK ports ` WARNING, port 0 is on remote NUMA node to polling thread. Performance will not be optimal. WARNING, port 1 is on remote NUMA node to polling thread. Performance will not be optimal.`
4. application has started on CPU logical thread 0 and waiting for user to interrupt the same ` Core 0 forwarding packets. [Ctrl+C to quit]`

With these observations the application is running with DPDK ports. 

With respect to queries
1. surprisingly cards are not showing up even in ifconfig - Braodcom NIC card (PF or VF) when bind with DPDK get removed from from kernel `bxnt` and added to `igb_uio|vfio_pci|uio_pci_generic`. This means there is no kernel netlink device attached to the NIC port.
2. dpdk-pmdinfo.py ./a.out But I am not seeing any output - as mentioned above this could be because the application is linked in shared library mode.
3. `gcc main.c `pkg-config --cflags libdpdk --libs libdpdk` Hence it is statically linked code. - this understanding is incorrect as the command to link statically is `pkg-config --static --cflags --libs libdpdk`.
4. Unlike other examples like l2fwd, l3fwd, testpmd there is procedure or call to function to display PMD stats for NIC ports

Hence, I recommend to spent some time https://doc.dpdk.org/guides/sample_app_ug/skeleton.html and then 
1. if the intention is build with static library, use the right command to to built the application.
2. Do not expect Broadcom NIC to make both Kernel netlink and DPDK available at the same time for PCIe id.
3. Use `log-level=pmd,8` to get more PMD related information.
4. Send traffic DPDK ports, and use `dpdk-procinfo` with option `--stats` to get the stats via DPDK secondary app.

Note: It could be NIC related issue you have to reach out Broadcom as NIC vendor for your current use case.

Regards
Vipin Varghese 

> -----Original Message-----
> From: users-request@dpdk.org <users-request@dpdk.org>
> Sent: Friday, August 26, 2022 3:30 PM
> To: users@dpdk.org
> Subject: users Digest, Vol 353, Issue 5
> 
> [CAUTION: External Email]
> 
> Send users mailing list submissions to
>         users@dpdk.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail
> s.dpdk.org%2Flistinfo%2Fusers&amp;data=05%7C01%7Cvipin.varghese%40a
> md.com%7C3b2eb0cad85d4133795e08da8749bf40%7C3dd8961fe4884e608e1
> 1a82d994e183d%7C0%7C0%7C637971048067638133%7CUnknown%7CTWFpb
> GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> 6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=WhU4xEpP2iZ8T3fp%2BKZjhGr4
> 47C6PUh5kRMAeNKNUyU%3D&amp;reserved=0
> or, via email, send a message with subject or body 'help' to
>         users-request@dpdk.org
> 
> You can reach the person managing the list at
>         users-owner@dpdk.org
> 
> When replying, please edit your Subject line so it is more specific than "Re:
> Contents of users digest..."
> 
> 
> Today's Topics:
> 
>    1. DPD-22.07 bind error=-19 (sysman@scaled.cloud)
>    2. Re: skeleton code failing (Lokesh Chakka)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Thu, 25 Aug 2022 13:50:16 +0300
> From: sysman@scaled.cloud
> To: users@dpdk.org
> Subject: DPD-22.07 bind error=-19
> Message-ID: <cdfd3482ad4b4b19ed34fdc21a92c1ea@scaled.cloud>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
> 
> Hello All!
> 
>   Can someone help me to resolved situation:
> 
> ubuntu 20 lts
> dpdk 22-0
> 
> 
> root@noserve1:/home/user/Works/tootoo2# dpdk-devbind.py -s
> 
> Network devices using kernel driver
> ===================================
> 0000:05:00.0 'I211 Gigabit Network Connection 1539' if=enp5s0 drv=igb
> unused=vfio-pci,uio_pci_generic *Active*
> 0000:06:00.0 'I211 Gigabit Network Connection 1539' if=enp6s0 drv=igb
> unused=vfio-pci,uio_pci_generic
> 0000:07:00.0 'I211 Gigabit Network Connection 1539' if=enp7s0 drv=igb
> unused=vfio-pci,uio_pci_generic
> 0000:08:00.0 'I211 Gigabit Network Connection 1539' if=enp8s0 drv=igb
> unused=vfio-pci,uio_pci_generic
> 0000:09:00.0 'I211 Gigabit Network Connection 1539' if=enp9s0 drv=igb
> unused=vfio-pci,uio_pci_generic
> 0000:0a:00.0 'I211 Gigabit Network Connection 1539' if=enp10s0 drv=igb
> unused=vfio-pci,uio_pci_generic
> 
> Other Network devices
> =====================
> 0000:01:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> unused=i40e,vfio-pci,uio_pci_generic
> 0000:01:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> unused=i40e,vfio-pci,uio_pci_generic
> 0000:01:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> unused=i40e,vfio-pci,uio_pci_generic
> 0000:01:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> unused=i40e,vfio-pci,uio_pci_generic
> 
> 
> When i try to bind:
> 
> 
> root@noserve1:/home/user/Works/tootoo2# dpdk-devbind.py -b i40e
> 0000:01:00.0
> Error: bind failed for 0000:01:00.0 - Cannot bind to driver i40e: [Errno 19] No
> such device root@noserve1:/home/user/Works/tootoo2# dpdk-devbind.py
> -b uio_pci_generic 0000:01:00.0
> Error: bind failed for 0000:01:00.0 - Cannot bind to driver
> uio_pci_generic: [Errno 19] No such device
> 
> 
> 
> Thanks in advance!
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Fri, 26 Aug 2022 10:05:22 +0530
> From: Lokesh Chakka <lvenkatakumarchakka@gmail.com>
> To: David Marchand <david.marchand@redhat.com>
> Cc: users <users@dpdk.org>
> Subject: Re: skeleton code failing
> Message-ID:
>         <CACh--shYBbmtc0-
> Bp8poVKVMnfsoSpNxwYEpEHiMHkjt8waM5g@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hello,
> 
> As I am not able to get sufficient support from the intel support team or from
> the dpdk forum, the support team from intel asked me to raise a ticket in
> Intel Premium Support. Can someone help me get registered there as it is
> asking for an intel agent.
> 
> Thanks & Regards
> --
> Lokesh Chakka.
> 
> 
> On Wed, Jul 13, 2022 at 12:56 PM Lokesh Chakka <
> lvenkatakumarchakka@gmail.com> wrote:
> 
> > Dear David,
> >
> > following is some more stuff i did
> > ==================================================
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 $ sudo dpdk-devbind.py -b
> > vfio-pci 83:00.1 $ sudo dpdk-devbind.py -b uio_pci_generic 83:00.0
> > 83:00.1
> > Error: Driver 'uio_pci_generic' is not loaded.
> > $ sudo dpdk-devbind.py -b igb_uio 83:00.0 83:00.1
> > Error: Driver 'igb_uio' is not loaded.
> > $ sudo dpdk-devbind.py -b vfio-pci 83:00.0 83:00.1
> > Notice: 0000:83:00.0 already bound to driver vfio-pci, skipping
> > Notice: 0000:83:00.1 already bound to driver vfio-pci, skipping
> > ==================================================
> > ~/Desktop/dpdk_examples/skeleton$ gcc main.c -g `pkg-config --cflags
> > libdpdk --libs libdpdk`
> > lokesh@lokesh-ProLiant-DL385-
> Gen10:~/Desktop/dpdk_examples/skeleton$
> > sudo ./a.out
> > EAL: Detected CPU lcores: 64
> > EAL: Detected NUMA nodes: 4
> > EAL: Detected shared linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: No available 1048576 kB hugepages reported
> > EAL: VFIO support initialized
> > EAL: Using IOMMU type 1 (Type 1)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.0
> > (socket 2)
> > EAL: Probe PCI driver: net_bnxt (14e4:1750) device: 0000:83:00.1
> > (socket 2)
> > TELEMETRY: No legacy callbacks, legacy socket not created Port 0 MAC:
> > bc 97 e1 ce 84 f0 Port 1 MAC: bc 97 e1 ce 84 f1
> >
> > WARNING: Too many lcores enabled. Only 1 used.
> > WARNING, port 0 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> > WARNING, port 1 is on remote NUMA node to polling thread.
> > Performance will not be optimal.
> >
> > Core 0 forwarding packets. [Ctrl+C to quit] ^C
> > ==================================================
> > After a few seconds, I presses ctrl+c
> >
> > surprisingly cards are not showing up even in ifconfig.
> >
> >
> >
> > Thanks & Regards
> > --
> > Lokesh Chakka.
> >
> >
> > On Wed, Jul 13, 2022 at 12:43 PM Lokesh Chakka <
> > lvenkatakumarchakka@gmail.com> wrote:
> >
> >> Dear David,
> >>
> >> =================================================
> >> $ lspci | grep -i broadcom
> >> 83:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> >> NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
> >> 83:00.1 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
> >> NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11) $
> lspci
> >> -n -s 83:00.0
> >> 83:00.0 0200: 14e4:1750 (rev 11)
> >> =================================================
> >>
> >> I am compiling my code like this :
> >> =================================================
> >> gcc main.c `pkg-config --cflags libdpdk --libs libdpdk`
> >> =================================================
> >>
> >> Hence it is statically linked code.
> >> If I try
> >> $ dpdk-pmdinfo.py ./a.out
> >>
> >> But I am not seeing any output
> >>
> >>
> >>
> >> Thanks & Regards
> >> --
> >> Lokesh Chakka.
> >>
> >>
> >> On Wed, Jul 13, 2022 at 12:22 PM David Marchand <
> >> david.marchand@redhat.com> wrote:
> >>
> >>> On Wed, Jul 13, 2022 at 7:35 AM Lokesh Chakka
> >>> <lvenkatakumarchakka@gmail.com> wrote:
> >>> > Would like to understand if I am missing something. I am new to
> >>> > this
> >>> platform.
> >>> > rte_eth_dev_count_avail is returning zero.
> >>> > OS is Ubuntu 22.04. DPDK is latest version.
> >>> > Cards are being detected by Linux. Ifconfig is showing the cards up.
> >>> LED is also glowing.
> >>>
> >>> Indeed, DPDK provides a userspace driver for some NetXtreme nics
> >>> (which is net/bnxt).
> >>> This userspace driver does not rely on the bnxt Linux kernel driver.
> >>> IOW, this card being detecting and working with the Linux kernel
> >>> does not automatically mean that this nic can work with DPDK.
> >>>
> >>> We need more info on your nic, first.
> >>>
> >>> Can you share the pci id of this nic (like running lspci -n -s
> >>> $pci_address)?
> >>> It should be a 14e4:XXXX.
> >>>
> >>> Then you can check this XXXX against what your dpdk application
> supports.
> >>>
> >>> If it is a statically linked application, you can run:
> >>> $ dpdk-pmdinfo.py /path/to/your/application
> >>>
> >>> Else, if your application is dynamically linked against DPDK driver,
> >>> you can run this command against the net/bnxt driver .so.22 (for
> >>> 21.11 and later releases):
> >>> $ dpdk-pmdinfo.py /path/to/your/dpdk/drivers/librte_net_bnxt.so.22
> >>>
> >>> You should get a list of supported NetXtreme nics, like:
> >>>
> >>> [snipped some other drivers compiled in my application] PMD NAME:
> >>> net_bnxt PMD HW SUPPORT:
> >>>  Broadcom Inc. and subsidiaries (14e4) : BCM5745X NetXtreme-E RDMA
> >>> Virtual Function (1606) (All Subdevices)  Broadcom Inc. and
> >>> subsidiaries (14e4) : BCM5745X NetXtreme-E Ethernet Virtual Function
> >>> (1609) (All Subdevices)  Broadcom Inc. and subsidiaries (14e4) :
> >>> BCM57454 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb Ethernet (1614)
> (All
> >>> Subdevices)  Broadcom Inc. and subsidiaries (14e4) : NetXtreme-E
> >>> RDMA Virtual Function (16c1) (All Subdevices)  Broadcom Inc. and
> >>> subsidiaries (14e4) : NetXtreme-C Ethernet Virtual Function (16cb)
> >>> (All Subdevices) [snipped the rest]
> >>>
> >>> I hope you can find a (XXXX) corresponding to your NIC.
> >>>
> >>>
> >>> --
> >>> David Marchand
> >>>
> >>>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail
> s.dpdk.org%2Farchives%2Fusers%2Fattachments%2F20220826%2F09fcb44d
> %2Fattachment-
> 0001.htm&amp;data=05%7C01%7Cvipin.varghese%40amd.com%7C3b2eb0ca
> d85d4133795e08da8749bf40%7C3dd8961fe4884e608e11a82d994e183d%7C0
> %7C0%7C637971048067638133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> 4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> 7C%7C%7C&amp;sdata=SfVPC2z%2BqkUkKTCfsY6A7FB02znexfvnKPSyKkk9
> MpA%3D&amp;reserved=0>
> 
> End of users Digest, Vol 353, Issue 5
> *************************************