* [dpdk-users] "eth_i40e_dev_init(): Failed to init adminq: -66" problem
@ 2020-04-10 8:29 Sunghyuk Kay
[not found] ` <CAJyVW49LOPQiVnorQ4=YECjDu4zfAci_7JmQx27hdA=TV9a2wQ@mail.gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Sunghyuk Kay @ 2020-04-10 8:29 UTC (permalink / raw)
To: users
Hi all,
I'm setting DPDK on our hardware to do some experiment, but suffering from .
I was trying to use DPDK on Ubuntu 18.04 with Intel XL710 Ethernet Adapter
with igb_uio driver. I've compiled DPDK from source, and bound the driver
to NIC.
But when I run `testpmd` application, it returns an error like this.
```
$ sudo ./build/app/testpmd
EAL: Detected 128 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:81:00.0 on NUMA socket 0
EAL: probe driver: 8086:1583 net_i40e
eth_i40e_dev_init(): Failed to init adminq: -66
EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.0 cannot
be used
EAL: PCI device 0000:81:00.1 on NUMA socket 0
EAL: probe driver: 8086:1583 net_i40e
eth_i40e_dev_init(): Failed to init adminq: -66
EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.1 cannot
be used
EAL: PCI device 0000:c4:00.0 on NUMA socket 0
EAL: probe driver: 8086:1533 net_e1000_igb
EAL: PCI device 0000:c5:00.0 on NUMA socket 0
EAL: probe driver: 8086:1533 net_e1000_igb
testpmd: No probed ethernet devices
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=1163456, size=2176,
socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Done
No commandline core given, start packet forwarding
io packet forwarding - ports=0 - cores=0 - streams=0 - NUMA support
enabled, MP allocation mode: native
io packet forwarding packets/burst=32
nb forwarding cores=1 - nb forwarding ports=0
Press enter to exit
Telling cores to stop...
Waiting for lcores to finish...
+++++++++++++++ Accumulated forward statistics for all
ports+++++++++++++++
RX-packets: 0 RX-dropped: 0 RX-total: 0
TX-packets: 0 TX-dropped: 0 TX-total: 0
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Done.
Bye...
```
I tried to find what return code -66 from adminq means, but I couldn't find
it.
Some say this might have to do with Intel IOMMU pass-through, but I'm not
sure that's the case.
Has anyone had this issue before? Maybe some tip on this issue would be
very helpful.
Thank you,
Sunghyuk
^ permalink raw reply [flat|nested] 4+ messages in thread[parent not found: <CAJyVW49LOPQiVnorQ4=YECjDu4zfAci_7JmQx27hdA=TV9a2wQ@mail.gmail.com>]
* Re: [dpdk-users] "eth_i40e_dev_init(): Failed to init adminq: -66" problem [not found] ` <CAJyVW49LOPQiVnorQ4=YECjDu4zfAci_7JmQx27hdA=TV9a2wQ@mail.gmail.com> @ 2020-04-10 13:31 ` Sunghyuk Kay 2020-04-10 14:29 ` Dheeraj Dang 0 siblings, 1 reply; 4+ messages in thread From: Sunghyuk Kay @ 2020-04-10 13:31 UTC (permalink / raw) To: Dheeraj Dang; +Cc: users Hi, Thanks for the solution. Actually, I was trying with igb_uio driver, rather than vfio-pci. Would that make a difference? Thank you, Sunghyuk 2020년 4월 10일 (금) 오후 9:54, Dheeraj Dang <dheerajdang138@gmail.com>님이 작성: > Hi, > > I think you are using virtual functions while running your test. If yes, > please try the following commands > > Make changes in GRUB setting, update grub and reboot the machine. > GRUB_CMDLINE_LINUX="intel_iommu=on amd_iommu=on" > When machine is up, check that iommu is enabled in the output of cat > /proc/cmdline. If iommu is enabled, then run the following commands. > modprobe vfio enable_unsafe_noiommu_mode=Y modprobe vfio-pci > > Bind your dpdk interfaces to vfio-pci (*dpdk-devbind.py -b vfio-pci > <PCI_ID>*) and run the test. > > Regards > Dheeraj Dang > > > On Fri, Apr 10, 2020 at 6:16 PM Sunghyuk Kay <sunghyuk.kay@kaist.ac.kr> > wrote: > >> Hi all, >> I'm setting DPDK on our hardware to do some experiment, but suffering >> from . >> >> I was trying to use DPDK on Ubuntu 18.04 with Intel XL710 Ethernet Adapter >> with igb_uio driver. I've compiled DPDK from source, and bound the driver >> to NIC. >> But when I run `testpmd` application, it returns an error like this. >> >> ``` >> $ sudo ./build/app/testpmd >> EAL: Detected 128 lcore(s) >> EAL: Detected 1 NUMA nodes >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >> EAL: Selected IOVA mode 'PA' >> EAL: No available hugepages reported in hugepages-1048576kB >> EAL: Probing VFIO support... >> EAL: PCI device 0000:81:00.0 on NUMA socket 0 >> EAL: probe driver: 8086:1583 net_i40e >> eth_i40e_dev_init(): Failed to init adminq: -66 >> EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.0 cannot >> be used >> EAL: PCI device 0000:81:00.1 on NUMA socket 0 >> EAL: probe driver: 8086:1583 net_i40e >> eth_i40e_dev_init(): Failed to init adminq: -66 >> EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.1 cannot >> be used >> EAL: PCI device 0000:c4:00.0 on NUMA socket 0 >> EAL: probe driver: 8086:1533 net_e1000_igb >> EAL: PCI device 0000:c5:00.0 on NUMA socket 0 >> EAL: probe driver: 8086:1533 net_e1000_igb >> testpmd: No probed ethernet devices >> testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=1163456, >> size=2176, >> socket=0 >> testpmd: preferred mempool ops selected: ring_mp_mc >> Done >> No commandline core given, start packet forwarding >> io packet forwarding - ports=0 - cores=0 - streams=0 - NUMA support >> enabled, MP allocation mode: native >> >> io packet forwarding packets/burst=32 >> nb forwarding cores=1 - nb forwarding ports=0 >> Press enter to exit >> >> Telling cores to stop... >> Waiting for lcores to finish... >> >> +++++++++++++++ Accumulated forward statistics for all >> ports+++++++++++++++ >> RX-packets: 0 RX-dropped: 0 RX-total: 0 >> TX-packets: 0 TX-dropped: 0 TX-total: 0 >> >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> Done. >> >> Bye... >> ``` >> I tried to find what return code -66 from adminq means, but I couldn't >> find >> it. >> Some say this might have to do with Intel IOMMU pass-through, but I'm not >> sure that's the case. >> >> Has anyone had this issue before? Maybe some tip on this issue would be >> very helpful. >> >> Thank you, >> Sunghyuk >> > -- 감사합니다 계성혁 드림 ========= 계성혁 (Kay, Sung Hyuk) KAIST Concurrency and Parallelism Laboratory Masters Student @ Software Graduate Program Mobile: 010-2794-6906 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-users] "eth_i40e_dev_init(): Failed to init adminq: -66" problem 2020-04-10 13:31 ` Sunghyuk Kay @ 2020-04-10 14:29 ` Dheeraj Dang 2020-04-16 4:00 ` Sunghyuk Kay 0 siblings, 1 reply; 4+ messages in thread From: Dheeraj Dang @ 2020-04-10 14:29 UTC (permalink / raw) To: Sunghyuk Kay; +Cc: users Hi, If you want to use igb_uio instead of vfio-pci, then you have to make the following changes in the GRUB GRUB_CMDLINE_LINUX="iommu=pt, intel_iommu=on" update grub settings and reboot the machine. When we are running DPDK on virtual functions, then it is recommended to use vfio-pci instead of igb_uio. You can refer this link http://mails.dpdk.org/archives/dev/2014-August/004610.html for more details. Regards Dheeraj Dang On Fri, Apr 10, 2020 at 7:01 PM Sunghyuk Kay <sunghyuk.kay@kaist.ac.kr> wrote: > Hi, > Thanks for the solution. > > Actually, I was trying with igb_uio driver, rather than vfio-pci. > Would that make a difference? > > Thank you, > Sunghyuk > > 2020년 4월 10일 (금) 오후 9:54, Dheeraj Dang <dheerajdang138@gmail.com>님이 작성: > >> Hi, >> >> I think you are using virtual functions while running your test. If yes, >> please try the following commands >> >> Make changes in GRUB setting, update grub and reboot the machine. >> GRUB_CMDLINE_LINUX="intel_iommu=on amd_iommu=on" >> When machine is up, check that iommu is enabled in the output of cat >> /proc/cmdline. If iommu is enabled, then run the following commands. >> modprobe vfio enable_unsafe_noiommu_mode=Y modprobe vfio-pci >> >> Bind your dpdk interfaces to vfio-pci (*dpdk-devbind.py -b vfio-pci >> <PCI_ID>*) and run the test. >> >> Regards >> Dheeraj Dang >> >> >> On Fri, Apr 10, 2020 at 6:16 PM Sunghyuk Kay <sunghyuk.kay@kaist.ac.kr> >> wrote: >> >>> Hi all, >>> I'm setting DPDK on our hardware to do some experiment, but suffering >>> from . >>> >>> I was trying to use DPDK on Ubuntu 18.04 with Intel XL710 Ethernet >>> Adapter >>> with igb_uio driver. I've compiled DPDK from source, and bound the driver >>> to NIC. >>> But when I run `testpmd` application, it returns an error like this. >>> >>> ``` >>> $ sudo ./build/app/testpmd >>> EAL: Detected 128 lcore(s) >>> EAL: Detected 1 NUMA nodes >>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >>> EAL: Selected IOVA mode 'PA' >>> EAL: No available hugepages reported in hugepages-1048576kB >>> EAL: Probing VFIO support... >>> EAL: PCI device 0000:81:00.0 on NUMA socket 0 >>> EAL: probe driver: 8086:1583 net_i40e >>> eth_i40e_dev_init(): Failed to init adminq: -66 >>> EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.0 >>> cannot >>> be used >>> EAL: PCI device 0000:81:00.1 on NUMA socket 0 >>> EAL: probe driver: 8086:1583 net_i40e >>> eth_i40e_dev_init(): Failed to init adminq: -66 >>> EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.1 >>> cannot >>> be used >>> EAL: PCI device 0000:c4:00.0 on NUMA socket 0 >>> EAL: probe driver: 8086:1533 net_e1000_igb >>> EAL: PCI device 0000:c5:00.0 on NUMA socket 0 >>> EAL: probe driver: 8086:1533 net_e1000_igb >>> testpmd: No probed ethernet devices >>> testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=1163456, >>> size=2176, >>> socket=0 >>> testpmd: preferred mempool ops selected: ring_mp_mc >>> Done >>> No commandline core given, start packet forwarding >>> io packet forwarding - ports=0 - cores=0 - streams=0 - NUMA support >>> enabled, MP allocation mode: native >>> >>> io packet forwarding packets/burst=32 >>> nb forwarding cores=1 - nb forwarding ports=0 >>> Press enter to exit >>> >>> Telling cores to stop... >>> Waiting for lcores to finish... >>> >>> +++++++++++++++ Accumulated forward statistics for all >>> ports+++++++++++++++ >>> RX-packets: 0 RX-dropped: 0 RX-total: 0 >>> TX-packets: 0 TX-dropped: 0 TX-total: 0 >>> >>> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> Done. >>> >>> Bye... >>> ``` >>> I tried to find what return code -66 from adminq means, but I couldn't >>> find >>> it. >>> Some say this might have to do with Intel IOMMU pass-through, but I'm not >>> sure that's the case. >>> >>> Has anyone had this issue before? Maybe some tip on this issue would be >>> very helpful. >>> >>> Thank you, >>> Sunghyuk >>> >> -- > 감사합니다 > 계성혁 드림 > ========= > 계성혁 (Kay, Sung Hyuk) > KAIST Concurrency and Parallelism Laboratory > Masters Student @ Software Graduate Program > Mobile: 010-2794-6906 > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-users] "eth_i40e_dev_init(): Failed to init adminq: -66" problem 2020-04-10 14:29 ` Dheeraj Dang @ 2020-04-16 4:00 ` Sunghyuk Kay 0 siblings, 0 replies; 4+ messages in thread From: Sunghyuk Kay @ 2020-04-16 4:00 UTC (permalink / raw) To: Dheeraj Dang; +Cc: users Hi, sorry for the late reply. I had done more digging on the message, and I guess the issue was because of NIC's low firmware version. (I found IO_PAGE_FAULT on dmesg) Since I didn't want to screw up with the current environment (including firmware), I managed to run DPDK on other hardware with your advice on GRUB settings, so I might be able to see what my real problem was later in the future :) Thanks for the kind response. Regards, Sunghyuk On Fri, Apr 10, 2020 at 11:30 PM Dheeraj Dang <dheerajdang138@gmail.com> wrote: > Hi, > > If you want to use igb_uio instead of vfio-pci, then you have to make the > following changes in the GRUB > GRUB_CMDLINE_LINUX="iommu=pt, intel_iommu=on" > > update grub settings and reboot the machine. > > When we are running DPDK on virtual functions, then it is recommended to > use vfio-pci instead of igb_uio. > You can refer this link > http://mails.dpdk.org/archives/dev/2014-August/004610.html for more > details. > > Regards > Dheeraj Dang > > > On Fri, Apr 10, 2020 at 7:01 PM Sunghyuk Kay <sunghyuk.kay@kaist.ac.kr> > wrote: > >> Hi, >> Thanks for the solution. >> >> Actually, I was trying with igb_uio driver, rather than vfio-pci. >> Would that make a difference? >> >> Thank you, >> Sunghyuk >> >> 2020년 4월 10일 (금) 오후 9:54, Dheeraj Dang <dheerajdang138@gmail.com>님이 작성: >> >>> Hi, >>> >>> I think you are using virtual functions while running your test. If yes, >>> please try the following commands >>> >>> Make changes in GRUB setting, update grub and reboot the machine. >>> GRUB_CMDLINE_LINUX="intel_iommu=on amd_iommu=on" >>> When machine is up, check that iommu is enabled in the output of cat >>> /proc/cmdline. If iommu is enabled, then run the following commands. >>> modprobe vfio enable_unsafe_noiommu_mode=Y modprobe vfio-pci >>> >>> Bind your dpdk interfaces to vfio-pci (*dpdk-devbind.py -b vfio-pci >>> <PCI_ID>*) and run the test. >>> >>> Regards >>> Dheeraj Dang >>> >>> >>> On Fri, Apr 10, 2020 at 6:16 PM Sunghyuk Kay <sunghyuk.kay@kaist.ac.kr> >>> wrote: >>> >>>> Hi all, >>>> I'm setting DPDK on our hardware to do some experiment, but suffering >>>> from . >>>> >>>> I was trying to use DPDK on Ubuntu 18.04 with Intel XL710 Ethernet >>>> Adapter >>>> with igb_uio driver. I've compiled DPDK from source, and bound the >>>> driver >>>> to NIC. >>>> But when I run `testpmd` application, it returns an error like this. >>>> >>>> ``` >>>> $ sudo ./build/app/testpmd >>>> EAL: Detected 128 lcore(s) >>>> EAL: Detected 1 NUMA nodes >>>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >>>> EAL: Selected IOVA mode 'PA' >>>> EAL: No available hugepages reported in hugepages-1048576kB >>>> EAL: Probing VFIO support... >>>> EAL: PCI device 0000:81:00.0 on NUMA socket 0 >>>> EAL: probe driver: 8086:1583 net_i40e >>>> eth_i40e_dev_init(): Failed to init adminq: -66 >>>> EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.0 >>>> cannot >>>> be used >>>> EAL: PCI device 0000:81:00.1 on NUMA socket 0 >>>> EAL: probe driver: 8086:1583 net_i40e >>>> eth_i40e_dev_init(): Failed to init adminq: -66 >>>> EAL: ethdev initialisation failedEAL: Requested device 0000:81:00.1 >>>> cannot >>>> be used >>>> EAL: PCI device 0000:c4:00.0 on NUMA socket 0 >>>> EAL: probe driver: 8086:1533 net_e1000_igb >>>> EAL: PCI device 0000:c5:00.0 on NUMA socket 0 >>>> EAL: probe driver: 8086:1533 net_e1000_igb >>>> testpmd: No probed ethernet devices >>>> testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=1163456, >>>> size=2176, >>>> socket=0 >>>> testpmd: preferred mempool ops selected: ring_mp_mc >>>> Done >>>> No commandline core given, start packet forwarding >>>> io packet forwarding - ports=0 - cores=0 - streams=0 - NUMA support >>>> enabled, MP allocation mode: native >>>> >>>> io packet forwarding packets/burst=32 >>>> nb forwarding cores=1 - nb forwarding ports=0 >>>> Press enter to exit >>>> >>>> Telling cores to stop... >>>> Waiting for lcores to finish... >>>> >>>> +++++++++++++++ Accumulated forward statistics for all >>>> ports+++++++++++++++ >>>> RX-packets: 0 RX-dropped: 0 RX-total: 0 >>>> TX-packets: 0 TX-dropped: 0 TX-total: 0 >>>> >>>> >>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> >>>> Done. >>>> >>>> Bye... >>>> ``` >>>> I tried to find what return code -66 from adminq means, but I couldn't >>>> find >>>> it. >>>> Some say this might have to do with Intel IOMMU pass-through, but I'm >>>> not >>>> sure that's the case. >>>> >>>> Has anyone had this issue before? Maybe some tip on this issue would be >>>> very helpful. >>>> >>>> Thank you, >>>> Sunghyuk >>>> >>> -- >> 감사합니다 >> 계성혁 드림 >> ========= >> 계성혁 (Kay, Sung Hyuk) >> KAIST Concurrency and Parallelism Laboratory >> Masters Student @ Software Graduate Program >> Mobile: 010-2794-6906 >> > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-04-16 4:01 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-10 8:29 [dpdk-users] "eth_i40e_dev_init(): Failed to init adminq: -66" problem Sunghyuk Kay
[not found] ` <CAJyVW49LOPQiVnorQ4=YECjDu4zfAci_7JmQx27hdA=TV9a2wQ@mail.gmail.com>
2020-04-10 13:31 ` Sunghyuk Kay
2020-04-10 14:29 ` Dheeraj Dang
2020-04-16 4:00 ` Sunghyuk Kay
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).