We are seeing following error, no device is detected.... ========== EAL: Detected 8 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: VFIO support initialized EAL: PCI device 0000:04:00.0 on NUMA socket 0 EAL: probe driver: 8086:1528 net_ixgbe EAL: 0000:04:00.0 failed to select IOMMU type EAL: Requested device 0000:04:00.0 cannot be used EAL: PCI device 0000:04:00.1 on NUMA socket 0 EAL: probe driver: 8086:1528 net_ixgbe EAL: 0000:04:00.1 failed to select IOMMU type EAL: Requested device 0000:04:00.1 cannot be used ================== dmesg [55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. [55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. ====== BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3 intel_iommu=on Can anybody suggest how to work around the issue ? Thanks, Regards, Venu
Hi Venumadhav, > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Venumadhav Josyula > Sent: Thursday, November 28, 2019 3:01 PM > To: dev@dpdk.org; users@dpdk.org > Cc: Venumadhav Josyula <vjosyula@parallelwireless.com> > Subject: [dpdk-dev] seeing a problem dpdk startup > > We are seeing following error, no device is detected.... > ========== > EAL: Detected 8 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: No free hugepages reported in hugepages-1048576kB > EAL: Probing VFIO support... > EAL: VFIO support initialized > EAL: PCI device 0000:04:00.0 on NUMA socket 0 > EAL: probe driver: 8086:1528 net_ixgbe > EAL: 0000:04:00.0 failed to select IOMMU type > EAL: Requested device 0000:04:00.0 cannot be used > EAL: PCI device 0000:04:00.1 on NUMA socket 0 > EAL: probe driver: 8086:1528 net_ixgbe > EAL: 0000:04:00.1 failed to select IOMMU type > EAL: Requested device 0000:04:00.1 cannot be used > > ================== > dmesg > > [55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain > attach due to platform RMRR requirement. Contact your platform vendor. > [55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU domain > attach due to platform RMRR requirement. Contact your platform vendor. > > ====== > BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3 > intel_iommu=on > > > Can anybody suggest how to work around the issue ? Did you check if the device is bound to DPDK? Here is an example on my host: gavin@net-arm-thunderx2-01:~/dpdk$ ./usertools/dpdk-devbind.py -s Network devices using DPDK-compatible driver ============================================ 0000:0f:00.0 'Ethernet Controller XL710 for 40GbE QSFP+ 1583' drv=vfio-pci unused=i40e 0000:8a:00.0 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=vfio-pci unused=i40e 0000:8a:00.1 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=vfio-pci unused=i40e > > Thanks, > Regards, > Venu
Hi Gavin,
It is
[root@localhost scripts]# ./dpdk-devbind.py --status-dev net
Network devices using DPDK-compatible driver
============================================
0000:04:00.0 'Ethernet Controller 10-Gigabit X540-AT2 1528' drv=vfio-pci
unused=ixgbe
0000:04:00.1 'Ethernet Controller 10-Gigabit X540-AT2 1528' drv=vfio-pci
unused=ixgbe
Network devices using kernel driver
===================================
0000:02:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno1 drv=tg3
unused=vfio-pci *Active*
0000:02:00.1 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno2 drv=tg3
unused=vfio-pci
0000:02:00.2 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno3 drv=tg3
unused=vfio-pci
0000:02:00.3 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno4 drv=tg3
unused=vfio-pci
[root@localhost scripts]#
I understand from the that IOMMU type selection is failing, is it possible
to somehow default it to either igb_uio ?
Thanks,
Regards,
Venu
On Thu, 28 Nov 2019 at 13:09, Gavin Hu (Arm Technology China) <
Gavin.Hu@arm.com> wrote:
> Hi Venumadhav,
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Venumadhav Josyula
> > Sent: Thursday, November 28, 2019 3:01 PM
> > To: dev@dpdk.org; users@dpdk.org
> > Cc: Venumadhav Josyula <vjosyula@parallelwireless.com>
> > Subject: [dpdk-dev] seeing a problem dpdk startup
> >
> > We are seeing following error, no device is detected....
> > ==========
> > EAL: Detected 8 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: No free hugepages reported in hugepages-1048576kB
> > EAL: Probing VFIO support...
> > EAL: VFIO support initialized
> > EAL: PCI device 0000:04:00.0 on NUMA socket 0
> > EAL: probe driver: 8086:1528 net_ixgbe
> > EAL: 0000:04:00.0 failed to select IOMMU type
> > EAL: Requested device 0000:04:00.0 cannot be used
> > EAL: PCI device 0000:04:00.1 on NUMA socket 0
> > EAL: probe driver: 8086:1528 net_ixgbe
> > EAL: 0000:04:00.1 failed to select IOMMU type
> > EAL: Requested device 0000:04:00.1 cannot be used
> >
> > ==================
> > dmesg
> >
> > [55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU
> domain
> > attach due to platform RMRR requirement. Contact your platform vendor.
> > [55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU
> domain
> > attach due to platform RMRR requirement. Contact your platform vendor.
> >
> > ======
> > BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3
> > intel_iommu=on
> >
> >
> > Can anybody suggest how to work around the issue ?
> Did you check if the device is bound to DPDK?
> Here is an example on my host:
> gavin@net-arm-thunderx2-01:~/dpdk$ ./usertools/dpdk-devbind.py -s
>
> Network devices using DPDK-compatible driver
> ============================================
> 0000:0f:00.0 'Ethernet Controller XL710 for 40GbE QSFP+ 1583' drv=vfio-pci
> unused=i40e
> 0000:8a:00.0 'Ethernet Controller XXV710 for 25GbE SFP28 158b'
> drv=vfio-pci unused=i40e
> 0000:8a:00.1 'Ethernet Controller XXV710 for 25GbE SFP28 158b'
> drv=vfio-pci unused=i40e
> >
> > Thanks,
> > Regards,
> > Venu
>
>We are seeing following error, no device is detected.... >========== >EAL: Detected 8 lcore(s) >EAL: Detected 2 NUMA nodes >EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >EAL: No free hugepages reported in hugepages-1048576kB >EAL: Probing VFIO support... >EAL: VFIO support initialized >EAL: PCI device 0000:04:00.0 on NUMA socket 0 >EAL: probe driver: 8086:1528 net_ixgbe >EAL: 0000:04:00.0 failed to select IOMMU type >EAL: Requested device 0000:04:00.0 cannot be used >EAL: PCI device 0000:04:00.1 on NUMA socket 0 >EAL: probe driver: 8086:1528 net_ixgbe >EAL: 0000:04:00.1 failed to select IOMMU type >EAL: Requested device 0000:04:00.1 cannot be used > >================== >dmesg > >[55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain >attach due to platform RMRR requirement. Contact your platform vendor. >[55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU domain >attach due to platform RMRR requirement. Contact your platform vendor. Intel Vt-d document addresses RMRR, but not that clearly. Check BIOS setup, no link share, for example, a nic detect by linux kernel, it was shared by BIOS for IPMI. If change BIOS setup don't solve your problem, contact platform vendor update nic firmware, a server made by HP, check HP's website for firmware. > >====== >BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3 >intel_iommu=on > > >Can anybody suggest how to work around the issue ? > >Thanks, >Regards, >Venu
Hi Hui,
I understand that and we are exploring that.
But is there no work around there where in it defaults to igb_uio driver
something like that..... Basic idea is we should not be situation of no
port detected at all under this situation ?
Thanks,
Regards,
Venu
On Thu, 28 Nov 2019 at 13:15, Hui Wei <huiweics@hotmail.com> wrote:
>
> >We are seeing following error, no device is detected....
> >==========
> >EAL: Detected 8 lcore(s)
> >EAL: Detected 2 NUMA nodes
> >EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> >EAL: No free hugepages reported in hugepages-1048576kB
> >EAL: Probing VFIO support...
> >EAL: VFIO support initialized
> >EAL: PCI device 0000:04:00.0 on NUMA socket 0
> >EAL: probe driver: 8086:1528 net_ixgbe
> >EAL: 0000:04:00.0 failed to select IOMMU type
> >EAL: Requested device 0000:04:00.0 cannot be used
> >EAL: PCI device 0000:04:00.1 on NUMA socket 0
> >EAL: probe driver: 8086:1528 net_ixgbe
> >EAL: 0000:04:00.1 failed to select IOMMU type
> >EAL: Requested device 0000:04:00.1 cannot be used
> >
> >==================
> >dmesg
> >
> >[55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU
> domain
> >attach due to platform RMRR requirement. Contact your platform vendor.
> >[55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU
> domain
> >attach due to platform RMRR requirement. Contact your platform vendor.
>
> Intel Vt-d document addresses RMRR, but not that clearly. Check BIOS
> setup,
> no link share, for example, a nic detect by linux kernel, it was shared by
> BIOS for
> IPMI. If change BIOS setup don't solve your problem, contact platform
> vendor update
> nic firmware, a server made by HP, check HP's website for firmware.
>
> >
> >======
> >BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3
> >intel_iommu=on
> >
> >
> >Can anybody suggest how to work around the issue ?
> >
> >Thanks,
> >Regards,
> >Venu
I think BIOS turn off vt-d, use igb_uio can work around.
>Hi Hui,
>
>I understand that and we are exploring that.
>
>But is there no work around there where in it defaults to igb_uio driver
>something like that..... Basic idea is we should not be situation of no
>port detected at all under this situation ?
>
>Thanks,
>Regards,
>Venu
>
>On Thu, 28 Nov 2019 at 13:15, Hui Wei <huiweics@hotmail.com> wrote:
>
>>
>> >We are seeing following error, no device is detected....
>> >==========
>> >EAL: Detected 8 lcore(s)
>> >EAL: Detected 2 NUMA nodes
>> >EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> >EAL: No free hugepages reported in hugepages-1048576kB
>> >EAL: Probing VFIO support...
>> >EAL: VFIO support initialized
>> >EAL: PCI device 0000:04:00.0 on NUMA socket 0
>> >EAL: probe driver: 8086:1528 net_ixgbe
>> >EAL: 0000:04:00.0 failed to select IOMMU type
>> >EAL: Requested device 0000:04:00.0 cannot be used
>> >EAL: PCI device 0000:04:00.1 on NUMA socket 0
>> >EAL: probe driver: 8086:1528 net_ixgbe
>> >EAL: 0000:04:00.1 failed to select IOMMU type
>> >EAL: Requested device 0000:04:00.1 cannot be used
>> >
>> >==================
>> >dmesg
>> >
>> >[55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU
>> domain
>> >attach due to platform RMRR requirement. Contact your platform vendor.
>> >[55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU
>> domain
>> >attach due to platform RMRR requirement. Contact your platform vendor.
>>
>> Intel Vt-d document addresses RMRR, but not that clearly. Check BIOS
>> setup,
>> no link share, for example, a nic detect by linux kernel, it was shared by
>> BIOS for
>> IPMI. If change BIOS setup don't solve your problem, contact platform
>> vendor update
>> nic firmware, a server made by HP, check HP's website for firmware.
>>
>> >
>> >======
>> >BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3
>> >intel_iommu=on
>> >
>> >
>> >Can anybody suggest how to work around the issue ?
>> >
>> >Thanks,
>> >Regards,
>> >Venu
Please see the HPE customer advisory at https://support.hpe.com/hpsc/doc/public/display?sp4ts.oid=7271241&docId=emr_na-c04781229&docLocale=en_US. You will need to disable the shared memory features for the NIC card. - Lee -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hui Wei Sent: Thursday, November 28, 2019 1:01 AM To: Venumadhav Josyula <vjosyula@gmail.com> Cc: dev <dev@dpdk.org>; users <users@dpdk.org>; Venumadhav Josyula <vjosyula@parallelwireless.com> Subject: Re: [dpdk-dev] seeing a problem dpdk startup I think BIOS turn off vt-d, use igb_uio can work around. >Hi Hui, > >I understand that and we are exploring that. > >But is there no work around there where in it defaults to igb_uio driver >something like that..... Basic idea is we should not be situation of no >port detected at all under this situation ? > >Thanks, >Regards, >Venu > >On Thu, 28 Nov 2019 at 13:15, Hui Wei <huiweics@hotmail.com> wrote: > >> >> >We are seeing following error, no device is detected.... >> >========== >> >EAL: Detected 8 lcore(s) >> >EAL: Detected 2 NUMA nodes >> >EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >> >EAL: No free hugepages reported in hugepages-1048576kB >> >EAL: Probing VFIO support... >> >EAL: VFIO support initialized >> >EAL: PCI device 0000:04:00.0 on NUMA socket 0 >> >EAL: probe driver: 8086:1528 net_ixgbe >> >EAL: 0000:04:00.0 failed to select IOMMU type >> >EAL: Requested device 0000:04:00.0 cannot be used >> >EAL: PCI device 0000:04:00.1 on NUMA socket 0 >> >EAL: probe driver: 8086:1528 net_ixgbe >> >EAL: 0000:04:00.1 failed to select IOMMU type >> >EAL: Requested device 0000:04:00.1 cannot be used >> > >> >================== >> >dmesg >> > >> >[55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU >> domain >> >attach due to platform RMRR requirement. Contact your platform vendor. >> >[55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU >> domain >> >attach due to platform RMRR requirement. Contact your platform vendor. >> >> Intel Vt-d document addresses RMRR, but not that clearly. Check BIOS >> setup, >> no link share, for example, a nic detect by linux kernel, it was shared by >> BIOS for >> IPMI. If change BIOS setup don't solve your problem, contact platform >> vendor update >> nic firmware, a server made by HP, check HP's website for firmware. >> >> > >> >====== >> >BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3 >> >intel_iommu=on >> > >> > >> >Can anybody suggest how to work around the issue ? >> > >> >Thanks, >> >Regards, >> >Venu
Hi Roberts,
Thanks for your reply. Basically, our BIOS is old we do not have that
feature in our BIOS , we need to upgrade it.
But after enabling will it work, can you confirm that based on your
experience or somebody informing you success story.
Thanks,
Regards,
Venu
On Thu, 28 Nov 2019 at 19:45, Roberts, Lee A. <lee.roberts@hpe.com> wrote:
> Please see the HPE customer advisory at
> https://support.hpe.com/hpsc/doc/public/display?sp4ts.oid=7271241&docId=emr_na-c04781229&docLocale=en_US
> .
> You will need to disable the shared memory features for the NIC card.
>
> - Lee
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hui Wei
> Sent: Thursday, November 28, 2019 1:01 AM
> To: Venumadhav Josyula <vjosyula@gmail.com>
> Cc: dev <dev@dpdk.org>; users <users@dpdk.org>; Venumadhav Josyula <
> vjosyula@parallelwireless.com>
> Subject: Re: [dpdk-dev] seeing a problem dpdk startup
>
> I think BIOS turn off vt-d, use igb_uio can work around.
>
> >Hi Hui,
> >
> >I understand that and we are exploring that.
> >
> >But is there no work around there where in it defaults to igb_uio driver
> >something like that..... Basic idea is we should not be situation of no
> >port detected at all under this situation ?
> >
> >Thanks,
> >Regards,
> >Venu
> >
> >On Thu, 28 Nov 2019 at 13:15, Hui Wei <huiweics@hotmail.com> wrote:
> >
> >>
> >> >We are seeing following error, no device is detected....
> >> >==========
> >> >EAL: Detected 8 lcore(s)
> >> >EAL: Detected 2 NUMA nodes
> >> >EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> >> >EAL: No free hugepages reported in hugepages-1048576kB
> >> >EAL: Probing VFIO support...
> >> >EAL: VFIO support initialized
> >> >EAL: PCI device 0000:04:00.0 on NUMA socket 0
> >> >EAL: probe driver: 8086:1528 net_ixgbe
> >> >EAL: 0000:04:00.0 failed to select IOMMU type
> >> >EAL: Requested device 0000:04:00.0 cannot be used
> >> >EAL: PCI device 0000:04:00.1 on NUMA socket 0
> >> >EAL: probe driver: 8086:1528 net_ixgbe
> >> >EAL: 0000:04:00.1 failed to select IOMMU type
> >> >EAL: Requested device 0000:04:00.1 cannot be used
> >> >
> >> >==================
> >> >dmesg
> >> >
> >> >[55455.259493] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU
> >> domain
> >> >attach due to platform RMRR requirement. Contact your platform vendor.
> >> >[55455.260947] vfio-pci 0000:04:00.1: Device is ineligible for IOMMU
> >> domain
> >> >attach due to platform RMRR requirement. Contact your platform vendor.
> >>
> >> Intel Vt-d document addresses RMRR, but not that clearly. Check BIOS
> >> setup,
> >> no link share, for example, a nic detect by linux kernel, it was shared
> by
> >> BIOS for
> >> IPMI. If change BIOS setup don't solve your problem, contact platform
> >> vendor update
> >> nic firmware, a server made by HP, check HP's website for firmware.
> >>
> >> >
> >> >======
> >> >BOOT_IMAGE=/boot/vmlinuz-3.10.0-957.el7.pw.1.x86_64 root=/dev/sda3
> >> >intel_iommu=on
> >> >
> >> >
> >> >Can anybody suggest how to work around the issue ?
> >> >
> >> >Thanks,
> >> >Regards,
> >> >Venu
>
On Thu, 28 Nov 2019 19:50:41 +0530 Venumadhav Josyula <vjosyula@gmail.com> wrote: > Hi Roberts, > > Thanks for your reply. Basically, our BIOS is old we do not have that > feature in our BIOS , we need to upgrade it. > > But after enabling will it work, can you confirm that based on your > experience or somebody informing you success story. > > Thanks, > Regards, > Venu This may help explain the issues. https://access.redhat.com/sites/default/files/attachments/rmrr-wp1.pdf