From: Stephen Hemminger <stephen@networkplumber.org>
To: Bruce Richardson <bruce.richardson@intel.com>
Cc: Nagendra Prabhu A T <nagendraprabhu.a.t@gmail.com>,
<anatoly.burakov@intel.com>, <dev@dpdk.org>
Subject: Re: unable to bind to vfio-pci
Date: Tue, 21 Feb 2023 09:03:52 -0800 [thread overview]
Message-ID: <20230221090352.739881a0@hermes.local> (raw)
In-Reply-To: <Y/Tw8wuVuUyLor4O@bricha3-MOBL.ger.corp.intel.com>
On Tue, 21 Feb 2023 16:27:31 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> On Tue, Feb 21, 2023 at 09:46:30PM +0530, Nagendra Prabhu A T wrote:
> > These are the messages i am seeing dmesg
> > [16588.298660] vfio-pci: probe of 0000:8a:00.2 failed with error -22
> > [16591.574630] i40e 0000:8a:00.3: i40e_ptp_stop: removed PHC on
> > enp138s0f3
> > [16593.138613] vfio-pci: probe of 0000:8a:00.3 failed with error -22
> > [16593.138674] vfio-pci: probe of 0000:8a:00.3 failed with error -22
> > [16627.551866] vfio-pci: probe of 0000:51:00.0 failed with error -22
> > [16627.551922] vfio-pci: probe of 0000:51:00.0 failed with error -22
> > [16716.344530] vfio-pci: probe of 0000:51:00.0 failed with error -22
> > [16716.344585] vfio-pci: probe of 0000:51:00.0 failed with error -22
> > [17859.876566] vfio-pci: probe of 0000:51:00.0 failed with error -22
> > [17859.876622] vfio-pci: probe of 0000:51:00.0 failed with error -22
> > root@nat-kvm:/home/nat# cat /boot/config-$(uname -r) | grep NOIOMMU
> > CONFIG_VFIO_NOIOMMU=y
> > root@nat-kvm:/home/nat#
> > Interface type :
> > Other Network devices
> > =====================
> > 0000:51:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:51:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:51:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:51:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:8a:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:8a:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:8a:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> > 0000:8a:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572'
> > unused=i40e,vfio-pci
> >
>
> Please try the following:
>
> echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
> dpdk-devbind.py -b vfio-pci 8a:00.*
>
> Does anything change then?
>
> /Bruce
>
> PS: For mailing list discussions, please don't top-post. Replies are easier
> to follow if posted immediately below the relevant content.
>
> > On Tue, Feb 21, 2023 at 9:37 PM Bruce Richardson
> > <[1]bruce.richardson@intel.com> wrote:
> >
> > On Tue, Feb 21, 2023 at 09:30:15PM +0530, Nagendra Prabhu A T wrote:
> > > HI Bruce,
> > > Thanks for your reply.
> > > I have removed the binding of all other ports. So , i can
> > confirm that
> > > #2 is taken care of.
> > > I tried using a different driver "uio_pci_generic". I still see
> > the
> > > same issue for that driver as well.
> > > The problem seems to be because the write operation is not
> > happening in
> > > "bind" file.
> > > Infact tried doing the following step manually. That also
> > failed
> > > echo sh -c 0000:51:00.0 >
> > /sys/bus/pci/drivers/uio_pci_generic/bind
> > > bash: echo: write error: No such device
> > The "sh -c" doesn't look right in that command. It should just echo
> > the
> > device id to the bind file. However, it's generally recommended just
> > to use
> > the devbind script, as more writes to sysfs may be necessary first,
> > e.g. to
> > register the device ids first.
> > > I have required permission for the file.
> > > I tried doing with "sudo"
> > > I dont have selinux running
> > > So not sure whats blocking me. Infact , even
> > > "cat /sys/bus/pci/drivers/uio_pci_generic/bind" is failing
> > > Can you please help.
> > To track down issues other than the obvious, I think we need more
> > information. For example, what NIC card is this? Are there any
> > useful error
> > messages in dmesg?
> > /Bruce
> >
> > References
> >
> > 1. mailto:bruce.richardson@intel.com
Another possibility is that the hardware shares IOMMU channels.
One machine I was using had multiple PCI devices sharing same IOMMU channel and kernel
would not let it work unless all of them were unbound from kernel.
next prev parent reply other threads:[~2023-02-21 17:03 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-21 14:35 Nagendra Prabhu A T
2023-02-21 14:45 ` Bruce Richardson
2023-02-21 16:00 ` Nagendra Prabhu A T
2023-02-21 16:06 ` Bruce Richardson
2023-02-21 16:16 ` Nagendra Prabhu A T
2023-02-21 16:27 ` Bruce Richardson
2023-02-21 17:03 ` Stephen Hemminger [this message]
2023-02-21 17:27 ` Nagendra Prabhu A T
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230221090352.739881a0@hermes.local \
--to=stephen@networkplumber.org \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=nagendraprabhu.a.t@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).