[dpdk-dev] unable to bind to vfio-pci

[dpdk-dev] unable to bind to vfio-pci

[Sarosh Arif]

I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
but am unable to do so. The reason behind this is that I am unable to
write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
I tried a couple of things such as setting iommu=pt and intel_iommu=on
and ensured vt-d is enabled.
Along with this I have made sure that the vfio-pci module is correctly
loaded. I have also tried

chmod 666 /sys/bus/pci/drivers/vfio-pci/bind

So that I have permissions to write in this file.

The error I get when I use usertools/dpdk-devbind.py to bind is this:
Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci

The details of 0000:b7:00.1 are as follows:
Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e

I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.

What more can be done to resolve this?

Regards,
Sarosh Arif

Re: [dpdk-dev] unable to bind to vfio-pci

[Dybkowski, AdamX]

Hi.

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Sarosh Arif
> Sent: Thursday, 17 September, 2020 08:53
> To: dev@dpdk.org
> Subject: [dpdk-dev] unable to bind to vfio-pci
> 
> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py but
> am unable to do so. The reason behind this is that I am unable to write in
> /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions I tried a couple
> of things such as setting iommu=pt and intel_iommu=on and ensured vt-d is
> enabled.

[Adam] First thing: to check if VT-d is enabled in Linux, you can read MSR 0x3a register. Its lowest bit set will show VT-d works:

(optionally on Ubuntu: apt-get install msr-tools)
modprobe msr
rdmsr 0x3a
3 or 5 -> OK

Other possible way to check for VT-d (IOMMU) working is to view the booting log (dmesg):

dmesg | grep DMAR
"DMAR: IOMMU enabled" -> OK

Both of these should be met in order to use vfio-pci in the next step.

> Along with this I have made sure that the vfio-pci module is correctly loaded.
> I have also tried
> 
> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> 
> So that I have permissions to write in this file.
> 
> The error I get when I use usertools/dpdk-devbind.py to bind is this:
> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> 
> The details of 0000:b7:00.1 are as follows:
> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> 
> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> 
> What more can be done to resolve this?
> 
> Regards,
> Sarosh Arif

Adam Dybkowski

Re: [dpdk-dev] unable to bind to vfio-pci

[Sarosh Arif]

Hello,

On Thu, Sep 17, 2020 at 12:32 PM Dybkowski, AdamX
<adamx.dybkowski@intel.com> wrote:
>
> Hi.
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Sarosh Arif
> > Sent: Thursday, 17 September, 2020 08:53
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] unable to bind to vfio-pci
> >
> > I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py but
> > am unable to do so. The reason behind this is that I am unable to write in
> > /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions I tried a couple
> > of things such as setting iommu=pt and intel_iommu=on and ensured vt-d is
> > enabled.
>
> [Adam] First thing: to check if VT-d is enabled in Linux, you can read MSR 0x3a register. Its lowest bit set will show VT-d works:
>
> (optionally on Ubuntu: apt-get install msr-tools)
> modprobe msr
> rdmsr 0x3a
> 3 or 5 -> OK
>
> Other possible way to check for VT-d (IOMMU) working is to view the booting log (dmesg):
>
> dmesg | grep DMAR
> "DMAR: IOMMU enabled" -> OK
>
I used this method to check and vt-d is enabled.
Is there anything else that I can do?
> Both of these should be met in order to use vfio-pci in the next step.
>
> > Along with this I have made sure that the vfio-pci module is correctly loaded.
> > I have also tried
> >
> > chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> >
> > So that I have permissions to write in this file.
> >
> > The error I get when I use usertools/dpdk-devbind.py to bind is this:
> > Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> >
> > The details of 0000:b7:00.1 are as follows:
> > Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> >
> > I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> >
> > What more can be done to resolve this?
> >
> > Regards,
> > Sarosh Arif
>
> Adam Dybkowski
>

Re: [dpdk-dev] unable to bind to vfio-pci

[Bruce Richardson]

On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
> but am unable to do so. The reason behind this is that I am unable to
> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
> I tried a couple of things such as setting iommu=pt and intel_iommu=on
> and ensured vt-d is enabled.
> Along with this I have made sure that the vfio-pci module is correctly
> loaded. I have also tried
> 
> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> 
> So that I have permissions to write in this file.
> 
> The error I get when I use usertools/dpdk-devbind.py to bind is this:
> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> 
> The details of 0000:b7:00.1 are as follows:
> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> 
> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> 
> What more can be done to resolve this?
>
Since you describe changing permissions on the "bind" file, are you trying
to run dpdk-devbind.py as a non-root user? Does it work as root? 

Re: [dpdk-dev] unable to bind to vfio-pci

[Sarosh Arif]

On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
> > I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
> > but am unable to do so. The reason behind this is that I am unable to
> > write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
> > I tried a couple of things such as setting iommu=pt and intel_iommu=on
> > and ensured vt-d is enabled.
> > Along with this I have made sure that the vfio-pci module is correctly
> > loaded. I have also tried
> >
> > chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> >
> > So that I have permissions to write in this file.
> >
> > The error I get when I use usertools/dpdk-devbind.py to bind is this:
> > Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> >
> > The details of 0000:b7:00.1 are as follows:
> > Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> >
> > I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> >
> > What more can be done to resolve this?
> >
> Since you describe changing permissions on the "bind" file, are you trying
> to run dpdk-devbind.py as a non-root user? Does it work as root?
I am running it as a root user. It does not work as a root user.

Re: [dpdk-dev] unable to bind to vfio-pci

[Burakov, Anatoly]

On 17-Sep-20 10:21 AM, Sarosh Arif wrote:
> On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
>>
>> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
>>> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
>>> but am unable to do so. The reason behind this is that I am unable to
>>> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
>>> I tried a couple of things such as setting iommu=pt and intel_iommu=on
>>> and ensured vt-d is enabled.
>>> Along with this I have made sure that the vfio-pci module is correctly
>>> loaded. I have also tried
>>>
>>> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
>>>
>>> So that I have permissions to write in this file.
>>>
>>> The error I get when I use usertools/dpdk-devbind.py to bind is this:
>>> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
>>>
>>> The details of 0000:b7:00.1 are as follows:
>>> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
>>>
>>> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
>>>
>>> What more can be done to resolve this?
>>>
>> Since you describe changing permissions on the "bind" file, are you trying
>> to run dpdk-devbind.py as a non-root user? Does it work as root?
> I am running it as a root user. It does not work as a root user.
> 

Does "dmesg | tail" say anything of interest?

-- 
Thanks,
Anatoly

Re: [dpdk-dev] unable to bind to vfio-pci

[Sarosh Arif]

On Thu, Sep 17, 2020 at 2:35 PM Burakov, Anatoly
<anatoly.burakov@intel.com> wrote:
>
> On 17-Sep-20 10:21 AM, Sarosh Arif wrote:
> > On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> >>
> >> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
> >>> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
> >>> but am unable to do so. The reason behind this is that I am unable to
> >>> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
> >>> I tried a couple of things such as setting iommu=pt and intel_iommu=on
> >>> and ensured vt-d is enabled.
> >>> Along with this I have made sure that the vfio-pci module is correctly
> >>> loaded. I have also tried
> >>>
> >>> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> >>>
> >>> So that I have permissions to write in this file.
> >>>
> >>> The error I get when I use usertools/dpdk-devbind.py to bind is this:
> >>> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> >>>
> >>> The details of 0000:b7:00.1 are as follows:
> >>> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> >>>
> >>> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> >>>
> >>> What more can be done to resolve this?
> >>>
> >> Since you describe changing permissions on the "bind" file, are you trying
> >> to run dpdk-devbind.py as a non-root user? Does it work as root?
> > I am running it as a root user. It does not work as a root user.
> >
>
> Does "dmesg | tail" say anything of interest?
This is the output of dmesg | tail:
[136286.136271] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
[221230.023654] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Up 10 Gbps,
Flow Control: RX/TX
[221230.024134] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
[249273.956525] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Up 10 Gbps,
Flow Control: RX/TX
[249273.957003] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
[314864.386303] EXT4-fs (sda1): mounting ext3 file system using the
ext4 subsystem
[314867.734973] EXT4-fs (sda1): mounted filesystem with ordered data
mode. Opts: (null)
[332584.888223] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Up 10 Gbps,
Flow Control: RX/TX
[332584.888700] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
[358429.954026] VFIO - User Level meta-driver version: 0.3
>
> --
> Thanks,
> Anatoly

Re: [dpdk-dev] unable to bind to vfio-pci

[Burakov, Anatoly]

On 21-Sep-20 12:54 PM, Sarosh Arif wrote:
> On Thu, Sep 17, 2020 at 2:35 PM Burakov, Anatoly
> <anatoly.burakov@intel.com> wrote:
>>
>> On 17-Sep-20 10:21 AM, Sarosh Arif wrote:
>>> On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
>>> <bruce.richardson@intel.com> wrote:
>>>>
>>>> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
>>>>> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
>>>>> but am unable to do so. The reason behind this is that I am unable to
>>>>> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
>>>>> I tried a couple of things such as setting iommu=pt and intel_iommu=on
>>>>> and ensured vt-d is enabled.
>>>>> Along with this I have made sure that the vfio-pci module is correctly
>>>>> loaded. I have also tried
>>>>>
>>>>> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
>>>>>
>>>>> So that I have permissions to write in this file.
>>>>>
>>>>> The error I get when I use usertools/dpdk-devbind.py to bind is this:
>>>>> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
>>>>>
>>>>> The details of 0000:b7:00.1 are as follows:
>>>>> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
>>>>>
>>>>> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
>>>>>
>>>>> What more can be done to resolve this?
>>>>>
>>>> Since you describe changing permissions on the "bind" file, are you trying
>>>> to run dpdk-devbind.py as a non-root user? Does it work as root?
>>> I am running it as a root user. It does not work as a root user.
>>>
>>
>> Does "dmesg | tail" say anything of interest?
> This is the output of dmesg | tail:
> [136286.136271] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
> [221230.023654] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Up 10 Gbps,
> Flow Control: RX/TX
> [221230.024134] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
> [249273.956525] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Up 10 Gbps,
> Flow Control: RX/TX
> [249273.957003] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
> [314864.386303] EXT4-fs (sda1): mounting ext3 file system using the
> ext4 subsystem
> [314867.734973] EXT4-fs (sda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [332584.888223] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Up 10 Gbps,
> Flow Control: RX/TX
> [332584.888700] ixgbe 0000:65:00.0 enp101s0f0: NIC Link is Down
> [358429.954026] VFIO - User Level meta-driver version: 0.3

So, nothing useful :)

To me, it starts to sound more and more like some kind of security 
measure. Either it's something like AppArmor/SELinux preventing you from 
binding the drivers, or maybe it's something like Secure Boot, or some 
other security-related feature. A "permission denied" error is usually 
indicative of such things.

I'm not really an expert on this so i can't tell you off the top of my 
head what to check, but my first stop would have been either Secure 
Boot-related settings, or SELinux/AppArmor logs.

>>
>> --
>> Thanks,
>> Anatoly


-- 
Thanks,
Anatoly

Re: [dpdk-dev] unable to bind to vfio-pci

[Bruce Richardson]

On Thu, Sep 17, 2020 at 02:21:27PM +0500, Sarosh Arif wrote:
> On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
> > > I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
> > > but am unable to do so. The reason behind this is that I am unable to
> > > write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
> > > I tried a couple of things such as setting iommu=pt and intel_iommu=on
> > > and ensured vt-d is enabled.
> > > Along with this I have made sure that the vfio-pci module is correctly
> > > loaded. I have also tried
> > >
> > > chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> > >
> > > So that I have permissions to write in this file.
> > >
> > > The error I get when I use usertools/dpdk-devbind.py to bind is this:
> > > Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> > >
> > > The details of 0000:b7:00.1 are as follows:
> > > Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> > >
> > > I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> > >
> > > What more can be done to resolve this?
> > >
> > Since you describe changing permissions on the "bind" file, are you trying
> > to run dpdk-devbind.py as a non-root user? Does it work as root?
> I am running it as a root user. It does not work as a root user.

One possible problem that it could be, is that you will need to ensure that
any other ports on the same device are either similarly bound to vfio-pci
or not bound to any driver. You can't have e.g. a 2-port X722 NIC where one
port is bound to the kernel driver, while another is bound to vfio in
userspace.

Re: [dpdk-dev] unable to bind to vfio-pci

[Burakov, Anatoly]

On 17-Sep-20 11:22 AM, Bruce Richardson wrote:
> On Thu, Sep 17, 2020 at 02:21:27PM +0500, Sarosh Arif wrote:
>> On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
>> <bruce.richardson@intel.com> wrote:
>>>
>>> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
>>>> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
>>>> but am unable to do so. The reason behind this is that I am unable to
>>>> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
>>>> I tried a couple of things such as setting iommu=pt and intel_iommu=on
>>>> and ensured vt-d is enabled.
>>>> Along with this I have made sure that the vfio-pci module is correctly
>>>> loaded. I have also tried
>>>>
>>>> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
>>>>
>>>> So that I have permissions to write in this file.
>>>>
>>>> The error I get when I use usertools/dpdk-devbind.py to bind is this:
>>>> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
>>>>
>>>> The details of 0000:b7:00.1 are as follows:
>>>> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
>>>>
>>>> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
>>>>
>>>> What more can be done to resolve this?
>>>>
>>> Since you describe changing permissions on the "bind" file, are you trying
>>> to run dpdk-devbind.py as a non-root user? Does it work as root?
>> I am running it as a root user. It does not work as a root user.
> 
> One possible problem that it could be, is that you will need to ensure that
> any other ports on the same device are either similarly bound to vfio-pci
> or not bound to any driver. You can't have e.g. a 2-port X722 NIC where one
> port is bound to the kernel driver, while another is bound to vfio in
> userspace.
> 

I don't think this would result in a bind error; rather, you'd see the 
IOMMU group as non-viable when attempting to use it with VFIO, but the 
binding itself should succeed.

-- 
Thanks,
Anatoly

Re: [dpdk-dev] unable to bind to vfio-pci

[Sarosh Arif]

On Thu, Sep 17, 2020 at 4:50 PM Burakov, Anatoly
<anatoly.burakov@intel.com> wrote:
>
> On 17-Sep-20 11:22 AM, Bruce Richardson wrote:
> > On Thu, Sep 17, 2020 at 02:21:27PM +0500, Sarosh Arif wrote:
> >> On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
> >> <bruce.richardson@intel.com> wrote:
> >>>
> >>> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
> >>>> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
> >>>> but am unable to do so. The reason behind this is that I am unable to
> >>>> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
> >>>> I tried a couple of things such as setting iommu=pt and intel_iommu=on
> >>>> and ensured vt-d is enabled.
> >>>> Along with this I have made sure that the vfio-pci module is correctly
> >>>> loaded. I have also tried
> >>>>
> >>>> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
> >>>>
> >>>> So that I have permissions to write in this file.
> >>>>
> >>>> The error I get when I use usertools/dpdk-devbind.py to bind is this:
> >>>> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
> >>>>
> >>>> The details of 0000:b7:00.1 are as follows:
> >>>> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
> >>>>
> >>>> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
> >>>>
> >>>> What more can be done to resolve this?
> >>>>
> >>> Since you describe changing permissions on the "bind" file, are you trying
> >>> to run dpdk-devbind.py as a non-root user? Does it work as root?
> >> I am running it as a root user. It does not work as a root user.
> >
> > One possible problem that it could be, is that you will need to ensure that
> > any other ports on the same device are either similarly bound to vfio-pci
> > or not bound to any driver. You can't have e.g. a 2-port X722 NIC where one
> > port is bound to the kernel driver, while another is bound to vfio in
> > userspace.
> >
I unbinded all other ports from the kernel driver yet I could not bind
to vfio. The problem seems to be the inability to write in
/sys/bus/pci/drivers/vfio-pci/bind and
/sys/bus/pci/drivers/vfio-pci/new_id. When I manually try to echo in
these files I get "Permission denied".
>
> I don't think this would result in a bind error; rather, you'd see the
> IOMMU group as non-viable when attempting to use it with VFIO, but the
> binding itself should succeed.
>
> --
> Thanks,
> Anatoly

Re: [dpdk-dev] unable to bind to vfio-pci

[Burakov, Anatoly]

On 21-Sep-20 12:10 PM, Sarosh Arif wrote:
> On Thu, Sep 17, 2020 at 4:50 PM Burakov, Anatoly
> <anatoly.burakov@intel.com> wrote:
>>
>> On 17-Sep-20 11:22 AM, Bruce Richardson wrote:
>>> On Thu, Sep 17, 2020 at 02:21:27PM +0500, Sarosh Arif wrote:
>>>> On Thu, Sep 17, 2020 at 2:17 PM Bruce Richardson
>>>> <bruce.richardson@intel.com> wrote:
>>>>>
>>>>> On Thu, Sep 17, 2020 at 11:52:58AM +0500, Sarosh Arif wrote:
>>>>>> I have been trying to bind to vfio-pci using usertools/dpdk-devbind.py
>>>>>> but am unable to do so. The reason behind this is that I am unable to
>>>>>> write in /sys/bus/pci/drivers/vfio-pci/bind. Upon searching solutions
>>>>>> I tried a couple of things such as setting iommu=pt and intel_iommu=on
>>>>>> and ensured vt-d is enabled.
>>>>>> Along with this I have made sure that the vfio-pci module is correctly
>>>>>> loaded. I have also tried
>>>>>>
>>>>>> chmod 666 /sys/bus/pci/drivers/vfio-pci/bind
>>>>>>
>>>>>> So that I have permissions to write in this file.
>>>>>>
>>>>>> The error I get when I use usertools/dpdk-devbind.py to bind is this:
>>>>>> Error: bind failed for 0000:b7:00.1 - Cannot bind to driver vfio-pci
>>>>>>
>>>>>> The details of 0000:b7:00.1 are as follows:
>>>>>> Ethernet Connection X722 for 10GBASE-T 37d2' if=eno6 drv=i40e
>>>>>>
>>>>>> I have also unbinded The pci bridge to which 0000:b7:00.1 was connected.
>>>>>>
>>>>>> What more can be done to resolve this?
>>>>>>
>>>>> Since you describe changing permissions on the "bind" file, are you trying
>>>>> to run dpdk-devbind.py as a non-root user? Does it work as root?
>>>> I am running it as a root user. It does not work as a root user.
>>>
>>> One possible problem that it could be, is that you will need to ensure that
>>> any other ports on the same device are either similarly bound to vfio-pci
>>> or not bound to any driver. You can't have e.g. a 2-port X722 NIC where one
>>> port is bound to the kernel driver, while another is bound to vfio in
>>> userspace.
>>>
> I unbinded all other ports from the kernel driver yet I could not bind
> to vfio. The problem seems to be the inability to write in
> /sys/bus/pci/drivers/vfio-pci/bind and
> /sys/bus/pci/drivers/vfio-pci/new_id. When I manually try to echo in
> these files I get "Permission denied".

Does dmesg | tail indicate anything?

>>
>> I don't think this would result in a bind error; rather, you'd see the
>> IOMMU group as non-viable when attempting to use it with VFIO, but the
>> binding itself should succeed.
>>
>> --
>> Thanks,
>> Anatoly


-- 
Thanks,
Anatoly