From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1EE5A41D4F; Thu, 23 Feb 2023 10:00:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4924943122; Thu, 23 Feb 2023 10:00:08 +0100 (CET) Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by mails.dpdk.org (Postfix) with ESMTP id 759AF40E0F for ; Tue, 21 Feb 2023 17:16:42 +0100 (CET) Received: by mail-pl1-f178.google.com with SMTP id h14so5726707plf.10 for ; Tue, 21 Feb 2023 08:16:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=DKdVx8qtQmXActAHgFYlzr+2wdCvpFIf5qFVlUUfNEc=; b=pDR1rzZOJyVsCUQkZnWXPiZE2LuvNEsSN2mPHaw3jwyTXwemZ8eQ18YP5yeqSFsXrv IqEqAzn2285E76L2Qju1TjgPhEikF+M3NHD7dKi0yDI93hxBdg5juBg4amdGnnq/ChlG aHEDLEKWs7FQdzFEkA0ukopxSILzyUOPX0odQ8a+0LHPIM7u6ZlwMMSxqjb2XwCb79PR 2uL3pN9++kCRgH7c0cWX+V97+pAHHpGU255z55dQ7crDFyVUbSi5TBcVdKoCvYhlOi0d FX2Pu58JmWmo4xIyzCeKBuyU0wABggzbLu+rV9TaaMPAkpnQw6llbQeL5+GPAvyujpvH sBYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DKdVx8qtQmXActAHgFYlzr+2wdCvpFIf5qFVlUUfNEc=; b=G6jKpFcvL4GC4WYUagZ7orPudy48Z7Lg0UBqWkgUU4f/lh1Ao7IyMh+qtQQRB1bafm HtvAowZbWaE1DberYAtK+J+Wdm9gUkTvgc+3HrAPIeifyw35ym7iTu0JI3fkAbBx0jQp uCAgIfCVMB5mGgaWkMAKqpHSaet9T3aOLDgCciVhSjwTc1rjibxEQXpZ24v9+82Dzl9S 8KjFw3LEc7mel0mLqjDoqOl7G/kkwWp2U8keTlyRw1T+7XjWls9JRrXueieQ1Ihzg+GO 3V5qqEqUDf0ookDGsjqwMx7khU+iiRMMtZ6B8yNkzi3mnPsCMI7XA2lXeBlY+m0oWfPH dMpg== X-Gm-Message-State: AO0yUKWujvA/CjYuQbQPM+EZ6TzWmRUa7vlzWWZ5CCIjJurG5oC/nyEF 0X4Jpv30lLXzszLIeROi4rdA74LH/Uhi6pGkKP4SjriI+aWAkw== X-Google-Smtp-Source: AK7set+HIn9/qUSi+RjNm9ScWx1DhO8359GkB4Ram6Vjx7vkqeZ2ChddFoJqLNBTX5Aap5td3WVballIvMEcOvvPCio= X-Received: by 2002:a17:90a:e7cc:b0:236:6dea:8794 with SMTP id kb12-20020a17090ae7cc00b002366dea8794mr3093633pjb.22.1676996201519; Tue, 21 Feb 2023 08:16:41 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Nagendra Prabhu A T Date: Tue, 21 Feb 2023 21:46:30 +0530 Message-ID: Subject: Re: unable to bind to vfio-pci To: Bruce Richardson Cc: anatoly.burakov@intel.com, dev@dpdk.org Content-Type: multipart/alternative; boundary="000000000000e19cf505f5381afd" X-Mailman-Approved-At: Thu, 23 Feb 2023 10:00:05 +0100 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000e19cf505f5381afd Content-Type: text/plain; charset="UTF-8" These are the messages i am seeing dmesg [16588.298660] vfio-pci: probe of 0000:8a:00.2 failed with error -22 [16591.574630] i40e 0000:8a:00.3: i40e_ptp_stop: removed PHC on enp138s0f3 [16593.138613] vfio-pci: probe of 0000:8a:00.3 failed with error -22 [16593.138674] vfio-pci: probe of 0000:8a:00.3 failed with error -22 [16627.551866] vfio-pci: probe of 0000:51:00.0 failed with error -22 [16627.551922] vfio-pci: probe of 0000:51:00.0 failed with error -22 [16716.344530] vfio-pci: probe of 0000:51:00.0 failed with error -22 [16716.344585] vfio-pci: probe of 0000:51:00.0 failed with error -22 [17859.876566] vfio-pci: probe of 0000:51:00.0 failed with error -22 [17859.876622] vfio-pci: probe of 0000:51:00.0 failed with error -22 root@nat-kvm:/home/nat# cat /boot/config-$(uname -r) | grep NOIOMMU CONFIG_VFIO_NOIOMMU=y root@nat-kvm:/home/nat# Interface type : Other Network devices ===================== 0000:51:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:51:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:51:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:51:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:8a:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:8a:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:8a:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci 0000:8a:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=i40e,vfio-pci On Tue, Feb 21, 2023 at 9:37 PM Bruce Richardson wrote: > On Tue, Feb 21, 2023 at 09:30:15PM +0530, Nagendra Prabhu A T wrote: > > HI Bruce, > > Thanks for your reply. > > I have removed the binding of all other ports. So , i can confirm that > > #2 is taken care of. > > I tried using a different driver "uio_pci_generic". I still see the > > same issue for that driver as well. > > The problem seems to be because the write operation is not happening > in > > "bind" file. > > Infact tried doing the following step manually. That also failed > > echo sh -c 0000:51:00.0 > /sys/bus/pci/drivers/uio_pci_generic/bind > > bash: echo: write error: No such device > > The "sh -c" doesn't look right in that command. It should just echo the > device id to the bind file. However, it's generally recommended just to use > the devbind script, as more writes to sysfs may be necessary first, e.g. to > register the device ids first. > > > I have required permission for the file. > > I tried doing with "sudo" > > I dont have selinux running > > So not sure whats blocking me. Infact , even > > "cat /sys/bus/pci/drivers/uio_pci_generic/bind" is failing > > Can you please help. > > To track down issues other than the obvious, I think we need more > information. For example, what NIC card is this? Are there any useful error > messages in dmesg? > > /Bruce > --000000000000e19cf505f5381afd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
These are the messages i= am seeing dmesg

[16588.298660] vfio-pci: probe of = 0000:8a:00.2 failed with error -22
[16591.574630] i40e 0000:8a:00= .3: i40e_ptp_stop: removed PHC on enp138s0f3
[16593.138613] vfio-= pci: probe of 0000:8a:00.3 failed with error -22
[16593.138674] v= fio-pci: probe of 0000:8a:00.3 failed with error -22
[16627.55186= 6] vfio-pci: probe of 0000:51:00.0 failed with error -22
[16627.5= 51922] vfio-pci: probe of 0000:51:00.0 failed with error -22
[167= 16.344530] vfio-pci: probe of 0000:51:00.0 failed with error -22
= [16716.344585] vfio-pci: probe of 0000:51:00.0 failed with error -22
<= div>[17859.876566] vfio-pci: probe of 0000:51:00.0 failed with error -22
[17859.876622] vfio-pci: probe of 0000:51:00.0 failed with error -2= 2
root@nat-kvm:/home/nat# cat /boot/config-$(uname -r) | grep NOI= OMMU
CONFIG_VFIO_NOIOMMU=3Dy
root@nat-kvm:/home/nat#



Interface type :=

Other Network devices
=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
0000:51:00.0 = 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=3Di40e,vfio-p= ci
0000:51:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572= ' unused=3Di40e,vfio-pci
0000:51:00.2 'Ethernet Controlle= r X710 for 10GbE SFP+ 1572' unused=3Di40e,vfio-pci
0000:51:00= .3 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=3Di40e,vfi= o-pci
0000:8a:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1= 572' unused=3Di40e,vfio-pci
0000:8a:00.1 'Ethernet Contro= ller X710 for 10GbE SFP+ 1572' unused=3Di40e,vfio-pci
0000:8a= :00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' unused=3Di40e,= vfio-pci
0000:8a:00.3 'Ethernet Controller X710 for 10GbE SFP= + 1572' unused=3Di40e,vfio-pci


On= Tue, Feb 21, 2023 at 9:37 PM Bruce Richardson <bruce.richardson@intel.com> wrote:
On Tue, Feb 21, 2023 at 09:= 30:15PM +0530, Nagendra Prabhu A T wrote:
>=C2=A0 =C2=A0 HI Bruce,
>=C2=A0 =C2=A0 Thanks for your reply.
>=C2=A0 =C2=A0 I have removed the binding of all other ports. So , i can= confirm that
>=C2=A0 =C2=A0 #2 is taken care of.
>=C2=A0 =C2=A0 I tried using a different driver "uio_pci_generic&qu= ot;. I still see the
>=C2=A0 =C2=A0 same issue for that driver as well.
>=C2=A0 =C2=A0 The problem seems to be because the write operation is no= t happening in
>=C2=A0 =C2=A0 "bind" file.
>=C2=A0 =C2=A0 Infact tried doing the following step manually. That also= failed
>=C2=A0 =C2=A0 echo sh -c 0000:51:00.0 > /sys/bus/pci/drivers/uio_pci= _generic/bind
>=C2=A0 =C2=A0 bash: echo: write error: No such device

The "sh -c" doesn't look right in that command. It should jus= t echo the
device id to the bind file. However, it's generally recommended just to= use
the devbind script, as more writes to sysfs may be necessary first, e.g. to=
register the device ids first.

>=C2=A0 =C2=A0 I have required permission for the file.
>=C2=A0 =C2=A0 I tried doing with "sudo"
>=C2=A0 =C2=A0 I dont have selinux running
>=C2=A0 =C2=A0 So not sure whats blocking me. Infact , even
>=C2=A0 =C2=A0 "cat /sys/bus/pci/drivers/uio_pci_generic/bind"= is failing
>=C2=A0 =C2=A0 Can you please help.

To track down issues other than the obvious, I think we need more
information. For example, what NIC card is this? Are there any useful error=
messages in dmesg?

/Bruce
--000000000000e19cf505f5381afd--