From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 8564CB0B4 for ; Tue, 17 Jun 2014 18:46:03 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP; 17 Jun 2014 09:46:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,495,1400050800"; d="scan'208";a="549292747" Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159]) by fmsmga001.fm.intel.com with ESMTP; 17 Jun 2014 09:41:32 -0700 Received: from irsmsx107.ger.corp.intel.com (163.33.3.99) by IRSMSX104.ger.corp.intel.com (163.33.3.159) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 17 Jun 2014 17:38:39 +0100 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.58]) by IRSMSX107.ger.corp.intel.com ([169.254.10.208]) with mapi id 14.03.0123.003; Tue, 17 Jun 2014 17:38:39 +0100 From: "Richardson, Bruce" To: "Richardson, Bruce" , "Burakov, Anatoly" , "dev@dpdk.org" Thread-Topic: vfio detection Thread-Index: Ac+Jr+QsX/7njye/SDa+CSH3FJFocAAVxw1QABBHEgAAAI5ncA== Date: Tue, 17 Jun 2014 16:38:38 +0000 Message-ID: <59AF69C657FD0841A61C55336867B5B01AA37143@IRSMSX103.ger.corp.intel.com> References: <59AF69C657FD0841A61C55336867B5B01AA36B6E@IRSMSX103.ger.corp.intel.com> <59AF69C657FD0841A61C55336867B5B01AA370A7@IRSMSX103.ger.corp.intel.com> In-Reply-To: <59AF69C657FD0841A61C55336867B5B01AA370A7@IRSMSX103.ger.corp.intel.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] vfio detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 16:46:05 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Richardson, Bruce > Sent: Tuesday, June 17, 2014 9:29 AM > To: Burakov, Anatoly; dev@dpdk.org > Subject: Re: [dpdk-dev] vfio detection >=20 > > -----Original Message----- > > From: Burakov, Anatoly > > Sent: Tuesday, June 17, 2014 1:40 AM > > To: Richardson, Bruce; dev@dpdk.org > > Subject: RE: vfio detection > > > > Hi Bruce, > > > > > I have a number of NIC ports which were working correctly yesterday a= nd > are > > > bound correctly to the igb_uio driver - and I want to keep using them > > > through the igb_uio driver for now, not vfio. However, whenever I run= a > > > dpdk application today, I find that the vfio kernel module is getting= loaded > > > each time - even after I manually remove it, and verify that it has b= een > > > removed by checking lsmod. Is this expected? If so, why are we loadin= g the > > > vfio driver when I just want to continue using igb_uio which works fi= ne? > > > > Can you elaborate a bit on what do you mean by "loading vfio driver"? D= o you > > mean the vfio-pci kernel gets loaded by DPDK? I certainly didn't put in= any code > > that would automatically load that driver, and certainly not binding de= vices to > it. >=20 > The kernel module called just "vfio" is constantly getting reloaded, and = there is > always a "/dev/vfio" directory, which triggers the vfio code handling eve= ry time I > run dpdk. >=20 > > > > > Secondly, then, when testpmd or any other app loads, it automatically= tries > > > to map the NIC using vfio and then aborts on the very first NIC port = when it > > > fails to do so. > > > > This shouldn't happen, unless you have a device bound to VFIO and have > another > > device in the same IOMMU group that is bound to something else. Can you > > provide a log of what you are seeing? >=20 > Log of testpmd run attached. Log got stripped from mail, including below instead. Script started on Tue 17 Jun 2014 17:23:54 IST bruce@silpixa00372841:dpdk.org$ ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:84:00.0 'Ethernet Server Adapter X520-4' drv=3Digb_uio unused=3Dixgbe 0000:87:00.0 'Ethernet Server Adapter X520-4' drv=3Digb_uio unused=3Dixgbe 0000:8b:00.0 'Ethernet Server Adapter X520-4' drv=3Digb_uio unused=3Dixgbe 0000:8e:00.0 'Ethernet Server Adapter X520-4' drv=3Digb_uio unused=3Dixgbe Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:04:00.0 'I350 Gigabit Network Connection' if=3Dem0 drv=3Digb unused=3D= igb_uio *Active* 0000:04:00.1 'I350 Gigabit Network Connection' if=3Dens2f1 drv=3Digb unused= =3Digb_uio=20 0000:04:00.2 'I350 Gigabit Network Connection' if=3Dens2f2 drv=3Digb unused= =3Digb_uio=20 0000:04:00.3 'I350 Gigabit Network Connection' if=3Dens2f3 drv=3Digb unused= =3Digb_uio=20 Other network devices =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:0a:00.1 'DH8900CC Null Device' unused=3Digb_uio 0000:0b:00.1 'DH8900CC Null Device' unused=3Digb_uio 0000:0c:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=3Dixgb= e,igb_uio 0000:0c:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=3Dixgb= e,igb_uio 0000:84:00.1 'Ethernet Server Adapter X520-4' unused=3Dixgbe,igb_uio 0000:87:00.1 'Ethernet Server Adapter X520-4' unused=3Dixgbe,igb_uio 0000:8b:00.1 'Ethernet Server Adapter X520-4' unused=3Dixgbe,igb_uio 0000:8e:00.1 'Ethernet Server Adapter X520-4' unused=3Dixgbe,igb_uio bruce@silpixa00372841:dpdk.org$ sudo ./x86_64-native-linuxapp-gcc/app/testp= md -c F00 -n 4 -- -i EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Detected lcore 8 as core 0 on socket 1 EAL: Detected lcore 9 as core 1 on socket 1 EAL: Detected lcore 10 as core 2 on socket 1 EAL: Detected lcore 11 as core 3 on socket 1 EAL: Detected lcore 12 as core 4 on socket 1 EAL: Detected lcore 13 as core 5 on socket 1 EAL: Detected lcore 14 as core 6 on socket 1 EAL: Detected lcore 15 as core 7 on socket 1 EAL: Detected lcore 16 as core 0 on socket 0 EAL: Detected lcore 17 as core 1 on socket 0 EAL: Detected lcore 18 as core 2 on socket 0 EAL: Detected lcore 19 as core 3 on socket 0 EAL: Detected lcore 20 as core 4 on socket 0 EAL: Detected lcore 21 as core 5 on socket 0 EAL: Detected lcore 22 as core 6 on socket 0 EAL: Detected lcore 23 as core 7 on socket 0 EAL: Detected lcore 24 as core 0 on socket 1 EAL: Detected lcore 25 as core 1 on socket 1 EAL: Detected lcore 26 as core 2 on socket 1 EAL: Detected lcore 27 as core 3 on socket 1 EAL: Detected lcore 28 as core 4 on socket 1 EAL: Detected lcore 29 as core 5 on socket 1 EAL: Detected lcore 30 as core 6 on socket 1 EAL: Detected lcore 31 as core 7 on socket 1 EAL: Support maximum 64 logical core(s) by configuration. EAL: Detected 32 lcore(s) EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs foun= d for that size EAL: Setting up memory... EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x7fe5c0000000 (size =3D 0x80000000) EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x7fe500000000 (size =3D 0x80000000) EAL: Requesting 2 pages of size 1024MB from socket 0 EAL: Requesting 2 pages of size 1024MB from socket 1 EAL: TSC frequency is ~2693503 KHz EAL: Master core 8 is ready (tid=3De877c880) EAL: Core 11 is ready (tid=3De6741700) EAL: Core 10 is ready (tid=3De6f42700) EAL: Core 9 is ready (tid=3De7743700) EAL: PCI device 0000:04:00.0 on NUMA socket 0 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: unknown IOMMU driver! EAL: 0000:04:00.0 cannot open VFIO container! EAL: Error - exiting with code: 1 Cause: Requested device 0000:04:00.0 cannot be used bruce@silpixa00372841:dpdk.org$ exit exit Script done on Tue 17 Jun 2014 17:25:29 IST