From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id 95CF5AFCE for ; Tue, 17 Jun 2014 19:41:47 +0200 (CEST) Received: from hmsreliant.think-freely.org ([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1WwxOO-0004Bb-7m; Tue, 17 Jun 2014 13:42:02 -0400 Date: Tue, 17 Jun 2014 13:41:55 -0400 From: Neil Horman To: "Richardson, Bruce" Message-ID: <20140617174155.GF8539@hmsreliant.think-freely.org> References: <59AF69C657FD0841A61C55336867B5B01AA36B6E@IRSMSX103.ger.corp.intel.com> <59AF69C657FD0841A61C55336867B5B01AA370A7@IRSMSX103.ger.corp.intel.com> <59AF69C657FD0841A61C55336867B5B01AA37143@IRSMSX103.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <59AF69C657FD0841A61C55336867B5B01AA37143@IRSMSX103.ger.corp.intel.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-Spam-Status: No Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] vfio detection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 17:41:48 -0000 On Tue, Jun 17, 2014 at 04:38:38PM +0000, Richardson, Bruce wrote: > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Richardson, Bruce > > Sent: Tuesday, June 17, 2014 9:29 AM > > To: Burakov, Anatoly; dev@dpdk.org > > Subject: Re: [dpdk-dev] vfio detection > > > > > -----Original Message----- > > > From: Burakov, Anatoly > > > Sent: Tuesday, June 17, 2014 1:40 AM > > > To: Richardson, Bruce; dev@dpdk.org > > > Subject: RE: vfio detection > > > > > > Hi Bruce, > > > > > > > I have a number of NIC ports which were working correctly yesterday and > > are > > > > bound correctly to the igb_uio driver - and I want to keep using them > > > > through the igb_uio driver for now, not vfio. However, whenever I run a > > > > dpdk application today, I find that the vfio kernel module is getting loaded > > > > each time - even after I manually remove it, and verify that it has been > > > > removed by checking lsmod. Is this expected? If so, why are we loading the > > > > vfio driver when I just want to continue using igb_uio which works fine? > > > > > > Can you elaborate a bit on what do you mean by "loading vfio driver"? Do you > > > mean the vfio-pci kernel gets loaded by DPDK? I certainly didn't put in any code > > > that would automatically load that driver, and certainly not binding devices to > > it. > > > > The kernel module called just "vfio" is constantly getting reloaded, and there is > > always a "/dev/vfio" directory, which triggers the vfio code handling every time I > > run dpdk. > > > > > > > > > Secondly, then, when testpmd or any other app loads, it automatically tries > > > > to map the NIC using vfio and then aborts on the very first NIC port when it > > > > fails to do so. > > > > > > This shouldn't happen, unless you have a device bound to VFIO and have > > another > > > device in the same IOMMU group that is bound to something else. Can you > > > provide a log of what you are seeing? > > > > Log of testpmd run attached. > > Log got stripped from mail, including below instead. > > Script started on Tue 17 Jun 2014 17:23:54 IST > > bruce@silpixa00372841:dpdk.org$ ./tools/dpdk_nic_bind.py --status > > Network devices using DPDK-compatible driver > ============================================ > 0000:84:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe > 0000:87:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe > 0000:8b:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe > 0000:8e:00.0 'Ethernet Server Adapter X520-4' drv=igb_uio unused=ixgbe > > Network devices using kernel driver > =================================== > 0000:04:00.0 'I350 Gigabit Network Connection' if=em0 drv=igb unused=igb_uio *Active* > 0000:04:00.1 'I350 Gigabit Network Connection' if=ens2f1 drv=igb unused=igb_uio > 0000:04:00.2 'I350 Gigabit Network Connection' if=ens2f2 drv=igb unused=igb_uio > 0000:04:00.3 'I350 Gigabit Network Connection' if=ens2f3 drv=igb unused=igb_uio > > Other network devices > ===================== > 0000:0a:00.1 'DH8900CC Null Device' unused=igb_uio > 0000:0b:00.1 'DH8900CC Null Device' unused=igb_uio > 0000:0c:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio > 0000:0c:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio > 0000:84:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio > 0000:87:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio > 0000:8b:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio > 0000:8e:00.1 'Ethernet Server Adapter X520-4' unused=ixgbe,igb_uio > > > bruce@silpixa00372841:dpdk.org$ sudo ./x86_64-native-linuxapp-gcc/app/testpmd -c F00 -n 4 -- -i > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 6 on socket 0 > EAL: Detected lcore 7 as core 7 on socket 0 > EAL: Detected lcore 8 as core 0 on socket 1 > EAL: Detected lcore 9 as core 1 on socket 1 > EAL: Detected lcore 10 as core 2 on socket 1 > EAL: Detected lcore 11 as core 3 on socket 1 > EAL: Detected lcore 12 as core 4 on socket 1 > EAL: Detected lcore 13 as core 5 on socket 1 > EAL: Detected lcore 14 as core 6 on socket 1 > EAL: Detected lcore 15 as core 7 on socket 1 > EAL: Detected lcore 16 as core 0 on socket 0 > EAL: Detected lcore 17 as core 1 on socket 0 > EAL: Detected lcore 18 as core 2 on socket 0 > EAL: Detected lcore 19 as core 3 on socket 0 > EAL: Detected lcore 20 as core 4 on socket 0 > EAL: Detected lcore 21 as core 5 on socket 0 > EAL: Detected lcore 22 as core 6 on socket 0 > EAL: Detected lcore 23 as core 7 on socket 0 > EAL: Detected lcore 24 as core 0 on socket 1 > EAL: Detected lcore 25 as core 1 on socket 1 > EAL: Detected lcore 26 as core 2 on socket 1 > EAL: Detected lcore 27 as core 3 on socket 1 > EAL: Detected lcore 28 as core 4 on socket 1 > EAL: Detected lcore 29 as core 5 on socket 1 > EAL: Detected lcore 30 as core 6 on socket 1 > EAL: Detected lcore 31 as core 7 on socket 1 > EAL: Support maximum 64 logical core(s) by configuration. > EAL: Detected 32 lcore(s) > EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size > EAL: Setting up memory... > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Virtual area found at 0x7fe5c0000000 (size = 0x80000000) > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Virtual area found at 0x7fe500000000 (size = 0x80000000) > EAL: Requesting 2 pages of size 1024MB from socket 0 > EAL: Requesting 2 pages of size 1024MB from socket 1 > EAL: TSC frequency is ~2693503 KHz > EAL: Master core 8 is ready (tid=e877c880) > EAL: Core 11 is ready (tid=e6741700) > EAL: Core 10 is ready (tid=e6f42700) > EAL: Core 9 is ready (tid=e7743700) > EAL: PCI device 0000:04:00.0 on NUMA socket 0 > EAL: probe driver: 8086:1521 rte_igb_pmd > EAL: unknown IOMMU driver! > EAL: 0000:04:00.0 cannot open VFIO container! > EAL: Error - exiting with code: 1 > Cause: Requested device 0000:04:00.0 cannot be used > > > bruce@silpixa00372841:dpdk.org$ exit > exit > > Script done on Tue 17 Jun 2014 17:25:29 IST > > Can you attach the lspci output for this system? Neil