From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.unipi.it (smtp3.unipi.it [131.114.21.153]) by dpdk.org (Postfix) with ESMTP id 78C0B2904 for ; Thu, 20 Apr 2017 18:21:35 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by smtp.unipi.it (Postfix) with ESMTP id D9D71C312C for ; Thu, 20 Apr 2017 18:21:34 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at unipi.it Received: from mail-oi0-f48.google.com (mail-oi0-f48.google.com [209.85.218.48]) (Authenticated User) by smtp.unipi.it (Postfix) with ESMTPSA id BAA46C3128 for ; Thu, 20 Apr 2017 18:21:32 +0200 (CEST) Received: by mail-oi0-f48.google.com with SMTP id j201so57471694oih.2 for ; Thu, 20 Apr 2017 09:21:32 -0700 (PDT) X-Gm-Message-State: AN3rC/7wpBh795/3Y0U4VcUF4j0J/CnNB+jteJrdHzAEwI96eAOKWGuR 7pgiQQW9pNQHyG8SJeSRcqQdB+wRuw== X-Received: by 10.157.26.78 with SMTP id u14mr5249999otu.211.1492705291345; Thu, 20 Apr 2017 09:21:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.5.10 with HTTP; Thu, 20 Apr 2017 09:21:30 -0700 (PDT) In-Reply-To: References: From: Vincenzo Maffione Date: Thu, 20 Apr 2017 18:21:30 +0200 X-Gmail-Original-Message-ID: Message-ID: To: Shivaram Mysore Cc: users@dpdk.org, Giuseppe Lettieri Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Cannot attach DPDK NIC port to OVS X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Apr 2017 16:21:35 -0000 vfio-pci is loaded and bound to the NIC, as shown by dpdk-devbind. $ sudo tools/dpdk-devbind.py --status Network devices using DPDK-compatible driver ============================================ 0000:01:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=vfio-pci unused=ixgbe Network devices using kernel driver =================================== 0000:00:19.0 'Ethernet Connection (2) I218-V' if=eno1 drv=e1000e unused=vfio-pci 0000:01:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=enp1s0f1 drv=ixgbe unused=vfio-pci 0000:06:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller' if=enp6s0 drv=r8169 unused=vfio-pci 0000:07:00.0 '82574L Gigabit Network Connection' if=enp7s0 drv=e1000e unused=vfio-pci [...] And I get: Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: PCI device 0000:01:00.0 on NUMA socket -1 Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: probe driver: 8086:10fb net_ixgbe Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: Requested device 0000:01:00.0 cannot be used Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: Driver cannot attach the device (0000:01:00.0) Apr 20 18:19:22 arch-server ovs-vswitchd[650]: 2017-04-20T16:19:22Z|00027|netdev_dpdk|WARN|Error attaching device '0000:01:00.0' to DPDK Apr 20 18:19:22 arch-server ovs-vswitchd[650]: 2017-04-20T16:19:22Z|00028|netdev|WARN|dpdk-p1: could not set configuration (Invalid argumen Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: PCI device 0000:01:00.0 on NUMA socket -1 Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: probe driver: 8086:10fb net_ixgbe Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: Requested device 0000:01:00.0 cannot be used Apr 20 18:19:22 arch-server ovs-vswitchd[650]: EAL: Driver cannot attach the device (0000:01:00.0) Apr 20 18:19:22 arch-server ovs-vswitchd[650]: ovs|00027|netdev_dpdk|WARN|Error attaching device '0000:01:00.0' to DPDK Apr 20 18:19:22 arch-server ovs-vswitchd[650]: ovs|00028|netdev|WARN|dpdk-p1: could not set configuration (Invalid argument) Thanks, Vincenzo 2017-04-20 18:13 GMT+02:00 Shivaram Mysore : > I have a feeling that vfio-pci is not installed. Note that you will have > to edit /etc/modules to load on boot. > > You can also check by > # lsmod | grep vfio > > To load on demand - use > # modprobe vfio-pci > > > > On Thu, Apr 20, 2017 at 9:03 AM, Vincenzo Maffione > wrote: > >> Hi, >> I installed OVS 2.7.0 and use uio_pci_generic in place of vfio-pci, and >> the error disappeared. >> >> Thanks, >> Vincenzo >> >> 2017-04-15 17:56 GMT+02:00 Shivaram Mysore : >> >>> You should look at emails from -(Hint: search DPDK in subject) >>> >>> https://mail.openvswitch.org/pipermail/ovs-discuss/2017-Marc >>> h/thread.html >>> https://mail.openvswitch.org/pipermail/ovs-discuss/2017-Apri >>> l/thread.html >>> https://mail.openvswitch.org/pipermail/ovs-discuss/2017-Apri >>> l/044155.html >>> >>> If possible, I would recommend using Ubuntu 17.04 with OVS 2.7 with DPDK >>> packages from http://packages.wand.net.nz/ which can simplify your >>> deployment. >>> >>> /Shivaram >>> >>> On Fri, Apr 14, 2017 at 4:04 AM, Vincenzo Maffione < >>> vincenzo.maffione@ing.unipi.it> wrote: >>> >>>> Hi, >>>> I would like to attach one of my ixgbe NIC ports to an OpenVSwitch >>>> instance, but I keep getting an unexpected error, see below. >>>> >>>> I've successfully bound both my ixgbe NIC ports to DPDK, using VFIO: >>>> >>>> # tools/dpdk-devbind.py --status >>>> >>>> Network devices using DPDK-compatible driver >>>> ============================================ >>>> 0000:01:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' >>>> drv=vfio-pci >>>> unused=ixgbe >>>> 0000:01:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' >>>> drv=vfio-pci >>>> unused=ixgbe >>>> >>>> Network devices using kernel driver >>>> =================================== >>>> 0000:00:19.0 'Ethernet Connection (2) I218-V' if=eno1 drv=e1000e >>>> unused=vfio-pci >>>> 0000:06:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller' >>>> if=enp6s0 drv=r8169 unused=vfio-pci >>>> 0000:07:00.0 '82574L Gigabit Network Connection' if=enp7s0 drv=e1000e >>>> unused=vfio-pci >>>> >>>> Other network devices >>>> ===================== >>>> >>>> >>>> Crypto devices using DPDK-compatible driver >>>> =========================================== >>>> >>>> >>>> Crypto devices using kernel driver >>>> ================================== >>>> >>>> >>>> Other crypto devices >>>> ==================== >>>> >>>> >>>> >>>> Then I created a bridge >>>> >>>> # ovs-vsctl add-br obr1 -- set bridge obr1 datapath_type=netdev >>>> >>>> But then when trying to add the dpdk NIC: >>>> >>>> # ovs-vsctl add-port obr1 dpdk-p0 -- set Interface dpdk-p0 type=dpdk >>>> options:dpdk-devargs=0000:01:00.0 >>>> >>>> the command fails, and I get this in the log: >>>> >>>> ovs-vswitchd[353]: 2017-04-14T10:54:06Z|00026|bridge|WARN|could not >>>> open >>>> network device dpdk-p0 (No such device) >>>> ovs-vswitchd[353]: ovs|00026|bridge|WARN|could not open network device >>>> dpdk-p0 (No such device) >>>> >>>> Note that I've successfully managed to setup an OVS instance with two >>>> DPDK >>>> vhost-user ports (each one assigned to a QEMU VM): this is why I'm >>>> confident my OVS is built and configured correctly to work with DPDK. >>>> Also, >>>> DPDK by itself is working as expected, I can use dpdk pktgen. >>>> Moreover, OVS log shows that DPDK is recognized >>>> >>>> [...] >>>> ovs|00005|dpdk|INFO|DPDK Enabled, initializing >>>> [...] >>>> >>>> Note that I'm carefully following the documentation here >>>> http://docs.openvswitch.org/en/latest/intro/install/dpdk/ and here >>>> http://docs.openvswitch.org/en/latest/howto/dpdk/ >>>> >>>> More info: >>>> >>>> DPDK version: 16.11 >>>> OVS version: 2.6.1, configured with --with-dpdk >>>> Host: Archlinux >>>> /proc/cmdline: BOOT_IMAGE=../vmlinuz-linux-vfio >>>> root=UUID=1c777330-3f00-4ab8-8d5d-d4b2b5d860b0 rw intel_iommu=on >>>> hugepages=2048 pci=assign-busses pcie_acs_override=downstream >>>> initrd=../initramfs-linux-vfio.img >>>> >>>> Can anybody help? I'm at a loss here. >>>> >>>> Thanks, >>>> Vincenzo >>>> >>> >>> >> >> >> -- >> Vincenzo Maffione >> > >