From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id D4F6C91FC for ; Thu, 12 Nov 2015 13:57:27 +0100 (CET) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 380D4A0B31; Thu, 12 Nov 2015 12:57:27 +0000 (UTC) Received: from sopuli.koti.laiskiainen.org (vpn1-5-7.ams2.redhat.com [10.36.5.7]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tACCvPLi032126; Thu, 12 Nov 2015 07:57:26 -0500 To: Bruce Richardson , "Montorsi, Francesco" References: <964049bfb9054699a2e4520c6758a7ee@bilemail1.empirix.com> <20151111162853.GA38496@bricha3-MOBL3> From: Panu Matilainen Message-ID: <56448CB5.8070102@redhat.com> Date: Thu, 12 Nov 2015 14:57:25 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151111162853.GA38496@bricha3-MOBL3> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Permanently binding NIC ports with DPDK drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2015 12:57:28 -0000 On 11/11/2015 06:28 PM, Bruce Richardson wrote: > On Wed, Nov 11, 2015 at 04:13:01PM +0000, Montorsi, Francesco wrote: >> Hi, >> Is there a way to permanently (i.e., have the configuration automatically applied after reboot) bind a NIC port to DPDK? >> >> In case there's none, I'm thinking to save in my software a list of the NIC ports chosen by the user for use with DPDK and then, upon software startup to just do >> for (int i=0; i < ...; i++) >> system("dpdk_nic_bind.py --bind=igb_uio " + PCI_device_chosen[i]); >> Do you see any problem with that? >> >> Thanks! >> Francesco Montorsi >> > > Hi Francesco, > > I'm not aware of any way to make the bindings permanent across reboots. What you > have suggested will work, but there are probably better ways to do the same thing. > For example, a couple of lines in an rc.local script can reapply the bindings at > boot for you. I'm sure others can suggest other ways of having the same effect, > for example, there may be a way to automatically do this using udev or systemd > or some such package. I've been looking into this recently, here's what I have so far: http://laiskiainen.org/git/?p=driverctl.git For the impatient, "make rpm" should produce something usable for recent Fedora/RHEL systems, usage looks somewhat like this: Find devices currently driven by ixgbe driver: # driverctl -v list-devices | grep ixgbe 0000:01:00.0 ixgbe (Ethernet 10G 4P X520/I350 rNDC) 0000:01:00.1 ixgbe (Ethernet 10G 4P X520/I350 rNDC) Change them to use the vfio-pci driver permanently: # driverctl set-override 0000:01:00.0 vfio-pci # driverctl set-override 0000:01:00.1 vfio-pci Find devices with driver overrides: [root@wsfd-netdev32 ~]# driverctl -v list-devices|grep \* 0000:01:00.0 vfio-pci [*] (Ethernet 10G 4P X520/I350 rNDC) 0000:01:00.1 vfio-pci [*] (Ethernet 10G 4P X520/I350 rNDC) Remove the permanent driver override for device 0000:01:00.1: # driverctl unset-override 0000:01:00.1 In addition it has udev rules to export vfio and uio devices on systemd level, eg the above looks like this with normal drivers: # systemctl |grep 0000:01:00 sys-devices-pci0000:00-0000:00:03.0-0000:01:00.0-net-em1.device loaded active plugged Ethernet 10G 4P X520/I350 rNDC sys-devices-pci0000:00-0000:00:03.0-0000:01:00.1-net-em2.device loaded active plugged Ethernet 10G 4P X520/I350 rNDC When changed to vfio, with upstream systemd/udev rules they would just disappear entirely, but with the driverctl rules they become: # systemctl |grep 0000:01:00 sys-devices-pci0000:00-0000:00:03.0-0000:01:00.0-vfio.device loaded active plugged /sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/vfio sys-devices-pci0000:00-0000:00:03.0-0000:01:00.1-vfio.device loaded active plugged /sys/devices/pci0000:00/0000:00:03.0/0000:01:00.1/vfio - Panu -