From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ee0-x22d.google.com (mail-ee0-x22d.google.com [IPv6:2a00:1450:4013:c00::22d]) by dpdk.org (Postfix) with ESMTP id 1C8B66932 for ; Tue, 4 Feb 2014 12:30:14 +0100 (CET) Received: by mail-ee0-f45.google.com with SMTP id b15so4221220eek.4 for ; Tue, 04 Feb 2014 03:31:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2BNXwtfWRKLDaFnzRixcaf+1e9Pc99BnRlgtKtl4iVU=; b=heAZxDrV8tbr+EAOs4GsdIjJVkSKfBgx+1n/H5y4F8SdMo1xIfp4pNfOuQ+nSdXjW5 OS1dtp+1J0mTsBpea1XHV9xUgRd4PAX0Fpt6hM7hq0Km2l8BnO7rHKb9Ncnctku+f1/L EPM0r0prlmLaTFAB/LcKhh34bOSLzBoOUlS0Yk5kpZa6RKaO5SCMSmEEIU4CGlH05+Ub 1IoRUnX6rZoYjyGpULAFNtt8t4NH9ws/xvESnqP4RS0MH/JnvfmyjFSeMjl54OQkHzT/ 9/tBc7aEN38uRM8U5iIk63a3LQCh1EpkMLmcFMLIVg/3BoNo+ZHhhygd7lEfezbFS9oU +WLw== MIME-Version: 1.0 X-Received: by 10.14.179.73 with SMTP id g49mr2620368eem.71.1391513494775; Tue, 04 Feb 2014 03:31:34 -0800 (PST) Received: by 10.14.147.5 with HTTP; Tue, 4 Feb 2014 03:31:34 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Feb 2014 12:31:34 +0100 Message-ID: From: Mats Liljegren To: jigsaw Content-Type: text/plain; charset=UTF-8 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] How to debug packet sends to virtual functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Feb 2014 11:30:14 -0000 Hi Qinglai, How did you configure the PCI passthrough in KVM? Did you use command line parameter as described in the instructions, or do you use virsh XML files, or maybe using virt-manager? The steps that works best (i.e. can receive but not transmit) are: 1) sudo modprobe ixgbe max_vfs=2 2) sudo ip link set em1 vf 0 mac 3) sudo ifconfig em1 up 4) sudo modprobe pci-stub 5) echo "8086 1515" | sudo tee /sys/bus/pci/drivers/pci-stub/new_id 6) echo "0000:02:10.0" | sudo tee /sys/bus/pci/devices/0000\:02\:10.0/driver/unbind 7) echo "0000:02:10.0" | sudo tee /sys/bus/pci/drivers/pci-stub/bind 8) virsh create vm vm.xml The guest then does: 5) sudo modprobe uio 6) sudo insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio 0000:02:10.0 7) sudo $RTE_SDK/tools/pci_unbind.py -b igb_uio 8) sudo The virsh xml file has the following entry for the virtual function:
I actually have two virtual functions and two physical functions, but I just repeat above for each. Regards Mats On Tue, Feb 4, 2014 at 12:14 PM, jigsaw wrote: > Hi Mats, > > I didn't have any deviation. What I did is just loading ixgbe (with > extra params for vf, as you mentioned in your first email), > and DPDK is up and running in guest. Of course I also followed section > 8.10 of the DPDK release notes. > > I can switch between DPDK and ixgebvf in guest at runtime and > everything works fine. > > Sorry I can't help to debug coz I have only 82599EB at hand. > > -Qinglai > > On Tue, Feb 4, 2014 at 12:45 PM, Mats Liljegren > wrote: >> Hi Qinglai, >> >> Thanks for the response! >> >> My previous attempt was with ixgbe loaded in the host. I also needed >> to load ixgbevf, but this seems to be because of a short-coming in >> libvirt. Maybe loading ixgbevf and then unbind it when running the >> guest is what causes my problems. I could receive packets with this >> setup, but not transmit. I used extra debug to syslog, and it showed >> that packets was placed in the transmit queue, but these packets was >> never sent. >> >> I'll see if I can get this working without loading ixgbevf in the host. >> >> What instructions did you follow to get this working? Did you do any >> deviation from the instructions? >> >> Regards >> Mats >> >> On Tue, Feb 4, 2014 at 11:26 AM, jigsaw wrote: >>> Hi Mats, >>> >>> I've tried vf with 82599EB and it works fine. As long as the VF is >>> visible in guest, DPDK's VF driver should work just as ixgbevf, which >>> shares more or less the same code. >>> >>> I don't understand why you would expect DPDK@guest to work as VF, >>> while the host has no ixgbe loaded. >>> To make further debug I'd suggest compile ixgbe driver with your own >>> syslogs. At least you will be able to see the signalling between vf >>> and ixgbe drivers. >>> >>> -Qinglai >>> >>> >>> On Tue, Feb 4, 2014 at 12:08 PM, Mats Liljegren >>> wrote: >>>> This is my fourth mail in my desperate attempt to get DPDK running in >>>> KVM and no comments so far, not even any "it works for me". Am I the >>>> only one crazy enough to believe that this can be done? >>>> >>>> Anyway, out of desperation I tried to get it running without having >>>> ixgbe or ixgbevf kernel modules loaded in host nor guest. I followed >>>> the instructions in the Programmer's Guide, chapter "Setting Up a KVM >>>> Virtual Machine Monitor", using the PMD version of the instructions. >>>> >>>> I get as far as being able to see my four virtual functions in the >>>> guest using "lspci". But starting the DPDK application gives me the >>>> following error: >>>> >>>> PMD: The MAC address is not valid. >>>> The most likely cause of this error is that the VM host >>>> has not assigned a valid MAC address to this VF device. >>>> Please consult the DPDK Release Notes (FAQ section) for >>>> a possible solution to this problem. >>>> >>>> This may be true, but without any kernel modules loaded, how am I >>>> supposed to change any MAC addresses? Can this be done from within >>>> DPDK? >>>> >>>> As a side-note, I did try to load ixgbevf in the guest, but it >>>> produced no interfaces. There was no error messages in the syslog >>>> though. >>>> >>>> Is it possible to get X540 working in a guest or should I switch hardware? >>>> >>>> Since the instructions assumes I know the command line to KVM to start >>>> my guest (which I do not), I cannot followed them precisely. I use >>>> virsh and XML file, and maybe I've misunderstood how to translate the >>>> pci-assign parameter to XML code. I currently use a entry, >>>> but I've also tried . Neither has been >>>> working for me so far, though the version got me as far as >>>> being able to receive packets at least, but not transmitting. >>>> >>>> Regards >>>> Mats >>>> >>>> >>>> On Mon, Feb 3, 2014 at 12:13 PM, Mats Liljegren >>>> wrote: >>>>> Never mind, I was hit by the infamous MAC spoofing... I got it working >>>>> on both the host and the guest using ixgbevf driver, so apparently the >>>>> cables are correctly attached. >>>>> >>>>> Using DPDK is still no-go. It can receive packets, but when sending >>>>> the packets the function returns success, but the driver reports >>>>> nothing (i.e. no errors, no sent packets, no nothing, except for >>>>> received packets of course). >>>>> >>>>> What could cause this behavior? >>>>> >>>>> Regards >>>>> Mats >>>>> >>>>> On Fri, Jan 31, 2014 at 7:30 PM, Mats Liljegren >>>>> wrote: >>>>>> I have a follow-up on this: >>>>>> >>>>>> ixgbe version 3.13.10-k >>>>>> ixgbevf version 2.7.12-k >>>>>> >>>>>> (These are what was provided by Ubuntu 13.10) >>>>>> >>>>>> I tried the following sequence on the host, before starting the guest: >>>>>> 1) sudo rmmod ixgbe >>>>>> 2) sudo modprobe ixgbe max_vfs=2 >>>>>> 3) sudo ifconfig em1 up # This is the physical function >>>>>> 4) sudo ifconfig em1_0 192.168.2.2 # This is the virtual function >>>>>> 5) ping 192.168.2.1 >>>>>> >>>>>> I can see that the ping request reaches its target, and a reply is >>>>>> sent back. But this reply is not received by the ping shell command. >>>>>> >>>>>> Why? >>>>>> >>>>>> Regards, >>>>>> Mats >>>>>> >>>>>> On Wed, Jan 29, 2014 at 6:56 PM, Mats Liljegren >>>>>> wrote: >>>>>>> I'm trying to get a modified version of the l2fwd example running, and >>>>>>> have problems with packets being silently thrown away. I can receive >>>>>>> packets, and my printf's indicates that the packets are being sent to >>>>>>> the correct port, using correct MAC address as source address. And >>>>>>> still, the packets are lost. >>>>>>> >>>>>>> Since the port is a virtual function, it seems like I cannot use >>>>>>> tcpdump on it to see the network traffic. There is nothing coming out >>>>>>> of the cable (activity light not flashing, the receiving end running >>>>>>> tcpdump does not see any traffic). >>>>>>> >>>>>>> I'm using two X540 with two virtual functions each. The application >>>>>>> runs in a KVM/Qemu environmen. >>>>>>> >>>>>>> Any suggestions how to debug this? >>>>>>> >>>>>>> Regards, >>>>>>> Mats