From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vk0-f46.google.com (mail-vk0-f46.google.com [209.85.213.46]) by dpdk.org (Postfix) with ESMTP id 1C4F42BE6 for ; Tue, 12 Apr 2016 07:51:47 +0200 (CEST) Received: by mail-vk0-f46.google.com with SMTP id k1so10833447vkb.0 for ; Mon, 11 Apr 2016 22:51:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=yEj2haoxAsoeomCTgFOjfPxnTkTmFBt7SNbUXHeydNQ=; b=E/GK2URQSWmXiTm4amjdYQ4eYy9AGrFXFUAJ5H46JBld9/TLXHMobHkYVqm1GU5OV8 KOSQfzcS+AYCufZ8b51flkApcQhS4rpLFXRCpoKiJkaLT+in3/HDqoe2ghofBsMPrXT+ Jv7WsfAxOl1BPG8iTpcZhqFZ5+2bS4XLJKUqu0s4WfDyvkZioEKQrETXYt4B/Qa/SXKT bHpO3Xhk1+OewWqnO7Ey4xGjctdQuwpuhR9/LGDHVTlxdHogYVngQ0zVmyEJM10AV5QU AtCkoi6aYBW/H2aikW4pZwQgmvY0wxA7DRabzeZ8rWOw64IwrUSS/SYJLMmQdRT2urfR hmUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=yEj2haoxAsoeomCTgFOjfPxnTkTmFBt7SNbUXHeydNQ=; b=cUUnEUiB5/gG2BIyKhI5FxtKMF4R3S6EuiOqExDz07xaJMP0yQS/a6/OjFLiesq0yJ a9hOeCiox7HlYlTohu9spBqF65aGgK9Wb4MdNslU3PtqfV6HEdp9jHde0zlorjwIkIuz NFwpozq2VYxRdhqDSz5lsGfe3eB67qZsI7iFpefjZPSy8Prrxa/ZUNFr3eexB5X0jsAx GmKaUEl0Z9NPehInVpwxAQ77fjtOckBZDONCuWxyrUqhGe+Dzh45f7z77Iq4C3f3IHay YC9B0cf4EsQ1XUQpJcMBePdS58WTRr+invNqy8WETyEwBLK7eXk/hsGHj9wNKS/q4Ady +ahg== X-Gm-Message-State: AOPr4FWiU/X2gK/NooNZbtDJCB8pnmn3t3M5Gz7bSR3f3HFcJhqpwoew2Kd+ivs5Wmw7AzaGfZqki/qEuYShDw== MIME-Version: 1.0 X-Received: by 10.176.6.197 with SMTP id g63mr667800uag.60.1460440306480; Mon, 11 Apr 2016 22:51:46 -0700 (PDT) Received: by 10.176.1.117 with HTTP; Mon, 11 Apr 2016 22:51:46 -0700 (PDT) In-Reply-To: References: <6A0DE07E22DDAD4C9103DF62FEBC09090343BA89@shsmsx102.ccr.corp.intel.com> <533710CFB86FA344BFBF2D6802E6028622F71044@SHSMSX101.ccr.corp.intel.com> Date: Tue, 12 Apr 2016 11:21:46 +0530 Message-ID: From: bharath paulraj To: "Rose, Gregory V" Cc: "Qiu, Michael" , "Zhang, Helin" , "Lu, Wenzhuo" , "Rowden, Aaron F" , "dev@dpdk.org" , "Jayakumar, Muthurajan" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Apr 2016 05:51:47 -0000 Hi Greg, Thanks for your response. I am using the old kernel (2.6.32-573.22.1.el6.x86_64). First I have to update the kernel to version-3.18.4 and will try what you have specified. I will keep you posted with the outcomes. Thanks, Bharath On Sat, Apr 9, 2016 at 2:36 AM, Rose, Gregory V wrote: > Bharath, > > > > Please try the following: > > > > In this example we are directing the traffic to queue 1 of VF ID 5. Firs= t > you must turn on ntuple. > > > > $ ethtool =E2=80=93K p5p2 ntuple on > > > > Then you can set the FD rule: > > > > $ ethtool =E2=80=93N p5p2 flow-type udp4 dst-port 4790 action 1 user-def = 5 > > > > When I do that this is the rule definition result: > > > > Filter: 2045 > > Rule Type: UDP over IPv4 > > Src IP addr: 0.0.0.0 mask: 255.255.255.255 > > Dest IP addr: 0.0.0.0 mask: 255.255.255.255 > > TOS: 0x0 mask: 0xff > > Src port: 0 mask: 0xffff > > Dest port: 4790 mask: 0x0 > > VLAN EtherType: 0x0 mask: 0xffff > > VLAN: 0x0 mask: 0xffff > > User-defined: 0x5 mask: 0xffffffffffffff00 > > Action: Direct to queue 1 > > > > Be sure to use a version of ethtool 3.0 or greater and a recent kernel = =E2=80=93 I > tested on ethtool version 3.15 with a 3.18.4 Linux kernel from kernel.org > and using the most recent ixgbe driver release. > > > > Let me know if this works for you. > > > > Thanks, > > > > - Greg > > > > *From:* Rose, Gregory V > *Sent:* Thursday, April 7, 2016 4:43 PM > *To:* bharath paulraj ; Qiu, Michael < > michael.qiu@intel.com> > *Cc:* Zhang, Helin ; Lu, Wenzhuo < > wenzhuo.lu@intel.com>; Rowden, Aaron F ; > dev@dpdk.org; Jayakumar, Muthurajan > > *Subject:* RE: [dpdk-dev] Reg: promiscuous mode on VF > > > > Bharath, > > > > I=E2=80=99m sorry for the delay but I am working on a response to this. = There is > a way to do what you need to do and I will get together some instructions > and respond by tomorrow. > > > > Thanks and regards, > > > > - Greg > > > > *From:* bharath paulraj [mailto:bharathpaul@gmail.com > ] > *Sent:* Thursday, April 7, 2016 3:40 AM > *To:* Qiu, Michael > *Cc:* Rose, Gregory V ; Zhang, Helin < > helin.zhang@intel.com>; Lu, Wenzhuo ; Rowden, Aaron > F ; dev@dpdk.org; Jayakumar, Muthurajan < > muthurajan.jayakumar@intel.com> > *Subject:* Re: [dpdk-dev] Reg: promiscuous mode on VF > > > > Hi Team, > > > > May I have some update on my previous mail? I am here stuck in flow > creation. > > > > Thanks, > > Bharath > > > > On Thu, Mar 31, 2016 at 4:13 PM, bharath paulraj > wrote: > > Hi Michael and All, > > > I am unable to set the rule to receive the packet on the VF. > Below is my setup. > > 1. Creating one virtual function with one queue, in one of my port, p2p1. > * modprobe ixgbe MQ=3D1 max_vfs=3D1 RSS=3D1 allow_unsupported_sfp=3D1 = * > 2. Below is the interface status after creating one virtual function. > [root@XXXX sriov]# ifconfig p2p1 > p2p1 Link encap:Ethernet HWaddr A0:36:9F:86:C2:74 > inet6 addr: fe80::a236:9fff:fe86:c274/64 Scope:Link > UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 > RX packets:2540 errors:0 dropped:0 overruns:0 frame:0 > TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:157456 (153.7 KiB) TX bytes:258 (258.0 b) > > [root@XXXX sriov]# ifconfig p2p1_0 > p2p1_0 Link encap:Ethernet HWaddr DA:61:95:CD:AF:35 > inet6 addr: fe80::d861:95ff:fecd:af35/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:12 errors:0 dropped:0 overruns:0 frame:0 > TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:360 (360.0 b) TX bytes:740 (740.0 b) > 3. Next I am enable ntuple > *ethtool -K p2p1 ntuple on * > 4. Now I am adding below rule > > *ethtool -N p2p1 flow-type udp4 dst-port 4789 action 0x100000000 --> VF > 0, queue 0 ethtool -N p2p1 flow-type udp4 dst-port 4790 action > 0x000000000 --> PF queue 0 * > 5. [root@XXX sriov]# ethtool -n p2p1 > 1 RX rings available > Total 2 rules > > Filter: 2044 > Rule Type: UDP over IPv4 > Src IP addr: 0.0.0.0 mask: 255.255.255.255 > Dest IP addr: 0.0.0.0 mask: 255.255.255.255 > TOS: 0x0 mask: 0xff > Src port: 0 mask: 0xffff > Dest port: 4790 mask: 0x0 > VLAN EtherType: 0x0 mask: 0xffff > VLAN: 0x0 mask: 0xffff > User-defined: 0x0 mask: 0xffffffffffffffff > Action: Direct to queue 0 > > Filter: 2045 > Rule Type: UDP over IPv4 > Src IP addr: 0.0.0.0 mask: 255.255.255.255 > Dest IP addr: 0.0.0.0 mask: 255.255.255.255 > TOS: 0x0 mask: 0xff > Src port: 0 mask: 0xffff > Dest port: 4789 mask: 0x0 > VLAN EtherType: 0x0 mask: 0xffff > VLAN: 0x0 mask: 0xffff > User-defined: 0x0 mask: 0xffffffffffffffff > Action: Direct to queue 0 > * >> Won't it show the VF queue numbers here?* > > 6. Start the VM over p2p1_0 > 7. Below is the Packet I am sending > a) Dest MAC - VF Mac, Src MAC - any untagged, src ip - 1.1.1.1 dest ip - > 2.2.2.2 src port - 100 dest port - 4789 > b) Dest MAC - VF Mac, Src MAC - any, untagged, src ip - 1.1.1.1 dest ip - > 2.2.2.2 src port - 100 dest port - 4790 > c) Dest MAC - VF Mac, untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src > port - 100 dest port - 4791 > > All the above testing is done on centOs-6.7 with ixgbe version - 4.3.13 > with patch you mentioned on 82599 Ethernet controller > Linux XXX 2.6.32-573.22.1.el6.x86_64 #1 SMP Wed Mar 23 03:35:39 UTC 2016 > x86_64 x86_64 x86_64 GNU/Linux > > > *Observation: * > If the packet matches the rule, I am not able to see the packet in the VF= , > instead I am able to see the packet in PF. > for the packets a) and b), I am able to see te packet only in PF. Even if > the packet destination MAC is VF's MAC, > I am able to see only in PF. > If the packet is not matching the rule, then I am able to see the packet > in VF, provided packet destination MAC is VF's MAC. > *Question: * > 1) Am I mapping the queues wrongly while adding the rules? > 2) How to Identify which VF using which Queues? > > Request you to provide some help on it. > > > > > > On Tue, Mar 22, 2016 at 1:03 PM, bharath paulraj > wrote: > > Thanks a lot Michael. Finally i am able to see some light. I will try th= e > same in our setup and will post you the results. > > > > Thanks, > > Bharath > > > > On Tue, Mar 22, 2016 at 12:09 PM, Qiu, Michael > wrote: > > Yes, we could let ovs using 82599 VF to do rx/tx. I don't know what's > your l2 bridge, but since ovs could work I think your bridge also could > work. But I only tested with one VF. > > Make sure below two patches (bifurcate driver) are included in your kerne= l: > > _https://patchwork.ozlabs.org/patch/476511/_ > _https://patchwork.ozlabs.org/patch/476516/_ > > Mostly, if your kernel version in 4.2 or newer, it should be included. > > After you create VF, before you passthrough the VF to guest: > > (vf +1) << 32 + queue-index, > > > 1. where vf is the VF index starting from 0 > 2. the queue-index is 0 if multi-queue support is not turned on, and > this value is [0,1] if multiple-queue is turned on > > > echo 1 > /sys/bus/pci/devices/0000\:05\:00.0/sriov_numvfs > ifconfig $(PF_INTF) up > ifconfig $(VF0_INFT) up > ip link set $(PF_INTF) promisc on > ethtool -K $(PF_INTF) ntuple on > ethtool -N $(PF_INTF) flow-type udp4 dst-port 4789 action 0x100000000 > (VF0 queue 0) > > Here we using flow director to all let packets according to the rules to > the VF, But I don't know if it could let the packets to other VFs at the > same time. > > Thanks, > Michael > > On 3/17/2016 2:43 PM, bharath paulraj wrote: > > Hi Lu, Helin, Greg, > > > > Many thanks for your response, which is really quick. Now, If I want > > to implement L2 bridging with Intel virtualization technologies, using > > 82599 controller, then Michael is my only hope, as getting the new > > kernel versions and upstream support will take considerable amount of > > time. > > > > Michael, Could you please share your experience on L2 bridging > > using Intel virtualization technologies. > > > > Thanks, > > Bharath > > > > On Wed, Mar 16, 2016 at 9:40 PM, Rose, Gregory V > > > wrote: > > > > Intel has not supported promiscuous mode for virtual functions due > > to the security concerns mentioned below. > > > > There will be upstream support in an upcoming Linux kernel for > > setting virtual functions as "trusted" and when that is available > > then Intel will allow virtual functions to enter unicast > > promiscuous mode on those Ethernet controllers that support > > promiscuous mode for virtual functions in the HW/FW. Be aware > > that not all Intel Ethernet controllers have support for unicast > > promiscuous mode for virtual functions. The only currently > > released product that does is the X710/XL710. > > > > The key take away is that unicast promiscuous mode for X710/XL710 > > virtual functions requires Linux kernel support, iproute2 package > > support and driver support. Only when all three of these are in > > place will the feature work. > > > > Thanks, > > > > - Greg > > > > -----Original Message----- > > From: Zhang, Helin > > Sent: Wednesday, March 16, 2016 9:04 AM > > To: bharath paulraj > >; Lu, Wenzhuo > >; Rowden, Aaron F > > >; > > Rose, Gregory V > > > > Cc: dev@dpdk.org ; Qiu, Michael > > >; Jayakumar, > > Muthurajan > > > > Subject: RE: [dpdk-dev] Reg: promiscuous mode on VF > > > > Hi Bharath > > > > For your question of "why intel does not support unicast > > promiscuos mode?", I'd ask Aaron or Greg to give answers. > > Thank you very much! > > > > Regards, > > Helin > > > > > -----Original Message----- > > > From: dev [mailto:dev-bounces@dpdk.org > > ] On Behalf Of bharath paulraj > > > Sent: Wednesday, March 16, 2016 11:29 PM > > > To: Lu, Wenzhuo > > > Cc: dev@dpdk.org > > > Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF > > > > > > Hi Lu, > > > > > > Many thanks for your response. Again I have few more queries. > > > If VF unicast promiscuous mode is not supported then can't we > > > implement a Layer 2 bridging functionality using intel > > virtualization > > > technologies? Or Is there any other way, say tweeking some hardwa= re > > > registers or drivers, which may help us in implementing Layer 2 > > bridging. > > > Also I would like to know, why intel does not support unicast > > promiscuos mode? > > > It could have been optional register settings and user should > > have had > > > a previleage to set or unset it. Besides, security reasons, is > there > > > any other big reason why Intel does not support this? > > > > > > Thanks, > > > Bharath Paulraj > > > > > > On Wed, Mar 16, 2016 at 6:15 AM, Lu, Wenzhuo > > > > > > > wrote: > > > > > > > Hi Bharath, > > > > > > > > > 2) Is the above supported for 82599 controller? If it is > > > > > supported > > > > in the NIC, > > > > > please provide the steps to enable. > > > > Talking about 82599, VF unicast promiscuous mode is not > supported. > > > > Only broadcast and multicast can be supported. > > > > > > > > > > > > > > Thanks, > > > > > Bharath Paulraj > > > > > > > > > > > > > > > > -- > > > Regards, > > > Bharath > > > > > > > > > > -- > > Regards, > > Bharath > > > > > > -- > > Regards, > > Bharath > > > > > > -- > > Regards, > > Bharath > > > > > > -- > > Regards, > > Bharath > --=20 Regards, Bharath