From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vk0-f41.google.com (mail-vk0-f41.google.com [209.85.213.41]) by dpdk.org (Postfix) with ESMTP id 4E0546A5B for ; Thu, 31 Mar 2016 12:43:56 +0200 (CEST) Received: by mail-vk0-f41.google.com with SMTP id k1so98190706vkb.0 for ; Thu, 31 Mar 2016 03:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=VWI+rfoVl4tAtiGgjTiTrnrldibdiXcRmn7IVlDLX6A=; b=QkzjkRKjJocBrzEOJhitgkiBHQCVBoDVbNt0YdaN49YC0a74X9bHRBB4RJte+aiZ/t oX0mD6IOtKdHZ4DRNZFjAX/BOZBhfDtZHp1TXhrkWxbPCnr+3mpTU2+tPXqkVWZquwQW Xu8lVEK/sqN8B/m+BQnlwBF0F36elRxmuy6Cs0Qn7JbufbjmV98erqr39twLZbOnoJls jCikPgSJRfFw6vvKdorqrgQcxQHBTUp4gjZ42tf15EHa2mxXAZ3wU3COiQxHwqvuiJH5 dkHhWDkPZv718EYNcIKvTyzzOpFWNkTTMiPR+VXB2Ffl8sASIfh3cKWhk+d+8IDVAhIb gusw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=VWI+rfoVl4tAtiGgjTiTrnrldibdiXcRmn7IVlDLX6A=; b=QJvaya7GrdJw5C1HpGQ/0DOYc0RNEMWIHcf5EAqqqi/gujfBxaYs79Lk+yGqz0kqS+ INNgIJ6HObkT6nIV4T0Tdssj7WxBRv/CWKh5ToqbPzlqxY3KdfwXPDzgu1zo6WYgeRJ5 C6ewBI+CZoe/fgw77h3JIaGK66yNkEa+TgY74zFYp7NiMZ/17kVmAE9gceAZAK3AdX7L aRUz+tXDo2Cni4XxViMnxmGc723xKL34n26Ao08SwJejhDhO/Rdbkyd1mM/kKmAB41HC N2WCFNnM7ppoKwdLKbJtBpN2xo5QEdWzniv9RmC1I2Q0V/BHYRVdgYHIJwARK+Nj7wuc i07Q== X-Gm-Message-State: AD7BkJKiAhxKDxJXkTpCpwQAOcdOmMEc2QUu/uYhuMbYh2z3yTtPUs1tYascwGmPgLnRi1YDKUC8WB2DtdXTzw== MIME-Version: 1.0 X-Received: by 10.159.37.137 with SMTP id 9mr2383889uaf.25.1459421035710; Thu, 31 Mar 2016 03:43:55 -0700 (PDT) Received: by 10.176.1.117 with HTTP; Thu, 31 Mar 2016 03:43:55 -0700 (PDT) In-Reply-To: References: <6A0DE07E22DDAD4C9103DF62FEBC09090343BA89@shsmsx102.ccr.corp.intel.com> <533710CFB86FA344BFBF2D6802E6028622F71044@SHSMSX101.ccr.corp.intel.com> Date: Thu, 31 Mar 2016 16:13:55 +0530 Message-ID: From: bharath paulraj To: "Qiu, Michael" Cc: "Rose, Gregory V" , "Zhang, Helin" , "Lu, Wenzhuo" , "Rowden, Aaron F" , "dev@dpdk.org" , "Jayakumar, Muthurajan" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Mar 2016 10:43:56 -0000 Hi Michael and All, I am unable to set the rule to receive the packet on the VF. Below is my setup. 1. Creating one virtual function with one queue, in one of my port, p2p1. * modprobe ixgbe MQ=1 max_vfs=1 RSS=1 allow_unsupported_sfp=1 * 2. Below is the interface status after creating one virtual function. [root@XXXX sriov]# ifconfig p2p1 p2p1 Link encap:Ethernet HWaddr A0:36:9F:86:C2:74 inet6 addr: fe80::a236:9fff:fe86:c274/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:2540 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:157456 (153.7 KiB) TX bytes:258 (258.0 b) [root@XXXX sriov]# ifconfig p2p1_0 p2p1_0 Link encap:Ethernet HWaddr DA:61:95:CD:AF:35 inet6 addr: fe80::d861:95ff:fecd:af35/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:12 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:360 (360.0 b) TX bytes:740 (740.0 b) 3. Next I am enable ntuple *ethtool -K p2p1 ntuple on * 4. Now I am adding below rule *ethtool -N p2p1 flow-type udp4 dst-port 4789 action 0x100000000 --> VF 0, queue 0 ethtool -N p2p1 flow-type udp4 dst-port 4790 action 0x000000000 --> PF queue 0 * 5. [root@XXX sriov]# ethtool -n p2p1 1 RX rings available Total 2 rules Filter: 2044 Rule Type: UDP over IPv4 Src IP addr: 0.0.0.0 mask: 255.255.255.255 Dest IP addr: 0.0.0.0 mask: 255.255.255.255 TOS: 0x0 mask: 0xff Src port: 0 mask: 0xffff Dest port: 4790 mask: 0x0 VLAN EtherType: 0x0 mask: 0xffff VLAN: 0x0 mask: 0xffff User-defined: 0x0 mask: 0xffffffffffffffff Action: Direct to queue 0 Filter: 2045 Rule Type: UDP over IPv4 Src IP addr: 0.0.0.0 mask: 255.255.255.255 Dest IP addr: 0.0.0.0 mask: 255.255.255.255 TOS: 0x0 mask: 0xff Src port: 0 mask: 0xffff Dest port: 4789 mask: 0x0 VLAN EtherType: 0x0 mask: 0xffff VLAN: 0x0 mask: 0xffff User-defined: 0x0 mask: 0xffffffffffffffff Action: Direct to queue 0 * >> Won't it show the VF queue numbers here?* 6. Start the VM over p2p1_0 7. Below is the Packet I am sending a) Dest MAC - VF Mac, Src MAC - any untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src port - 100 dest port - 4789 b) Dest MAC - VF Mac, Src MAC - any, untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src port - 100 dest port - 4790 c) Dest MAC - VF Mac, untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src port - 100 dest port - 4791 All the above testing is done on centOs-6.7 with ixgbe version - 4.3.13 with patch you mentioned on 82599 Ethernet controller Linux XXX 2.6.32-573.22.1.el6.x86_64 #1 SMP Wed Mar 23 03:35:39 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux *Observation: * If the packet matches the rule, I am not able to see the packet in the VF, instead I am able to see the packet in PF. for the packets a) and b), I am able to see te packet only in PF. Even if the packet destination MAC is VF's MAC, I am able to see only in PF. If the packet is not matching the rule, then I am able to see the packet in VF, provided packet destination MAC is VF's MAC. *Question: * 1) Am I mapping the queues wrongly while adding the rules? 2) How to Identify which VF using which Queues? Request you to provide some help on it. On Tue, Mar 22, 2016 at 1:03 PM, bharath paulraj wrote: > Thanks a lot Michael. Finally i am able to see some light. I will try the > same in our setup and will post you the results. > > Thanks, > Bharath > > On Tue, Mar 22, 2016 at 12:09 PM, Qiu, Michael > wrote: > >> Yes, we could let ovs using 82599 VF to do rx/tx. I don't know what's >> your l2 bridge, but since ovs could work I think your bridge also could >> work. But I only tested with one VF. >> >> Make sure below two patches (bifurcate driver) are included in your >> kernel: >> >> _https://patchwork.ozlabs.org/patch/476511/_ >> _https://patchwork.ozlabs.org/patch/476516/_ >> >> Mostly, if your kernel version in 4.2 or newer, it should be included. >> >> After you create VF, before you passthrough the VF to guest: >> >> (vf +1) << 32 + queue-index, >> >> >> 1. where vf is the VF index starting from 0 >> 2. the queue-index is 0 if multi-queue support is not turned on, and >> this value is [0,1] if multiple-queue is turned on >> >> >> echo 1 > /sys/bus/pci/devices/0000\:05\:00.0/sriov_numvfs >> ifconfig $(PF_INTF) up >> ifconfig $(VF0_INFT) up >> ip link set $(PF_INTF) promisc on >> ethtool -K $(PF_INTF) ntuple on >> ethtool -N $(PF_INTF) flow-type udp4 dst-port 4789 action 0x100000000 >> (VF0 queue 0) >> >> Here we using flow director to all let packets according to the rules to >> the VF, But I don't know if it could let the packets to other VFs at the >> same time. >> >> Thanks, >> Michael >> >> On 3/17/2016 2:43 PM, bharath paulraj wrote: >> > Hi Lu, Helin, Greg, >> > >> > Many thanks for your response, which is really quick. Now, If I want >> > to implement L2 bridging with Intel virtualization technologies, using >> > 82599 controller, then Michael is my only hope, as getting the new >> > kernel versions and upstream support will take considerable amount of >> > time. >> > >> > Michael, Could you please share your experience on L2 bridging >> > using Intel virtualization technologies. >> > >> > Thanks, >> > Bharath >> > >> > On Wed, Mar 16, 2016 at 9:40 PM, Rose, Gregory V >> > > wrote: >> > >> > Intel has not supported promiscuous mode for virtual functions due >> > to the security concerns mentioned below. >> > >> > There will be upstream support in an upcoming Linux kernel for >> > setting virtual functions as "trusted" and when that is available >> > then Intel will allow virtual functions to enter unicast >> > promiscuous mode on those Ethernet controllers that support >> > promiscuous mode for virtual functions in the HW/FW. Be aware >> > that not all Intel Ethernet controllers have support for unicast >> > promiscuous mode for virtual functions. The only currently >> > released product that does is the X710/XL710. >> > >> > The key take away is that unicast promiscuous mode for X710/XL710 >> > virtual functions requires Linux kernel support, iproute2 package >> > support and driver support. Only when all three of these are in >> > place will the feature work. >> > >> > Thanks, >> > >> > - Greg >> > >> > -----Original Message----- >> > From: Zhang, Helin >> > Sent: Wednesday, March 16, 2016 9:04 AM >> > To: bharath paulraj > > >; Lu, Wenzhuo > > >; Rowden, Aaron F >> > >; >> > Rose, Gregory V > > > >> > Cc: dev@dpdk.org ; Qiu, Michael >> > >; Jayakumar, >> > Muthurajan > > > >> > Subject: RE: [dpdk-dev] Reg: promiscuous mode on VF >> > >> > Hi Bharath >> > >> > For your question of "why intel does not support unicast >> > promiscuos mode?", I'd ask Aaron or Greg to give answers. >> > Thank you very much! >> > >> > Regards, >> > Helin >> > >> > > -----Original Message----- >> > > From: dev [mailto:dev-bounces@dpdk.org >> > ] On Behalf Of bharath paulraj >> > > Sent: Wednesday, March 16, 2016 11:29 PM >> > > To: Lu, Wenzhuo >> > > Cc: dev@dpdk.org >> > > Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF >> > > >> > > Hi Lu, >> > > >> > > Many thanks for your response. Again I have few more queries. >> > > If VF unicast promiscuous mode is not supported then can't we >> > > implement a Layer 2 bridging functionality using intel >> > virtualization >> > > technologies? Or Is there any other way, say tweeking some >> hardware >> > > registers or drivers, which may help us in implementing Layer 2 >> > bridging. >> > > Also I would like to know, why intel does not support unicast >> > promiscuos mode? >> > > It could have been optional register settings and user should >> > have had >> > > a previleage to set or unset it. Besides, security reasons, is >> there >> > > any other big reason why Intel does not support this? >> > > >> > > Thanks, >> > > Bharath Paulraj >> > > >> > > On Wed, Mar 16, 2016 at 6:15 AM, Lu, Wenzhuo >> > > >> > > wrote: >> > > >> > > > Hi Bharath, >> > > > >> > > > > 2) Is the above supported for 82599 controller? If it is >> > > > > supported >> > > > in the NIC, >> > > > > please provide the steps to enable. >> > > > Talking about 82599, VF unicast promiscuous mode is not >> supported. >> > > > Only broadcast and multicast can be supported. >> > > > >> > > > > >> > > > > Thanks, >> > > > > Bharath Paulraj >> > > > >> > > >> > > >> > > >> > > -- >> > > Regards, >> > > Bharath >> > >> > >> > >> > >> > -- >> > Regards, >> > Bharath >> >> > > > -- > Regards, > Bharath > -- Regards, Bharath