Re: [dpdk-dev] Reg: promiscuous mode on VF

["Rose, Gregory V" <gregory.v.rose@intel.com>]

Bharath,

Please try the following:

In this example we are directing the traffic to queue 1 of VF ID 5.  First you must turn on ntuple.

$ ethtool –K p5p2 ntuple on

Then you can set the FD rule:

$ ethtool –N p5p2 flow-type udp4 dst-port 4790 action 1 user-def 5

When I do that this is the rule definition result:

Filter: 2045
        Rule Type: UDP over IPv4
        Src IP addr: 0.0.0.0 mask: 255.255.255.255
        Dest IP addr: 0.0.0.0 mask: 255.255.255.255
        TOS: 0x0 mask: 0xff
        Src port: 0 mask: 0xffff
        Dest port: 4790 mask: 0x0
        VLAN EtherType: 0x0 mask: 0xffff
        VLAN: 0x0 mask: 0xffff
        User-defined: 0x5 mask: 0xffffffffffffff00
        Action: Direct to queue 1

Be sure to use a version of ethtool 3.0 or greater and a recent kernel – I tested on ethtool version 3.15 with a 3.18.4 Linux kernel from kernel.org and using the most recent ixgbe driver release.

Let me know if this works for you.

Thanks,

- Greg

From: Rose, Gregory V
Sent: Thursday, April 7, 2016 4:43 PM
To: bharath paulraj <bharathpaul@gmail.com>; Qiu, Michael <michael.qiu@intel.com>
Cc: Zhang, Helin <helin.zhang@intel.com>; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Rowden, Aaron F <aaron.f.rowden@intel.com>; dev@dpdk.org; Jayakumar, Muthurajan <muthurajan.jayakumar@intel.com>
Subject: RE: [dpdk-dev] Reg: promiscuous mode on VF

Bharath,

I’m sorry for the delay but I am working on a response to this.  There is a way to do what you need to do and I will get together some instructions and respond by tomorrow.

Thanks and regards,

- Greg

From: bharath paulraj [mailto:bharathpaul@gmail.com]
Sent: Thursday, April 7, 2016 3:40 AM
To: Qiu, Michael <michael.qiu@intel.com<mailto:michael.qiu@intel.com>>
Cc: Rose, Gregory V <gregory.v.rose@intel.com<mailto:gregory.v.rose@intel.com>>; Zhang, Helin <helin.zhang@intel.com<mailto:helin.zhang@intel.com>>; Lu, Wenzhuo <wenzhuo.lu@intel.com<mailto:wenzhuo.lu@intel.com>>; Rowden, Aaron F <aaron.f.rowden@intel.com<mailto:aaron.f.rowden@intel.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; Jayakumar, Muthurajan <muthurajan.jayakumar@intel.com<mailto:muthurajan.jayakumar@intel.com>>
Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF

Hi Team,

May I have some update on my previous mail? I am here stuck in flow creation.

Thanks,
Bharath

On Thu, Mar 31, 2016 at 4:13 PM, bharath paulraj <bharathpaul@gmail.com<mailto:bharathpaul@gmail.com>> wrote:
Hi Michael and All,

         I am unable to set the rule to receive the packet on the VF.
Below is my setup.

1. Creating one virtual function with one queue, in one of my port, p2p1.
    modprobe ixgbe MQ=1 max_vfs=1 RSS=1 allow_unsupported_sfp=1
2. Below is the interface status after creating one virtual function.
[root@XXXX sriov]# ifconfig p2p1
p2p1      Link encap:Ethernet  HWaddr A0:36:9F:86:C2:74
          inet6 addr: fe80::a236:9fff:fe86:c274/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500 Metric:1
          RX packets:2540 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:157456 (153.7 KiB)  TX bytes:258 (258.0 b)

[root@XXXX sriov]# ifconfig p2p1_0
p2p1_0    Link encap:Ethernet  HWaddr DA:61:95:CD:AF:35
          inet6 addr: fe80::d861:95ff:fecd:af35/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:12 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:360 (360.0 b)  TX bytes:740 (740.0 b)
3. Next I am enable ntuple
ethtool -K p2p1  ntuple on
4. Now I am adding below rule
ethtool -N p2p1   flow-type udp4 dst-port 4789 action 0x100000000 --> VF 0, queue 0
ethtool -N p2p1   flow-type udp4 dst-port 4790 action 0x000000000 --> PF queue 0
5. [root@XXX sriov]# ethtool -n p2p1
1 RX rings available
Total 2 rules

Filter: 2044
        Rule Type: UDP over IPv4
        Src IP addr: 0.0.0.0 mask: 255.255.255.255
        Dest IP addr: 0.0.0.0 mask: 255.255.255.255
        TOS: 0x0 mask: 0xff
        Src port: 0 mask: 0xffff
        Dest port: 4790 mask: 0x0
        VLAN EtherType: 0x0 mask: 0xffff
        VLAN: 0x0 mask: 0xffff
        User-defined: 0x0 mask: 0xffffffffffffffff
        Action: Direct to queue 0

Filter: 2045
        Rule Type: UDP over IPv4
        Src IP addr: 0.0.0.0 mask: 255.255.255.255
        Dest IP addr: 0.0.0.0 mask: 255.255.255.255
        TOS: 0x0 mask: 0xff
        Src port: 0 mask: 0xffff
        Dest port: 4789 mask: 0x0
        VLAN EtherType: 0x0 mask: 0xffff
        VLAN: 0x0 mask: 0xffff
        User-defined: 0x0 mask: 0xffffffffffffffff
        Action: Direct to queue 0
    >> Won't it show the VF queue numbers here?

6. Start the VM over p2p1_0
7. Below is the Packet I am sending
a) Dest MAC - VF Mac, Src MAC - any untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src port - 100 dest port - 4789
b) Dest MAC - VF Mac, Src MAC - any, untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src port - 100 dest port - 4790
c) Dest MAC - VF Mac, untagged, src ip - 1.1.1.1 dest ip - 2.2.2.2 src port - 100 dest port - 4791

All the above testing is done on centOs-6.7 with ixgbe version - 4.3.13 with patch you mentioned on 82599 Ethernet controller
Linux XXX 2.6.32-573.22.1.el6.x86_64 #1 SMP Wed Mar 23 03:35:39 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux


Observation:
If the packet matches the rule, I am not able to see the packet in the VF, instead I am able to see the packet in PF.
for the packets a) and b), I am able to see te packet only in PF. Even if the packet destination MAC is VF's MAC,
I am able to see only in PF.
If the packet is not matching the rule, then I am able to see the packet in VF, provided packet destination MAC is VF's MAC.
Question:
1) Am I mapping the queues wrongly while adding the rules?
2) How to Identify which VF using which Queues?

    Request you to provide some help on it.


On Tue, Mar 22, 2016 at 1:03 PM, bharath paulraj <bharathpaul@gmail.com<mailto:bharathpaul@gmail.com>> wrote:
Thanks a lot Michael.  Finally i am able to see some light. I will try the same in our setup and will post you the results.

Thanks,
Bharath

On Tue, Mar 22, 2016 at 12:09 PM, Qiu, Michael <michael.qiu@intel.com<mailto:michael.qiu@intel.com>> wrote:
Yes, we could let ovs using 82599 VF to do rx/tx. I don't know what's
your l2 bridge, but since ovs could work I think your bridge also could
work. But I only tested with one VF.

Make sure below two patches (bifurcate driver) are included in your kernel:

_https://patchwork.ozlabs.org/patch/476511/_
_https://patchwork.ozlabs.org/patch/476516/_

Mostly, if your kernel version in 4.2 or newer, it should be included.

After you create VF, before you passthrough the VF to guest:

(vf +1) << 32 + queue-index,


 1. where vf is the VF index starting from 0
 2. the queue-index is 0 if multi-queue support is not turned on, and
    this value is [0,1] if multiple-queue is turned on


echo 1 > /sys/bus/pci/devices/0000\:05\:00.0/sriov_numvfs
ifconfig $(PF_INTF) up
ifconfig $(VF0_INFT) up
ip link set $(PF_INTF) promisc on
ethtool -K $(PF_INTF) ntuple on
ethtool -N $(PF_INTF) flow-type udp4 dst-port 4789 action 0x100000000
(VF0 queue 0)

Here we using flow director to all let packets according to the rules to
the VF, But I don't know if it could let the packets to other VFs at the
same time.

Thanks,
Michael

On 3/17/2016 2:43 PM, bharath paulraj wrote:
> Hi Lu, Helin, Greg,
>
>   Many thanks for your response, which is really quick. Now, If I want
> to implement L2 bridging with Intel virtualization technologies, using
> 82599 controller, then Michael is my only hope, as getting the new
> kernel versions and upstream support will take considerable amount of
> time.
>
>    Michael, Could you please share your experience on L2 bridging
> using Intel virtualization technologies.
>
> Thanks,
> Bharath
>
> On Wed, Mar 16, 2016 at 9:40 PM, Rose, Gregory V
> <gregory.v.rose@intel.com<mailto:gregory.v.rose@intel.com> <mailto:gregory.v.rose@intel.com<mailto:gregory.v.rose@intel.com>>> wrote:
>
>     Intel has not supported promiscuous mode for virtual functions due
>     to the security concerns mentioned below.
>
>     There will be upstream support in an upcoming Linux kernel for
>     setting virtual functions as "trusted" and when that is available
>     then Intel will allow virtual functions to enter unicast
>     promiscuous mode on those Ethernet controllers that support
>     promiscuous mode for virtual functions in the HW/FW.  Be aware
>     that not all Intel Ethernet controllers have support for unicast
>     promiscuous mode for virtual functions.  The only currently
>     released product that does is the X710/XL710.
>
>     The key take away is that unicast promiscuous mode for X710/XL710
>     virtual functions requires Linux kernel support, iproute2 package
>     support and driver support.  Only when all three of these are in
>     place will the feature work.
>
>     Thanks,
>
>     - Greg
>
>     -----Original Message-----
>     From: Zhang, Helin
>     Sent: Wednesday, March 16, 2016 9:04 AM
>     To: bharath paulraj <bharathpaul@gmail.com<mailto:bharathpaul@gmail.com>
>     <mailto:bharathpaul@gmail.com<mailto:bharathpaul@gmail.com>>>; Lu, Wenzhuo <wenzhuo.lu@intel.com<mailto:wenzhuo.lu@intel.com>
>     <mailto:wenzhuo.lu@intel.com<mailto:wenzhuo.lu@intel.com>>>; Rowden, Aaron F
>     <aaron.f.rowden@intel.com<mailto:aaron.f.rowden@intel.com> <mailto:aaron.f.rowden@intel.com<mailto:aaron.f.rowden@intel.com>>>;
>     Rose, Gregory V <gregory.v.rose@intel.com<mailto:gregory.v.rose@intel.com>
>     <mailto:gregory.v.rose@intel.com<mailto:gregory.v.rose@intel.com>>>
>     Cc: dev@dpdk.org<mailto:dev@dpdk.org> <mailto:dev@dpdk.org<mailto:dev@dpdk.org>>; Qiu, Michael
>     <michael.qiu@intel.com<mailto:michael.qiu@intel.com> <mailto:michael.qiu@intel.com<mailto:michael.qiu@intel.com>>>; Jayakumar,
>     Muthurajan <muthurajan.jayakumar@intel.com<mailto:muthurajan.jayakumar@intel.com>
>     <mailto:muthurajan.jayakumar@intel.com<mailto:muthurajan.jayakumar@intel.com>>>
>     Subject: RE: [dpdk-dev] Reg: promiscuous mode on VF
>
>     Hi Bharath
>
>     For your question of "why intel does not support unicast
>     promiscuos mode?", I'd ask Aaron or Greg to give answers.
>     Thank you very much!
>
>     Regards,
>     Helin
>
>     > -----Original Message-----
>     > From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>
>     <mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>>] On Behalf Of bharath paulraj
>     > Sent: Wednesday, March 16, 2016 11:29 PM
>     > To: Lu, Wenzhuo
>     > Cc: dev@dpdk.org<mailto:dev@dpdk.org> <mailto:dev@dpdk.org<mailto:dev@dpdk.org>>
>     > Subject: Re: [dpdk-dev] Reg: promiscuous mode on VF
>     >
>     > Hi Lu,
>     >
>     > Many thanks for your response. Again I have few more queries.
>     > If VF unicast promiscuous mode is not supported then can't we
>     > implement a Layer 2 bridging functionality using intel
>     virtualization
>     > technologies? Or Is there any other way, say tweeking some hardware
>     > registers or drivers, which may help us in implementing Layer 2
>     bridging.
>     > Also I would like to know, why intel does not support unicast
>     promiscuos mode?
>     > It could have been optional register settings and user should
>     have had
>     > a previleage to set or unset it. Besides, security reasons, is there
>     > any other big reason why Intel does not support this?
>     >
>     > Thanks,
>     > Bharath Paulraj
>     >
>     > On Wed, Mar 16, 2016 at 6:15 AM, Lu, Wenzhuo
>     <wenzhuo.lu@intel.com<mailto:wenzhuo.lu@intel.com> <mailto:wenzhuo.lu@intel.com<mailto:wenzhuo.lu@intel.com>>>
>     > wrote:
>     >
>     > > Hi Bharath,
>     > >
>     > > >     2) Is the above supported for 82599 controller? If it is
>     > > > supported
>     > > in the NIC,
>     > > > please provide the steps to enable.
>     > > Talking about 82599, VF unicast promiscuous mode is not supported.
>     > > Only broadcast and multicast can be supported.
>     > >
>     > > >
>     > > > Thanks,
>     > > > Bharath Paulraj
>     > >
>     >
>     >
>     >
>     > --
>     > Regards,
>     > Bharath
>
>
>
>
> --
> Regards,
> Bharath



--
Regards,
Bharath



--
Regards,
Bharath



--
Regards,
Bharath