From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) by dpdk.org (Postfix) with ESMTP id 5517D5323 for ; Tue, 28 Jan 2014 04:12:12 +0100 (CET) Received: by mail-qc0-f174.google.com with SMTP id x13so9181753qcv.5 for ; Mon, 27 Jan 2014 19:13:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ukrivvf1+P8xf3ri7hEh3pI1VSdmaeLt4DaXlrqSIRo=; b=1BeuCcRvjJaiYxsVbClU5BGBAq7b9wv4j7f+/kk1bphTqA7LmVPZh9AsD74zTNl6mM OODfUy1x9sI7rQ9eqQDuM1EbGm2x6ahXTvvEOFXaI8T472p0eUD1iOLWIBFnEMQ0x4N5 +MOImsb0PgcAmHgBOY9b92sfqKdbwEuDYZyj4vf5SnUgA+0LtuxLpwkYXxdbOLAoFGUY Piagp0RDQGsZflCwzn9Uq4IJyOC5VR0+ThWNZjJKVl9cMZUru9C+LI19NJHMErMC/Ht4 BDtjR0KIip8aROQMkPBzoRPp95x7jFY80yDWMv0AI2F1MfdUfmz1YHQUcPfBYhUo5DWX nqBQ== MIME-Version: 1.0 X-Received: by 10.229.56.200 with SMTP id z8mr48184120qcg.1.1390878810504; Mon, 27 Jan 2014 19:13:30 -0800 (PST) Received: by 10.96.63.166 with HTTP; Mon, 27 Jan 2014 19:13:30 -0800 (PST) In-Reply-To: References: <20140123134356.522e822a@nehalam.linuxnetplumber.net> Date: Tue, 28 Jan 2014 03:13:30 +0000 Message-ID: From: James Yu To: Stephen Hemminger Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Passing VLAN traffic via l2fwd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jan 2014 03:12:12 -0000 Some more discovery. That flag PKT_TX_VLAN_PKT eventually send the following command to the chips (ixgbe_xmit_pkts(), tx_desc_vlan_flags_to_cmdtype()) . Is this actually adding a VLAN tag during tx ? #define IXGBE_ADVTXD_DCMD_VLE IXGBE_TXD_CMD_VLE /* VLAN pkt enable */ #define IXGBE_TXD_CMD_VLE 0x40000000 /* Add VLAN tag */ After I intentionally remove the tag before sending out the traffic, the Spirent can receive tagged packets. I am not sure the DPDK codes is adding a tag. I think the ixgbevf on the host is adding VLAN tag based on the configured tag ID. Is this expected ? How can I configure the host to not add tag since traffic from the VM is already tagged. Otherwise it becomes double tagged. Thanks James On Fri, Jan 24, 2014 at 6:07 AM, James Yu wrote: > Stephen, > > I added that to the l2fwd main.c and it is still not received by the > Spirent. I obtained the stat using rte_eth_stats_get() and printout the > following when I send only one VLAN pkt > ipackets: increased by 1 > opackets: no change (pkt not sent out by the device) > ierrors, oerrors, otxerror, otxdiscard: all 0 > > The same thing with 10 packets which increase 10 ipackets counts. > > I found out that the rte_eth_stats_get() obtains the stats from > ixgbevf_dev_stats_get(). Is this expected for SR-IOV that uses > ixgbe_ethdev.c ? > If that's correct, I think this tells me that the tagged packet is > received by the VF PMD codes but not sent out and has no error or packets > dropped. > > So it was not sent out of the VF PMD. But why there is not error or packet > dropped. Where did it go ? I am trying to find out where it got dropped and > how to fix it. > > I tried DPDK1.5.2r1. It exhibits the same issue even with the patch. > > Thanks > > James > > > > On Thu, Jan 23, 2014 at 1:43 PM, Stephen Hemminger < > stephen@networkplumber.org> wrote: > >> On Thu, 23 Jan 2014 11:27:14 -0800 >> James Yu wrote: >> >> > Hi Daniel, >> > >> > Untagged traffic could be looped back. I am doing a one-directional loop >> > back. In the Frame tabe of the StreamBlock editor, I did the following >> > stream1: >> > destination address: MAC address of the receiving port of the guest >> > VM. (obtained from the debug print out when l2fwd is started, or >> ifconfig >> > before l2fwd is started) >> > stream2: >> > destination address: MAC address of the sending port of the guest >> VM >> > (obtained from the debug print out when l2fwd is started, or ifconfig >> > before l2fwd is started) >> > >> > >> > The only thing I add is to tag both the SR-IOV VF ports on the host with >> > the same tag using ip command and add that tag number as the VID to the >> > Spirent streams. After that that traffic can reach l2fwd which reported >> > forwarded the packets on the periodic printout. However the packets did >> not >> > reach the Spirent. >> > >> > Also it is exercising rte_ixgbevf_pmd codes as indicated during starting >> > l2fwd. >> > >> > EAL: Master core 0 is ready (tid=b77ab7d0) >> > >> > EAL: Core 1 is ready (tid=adffeb70) >> > >> > EAL: probe driver: 8086:100e rte_em_pmd >> > >> > EAL: probe driver: 8086:100e rte_em_pmd >> > >> > *EAL: probe driver: 8086:10ed rte_ixgbevf_pmd* >> > I wonder do I have to change any codes in the DPDK library or call >> certain >> > routines to setup the VLAN in the DPDK stack during starting the l2fwd ? >> > >> > >> > Were you able to send VLAN traffic from your iXia and looped back to >> your >> > iXia ? >> > >> > >> > Thanks >> > >> > James >> > >> > >> > >> > >> > On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky < >> > daniel.kaminsky@infinitelocality.com> wrote: >> > >> > > Hi James, >> > > >> > > Did you try a non-tagged traffic? I suspect this is the limitation of >> the >> > > l2fwd or the requirements of Spirent. From my experience with Ixia, >> the L2 >> > > forward device must implement the ARP protocol (at least a subset of >> it). >> > > >> > > Also, notice that the l2fwd example sets the destination MAC address >> to >> > > "02:00:00:00:00:xx" which the Spirent might doesn't like. >> > > >> > > Daniel >> > > >> > > >> > > On Thu, Jan 23, 2014 at 7:08 AM, James Yu wrote: >> > > >> > >> I could not pass tagged traffic through the l2fwd program running >> inside a >> > >> CentOS VM. The l2fwd program reported sending out all the packets >> received >> > >> from one port to the other port. But the outside Spirent tester >> could not >> > >> receive a packet at all. I am wondering maybe the tagged packets are >> > >> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone >> knows >> > >> how to pass/send VLAN traffic through the DPDK ? >> > >> >> > >> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each >> > >> other >> > >> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues >> with >> > >> sending out tagged packets out of DPDK to the host. >> > >> >> > >> I have done the following: >> > >> >> > >> On RHEL6.1 host >> > >> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 >> and "ip >> > >> link set eth3 vf 0 vlan 3" for port 2 >> > >> [root@cent64x64-137189 ~]# ip link show eth2 >> > >> 96: eth2: mtu 1500 qdisc mq state >> UP >> > >> qlen >> > >> 1000 >> > >> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff >> > >> vf 0 MAC 52:54:00:f9:9a:90, vlan 3 >> > >> [root@cent64x64-137189 ~]# ip link show eth3 >> > >> 98: eth3: mtu 1500 qdisc mq state >> UP >> > >> qlen >> > >> 1000 >> > >> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff >> > >> vf 0 MAC 52:54:00:30:37:98, vlan 3 >> > >> >> > >> 2. virsh configure SR-IOV hostdev >> > >> >> > >> >> > >> >> > >>
> > >> function='0x0'/> >> > >> >> > >>
> > >> function='0x0'/> >> > >> >> > >> >> > >> >> > >> >> > >>
> > >> function='0x1'/> >> > >> >> > >>
> > >> function='0x0'/> >> > >> >> > >> >> > >> [root@cent64x64-137189 ~]# lspci |grep Eth >> > >> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit >> SFI/SFP+ >> > >> Network Connection (rev 01) >> > >> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit >> SFI/SFP+ >> > >> Network Connection (rev 01) >> > >> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> >> > >> >> > >> On guest VM >> > >> 1. check all the ports are started up and check their corresponding >> port >> > >> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd >> > >> [root@VirtualADX ~]# lspci |grep Eth >> > >> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit >> Ethernet >> > >> Controller (rev 03) >> > >> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit >> Ethernet >> > >> Controller (rev 03) >> > >> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet >> Controller >> > >> Virtual Function (rev 01) >> > >> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit >> Ethernet >> > >> Controller (rev 03) >> > >> >> > >> 2. run this command: >> > >> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b >> > >> 000:00:0a.0 -- -q 1 -p 3 >> > >> >> > >> Although the l2fwd forwards all received tagged packets, the >> Spirent >> > >> tester did not receive any packet at all. >> > >> >> > > >> > > >> >> The VLAN offload bit in the mbuf is different for TX and RX. >> Probably would work with something like this: >> >> diff --git a/examples/l2fwd/main.c b/examples/l2fwd/main.c >> index 05ef330..70dbf85 100644 >> --- a/examples/l2fwd/main.c >> +++ b/examples/l2fwd/main.c >> @@ -289,6 +289,9 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned >> portid) >> /* src addr */ >> ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr); >> >> + if (m->ol_flags & PKT_RX_VLAN_PKT) >> + m->ol_flags |= PKT_TX_VLAN_PKT; >> + >> l2fwd_send_packet(m, (uint8_t) dst_port); >> } >> >> >> >> >