* [dpdk-dev] Passing VLAN traffic via l2fwd
@ 2014-01-23 5:08 James Yu
2014-01-23 8:45 ` Daniel Kaminsky
0 siblings, 1 reply; 6+ messages in thread
From: James Yu @ 2014-01-23 5:08 UTC (permalink / raw)
To: dev
I could not pass tagged traffic through the l2fwd program running inside a
CentOS VM. The l2fwd program reported sending out all the packets received
from one port to the other port. But the outside Spirent tester could not
receive a packet at all. I am wondering maybe the tagged packets are
dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone knows
how to pass/send VLAN traffic through the DPDK ?
With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each other
without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues with
sending out tagged packets out of DPDK to the host.
I have done the following:
On RHEL6.1 host
1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 and "ip
link set eth3 vf 0 vlan 3" for port 2
[root@cent64x64-137189 ~]# ip link show eth2
96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:f9:9a:90, vlan 3
[root@cent64x64-137189 ~]# ip link show eth3
98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
1000
link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:30:37:98, vlan 3
2. virsh configure SR-IOV hostdev
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:f9:9a:90'/>
<source>
<address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08'
function='0x0'/>
</interface>
<interface type='hostdev' managed='yes'>
<mac address='52:54:00:30:37:98'/>
<source>
<address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09'
function='0x0'/>
</interface>
[root@cent64x64-137189 ~]# lspci |grep Eth
1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
On guest VM
1. check all the ports are started up and check their corresponding port
type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd
[root@VirtualADX ~]# lspci |grep Eth
00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
Controller (rev 03)
00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
Controller (rev 03)
00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
Controller (rev 03)
2. run this command:
/root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b
000:00:0a.0 -- -q 1 -p 3
Although the l2fwd forwards all received tagged packets, the Spirent
tester did not receive any packet at all.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Passing VLAN traffic via l2fwd
2014-01-23 5:08 [dpdk-dev] Passing VLAN traffic via l2fwd James Yu
@ 2014-01-23 8:45 ` Daniel Kaminsky
2014-01-23 19:27 ` James Yu
0 siblings, 1 reply; 6+ messages in thread
From: Daniel Kaminsky @ 2014-01-23 8:45 UTC (permalink / raw)
To: James Yu; +Cc: dev
Hi James,
Did you try a non-tagged traffic? I suspect this is the limitation of the
l2fwd or the requirements of Spirent. From my experience with Ixia, the L2
forward device must implement the ARP protocol (at least a subset of it).
Also, notice that the l2fwd example sets the destination MAC address to
"02:00:00:00:00:xx" which the Spirent might doesn't like.
Daniel
On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011@gmail.com> wrote:
> I could not pass tagged traffic through the l2fwd program running inside a
> CentOS VM. The l2fwd program reported sending out all the packets received
> from one port to the other port. But the outside Spirent tester could not
> receive a packet at all. I am wondering maybe the tagged packets are
> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone knows
> how to pass/send VLAN traffic through the DPDK ?
>
> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each other
> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues with
> sending out tagged packets out of DPDK to the host.
>
> I have done the following:
>
> On RHEL6.1 host
> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 and "ip
> link set eth3 vf 0 vlan 3" for port 2
> [root@cent64x64-137189 ~]# ip link show eth2
> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
> 1000
> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 52:54:00:f9:9a:90, vlan 3
> [root@cent64x64-137189 ~]# ip link show eth3
> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen
> 1000
> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
> vf 0 MAC 52:54:00:30:37:98, vlan 3
>
> 2. virsh configure SR-IOV hostdev
> <interface type='hostdev' managed='yes'>
> <mac address='52:54:00:f9:9a:90'/>
> <source>
> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
> function='0x0'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
> </interface>
> <interface type='hostdev' managed='yes'>
> <mac address='52:54:00:30:37:98'/>
> <source>
> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
> function='0x1'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
> function='0x0'/>
> </interface>
>
> [root@cent64x64-137189 ~]# lspci |grep Eth
> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
>
> On guest VM
> 1. check all the ports are started up and check their corresponding port
> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd
> [root@VirtualADX ~]# lspci |grep Eth
> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
> Controller (rev 03)
> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
> Controller (rev 03)
> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
> Controller (rev 03)
>
> 2. run this command:
> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b
> 000:00:0a.0 -- -q 1 -p 3
>
> Although the l2fwd forwards all received tagged packets, the Spirent
> tester did not receive any packet at all.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Passing VLAN traffic via l2fwd
2014-01-23 8:45 ` Daniel Kaminsky
@ 2014-01-23 19:27 ` James Yu
2014-01-23 21:43 ` Stephen Hemminger
0 siblings, 1 reply; 6+ messages in thread
From: James Yu @ 2014-01-23 19:27 UTC (permalink / raw)
To: Daniel Kaminsky; +Cc: dev
Hi Daniel,
Untagged traffic could be looped back. I am doing a one-directional loop
back. In the Frame tabe of the StreamBlock editor, I did the following
stream1:
destination address: MAC address of the receiving port of the guest
VM. (obtained from the debug print out when l2fwd is started, or ifconfig
before l2fwd is started)
stream2:
destination address: MAC address of the sending port of the guest VM
(obtained from the debug print out when l2fwd is started, or ifconfig
before l2fwd is started)
The only thing I add is to tag both the SR-IOV VF ports on the host with
the same tag using ip command and add that tag number as the VID to the
Spirent streams. After that that traffic can reach l2fwd which reported
forwarded the packets on the periodic printout. However the packets did not
reach the Spirent.
Also it is exercising rte_ixgbevf_pmd codes as indicated during starting
l2fwd.
EAL: Master core 0 is ready (tid=b77ab7d0)
EAL: Core 1 is ready (tid=adffeb70)
EAL: probe driver: 8086:100e rte_em_pmd
EAL: probe driver: 8086:100e rte_em_pmd
*EAL: probe driver: 8086:10ed rte_ixgbevf_pmd*
I wonder do I have to change any codes in the DPDK library or call certain
routines to setup the VLAN in the DPDK stack during starting the l2fwd ?
Were you able to send VLAN traffic from your iXia and looped back to your
iXia ?
Thanks
James
On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky <
daniel.kaminsky@infinitelocality.com> wrote:
> Hi James,
>
> Did you try a non-tagged traffic? I suspect this is the limitation of the
> l2fwd or the requirements of Spirent. From my experience with Ixia, the L2
> forward device must implement the ARP protocol (at least a subset of it).
>
> Also, notice that the l2fwd example sets the destination MAC address to
> "02:00:00:00:00:xx" which the Spirent might doesn't like.
>
> Daniel
>
>
> On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011@gmail.com> wrote:
>
>> I could not pass tagged traffic through the l2fwd program running inside a
>> CentOS VM. The l2fwd program reported sending out all the packets received
>> from one port to the other port. But the outside Spirent tester could not
>> receive a packet at all. I am wondering maybe the tagged packets are
>> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone knows
>> how to pass/send VLAN traffic through the DPDK ?
>>
>> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each
>> other
>> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues with
>> sending out tagged packets out of DPDK to the host.
>>
>> I have done the following:
>>
>> On RHEL6.1 host
>> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 and "ip
>> link set eth3 vf 0 vlan 3" for port 2
>> [root@cent64x64-137189 ~]# ip link show eth2
>> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> qlen
>> 1000
>> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
>> vf 0 MAC 52:54:00:f9:9a:90, vlan 3
>> [root@cent64x64-137189 ~]# ip link show eth3
>> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> qlen
>> 1000
>> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
>> vf 0 MAC 52:54:00:30:37:98, vlan 3
>>
>> 2. virsh configure SR-IOV hostdev
>> <interface type='hostdev' managed='yes'>
>> <mac address='52:54:00:f9:9a:90'/>
>> <source>
>> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
>> function='0x0'/>
>> </source>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>> function='0x0'/>
>> </interface>
>> <interface type='hostdev' managed='yes'>
>> <mac address='52:54:00:30:37:98'/>
>> <source>
>> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
>> function='0x1'/>
>> </source>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
>> function='0x0'/>
>> </interface>
>>
>> [root@cent64x64-137189 ~]# lspci |grep Eth
>> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
>> Network Connection (rev 01)
>> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
>> Network Connection (rev 01)
>> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>>
>>
>> On guest VM
>> 1. check all the ports are started up and check their corresponding port
>> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd
>> [root@VirtualADX ~]# lspci |grep Eth
>> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
>> Controller (rev 03)
>> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
>> Controller (rev 03)
>> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
>> Virtual Function (rev 01)
>> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
>> Controller (rev 03)
>>
>> 2. run this command:
>> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b
>> 000:00:0a.0 -- -q 1 -p 3
>>
>> Although the l2fwd forwards all received tagged packets, the Spirent
>> tester did not receive any packet at all.
>>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Passing VLAN traffic via l2fwd
2014-01-23 19:27 ` James Yu
@ 2014-01-23 21:43 ` Stephen Hemminger
2014-01-24 6:07 ` James Yu
0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2014-01-23 21:43 UTC (permalink / raw)
To: James Yu; +Cc: dev
On Thu, 23 Jan 2014 11:27:14 -0800
James Yu <ypyu2011@gmail.com> wrote:
> Hi Daniel,
>
> Untagged traffic could be looped back. I am doing a one-directional loop
> back. In the Frame tabe of the StreamBlock editor, I did the following
> stream1:
> destination address: MAC address of the receiving port of the guest
> VM. (obtained from the debug print out when l2fwd is started, or ifconfig
> before l2fwd is started)
> stream2:
> destination address: MAC address of the sending port of the guest VM
> (obtained from the debug print out when l2fwd is started, or ifconfig
> before l2fwd is started)
>
>
> The only thing I add is to tag both the SR-IOV VF ports on the host with
> the same tag using ip command and add that tag number as the VID to the
> Spirent streams. After that that traffic can reach l2fwd which reported
> forwarded the packets on the periodic printout. However the packets did not
> reach the Spirent.
>
> Also it is exercising rte_ixgbevf_pmd codes as indicated during starting
> l2fwd.
>
> EAL: Master core 0 is ready (tid=b77ab7d0)
>
> EAL: Core 1 is ready (tid=adffeb70)
>
> EAL: probe driver: 8086:100e rte_em_pmd
>
> EAL: probe driver: 8086:100e rte_em_pmd
>
> *EAL: probe driver: 8086:10ed rte_ixgbevf_pmd*
> I wonder do I have to change any codes in the DPDK library or call certain
> routines to setup the VLAN in the DPDK stack during starting the l2fwd ?
>
>
> Were you able to send VLAN traffic from your iXia and looped back to your
> iXia ?
>
>
> Thanks
>
> James
>
>
>
>
> On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky <
> daniel.kaminsky@infinitelocality.com> wrote:
>
> > Hi James,
> >
> > Did you try a non-tagged traffic? I suspect this is the limitation of the
> > l2fwd or the requirements of Spirent. From my experience with Ixia, the L2
> > forward device must implement the ARP protocol (at least a subset of it).
> >
> > Also, notice that the l2fwd example sets the destination MAC address to
> > "02:00:00:00:00:xx" which the Spirent might doesn't like.
> >
> > Daniel
> >
> >
> > On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011@gmail.com> wrote:
> >
> >> I could not pass tagged traffic through the l2fwd program running inside a
> >> CentOS VM. The l2fwd program reported sending out all the packets received
> >> from one port to the other port. But the outside Spirent tester could not
> >> receive a packet at all. I am wondering maybe the tagged packets are
> >> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone knows
> >> how to pass/send VLAN traffic through the DPDK ?
> >>
> >> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each
> >> other
> >> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues with
> >> sending out tagged packets out of DPDK to the host.
> >>
> >> I have done the following:
> >>
> >> On RHEL6.1 host
> >> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 and "ip
> >> link set eth3 vf 0 vlan 3" for port 2
> >> [root@cent64x64-137189 ~]# ip link show eth2
> >> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> >> qlen
> >> 1000
> >> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
> >> vf 0 MAC 52:54:00:f9:9a:90, vlan 3
> >> [root@cent64x64-137189 ~]# ip link show eth3
> >> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> >> qlen
> >> 1000
> >> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
> >> vf 0 MAC 52:54:00:30:37:98, vlan 3
> >>
> >> 2. virsh configure SR-IOV hostdev
> >> <interface type='hostdev' managed='yes'>
> >> <mac address='52:54:00:f9:9a:90'/>
> >> <source>
> >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
> >> function='0x0'/>
> >> </source>
> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> >> function='0x0'/>
> >> </interface>
> >> <interface type='hostdev' managed='yes'>
> >> <mac address='52:54:00:30:37:98'/>
> >> <source>
> >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
> >> function='0x1'/>
> >> </source>
> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
> >> function='0x0'/>
> >> </interface>
> >>
> >> [root@cent64x64-137189 ~]# lspci |grep Eth
> >> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> >> Network Connection (rev 01)
> >> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> >> Network Connection (rev 01)
> >> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> >> Virtual Function (rev 01)
> >> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> >> Virtual Function (rev 01)
> >>
> >>
> >> On guest VM
> >> 1. check all the ports are started up and check their corresponding port
> >> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd
> >> [root@VirtualADX ~]# lspci |grep Eth
> >> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
> >> Controller (rev 03)
> >> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
> >> Controller (rev 03)
> >> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> >> Virtual Function (rev 01)
> >> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> >> Virtual Function (rev 01)
> >> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet
> >> Controller (rev 03)
> >>
> >> 2. run this command:
> >> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b
> >> 000:00:0a.0 -- -q 1 -p 3
> >>
> >> Although the l2fwd forwards all received tagged packets, the Spirent
> >> tester did not receive any packet at all.
> >>
> >
> >
The VLAN offload bit in the mbuf is different for TX and RX.
Probably would work with something like this:
diff --git a/examples/l2fwd/main.c b/examples/l2fwd/main.c
index 05ef330..70dbf85 100644
--- a/examples/l2fwd/main.c
+++ b/examples/l2fwd/main.c
@@ -289,6 +289,9 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned portid)
/* src addr */
ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr);
+ if (m->ol_flags & PKT_RX_VLAN_PKT)
+ m->ol_flags |= PKT_TX_VLAN_PKT;
+
l2fwd_send_packet(m, (uint8_t) dst_port);
}
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Passing VLAN traffic via l2fwd
2014-01-23 21:43 ` Stephen Hemminger
@ 2014-01-24 6:07 ` James Yu
2014-01-28 3:13 ` James Yu
0 siblings, 1 reply; 6+ messages in thread
From: James Yu @ 2014-01-24 6:07 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
Stephen,
I added that to the l2fwd main.c and it is still not received by the
Spirent. I obtained the stat using rte_eth_stats_get() and printout the
following when I send only one VLAN pkt
ipackets: increased by 1
opackets: no change (pkt not sent out by the device)
ierrors, oerrors, otxerror, otxdiscard: all 0
The same thing with 10 packets which increase 10 ipackets counts.
I found out that the rte_eth_stats_get() obtains the stats from
ixgbevf_dev_stats_get(). Is this expected for SR-IOV that uses
ixgbe_ethdev.c ?
If that's correct, I think this tells me that the tagged packet is received
by the VF PMD codes but not sent out and has no error or packets dropped.
So it was not sent out of the VF PMD. But why there is not error or packet
dropped. Where did it go ? I am trying to find out where it got dropped and
how to fix it.
I tried DPDK1.5.2r1. It exhibits the same issue even with the patch.
Thanks
James
On Thu, Jan 23, 2014 at 1:43 PM, Stephen Hemminger <
stephen@networkplumber.org> wrote:
> On Thu, 23 Jan 2014 11:27:14 -0800
> James Yu <ypyu2011@gmail.com> wrote:
>
> > Hi Daniel,
> >
> > Untagged traffic could be looped back. I am doing a one-directional loop
> > back. In the Frame tabe of the StreamBlock editor, I did the following
> > stream1:
> > destination address: MAC address of the receiving port of the guest
> > VM. (obtained from the debug print out when l2fwd is started, or ifconfig
> > before l2fwd is started)
> > stream2:
> > destination address: MAC address of the sending port of the guest VM
> > (obtained from the debug print out when l2fwd is started, or ifconfig
> > before l2fwd is started)
> >
> >
> > The only thing I add is to tag both the SR-IOV VF ports on the host with
> > the same tag using ip command and add that tag number as the VID to the
> > Spirent streams. After that that traffic can reach l2fwd which reported
> > forwarded the packets on the periodic printout. However the packets did
> not
> > reach the Spirent.
> >
> > Also it is exercising rte_ixgbevf_pmd codes as indicated during starting
> > l2fwd.
> >
> > EAL: Master core 0 is ready (tid=b77ab7d0)
> >
> > EAL: Core 1 is ready (tid=adffeb70)
> >
> > EAL: probe driver: 8086:100e rte_em_pmd
> >
> > EAL: probe driver: 8086:100e rte_em_pmd
> >
> > *EAL: probe driver: 8086:10ed rte_ixgbevf_pmd*
> > I wonder do I have to change any codes in the DPDK library or call
> certain
> > routines to setup the VLAN in the DPDK stack during starting the l2fwd ?
> >
> >
> > Were you able to send VLAN traffic from your iXia and looped back to your
> > iXia ?
> >
> >
> > Thanks
> >
> > James
> >
> >
> >
> >
> > On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky <
> > daniel.kaminsky@infinitelocality.com> wrote:
> >
> > > Hi James,
> > >
> > > Did you try a non-tagged traffic? I suspect this is the limitation of
> the
> > > l2fwd or the requirements of Spirent. From my experience with Ixia,
> the L2
> > > forward device must implement the ARP protocol (at least a subset of
> it).
> > >
> > > Also, notice that the l2fwd example sets the destination MAC address to
> > > "02:00:00:00:00:xx" which the Spirent might doesn't like.
> > >
> > > Daniel
> > >
> > >
> > > On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011@gmail.com> wrote:
> > >
> > >> I could not pass tagged traffic through the l2fwd program running
> inside a
> > >> CentOS VM. The l2fwd program reported sending out all the packets
> received
> > >> from one port to the other port. But the outside Spirent tester could
> not
> > >> receive a packet at all. I am wondering maybe the tagged packets are
> > >> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone
> knows
> > >> how to pass/send VLAN traffic through the DPDK ?
> > >>
> > >> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each
> > >> other
> > >> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues
> with
> > >> sending out tagged packets out of DPDK to the host.
> > >>
> > >> I have done the following:
> > >>
> > >> On RHEL6.1 host
> > >> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1 and
> "ip
> > >> link set eth3 vf 0 vlan 3" for port 2
> > >> [root@cent64x64-137189 ~]# ip link show eth2
> > >> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> > >> qlen
> > >> 1000
> > >> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
> > >> vf 0 MAC 52:54:00:f9:9a:90, vlan 3
> > >> [root@cent64x64-137189 ~]# ip link show eth3
> > >> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> > >> qlen
> > >> 1000
> > >> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
> > >> vf 0 MAC 52:54:00:30:37:98, vlan 3
> > >>
> > >> 2. virsh configure SR-IOV hostdev
> > >> <interface type='hostdev' managed='yes'>
> > >> <mac address='52:54:00:f9:9a:90'/>
> > >> <source>
> > >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
> > >> function='0x0'/>
> > >> </source>
> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> > >> function='0x0'/>
> > >> </interface>
> > >> <interface type='hostdev' managed='yes'>
> > >> <mac address='52:54:00:30:37:98'/>
> > >> <source>
> > >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
> > >> function='0x1'/>
> > >> </source>
> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
> > >> function='0x0'/>
> > >> </interface>
> > >>
> > >> [root@cent64x64-137189 ~]# lspci |grep Eth
> > >> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+
> > >> Network Connection (rev 01)
> > >> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
> SFI/SFP+
> > >> Network Connection (rev 01)
> > >> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller
> > >> Virtual Function (rev 01)
> > >> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller
> > >> Virtual Function (rev 01)
> > >>
> > >>
> > >> On guest VM
> > >> 1. check all the ports are started up and check their corresponding
> port
> > >> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd
> > >> [root@VirtualADX ~]# lspci |grep Eth
> > >> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit
> Ethernet
> > >> Controller (rev 03)
> > >> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit
> Ethernet
> > >> Controller (rev 03)
> > >> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller
> > >> Virtual Function (rev 01)
> > >> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet
> Controller
> > >> Virtual Function (rev 01)
> > >> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit
> Ethernet
> > >> Controller (rev 03)
> > >>
> > >> 2. run this command:
> > >> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b
> > >> 000:00:0a.0 -- -q 1 -p 3
> > >>
> > >> Although the l2fwd forwards all received tagged packets, the
> Spirent
> > >> tester did not receive any packet at all.
> > >>
> > >
> > >
>
> The VLAN offload bit in the mbuf is different for TX and RX.
> Probably would work with something like this:
>
> diff --git a/examples/l2fwd/main.c b/examples/l2fwd/main.c
> index 05ef330..70dbf85 100644
> --- a/examples/l2fwd/main.c
> +++ b/examples/l2fwd/main.c
> @@ -289,6 +289,9 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned
> portid)
> /* src addr */
> ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr);
>
> + if (m->ol_flags & PKT_RX_VLAN_PKT)
> + m->ol_flags |= PKT_TX_VLAN_PKT;
> +
> l2fwd_send_packet(m, (uint8_t) dst_port);
> }
>
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Passing VLAN traffic via l2fwd
2014-01-24 6:07 ` James Yu
@ 2014-01-28 3:13 ` James Yu
0 siblings, 0 replies; 6+ messages in thread
From: James Yu @ 2014-01-28 3:13 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
Some more discovery.
That flag PKT_TX_VLAN_PKT eventually send the following command to the
chips (ixgbe_xmit_pkts(), tx_desc_vlan_flags_to_cmdtype()) . Is this
actually adding a VLAN tag during tx ?
#define IXGBE_ADVTXD_DCMD_VLE IXGBE_TXD_CMD_VLE /* VLAN pkt
enable */
#define IXGBE_TXD_CMD_VLE 0x40000000 /* Add VLAN tag */
After I intentionally remove the tag before sending out the traffic, the
Spirent can receive tagged packets. I am not sure the DPDK codes is adding
a tag. I think the ixgbevf on the host is adding VLAN tag based on the
configured tag ID.
Is this expected ? How can I configure the host to not add tag since
traffic from the VM is already tagged. Otherwise it becomes double tagged.
Thanks
James
On Fri, Jan 24, 2014 at 6:07 AM, James Yu <ypyu2011@gmail.com> wrote:
> Stephen,
>
> I added that to the l2fwd main.c and it is still not received by the
> Spirent. I obtained the stat using rte_eth_stats_get() and printout the
> following when I send only one VLAN pkt
> ipackets: increased by 1
> opackets: no change (pkt not sent out by the device)
> ierrors, oerrors, otxerror, otxdiscard: all 0
>
> The same thing with 10 packets which increase 10 ipackets counts.
>
> I found out that the rte_eth_stats_get() obtains the stats from
> ixgbevf_dev_stats_get(). Is this expected for SR-IOV that uses
> ixgbe_ethdev.c ?
> If that's correct, I think this tells me that the tagged packet is
> received by the VF PMD codes but not sent out and has no error or packets
> dropped.
>
> So it was not sent out of the VF PMD. But why there is not error or packet
> dropped. Where did it go ? I am trying to find out where it got dropped and
> how to fix it.
>
> I tried DPDK1.5.2r1. It exhibits the same issue even with the patch.
>
> Thanks
>
> James
>
>
>
> On Thu, Jan 23, 2014 at 1:43 PM, Stephen Hemminger <
> stephen@networkplumber.org> wrote:
>
>> On Thu, 23 Jan 2014 11:27:14 -0800
>> James Yu <ypyu2011@gmail.com> wrote:
>>
>> > Hi Daniel,
>> >
>> > Untagged traffic could be looped back. I am doing a one-directional loop
>> > back. In the Frame tabe of the StreamBlock editor, I did the following
>> > stream1:
>> > destination address: MAC address of the receiving port of the guest
>> > VM. (obtained from the debug print out when l2fwd is started, or
>> ifconfig
>> > before l2fwd is started)
>> > stream2:
>> > destination address: MAC address of the sending port of the guest
>> VM
>> > (obtained from the debug print out when l2fwd is started, or ifconfig
>> > before l2fwd is started)
>> >
>> >
>> > The only thing I add is to tag both the SR-IOV VF ports on the host with
>> > the same tag using ip command and add that tag number as the VID to the
>> > Spirent streams. After that that traffic can reach l2fwd which reported
>> > forwarded the packets on the periodic printout. However the packets did
>> not
>> > reach the Spirent.
>> >
>> > Also it is exercising rte_ixgbevf_pmd codes as indicated during starting
>> > l2fwd.
>> >
>> > EAL: Master core 0 is ready (tid=b77ab7d0)
>> >
>> > EAL: Core 1 is ready (tid=adffeb70)
>> >
>> > EAL: probe driver: 8086:100e rte_em_pmd
>> >
>> > EAL: probe driver: 8086:100e rte_em_pmd
>> >
>> > *EAL: probe driver: 8086:10ed rte_ixgbevf_pmd*
>> > I wonder do I have to change any codes in the DPDK library or call
>> certain
>> > routines to setup the VLAN in the DPDK stack during starting the l2fwd ?
>> >
>> >
>> > Were you able to send VLAN traffic from your iXia and looped back to
>> your
>> > iXia ?
>> >
>> >
>> > Thanks
>> >
>> > James
>> >
>> >
>> >
>> >
>> > On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky <
>> > daniel.kaminsky@infinitelocality.com> wrote:
>> >
>> > > Hi James,
>> > >
>> > > Did you try a non-tagged traffic? I suspect this is the limitation of
>> the
>> > > l2fwd or the requirements of Spirent. From my experience with Ixia,
>> the L2
>> > > forward device must implement the ARP protocol (at least a subset of
>> it).
>> > >
>> > > Also, notice that the l2fwd example sets the destination MAC address
>> to
>> > > "02:00:00:00:00:xx" which the Spirent might doesn't like.
>> > >
>> > > Daniel
>> > >
>> > >
>> > > On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011@gmail.com> wrote:
>> > >
>> > >> I could not pass tagged traffic through the l2fwd program running
>> inside a
>> > >> CentOS VM. The l2fwd program reported sending out all the packets
>> received
>> > >> from one port to the other port. But the outside Spirent tester
>> could not
>> > >> receive a packet at all. I am wondering maybe the tagged packets are
>> > >> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone
>> knows
>> > >> how to pass/send VLAN traffic through the DPDK ?
>> > >>
>> > >> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each
>> > >> other
>> > >> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues
>> with
>> > >> sending out tagged packets out of DPDK to the host.
>> > >>
>> > >> I have done the following:
>> > >>
>> > >> On RHEL6.1 host
>> > >> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1
>> and "ip
>> > >> link set eth3 vf 0 vlan 3" for port 2
>> > >> [root@cent64x64-137189 ~]# ip link show eth2
>> > >> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
>> UP
>> > >> qlen
>> > >> 1000
>> > >> link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
>> > >> vf 0 MAC 52:54:00:f9:9a:90, vlan 3
>> > >> [root@cent64x64-137189 ~]# ip link show eth3
>> > >> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
>> UP
>> > >> qlen
>> > >> 1000
>> > >> link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
>> > >> vf 0 MAC 52:54:00:30:37:98, vlan 3
>> > >>
>> > >> 2. virsh configure SR-IOV hostdev
>> > >> <interface type='hostdev' managed='yes'>
>> > >> <mac address='52:54:00:f9:9a:90'/>
>> > >> <source>
>> > >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
>> > >> function='0x0'/>
>> > >> </source>
>> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>> > >> function='0x0'/>
>> > >> </interface>
>> > >> <interface type='hostdev' managed='yes'>
>> > >> <mac address='52:54:00:30:37:98'/>
>> > >> <source>
>> > >> <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
>> > >> function='0x1'/>
>> > >> </source>
>> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
>> > >> function='0x0'/>
>> > >> </interface>
>> > >>
>> > >> [root@cent64x64-137189 ~]# lspci |grep Eth
>> > >> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+
>> > >> Network Connection (rev 01)
>> > >> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+
>> > >> Network Connection (rev 01)
>> > >> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >>
>> > >>
>> > >> On guest VM
>> > >> 1. check all the ports are started up and check their corresponding
>> port
>> > >> type. Use the two SR-IOV ports with device type as ixgbevf for l2fwd
>> > >> [root@VirtualADX ~]# lspci |grep Eth
>> > >> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>> Ethernet
>> > >> Controller (rev 03)
>> > >> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>> Ethernet
>> > >> Controller (rev 03)
>> > >> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>> Ethernet
>> > >> Controller (rev 03)
>> > >>
>> > >> 2. run this command:
>> > >> /root/l2fwd -c 3 -n 1 -b 000:00:03.0 -b 000:00:07.0 -b
>> > >> 000:00:0a.0 -- -q 1 -p 3
>> > >>
>> > >> Although the l2fwd forwards all received tagged packets, the
>> Spirent
>> > >> tester did not receive any packet at all.
>> > >>
>> > >
>> > >
>>
>> The VLAN offload bit in the mbuf is different for TX and RX.
>> Probably would work with something like this:
>>
>> diff --git a/examples/l2fwd/main.c b/examples/l2fwd/main.c
>> index 05ef330..70dbf85 100644
>> --- a/examples/l2fwd/main.c
>> +++ b/examples/l2fwd/main.c
>> @@ -289,6 +289,9 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned
>> portid)
>> /* src addr */
>> ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr);
>>
>> + if (m->ol_flags & PKT_RX_VLAN_PKT)
>> + m->ol_flags |= PKT_TX_VLAN_PKT;
>> +
>> l2fwd_send_packet(m, (uint8_t) dst_port);
>> }
>>
>>
>>
>>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-01-28 3:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-01-23 5:08 [dpdk-dev] Passing VLAN traffic via l2fwd James Yu
2014-01-23 8:45 ` Daniel Kaminsky
2014-01-23 19:27 ` James Yu
2014-01-23 21:43 ` Stephen Hemminger
2014-01-24 6:07 ` James Yu
2014-01-28 3:13 ` James Yu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).