* Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? [not found] ` <6a90755824944d70a5c239925eb25c73@XCH-RCD-013.cisco.com> @ 2016-05-25 19:23 ` Ananyev, Konstantin 2016-05-26 18:11 ` John Lo (loj) 0 siblings, 1 reply; 6+ messages in thread From: Ananyev, Konstantin @ 2016-05-25 19:23 UTC (permalink / raw) To: John Lo (loj), Wiles, Keith, Chandrasekar Kannan Cc: vpp-dev, Zhang, Helin, dev > I suppose this has to do with what is expected usage of the PKT_RX_VLAN_PKT offload flag. Is it set only for VLAN packets with the > VLAN stripped or should always be set if VLAN is/was present in the received packet. It seems that different DPDK drivers are > behaving differently which will make it really hard for VPP to take advantage of NIC and driver offload capability to provide better > performance. Yes, that's true ixgbe/igb from one side and i40e do raise PKT_RX_VLAN_PKT in a different manner. There is an attempt to make it unified across all supported devices: http://dpdk.org/dev/patchwork/patch/12938/ Though, I am not sure it will help you with your issue. As I understand, for you the desired behaviour is: If it is a vlan packet, keep the packet intact (don't strip the vlan) and raise PKT_RX_VLAN_PK inside mbuf->ol_flags. That's what ixgbe is doing with rte_eth_conf.rxmode.hw_vlan_strip==0. Correct? As far as I know, i40e HW doesn't provide such ability. i40e Receive HW Descriptor can only flag was VLAN tag stripped from the packet or not, but if stripping is disabled it wouldn't indicate in any way is VLAN tag is present inside the packet or not. I am CC-ing it to dpdk.org in case I am missing something here. Probably someone knows a way to make it work in that way. Konstantin > > If VPP cannot rely on offload flags for VLAN so packets always have to go through ethernet-input node, there is a performance cost. > For the 10GE case, before the inverse patch of the ixgbe driver, 10GE Rx-vector path removed support of offload flag with DPDK 16.04 > and so ethernet-input node is always used. The 10GE IPv4 throughput rate dropped from 6.17MPPSx2 to 4.92MPPSx2 for bidirectional > traffic (2 streams each with a single IP address as destination) on a single core on my server. Konstantin suggested at that time to use > scalar mode which does support offload flags properly. The scalar mode did by-pass ethernet-input and provided throughput of > 5.72MPPS. We ended up inverse patched the ixgbe driver to restore vector mode offload flag support as the original restriction (the > reason offload flag support was removed) would not affect VPP. > > I think for 40GE driver to provide offload flag support in vector mode but not give indication of presence of VLAN tag is just wrong. This > make the offload flag support useless for VPP. > > Regards, > John > > -----Original Message----- > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > Sent: Wednesday, May 25, 2016 11:30 AM > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > Cc: vpp-dev; Zhang, Helin > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > I see what you are getting at, Konstantin. The VPP init code does not > > enable VLAN strip for Intel NICs as VLAN tag must be in the packet for > > sub-interface lookup by ethernet-input node. I agree if we enable VLAN tag strip for the i40e driver, we can get around this problem > but then all packets will be considered as received on the main interface. > > I see... > As far as I know, when VLAN stripping is disabled, i40e RXD doesn't contain information does that packet contain a VLAN or not. > So, if enabling vlan stripping is not an option for you guys, then I don't see any other way on i40e to recognise is that VLAN packet or > not, except then parse the packet in SW. > Helin, please correct me here, if I am missing something here. > Thanks > Konstantin > > > > > Regards, > > John > > > > -----Original Message----- > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > Sent: Wednesday, May 25, 2016 10:35 AM > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > Cc: vpp-dev > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > Since this is the XL710 40GE NIC, I suppose the DPDK driver involved would be the i40e driver and not ixgbe for 10GE NICs. > > > > Yes, I understand that you are facing problem on i40e, not ixgbe. > > And the problem is that for i40e PKT_RX_VLAN_PKT flag is not set in mbuf->ol_flags, correct? > > That's why I asked: are you running it with rte_eth_conf.rxmode.hw_vlan_strip==0 or not? > > If yes, you can try with rte_eth_conf.rxmode.hw_vlan_strip=1 and see would it help you. > > > > > > > > As explained before, ixgbe driver had the inverse patch added. It > > > does recognize VLAN tag in the packet and set PKT_RX_VLAN_PKT offload flag properly: > > > > That patch has nothing to do with PKT_RX_VLAN_PKT and i40e. > > So I don't think it is related to that problem at all. > > Konstantin > > > > > > > > 00:01:02:132370: dpdk-input > > > TenGigabitEthernet5/0/0 rx queue 0 > > > buffer 0x44cff: current data 0, length 96, free-list 0, totlen-nifb 0, trace 0x0 > > > PKT MBUF: port 3, nb_segs 1, pkt_len 96 > > > buf_len 2176, data_len 96, ol_flags 0x1, > > > packet_type 0x210 > > > Packet Offload Flags > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > Packet Types > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > UDP: 28.0.0.1 -> 29.0.0.254 > > > tos 0x00, ttl 64, length 78, checksum 0x40a1 > > > fragment id 0x0000 > > > UDP: 63 -> 63 > > > length 58, checksum 0x5b76 > > > 00:01:02:132391: ethernet-input > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > 00:01:02:132408: error-drop > > > ethernet-input: unknown vlan > > > > > > You can see from above the packet was passed to ethernet-input node > > > and dropped because sub-interface for VLAN 1101 is not set up. > > > > > > Regards, > > > John > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > Sent: Wednesday, May 25, 2016 4:46 AM > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > Hi, > > > I don’t think ptype recognition on ixgbe vRX is somehow related to that. > > > Are you guys running FVL with rte_eth_conf.rxmode.hw_vlan_strip==0? > > > If so, can you set it to 1 and see would it help? > > > Konstantin > > > > > > From: vpp-dev-bounces@lists.fd.io > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > On Behalf Of John Lo (loj) > > > Sent: Wednesday, May 25, 2016 1:56 AM > > > To: Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > I did put in a reverse patch, provided by Damjan, for this in fd.io: > > > https://gerrit.fd.io/r/#/c/941/1/dpdk/dpdk-16.04_patches/0013-Revert > > > -i xgbe-fix-packet-type-from-vector-Rx.patch > > > > > > I know this fixed the 10GE NIC behavior. Perhaps the 40GE NIC is using a different DPDK driver? > > > > > > Regards, > > > John > > > > > > From: Wiles, Keith [mailto:keith.wiles@intel.com] > > > Sent: Tuesday, May 24, 2016 8:30 PM > > > To: John Lo (loj); Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > The commit ID is > > > > > > commit d9a2009a81089093645fea2e04b51dd37edf3e6f > > > Author: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > Date: Tue Mar 22 14:30:17 2016 +0000 > > > > > > ixgbe: fix packet type from vector Rx > > > > > > Current vector RX can't always set the packet_type properly. > > > To be more specific: > > > a) it never sets RTE_PTYPE_L2_ETHER > > > b) it doesn't handle tunnel ipv4/ipv6 case correctly. > > > c) it doesn't check is IXGBE_RXDADV_PKTTYPE_ETQF set or not. > > > While a) is pretty easy to fix, b) and c) are not that > > > straightforward > > > in terms of SIMD ops (specially b). > > > So far I wasn't able to make vRX support packet_type properly > > > without > > > noticeable performance loss. > > > So for now, just remove that functionality from vector RX and > > > update dev_supported_ptypes_get(). > > > > > > Fixes: 396254175854 ("mbuf: redefine packet type") > > > > > > Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > Acked-by: Cunming Liang <cunming.liang@intel.com> > > > > > > It appears the change was only down to the vector version in the RX routine, could move to the none-vector version I guess. > > > > > > I would think adding a patch or adding the code to VPP DPDK init to use the RX callback to setup things correctly. > > > > > > Regards, > > > Keith > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > <keith.wiles@intel.com> > > > Date: Tuesday, May 24, 2016 at 7:19 PM > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > <ckannan@console.to> > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > Here is the starting thread: > > > http://dpdk.org/ml/archives/dev/2016-April/037837.html > > > > > > > > > Regards, > > > Keith > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > <keith.wiles@intel.com> > > > Date: Tuesday, May 24, 2016 at 7:16 PM > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > <ckannan@console.to> > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > The flags in the rte_mbuf were changed a before 16.04, which removed > > > the ptype testing and possible setting this flag. I thought a patch to revert that change was done, did that not happen. > > > > > > The other solution is to add an option to DPDK to restore the > > > previous behavior for this type of problem. The only other way is to > > > have a installed in the PMD to setup the flags correctly for VPP. > > > This would require adding a routine to VPP and setting the call back > > > in the > > driver. > > > > > > I really thought a patch was created, does anyone know? > > > > > > Regards, > > > Keith > > > > > > > > > From: "John Lo (loj)" <loj@cisco.com> > > > Date: Tuesday, May 24, 2016 at 6:53 PM > > > To: Chandrasekar Kannan <ckannan@console.to> > > > Cc: Keith Wiles <keith.wiles@intel.com>, vpp-dev > > > <vpp-dev@lists.fd.io> > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > I believe the problem is due to 40GE DPDK driver not parsing VLAN > > > packet properly and setting the packet offload flag PKT_RX_VLAN_PKT, > > > causing dpdk-input node to send packet directly to ip4-input instead > > > of ethernet-input. If this flag is set, then the packet will be > > > passed to ethernet-input which will then parse the VLAN tag in the > > packet to perform sub-interface lookup and find the start of IP header > > in the packet properly. Following is an example trace of a correctly marked packet: > > > > > > 23:11:09:855916: dpdk-input > > > TenGigE0/0/0/0 rx queue 0 > > > buffer 0x23f6c0: current data 0, length 118, free-list 0, > > > totlen-nifb 0, trace 0x0 > > > PKT MBUF: port 0, nb_segs 1, pkt_len 118 > > > buf_len 2176, data_len 118, ol_flags 0x1, > > > packet_type 0x10 > > > Packet Offload Flags > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > Packet Types > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > > headers > > > IP4: 74:a0:2f:e2:d5:14 -> 90:e2:ba:9f:8b:cc vlan 101 > > > ICMP: 130.1.1.1 -> 130.1.1.2 > > > tos 0x00, ttl 255, length 100, checksum 0xac4b > > > fragment id 0x0948 > > > ICMP echo_request checksum 0x900d > > > > > > Hoping someone familiar with the 40GE DPDK driver can comment further. > > > > > > Regards, > > > John > > > > > > From: Chandrasekar Kannan [mailto:ckannan@console.to] > > > Sent: Tuesday, May 24, 2016 6:57 PM > > > To: John Lo (loj) > > > Cc: Wiles, Keith; vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > Yes. Subinterface has been configured like this ... > > > > > > vppctl set interface ip address del FortyGigabitEthernet83/0/0 all > > > vppctl create sub FortyGigabitEthernet83/0/0 900 vppctl set > > > interface ip address FortyGigabitEthernet83/0/0.900 6.0.0.1/24 > > > vppctl set interface ip address FortyGigabitEthernet83/0/0.900 > > > 7.0.0.1/24 vppctl set interface state FortyGigabitEthernet83/0/0 up > > > vppctl set interface state FortyGigabitEthernet83/0/0.900 up vppctl > > > set ip arp static > > > FortyGigabitEthernet83/0/0.900 7.0.0.2 3c:fd:fe:9c:c3:88 > > > > > > -------------------------------------------------- > > > Packet 4 > > > > > > 00:02:23:366879: dpdk-input > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > totlen-nifb 0, trace 0x3 > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > packet_type 0x291 > > > Packet Types > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > without extension headers > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > UDP: 6.0.0.2 -> 7.0.0.2 > > > tos 0x00, ttl 64, length 46, checksum 0x2dbc > > > fragment id 0x0000, flags DONT_FRAGMENT > > > UDP: 1024 -> 2001 > > > length 26, checksum 0x0000 > > > 00:02:23:366894: ip4-input-no-checksum > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.45.188 -> 6.0.0.2 > > > version 0, header length 12 > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be 0xaf4d) > > > fragment id 0x4500 offset 368, flags > > > 00:02:23:366910: error-drop > > > ip4-input: ip4 length > l2 length > > > > > > -------------------------------------------------------------- > > > > > > > > > > > > On Tue, May 24, 2016 at 3:12 PM, John Lo (loj) <loj@cisco.com> wrote: > > > Was a sub-interface created with the expected VLAN tag 900? An > > > example of creating a sub-interface and bring it up would be something like: > > > > > > create sub GigabitEthernet1/0/0 900 > > > set int state GigabitEthernet1/0/0 up set int state > > > GigabitEthernet1/0/0.900 up > > > > > > Regards, > > > John > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > On Behalf Of Wiles, Keith > > > Sent: Tuesday, May 24, 2016 5:59 PM > > > To: Chandrasekar Kannan; vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > I am starting to think the unknown VLAN is not accounted for in the processing of the input frame in the VPP DPDK interface code. > > > vnet/vnet/devices/dpdk/node.c > > > > > > The version and header length are wrong in the output below, which > > > leads me to believe the starting IPv4 header location is wrong. I > > > have not parsed all of the code in node.c:dpdk_device_input() yet, > > > but I do not think the l3_offset is being adjusted to the correct > > offset???? > > > > > > Regards, > > > Keith > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Chandrasekar Kannan > > > <ckannan@console.to> > > > Date: Tuesday, May 24, 2016 at 3:40 PM > > > To: vpp-dev <vpp-dev@lists.fd.io> > > > Subject: [vpp-dev] VLAN packets dropped... ? > > > > > > Hi, > > > > > > I was trying to setup VLAN subinterfaces with VPP but it appears the > > > packets are dropped at ingress. I'm tried with both pktgen-dpdk and > > > trex for traffic generation just to rule out any corruption with > > pkt generation. But both are producing the same errors with vpp. > > > > > > yaml file from trex... > > > > > > ------------------------------ > > > [root@node04 scripts]# cat cap2/ipv4_vlan.yaml > > > - duration : 10 > > > generator : > > > distribution : "seq" > > > clients_start : "16.0.0.1" > > > clients_end : "16.0.1.255" > > > servers_start : "48.0.0.1" > > > servers_end : "48.0.0.255" > > > clients_per_gb : 201 > > > min_clients : 101 > > > dual_port_mask : "1.0.0.0" > > > tcp_aging : 1 > > > udp_aging : 1 > > > mac : [0x0,0x0,0x0,0x1,0x0,0x00] > > > vlan : { enable : 1 , vlan0 : 900 , vlan1 : 1100 } > > > cap_info : > > > - name: cap2/udp_64B.pcap > > > cps : 1000.0 > > > ipg : 10000 > > > rtt : 10000 > > > w : 4 > > > limit : 20 > > > [root@node04 scripts]# > > > --------------------------------------------------------------- > > > > > > vpp trace... > > > > > > 00:02:21:576810: dpdk-input > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > totlen-nifb 0, trace 0x1 > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > packet_type 0x291 > > > Packet Types > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > without extension headers > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > UDP: 16.0.1.0 -> 48.0.0.128 > > > tos 0x00, ttl 64, length 46, checksum 0xf93f > > > fragment id 0x0000, flags DONT_FRAGMENT > > > UDP: 1024 -> 2001 > > > length 26, checksum 0x0000 > > > 00:02:21:576817: ip4-input-no-checksum > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.249.63 -> 16.0.1.0 > > > version 0, header length 12 > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be 0xaf4d) > > > fragment id 0x4500 offset 368, flags > > > 00:02:21:576828: error-drop > > > ip4-input: ip4 length > l2 length > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? 2016-05-25 19:23 ` [dpdk-dev] [vpp-dev] VLAN packets dropped... ? Ananyev, Konstantin @ 2016-05-26 18:11 ` John Lo (loj) 2016-05-26 22:56 ` John Daley (johndale) 0 siblings, 1 reply; 6+ messages in thread From: John Lo (loj) @ 2016-05-26 18:11 UTC (permalink / raw) To: Ananyev, Konstantin, Wiles, Keith, Chandrasekar Kannan Cc: vpp-dev, Zhang, Helin, dev Hi Konstantin, Thanks for the link to DPDK discussion wrt this VLAN offload flag PKT_RX_VLAN_PKT. It seems the proposal was to deprecate PKT_RX_VLAN_PKT and introduce two new flags PKT_RX_VLAN_STRIPPED and PKT_RX_QINQ_STRIPPED. It would be a really good idea to keep PKT_RX_VLAN_PKT to indicate at least one VLAN tag is present on the packet. For the case of i40e where its HW cannot detect VLAN tag if strip is not enabled, it should be reasonable for the i40e DPDK driver software to make a check and set this flag. I would think the overhead to check the 1st ethertype field in the MAC header against dot1q or dot1ad values in software be pretty minimal. Regards, John -----Original Message----- From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] Sent: Wednesday, May 25, 2016 3:23 PM To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan Cc: vpp-dev; Zhang, Helin; dev@dpdk.org Subject: RE: [vpp-dev] VLAN packets dropped... ? > I suppose this has to do with what is expected usage of the > PKT_RX_VLAN_PKT offload flag. Is it set only for VLAN packets with the > VLAN stripped or should always be set if VLAN is/was present in the > received packet. It seems that different DPDK drivers are behaving differently which will make it really hard for VPP to take advantage of NIC and driver offload capability to provide better performance. Yes, that's true ixgbe/igb from one side and i40e do raise PKT_RX_VLAN_PKT in a different manner. There is an attempt to make it unified across all supported devices: http://dpdk.org/dev/patchwork/patch/12938/ Though, I am not sure it will help you with your issue. As I understand, for you the desired behaviour is: If it is a vlan packet, keep the packet intact (don't strip the vlan) and raise PKT_RX_VLAN_PK inside mbuf->ol_flags. That's what ixgbe is doing with rte_eth_conf.rxmode.hw_vlan_strip==0. Correct? As far as I know, i40e HW doesn't provide such ability. i40e Receive HW Descriptor can only flag was VLAN tag stripped from the packet or not, but if stripping is disabled it wouldn't indicate in any way is VLAN tag is present inside the packet or not. I am CC-ing it to dpdk.org in case I am missing something here. Probably someone knows a way to make it work in that way. Konstantin > > If VPP cannot rely on offload flags for VLAN so packets always have to go through ethernet-input node, there is a performance cost. > For the 10GE case, before the inverse patch of the ixgbe driver, 10GE > Rx-vector path removed support of offload flag with DPDK 16.04 and so > ethernet-input node is always used. The 10GE IPv4 throughput rate > dropped from 6.17MPPSx2 to 4.92MPPSx2 for bidirectional traffic (2 > streams each with a single IP address as destination) on a single core > on my server. Konstantin suggested at that time to use scalar mode which does support offload flags properly. The scalar mode did by-pass ethernet-input and provided throughput of 5.72MPPS. We ended up inverse patched the ixgbe driver to restore vector mode offload flag support as the original restriction (the reason offload flag support was removed) would not affect VPP. > > I think for 40GE driver to provide offload flag support in vector mode > but not give indication of presence of VLAN tag is just wrong. This make the offload flag support useless for VPP. > > Regards, > John > > -----Original Message----- > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > Sent: Wednesday, May 25, 2016 11:30 AM > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > Cc: vpp-dev; Zhang, Helin > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > I see what you are getting at, Konstantin. The VPP init code does > > not enable VLAN strip for Intel NICs as VLAN tag must be in the > > packet for sub-interface lookup by ethernet-input node. I agree if > > we enable VLAN tag strip for the i40e driver, we can get around this > > problem > but then all packets will be considered as received on the main interface. > > I see... > As far as I know, when VLAN stripping is disabled, i40e RXD doesn't contain information does that packet contain a VLAN or not. > So, if enabling vlan stripping is not an option for you guys, then I > don't see any other way on i40e to recognise is that VLAN packet or not, except then parse the packet in SW. > Helin, please correct me here, if I am missing something here. > Thanks > Konstantin > > > > > Regards, > > John > > > > -----Original Message----- > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > Sent: Wednesday, May 25, 2016 10:35 AM > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > Cc: vpp-dev > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > Since this is the XL710 40GE NIC, I suppose the DPDK driver involved would be the i40e driver and not ixgbe for 10GE NICs. > > > > Yes, I understand that you are facing problem on i40e, not ixgbe. > > And the problem is that for i40e PKT_RX_VLAN_PKT flag is not set in mbuf->ol_flags, correct? > > That's why I asked: are you running it with rte_eth_conf.rxmode.hw_vlan_strip==0 or not? > > If yes, you can try with rte_eth_conf.rxmode.hw_vlan_strip=1 and see would it help you. > > > > > > > > As explained before, ixgbe driver had the inverse patch added. It > > > does recognize VLAN tag in the packet and set PKT_RX_VLAN_PKT offload flag properly: > > > > That patch has nothing to do with PKT_RX_VLAN_PKT and i40e. > > So I don't think it is related to that problem at all. > > Konstantin > > > > > > > > 00:01:02:132370: dpdk-input > > > TenGigabitEthernet5/0/0 rx queue 0 > > > buffer 0x44cff: current data 0, length 96, free-list 0, totlen-nifb 0, trace 0x0 > > > PKT MBUF: port 3, nb_segs 1, pkt_len 96 > > > buf_len 2176, data_len 96, ol_flags 0x1, > > > packet_type 0x210 > > > Packet Offload Flags > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > Packet Types > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > UDP: 28.0.0.1 -> 29.0.0.254 > > > tos 0x00, ttl 64, length 78, checksum 0x40a1 > > > fragment id 0x0000 > > > UDP: 63 -> 63 > > > length 58, checksum 0x5b76 > > > 00:01:02:132391: ethernet-input > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > 00:01:02:132408: error-drop > > > ethernet-input: unknown vlan > > > > > > You can see from above the packet was passed to ethernet-input > > > node and dropped because sub-interface for VLAN 1101 is not set up. > > > > > > Regards, > > > John > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > Sent: Wednesday, May 25, 2016 4:46 AM > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > Hi, > > > I don’t think ptype recognition on ixgbe vRX is somehow related to that. > > > Are you guys running FVL with rte_eth_conf.rxmode.hw_vlan_strip==0? > > > If so, can you set it to 1 and see would it help? > > > Konstantin > > > > > > From: vpp-dev-bounces@lists.fd.io > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > On Behalf Of John Lo (loj) > > > Sent: Wednesday, May 25, 2016 1:56 AM > > > To: Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > I did put in a reverse patch, provided by Damjan, for this in fd.io: > > > https://gerrit.fd.io/r/#/c/941/1/dpdk/dpdk-16.04_patches/0013-Reve > > > rt -i xgbe-fix-packet-type-from-vector-Rx.patch > > > > > > I know this fixed the 10GE NIC behavior. Perhaps the 40GE NIC is using a different DPDK driver? > > > > > > Regards, > > > John > > > > > > From: Wiles, Keith [mailto:keith.wiles@intel.com] > > > Sent: Tuesday, May 24, 2016 8:30 PM > > > To: John Lo (loj); Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > The commit ID is > > > > > > commit d9a2009a81089093645fea2e04b51dd37edf3e6f > > > Author: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > Date: Tue Mar 22 14:30:17 2016 +0000 > > > > > > ixgbe: fix packet type from vector Rx > > > > > > Current vector RX can't always set the packet_type properly. > > > To be more specific: > > > a) it never sets RTE_PTYPE_L2_ETHER > > > b) it doesn't handle tunnel ipv4/ipv6 case correctly. > > > c) it doesn't check is IXGBE_RXDADV_PKTTYPE_ETQF set or not. > > > While a) is pretty easy to fix, b) and c) are not that > > > straightforward > > > in terms of SIMD ops (specially b). > > > So far I wasn't able to make vRX support packet_type properly > > > without > > > noticeable performance loss. > > > So for now, just remove that functionality from vector RX and > > > update dev_supported_ptypes_get(). > > > > > > Fixes: 396254175854 ("mbuf: redefine packet type") > > > > > > Signed-off-by: Konstantin Ananyev > > > <konstantin.ananyev@intel.com> > > > Acked-by: Cunming Liang <cunming.liang@intel.com> > > > > > > It appears the change was only down to the vector version in the RX routine, could move to the none-vector version I guess. > > > > > > I would think adding a patch or adding the code to VPP DPDK init to use the RX callback to setup things correctly. > > > > > > Regards, > > > Keith > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > <keith.wiles@intel.com> > > > Date: Tuesday, May 24, 2016 at 7:19 PM > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > <ckannan@console.to> > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > Here is the starting thread: > > > http://dpdk.org/ml/archives/dev/2016-April/037837.html > > > > > > > > > Regards, > > > Keith > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > <keith.wiles@intel.com> > > > Date: Tuesday, May 24, 2016 at 7:16 PM > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > <ckannan@console.to> > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > The flags in the rte_mbuf were changed a before 16.04, which > > > removed the ptype testing and possible setting this flag. I thought a patch to revert that change was done, did that not happen. > > > > > > The other solution is to add an option to DPDK to restore the > > > previous behavior for this type of problem. The only other way is > > > to have a installed in the PMD to setup the flags correctly for VPP. > > > This would require adding a routine to VPP and setting the call > > > back in the > > driver. > > > > > > I really thought a patch was created, does anyone know? > > > > > > Regards, > > > Keith > > > > > > > > > From: "John Lo (loj)" <loj@cisco.com> > > > Date: Tuesday, May 24, 2016 at 6:53 PM > > > To: Chandrasekar Kannan <ckannan@console.to> > > > Cc: Keith Wiles <keith.wiles@intel.com>, vpp-dev > > > <vpp-dev@lists.fd.io> > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > I believe the problem is due to 40GE DPDK driver not parsing VLAN > > > packet properly and setting the packet offload flag > > > PKT_RX_VLAN_PKT, causing dpdk-input node to send packet directly > > > to ip4-input instead of ethernet-input. If this flag is set, then > > > the packet will be passed to ethernet-input which will then parse > > > the VLAN tag in the > > packet to perform sub-interface lookup and find the start of IP > > header in the packet properly. Following is an example trace of a correctly marked packet: > > > > > > 23:11:09:855916: dpdk-input > > > TenGigE0/0/0/0 rx queue 0 > > > buffer 0x23f6c0: current data 0, length 118, free-list 0, > > > totlen-nifb 0, trace 0x0 > > > PKT MBUF: port 0, nb_segs 1, pkt_len 118 > > > buf_len 2176, data_len 118, ol_flags 0x1, > > > packet_type 0x10 > > > Packet Offload Flags > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > Packet Types > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > > headers > > > IP4: 74:a0:2f:e2:d5:14 -> 90:e2:ba:9f:8b:cc vlan 101 > > > ICMP: 130.1.1.1 -> 130.1.1.2 > > > tos 0x00, ttl 255, length 100, checksum 0xac4b > > > fragment id 0x0948 > > > ICMP echo_request checksum 0x900d > > > > > > Hoping someone familiar with the 40GE DPDK driver can comment further. > > > > > > Regards, > > > John > > > > > > From: Chandrasekar Kannan [mailto:ckannan@console.to] > > > Sent: Tuesday, May 24, 2016 6:57 PM > > > To: John Lo (loj) > > > Cc: Wiles, Keith; vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > Yes. Subinterface has been configured like this ... > > > > > > vppctl set interface ip address del FortyGigabitEthernet83/0/0 all > > > vppctl create sub FortyGigabitEthernet83/0/0 900 vppctl set > > > interface ip address FortyGigabitEthernet83/0/0.900 6.0.0.1/24 > > > vppctl set interface ip address FortyGigabitEthernet83/0/0.900 > > > 7.0.0.1/24 vppctl set interface state FortyGigabitEthernet83/0/0 > > > up vppctl set interface state FortyGigabitEthernet83/0/0.900 up > > > vppctl set ip arp static > > > FortyGigabitEthernet83/0/0.900 7.0.0.2 3c:fd:fe:9c:c3:88 > > > > > > -------------------------------------------------- > > > Packet 4 > > > > > > 00:02:23:366879: dpdk-input > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > totlen-nifb 0, trace 0x3 > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > packet_type 0x291 > > > Packet Types > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > without extension headers > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > UDP: 6.0.0.2 -> 7.0.0.2 > > > tos 0x00, ttl 64, length 46, checksum 0x2dbc > > > fragment id 0x0000, flags DONT_FRAGMENT > > > UDP: 1024 -> 2001 > > > length 26, checksum 0x0000 > > > 00:02:23:366894: ip4-input-no-checksum > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.45.188 -> 6.0.0.2 > > > version 0, header length 12 > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > 0xaf4d) > > > fragment id 0x4500 offset 368, flags > > > 00:02:23:366910: error-drop > > > ip4-input: ip4 length > l2 length > > > > > > -------------------------------------------------------------- > > > > > > > > > > > > On Tue, May 24, 2016 at 3:12 PM, John Lo (loj) <loj@cisco.com> wrote: > > > Was a sub-interface created with the expected VLAN tag 900? An > > > example of creating a sub-interface and bring it up would be something like: > > > > > > create sub GigabitEthernet1/0/0 900 set int state > > > GigabitEthernet1/0/0 up set int state > > > GigabitEthernet1/0/0.900 up > > > > > > Regards, > > > John > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > On Behalf Of Wiles, Keith > > > Sent: Tuesday, May 24, 2016 5:59 PM > > > To: Chandrasekar Kannan; vpp-dev > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > I am starting to think the unknown VLAN is not accounted for in the processing of the input frame in the VPP DPDK interface code. > > > vnet/vnet/devices/dpdk/node.c > > > > > > The version and header length are wrong in the output below, which > > > leads me to believe the starting IPv4 header location is wrong. I > > > have not parsed all of the code in node.c:dpdk_device_input() yet, > > > but I do not think the l3_offset is being adjusted to the correct > > offset???? > > > > > > Regards, > > > Keith > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Chandrasekar > > > Kannan <ckannan@console.to> > > > Date: Tuesday, May 24, 2016 at 3:40 PM > > > To: vpp-dev <vpp-dev@lists.fd.io> > > > Subject: [vpp-dev] VLAN packets dropped... ? > > > > > > Hi, > > > > > > I was trying to setup VLAN subinterfaces with VPP but it appears > > > the packets are dropped at ingress. I'm tried with both > > > pktgen-dpdk and trex for traffic generation just to rule out any > > > corruption with > > pkt generation. But both are producing the same errors with vpp. > > > > > > yaml file from trex... > > > > > > ------------------------------ > > > [root@node04 scripts]# cat cap2/ipv4_vlan.yaml > > > - duration : 10 > > > generator : > > > distribution : "seq" > > > clients_start : "16.0.0.1" > > > clients_end : "16.0.1.255" > > > servers_start : "48.0.0.1" > > > servers_end : "48.0.0.255" > > > clients_per_gb : 201 > > > min_clients : 101 > > > dual_port_mask : "1.0.0.0" > > > tcp_aging : 1 > > > udp_aging : 1 > > > mac : [0x0,0x0,0x0,0x1,0x0,0x00] > > > vlan : { enable : 1 , vlan0 : 900 , vlan1 : 1100 } > > > cap_info : > > > - name: cap2/udp_64B.pcap > > > cps : 1000.0 > > > ipg : 10000 > > > rtt : 10000 > > > w : 4 > > > limit : 20 > > > [root@node04 scripts]# > > > --------------------------------------------------------------- > > > > > > vpp trace... > > > > > > 00:02:21:576810: dpdk-input > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > totlen-nifb 0, trace 0x1 > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > packet_type 0x291 > > > Packet Types > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > without extension headers > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > UDP: 16.0.1.0 -> 48.0.0.128 > > > tos 0x00, ttl 64, length 46, checksum 0xf93f > > > fragment id 0x0000, flags DONT_FRAGMENT > > > UDP: 1024 -> 2001 > > > length 26, checksum 0x0000 > > > 00:02:21:576817: ip4-input-no-checksum > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.249.63 -> 16.0.1.0 > > > version 0, header length 12 > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > 0xaf4d) > > > fragment id 0x4500 offset 368, flags > > > 00:02:21:576828: error-drop > > > ip4-input: ip4 length > l2 length > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? 2016-05-26 18:11 ` John Lo (loj) @ 2016-05-26 22:56 ` John Daley (johndale) 2016-06-01 18:44 ` Chandrasekar Kannan 0 siblings, 1 reply; 6+ messages in thread From: John Daley (johndale) @ 2016-05-26 22:56 UTC (permalink / raw) To: John Lo (loj), Ananyev, Konstantin, Wiles, Keith, Chandrasekar Kannan Cc: vpp-dev, Zhang, Helin, dev John, As just discussed, what you suggest was considered but there are other apps depending a different behavior of the flag, so we thought the best thing to do is deprecate it. That is part of what Olivier's patch discussed in http://www.dpdk.org/ml/archives/dev/2016-May/038786.html does. Adding a new ptype for VLAN tagged packets after the patch is accepted would allow VPP to check if the ptype is supported by the PMD and if so, use it to determine if the delivered packet actually has a VLAN tag in it. No need to know if stripping is enabled/disabled. If the ptype is not supported, the app would have check the packet in SW. > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of John Lo (loj) > Sent: Thursday, May 26, 2016 11:11 AM > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wiles, Keith > <keith.wiles@intel.com>; Chandrasekar Kannan <ckannan@console.to> > Cc: vpp-dev <vpp-dev@lists.fd.io>; Zhang, Helin <helin.zhang@intel.com>; > dev@dpdk.org > Subject: Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? > > Hi Konstantin, > > Thanks for the link to DPDK discussion wrt this VLAN offload flag > PKT_RX_VLAN_PKT. It seems the proposal was to deprecate > PKT_RX_VLAN_PKT and introduce two new flags PKT_RX_VLAN_STRIPPED > and PKT_RX_QINQ_STRIPPED. > > It would be a really good idea to keep PKT_RX_VLAN_PKT to indicate at least > one VLAN tag is present on the packet. For the case of i40e where its HW > cannot detect VLAN tag if strip is not enabled, it should be reasonable for the > i40e DPDK driver software to make a check and set this flag. I would think the > overhead to check the 1st ethertype field in the MAC header against dot1q > or dot1ad values in software be pretty minimal. > > Regards, > John > > > -----Original Message----- > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > Sent: Wednesday, May 25, 2016 3:23 PM > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > Cc: vpp-dev; Zhang, Helin; dev@dpdk.org > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > I suppose this has to do with what is expected usage of the > > PKT_RX_VLAN_PKT offload flag. Is it set only for VLAN packets with the > > VLAN stripped or should always be set if VLAN is/was present in the > > received packet. It seems that different DPDK drivers are behaving > differently which will make it really hard for VPP to take advantage of NIC > and driver offload capability to provide better performance. > > Yes, that's true ixgbe/igb from one side and i40e do raise PKT_RX_VLAN_PKT > in a different manner. > There is an attempt to make it unified across all supported devices: > http://dpdk.org/dev/patchwork/patch/12938/ > > Though, I am not sure it will help you with your issue. > As I understand, for you the desired behaviour is: > If it is a vlan packet, keep the packet intact (don't strip the vlan) and raise > PKT_RX_VLAN_PK inside mbuf->ol_flags. > That's what ixgbe is doing with rte_eth_conf.rxmode.hw_vlan_strip==0. > Correct? > As far as I know, i40e HW doesn't provide such ability. > i40e Receive HW Descriptor can only flag was VLAN tag stripped from the > packet or not, but if stripping is disabled it wouldn't indicate in any way is > VLAN tag is present inside the packet or not. > I am CC-ing it to dpdk.org in case I am missing something here. > Probably someone knows a way to make it work in that way. > Konstantin > > > > > If VPP cannot rely on offload flags for VLAN so packets always have to go > through ethernet-input node, there is a performance cost. > > For the 10GE case, before the inverse patch of the ixgbe driver, 10GE > > Rx-vector path removed support of offload flag with DPDK 16.04 and so > > ethernet-input node is always used. The 10GE IPv4 throughput rate > > dropped from 6.17MPPSx2 to 4.92MPPSx2 for bidirectional traffic (2 > > streams each with a single IP address as destination) on a single core > > on my server. Konstantin suggested at that time to use scalar mode which > does support offload flags properly. The scalar mode did by-pass ethernet- > input and provided throughput of 5.72MPPS. We ended up inverse patched > the ixgbe driver to restore vector mode offload flag support as the original > restriction (the reason offload flag support was removed) would not affect > VPP. > > > > I think for 40GE driver to provide offload flag support in vector mode > > but not give indication of presence of VLAN tag is just wrong. This make the > offload flag support useless for VPP. > > > > Regards, > > John > > > > -----Original Message----- > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > Sent: Wednesday, May 25, 2016 11:30 AM > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > Cc: vpp-dev; Zhang, Helin > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I see what you are getting at, Konstantin. The VPP init code does > > > not enable VLAN strip for Intel NICs as VLAN tag must be in the > > > packet for sub-interface lookup by ethernet-input node. I agree if > > > we enable VLAN tag strip for the i40e driver, we can get around this > > > problem > > but then all packets will be considered as received on the main interface. > > > > I see... > > As far as I know, when VLAN stripping is disabled, i40e RXD doesn't contain > information does that packet contain a VLAN or not. > > So, if enabling vlan stripping is not an option for you guys, then I > > don't see any other way on i40e to recognise is that VLAN packet or not, > except then parse the packet in SW. > > Helin, please correct me here, if I am missing something here. > > Thanks > > Konstantin > > > > > > > > Regards, > > > John > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > Sent: Wednesday, May 25, 2016 10:35 AM > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > Since this is the XL710 40GE NIC, I suppose the DPDK driver involved > would be the i40e driver and not ixgbe for 10GE NICs. > > > > > > Yes, I understand that you are facing problem on i40e, not ixgbe. > > > And the problem is that for i40e PKT_RX_VLAN_PKT flag is not set in > mbuf->ol_flags, correct? > > > That's why I asked: are you running it with > rte_eth_conf.rxmode.hw_vlan_strip==0 or not? > > > If yes, you can try with rte_eth_conf.rxmode.hw_vlan_strip=1 and see > would it help you. > > > > > > > > > > > As explained before, ixgbe driver had the inverse patch added. It > > > > does recognize VLAN tag in the packet and set PKT_RX_VLAN_PKT > offload flag properly: > > > > > > That patch has nothing to do with PKT_RX_VLAN_PKT and i40e. > > > So I don't think it is related to that problem at all. > > > Konstantin > > > > > > > > > > > 00:01:02:132370: dpdk-input > > > > TenGigabitEthernet5/0/0 rx queue 0 > > > > buffer 0x44cff: current data 0, length 96, free-list 0, totlen-nifb 0, trace > 0x0 > > > > PKT MBUF: port 3, nb_segs 1, pkt_len 96 > > > > buf_len 2176, data_len 96, ol_flags 0x1, > > > > packet_type 0x210 > > > > Packet Offload Flags > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > Packet Types > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > headers > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > UDP: 28.0.0.1 -> 29.0.0.254 > > > > tos 0x00, ttl 64, length 78, checksum 0x40a1 > > > > fragment id 0x0000 > > > > UDP: 63 -> 63 > > > > length 58, checksum 0x5b76 > > > > 00:01:02:132391: ethernet-input > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > 00:01:02:132408: error-drop > > > > ethernet-input: unknown vlan > > > > > > > > You can see from above the packet was passed to ethernet-input > > > > node and dropped because sub-interface for VLAN 1101 is not set up. > > > > > > > > Regards, > > > > John > > > > > > > > -----Original Message----- > > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > > Sent: Wednesday, May 25, 2016 4:46 AM > > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > Hi, > > > > I don’t think ptype recognition on ixgbe vRX is somehow related to that. > > > > Are you guys running FVL with > rte_eth_conf.rxmode.hw_vlan_strip==0? > > > > If so, can you set it to 1 and see would it help? > > > > Konstantin > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > > On Behalf Of John Lo (loj) > > > > Sent: Wednesday, May 25, 2016 1:56 AM > > > > To: Wiles, Keith; Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > I did put in a reverse patch, provided by Damjan, for this in fd.io: > > > > https://gerrit.fd.io/r/#/c/941/1/dpdk/dpdk-16.04_patches/0013-Reve > > > > rt -i xgbe-fix-packet-type-from-vector-Rx.patch > > > > > > > > I know this fixed the 10GE NIC behavior. Perhaps the 40GE NIC is using a > different DPDK driver? > > > > > > > > Regards, > > > > John > > > > > > > > From: Wiles, Keith [mailto:keith.wiles@intel.com] > > > > Sent: Tuesday, May 24, 2016 8:30 PM > > > > To: John Lo (loj); Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > The commit ID is > > > > > > > > commit d9a2009a81089093645fea2e04b51dd37edf3e6f > > > > Author: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > > Date: Tue Mar 22 14:30:17 2016 +0000 > > > > > > > > ixgbe: fix packet type from vector Rx > > > > > > > > Current vector RX can't always set the packet_type properly. > > > > To be more specific: > > > > a) it never sets RTE_PTYPE_L2_ETHER > > > > b) it doesn't handle tunnel ipv4/ipv6 case correctly. > > > > c) it doesn't check is IXGBE_RXDADV_PKTTYPE_ETQF set or not. > > > > While a) is pretty easy to fix, b) and c) are not that > > > > straightforward > > > > in terms of SIMD ops (specially b). > > > > So far I wasn't able to make vRX support packet_type properly > > > > without > > > > noticeable performance loss. > > > > So for now, just remove that functionality from vector RX and > > > > update dev_supported_ptypes_get(). > > > > > > > > Fixes: 396254175854 ("mbuf: redefine packet type") > > > > > > > > Signed-off-by: Konstantin Ananyev > > > > <konstantin.ananyev@intel.com> > > > > Acked-by: Cunming Liang <cunming.liang@intel.com> > > > > > > > > It appears the change was only down to the vector version in the RX > routine, could move to the none-vector version I guess. > > > > > > > > I would think adding a patch or adding the code to VPP DPDK init to use > the RX callback to setup things correctly. > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > > <keith.wiles@intel.com> > > > > Date: Tuesday, May 24, 2016 at 7:19 PM > > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > > <ckannan@console.to> > > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > Here is the starting thread: > > > > http://dpdk.org/ml/archives/dev/2016-April/037837.html > > > > > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > > <keith.wiles@intel.com> > > > > Date: Tuesday, May 24, 2016 at 7:16 PM > > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > > <ckannan@console.to> > > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > The flags in the rte_mbuf were changed a before 16.04, which > > > > removed the ptype testing and possible setting this flag. I thought a > patch to revert that change was done, did that not happen. > > > > > > > > The other solution is to add an option to DPDK to restore the > > > > previous behavior for this type of problem. The only other way is > > > > to have a installed in the PMD to setup the flags correctly for VPP. > > > > This would require adding a routine to VPP and setting the call > > > > back in the > > > driver. > > > > > > > > I really thought a patch was created, does anyone know? > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: "John Lo (loj)" <loj@cisco.com> > > > > Date: Tuesday, May 24, 2016 at 6:53 PM > > > > To: Chandrasekar Kannan <ckannan@console.to> > > > > Cc: Keith Wiles <keith.wiles@intel.com>, vpp-dev > > > > <vpp-dev@lists.fd.io> > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > I believe the problem is due to 40GE DPDK driver not parsing VLAN > > > > packet properly and setting the packet offload flag > > > > PKT_RX_VLAN_PKT, causing dpdk-input node to send packet directly > > > > to ip4-input instead of ethernet-input. If this flag is set, then > > > > the packet will be passed to ethernet-input which will then parse > > > > the VLAN tag in the > > > packet to perform sub-interface lookup and find the start of IP > > > header in the packet properly. Following is an example trace of a correctly > marked packet: > > > > > > > > 23:11:09:855916: dpdk-input > > > > TenGigE0/0/0/0 rx queue 0 > > > > buffer 0x23f6c0: current data 0, length 118, free-list 0, > > > > totlen-nifb 0, trace 0x0 > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 118 > > > > buf_len 2176, data_len 118, ol_flags 0x1, > > > > packet_type 0x10 > > > > Packet Offload Flags > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > Packet Types > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > > > headers > > > > IP4: 74:a0:2f:e2:d5:14 -> 90:e2:ba:9f:8b:cc vlan 101 > > > > ICMP: 130.1.1.1 -> 130.1.1.2 > > > > tos 0x00, ttl 255, length 100, checksum 0xac4b > > > > fragment id 0x0948 > > > > ICMP echo_request checksum 0x900d > > > > > > > > Hoping someone familiar with the 40GE DPDK driver can comment > further. > > > > > > > > Regards, > > > > John > > > > > > > > From: Chandrasekar Kannan [mailto:ckannan@console.to] > > > > Sent: Tuesday, May 24, 2016 6:57 PM > > > > To: John Lo (loj) > > > > Cc: Wiles, Keith; vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > Yes. Subinterface has been configured like this ... > > > > > > > > vppctl set interface ip address del FortyGigabitEthernet83/0/0 all > > > > vppctl create sub FortyGigabitEthernet83/0/0 900 vppctl set > > > > interface ip address FortyGigabitEthernet83/0/0.900 6.0.0.1/24 > > > > vppctl set interface ip address FortyGigabitEthernet83/0/0.900 > > > > 7.0.0.1/24 vppctl set interface state FortyGigabitEthernet83/0/0 > > > > up vppctl set interface state FortyGigabitEthernet83/0/0.900 up > > > > vppctl set ip arp static > > > > FortyGigabitEthernet83/0/0.900 7.0.0.2 3c:fd:fe:9c:c3:88 > > > > > > > > -------------------------------------------------- > > > > Packet 4 > > > > > > > > 00:02:23:366879: dpdk-input > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > totlen-nifb 0, trace 0x3 > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > packet_type 0x291 > > > > Packet Types > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > without extension headers > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > UDP: 6.0.0.2 -> 7.0.0.2 > > > > tos 0x00, ttl 64, length 46, checksum 0x2dbc > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > UDP: 1024 -> 2001 > > > > length 26, checksum 0x0000 > > > > 00:02:23:366894: ip4-input-no-checksum > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.45.188 -> 6.0.0.2 > > > > version 0, header length 12 > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > 0xaf4d) > > > > fragment id 0x4500 offset 368, flags > > > > 00:02:23:366910: error-drop > > > > ip4-input: ip4 length > l2 length > > > > > > > > -------------------------------------------------------------- > > > > > > > > > > > > > > > > On Tue, May 24, 2016 at 3:12 PM, John Lo (loj) <loj@cisco.com> wrote: > > > > Was a sub-interface created with the expected VLAN tag 900? An > > > > example of creating a sub-interface and bring it up would be something > like: > > > > > > > > create sub GigabitEthernet1/0/0 900 set int state > > > > GigabitEthernet1/0/0 up set int state > > > > GigabitEthernet1/0/0.900 up > > > > > > > > Regards, > > > > John > > > > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > > On Behalf Of Wiles, Keith > > > > Sent: Tuesday, May 24, 2016 5:59 PM > > > > To: Chandrasekar Kannan; vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > I am starting to think the unknown VLAN is not accounted for in the > processing of the input frame in the VPP DPDK interface code. > > > > vnet/vnet/devices/dpdk/node.c > > > > > > > > The version and header length are wrong in the output below, which > > > > leads me to believe the starting IPv4 header location is wrong. I > > > > have not parsed all of the code in node.c:dpdk_device_input() yet, > > > > but I do not think the l3_offset is being adjusted to the correct > > > offset???? > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Chandrasekar > > > > Kannan <ckannan@console.to> > > > > Date: Tuesday, May 24, 2016 at 3:40 PM > > > > To: vpp-dev <vpp-dev@lists.fd.io> > > > > Subject: [vpp-dev] VLAN packets dropped... ? > > > > > > > > Hi, > > > > > > > > I was trying to setup VLAN subinterfaces with VPP but it appears > > > > the packets are dropped at ingress. I'm tried with both > > > > pktgen-dpdk and trex for traffic generation just to rule out any > > > > corruption with > > > pkt generation. But both are producing the same errors with vpp. > > > > > > > > yaml file from trex... > > > > > > > > ------------------------------ > > > > [root@node04 scripts]# cat cap2/ipv4_vlan.yaml > > > > - duration : 10 > > > > generator : > > > > distribution : "seq" > > > > clients_start : "16.0.0.1" > > > > clients_end : "16.0.1.255" > > > > servers_start : "48.0.0.1" > > > > servers_end : "48.0.0.255" > > > > clients_per_gb : 201 > > > > min_clients : 101 > > > > dual_port_mask : "1.0.0.0" > > > > tcp_aging : 1 > > > > udp_aging : 1 > > > > mac : [0x0,0x0,0x0,0x1,0x0,0x00] > > > > vlan : { enable : 1 , vlan0 : 900 , vlan1 : 1100 } > > > > cap_info : > > > > - name: cap2/udp_64B.pcap > > > > cps : 1000.0 > > > > ipg : 10000 > > > > rtt : 10000 > > > > w : 4 > > > > limit : 20 > > > > [root@node04 scripts]# > > > > --------------------------------------------------------------- > > > > > > > > vpp trace... > > > > > > > > 00:02:21:576810: dpdk-input > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > totlen-nifb 0, trace 0x1 > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > packet_type 0x291 > > > > Packet Types > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > without extension headers > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > UDP: 16.0.1.0 -> 48.0.0.128 > > > > tos 0x00, ttl 64, length 46, checksum 0xf93f > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > UDP: 1024 -> 2001 > > > > length 26, checksum 0x0000 > > > > 00:02:21:576817: ip4-input-no-checksum > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.249.63 -> 16.0.1.0 > > > > version 0, header length 12 > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > 0xaf4d) > > > > fragment id 0x4500 offset 368, flags > > > > 00:02:21:576828: error-drop > > > > ip4-input: ip4 length > l2 length > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? 2016-05-26 22:56 ` John Daley (johndale) @ 2016-06-01 18:44 ` Chandrasekar Kannan 2016-06-01 20:02 ` John Lo (loj) 0 siblings, 1 reply; 6+ messages in thread From: Chandrasekar Kannan @ 2016-06-01 18:44 UTC (permalink / raw) To: John Daley (johndale) Cc: John Lo (loj), Ananyev, Konstantin, Wiles, Keith, vpp-dev, Zhang, Helin, dev Just checking back on this thread to see where things are. Are we saying this is a bug in dpdk or vpp ?. That part wasn't quite clear to me. -Chandra On Thu, May 26, 2016 at 3:56 PM, John Daley (johndale) <johndale@cisco.com> wrote: > John, > As just discussed, what you suggest was considered but there are other > apps depending a different behavior of the flag, so we thought the best > thing to do is deprecate it. That is part of what Olivier's patch discussed > in http://www.dpdk.org/ml/archives/dev/2016-May/038786.html does. Adding > a new ptype for VLAN tagged packets after the patch is accepted would allow > VPP to check if the ptype is supported by the PMD and if so, use it to > determine if the delivered packet actually has a VLAN tag in it. No need to > know if stripping is enabled/disabled. If the ptype is not supported, the > app would have check the packet in SW. > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of John Lo (loj) > > Sent: Thursday, May 26, 2016 11:11 AM > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wiles, Keith > > <keith.wiles@intel.com>; Chandrasekar Kannan <ckannan@console.to> > > Cc: vpp-dev <vpp-dev@lists.fd.io>; Zhang, Helin <helin.zhang@intel.com>; > > dev@dpdk.org > > Subject: Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? > > > > Hi Konstantin, > > > > Thanks for the link to DPDK discussion wrt this VLAN offload flag > > PKT_RX_VLAN_PKT. It seems the proposal was to deprecate > > PKT_RX_VLAN_PKT and introduce two new flags PKT_RX_VLAN_STRIPPED > > and PKT_RX_QINQ_STRIPPED. > > > > It would be a really good idea to keep PKT_RX_VLAN_PKT to indicate at > least > > one VLAN tag is present on the packet. For the case of i40e where its HW > > cannot detect VLAN tag if strip is not enabled, it should be reasonable > for the > > i40e DPDK driver software to make a check and set this flag. I would > think the > > overhead to check the 1st ethertype field in the MAC header against dot1q > > or dot1ad values in software be pretty minimal. > > > > Regards, > > John > > > > > > -----Original Message----- > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > Sent: Wednesday, May 25, 2016 3:23 PM > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > Cc: vpp-dev; Zhang, Helin; dev@dpdk.org > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > I suppose this has to do with what is expected usage of the > > > PKT_RX_VLAN_PKT offload flag. Is it set only for VLAN packets with the > > > VLAN stripped or should always be set if VLAN is/was present in the > > > received packet. It seems that different DPDK drivers are behaving > > differently which will make it really hard for VPP to take advantage of > NIC > > and driver offload capability to provide better performance. > > > > Yes, that's true ixgbe/igb from one side and i40e do raise > PKT_RX_VLAN_PKT > > in a different manner. > > There is an attempt to make it unified across all supported devices: > > http://dpdk.org/dev/patchwork/patch/12938/ > > > > Though, I am not sure it will help you with your issue. > > As I understand, for you the desired behaviour is: > > If it is a vlan packet, keep the packet intact (don't strip the vlan) > and raise > > PKT_RX_VLAN_PK inside mbuf->ol_flags. > > That's what ixgbe is doing with rte_eth_conf.rxmode.hw_vlan_strip==0. > > Correct? > > As far as I know, i40e HW doesn't provide such ability. > > i40e Receive HW Descriptor can only flag was VLAN tag stripped from the > > packet or not, but if stripping is disabled it wouldn't indicate in any > way is > > VLAN tag is present inside the packet or not. > > I am CC-ing it to dpdk.org in case I am missing something here. > > Probably someone knows a way to make it work in that way. > > Konstantin > > > > > > > > If VPP cannot rely on offload flags for VLAN so packets always have to > go > > through ethernet-input node, there is a performance cost. > > > For the 10GE case, before the inverse patch of the ixgbe driver, 10GE > > > Rx-vector path removed support of offload flag with DPDK 16.04 and so > > > ethernet-input node is always used. The 10GE IPv4 throughput rate > > > dropped from 6.17MPPSx2 to 4.92MPPSx2 for bidirectional traffic (2 > > > streams each with a single IP address as destination) on a single core > > > on my server. Konstantin suggested at that time to use scalar mode > which > > does support offload flags properly. The scalar mode did by-pass > ethernet- > > input and provided throughput of 5.72MPPS. We ended up inverse patched > > the ixgbe driver to restore vector mode offload flag support as the > original > > restriction (the reason offload flag support was removed) would not > affect > > VPP. > > > > > > I think for 40GE driver to provide offload flag support in vector mode > > > but not give indication of presence of VLAN tag is just wrong. This > make the > > offload flag support useless for VPP. > > > > > > Regards, > > > John > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > Sent: Wednesday, May 25, 2016 11:30 AM > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev; Zhang, Helin > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > I see what you are getting at, Konstantin. The VPP init code does > > > > not enable VLAN strip for Intel NICs as VLAN tag must be in the > > > > packet for sub-interface lookup by ethernet-input node. I agree if > > > > we enable VLAN tag strip for the i40e driver, we can get around this > > > > problem > > > but then all packets will be considered as received on the main > interface. > > > > > > I see... > > > As far as I know, when VLAN stripping is disabled, i40e RXD doesn't > contain > > information does that packet contain a VLAN or not. > > > So, if enabling vlan stripping is not an option for you guys, then I > > > don't see any other way on i40e to recognise is that VLAN packet or > not, > > except then parse the packet in SW. > > > Helin, please correct me here, if I am missing something here. > > > Thanks > > > Konstantin > > > > > > > > > > > Regards, > > > > John > > > > > > > > -----Original Message----- > > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > > Sent: Wednesday, May 25, 2016 10:35 AM > > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > > > > Since this is the XL710 40GE NIC, I suppose the DPDK driver > involved > > would be the i40e driver and not ixgbe for 10GE NICs. > > > > > > > > Yes, I understand that you are facing problem on i40e, not ixgbe. > > > > And the problem is that for i40e PKT_RX_VLAN_PKT flag is not set in > > mbuf->ol_flags, correct? > > > > That's why I asked: are you running it with > > rte_eth_conf.rxmode.hw_vlan_strip==0 or not? > > > > If yes, you can try with rte_eth_conf.rxmode.hw_vlan_strip=1 and see > > would it help you. > > > > > > > > > > > > > > As explained before, ixgbe driver had the inverse patch added. It > > > > > does recognize VLAN tag in the packet and set PKT_RX_VLAN_PKT > > offload flag properly: > > > > > > > > That patch has nothing to do with PKT_RX_VLAN_PKT and i40e. > > > > So I don't think it is related to that problem at all. > > > > Konstantin > > > > > > > > > > > > > > 00:01:02:132370: dpdk-input > > > > > TenGigabitEthernet5/0/0 rx queue 0 > > > > > buffer 0x44cff: current data 0, length 96, free-list 0, > totlen-nifb 0, trace > > 0x0 > > > > > PKT MBUF: port 3, nb_segs 1, pkt_len 96 > > > > > buf_len 2176, data_len 96, ol_flags 0x1, > > > > > packet_type 0x210 > > > > > Packet Offload Flags > > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > > Packet Types > > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > headers > > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > > UDP: 28.0.0.1 -> 29.0.0.254 > > > > > tos 0x00, ttl 64, length 78, checksum 0x40a1 > > > > > fragment id 0x0000 > > > > > UDP: 63 -> 63 > > > > > length 58, checksum 0x5b76 > > > > > 00:01:02:132391: ethernet-input > > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > > 00:01:02:132408: error-drop > > > > > ethernet-input: unknown vlan > > > > > > > > > > You can see from above the packet was passed to ethernet-input > > > > > node and dropped because sub-interface for VLAN 1101 is not set up. > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > -----Original Message----- > > > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > > > Sent: Wednesday, May 25, 2016 4:46 AM > > > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > > > Cc: vpp-dev > > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > Hi, > > > > > I don’t think ptype recognition on ixgbe vRX is somehow related to > that. > > > > > Are you guys running FVL with > > rte_eth_conf.rxmode.hw_vlan_strip==0? > > > > > If so, can you set it to 1 and see would it help? > > > > > Konstantin > > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > > > On Behalf Of John Lo (loj) > > > > > Sent: Wednesday, May 25, 2016 1:56 AM > > > > > To: Wiles, Keith; Chandrasekar Kannan > > > > > Cc: vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I did put in a reverse patch, provided by Damjan, for this in > fd.io: > > > > > https://gerrit.fd.io/r/#/c/941/1/dpdk/dpdk-16.04_patches/0013-Reve > > > > > rt -i xgbe-fix-packet-type-from-vector-Rx.patch > > > > > > > > > > I know this fixed the 10GE NIC behavior. Perhaps the 40GE NIC is > using a > > different DPDK driver? > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > From: Wiles, Keith [mailto:keith.wiles@intel.com] > > > > > Sent: Tuesday, May 24, 2016 8:30 PM > > > > > To: John Lo (loj); Chandrasekar Kannan > > > > > Cc: vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > The commit ID is > > > > > > > > > > commit d9a2009a81089093645fea2e04b51dd37edf3e6f > > > > > Author: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > > > Date: Tue Mar 22 14:30:17 2016 +0000 > > > > > > > > > > ixgbe: fix packet type from vector Rx > > > > > > > > > > Current vector RX can't always set the packet_type properly. > > > > > To be more specific: > > > > > a) it never sets RTE_PTYPE_L2_ETHER > > > > > b) it doesn't handle tunnel ipv4/ipv6 case correctly. > > > > > c) it doesn't check is IXGBE_RXDADV_PKTTYPE_ETQF set or not. > > > > > While a) is pretty easy to fix, b) and c) are not that > > > > > straightforward > > > > > in terms of SIMD ops (specially b). > > > > > So far I wasn't able to make vRX support packet_type properly > > > > > without > > > > > noticeable performance loss. > > > > > So for now, just remove that functionality from vector RX and > > > > > update dev_supported_ptypes_get(). > > > > > > > > > > Fixes: 396254175854 ("mbuf: redefine packet type") > > > > > > > > > > Signed-off-by: Konstantin Ananyev > > > > > <konstantin.ananyev@intel.com> > > > > > Acked-by: Cunming Liang <cunming.liang@intel.com> > > > > > > > > > > It appears the change was only down to the vector version in the RX > > routine, could move to the none-vector version I guess. > > > > > > > > > > I would think adding a patch or adding the code to VPP DPDK init > to use > > the RX callback to setup things correctly. > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > > > <keith.wiles@intel.com> > > > > > Date: Tuesday, May 24, 2016 at 7:19 PM > > > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > > > <ckannan@console.to> > > > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > Here is the starting thread: > > > > > http://dpdk.org/ml/archives/dev/2016-April/037837.html > > > > > > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > > > <keith.wiles@intel.com> > > > > > Date: Tuesday, May 24, 2016 at 7:16 PM > > > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > > > <ckannan@console.to> > > > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > The flags in the rte_mbuf were changed a before 16.04, which > > > > > removed the ptype testing and possible setting this flag. I > thought a > > patch to revert that change was done, did that not happen. > > > > > > > > > > The other solution is to add an option to DPDK to restore the > > > > > previous behavior for this type of problem. The only other way is > > > > > to have a installed in the PMD to setup the flags correctly for > VPP. > > > > > This would require adding a routine to VPP and setting the call > > > > > back in the > > > > driver. > > > > > > > > > > I really thought a patch was created, does anyone know? > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: "John Lo (loj)" <loj@cisco.com> > > > > > Date: Tuesday, May 24, 2016 at 6:53 PM > > > > > To: Chandrasekar Kannan <ckannan@console.to> > > > > > Cc: Keith Wiles <keith.wiles@intel.com>, vpp-dev > > > > > <vpp-dev@lists.fd.io> > > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I believe the problem is due to 40GE DPDK driver not parsing VLAN > > > > > packet properly and setting the packet offload flag > > > > > PKT_RX_VLAN_PKT, causing dpdk-input node to send packet directly > > > > > to ip4-input instead of ethernet-input. If this flag is set, then > > > > > the packet will be passed to ethernet-input which will then parse > > > > > the VLAN tag in the > > > > packet to perform sub-interface lookup and find the start of IP > > > > header in the packet properly. Following is an example trace of a > correctly > > marked packet: > > > > > > > > > > 23:11:09:855916: dpdk-input > > > > > TenGigE0/0/0/0 rx queue 0 > > > > > buffer 0x23f6c0: current data 0, length 118, free-list 0, > > > > > totlen-nifb 0, trace 0x0 > > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 118 > > > > > buf_len 2176, data_len 118, ol_flags 0x1, > > > > > packet_type 0x10 > > > > > Packet Offload Flags > > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > > Packet Types > > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > > > > headers > > > > > IP4: 74:a0:2f:e2:d5:14 -> 90:e2:ba:9f:8b:cc vlan 101 > > > > > ICMP: 130.1.1.1 -> 130.1.1.2 > > > > > tos 0x00, ttl 255, length 100, checksum 0xac4b > > > > > fragment id 0x0948 > > > > > ICMP echo_request checksum 0x900d > > > > > > > > > > Hoping someone familiar with the 40GE DPDK driver can comment > > further. > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > From: Chandrasekar Kannan [mailto:ckannan@console.to] > > > > > Sent: Tuesday, May 24, 2016 6:57 PM > > > > > To: John Lo (loj) > > > > > Cc: Wiles, Keith; vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > Yes. Subinterface has been configured like this ... > > > > > > > > > > vppctl set interface ip address del FortyGigabitEthernet83/0/0 all > > > > > vppctl create sub FortyGigabitEthernet83/0/0 900 vppctl set > > > > > interface ip address FortyGigabitEthernet83/0/0.900 6.0.0.1/24 > > > > > vppctl set interface ip address FortyGigabitEthernet83/0/0.900 > > > > > 7.0.0.1/24 vppctl set interface state FortyGigabitEthernet83/0/0 > > > > > up vppctl set interface state FortyGigabitEthernet83/0/0.900 up > > > > > vppctl set ip arp static > > > > > FortyGigabitEthernet83/0/0.900 7.0.0.2 3c:fd:fe:9c:c3:88 > > > > > > > > > > -------------------------------------------------- > > > > > Packet 4 > > > > > > > > > > 00:02:23:366879: dpdk-input > > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > > totlen-nifb 0, trace 0x3 > > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > > packet_type 0x291 > > > > > Packet Types > > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > > without extension headers > > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > > UDP: 6.0.0.2 -> 7.0.0.2 > > > > > tos 0x00, ttl 64, length 46, checksum 0x2dbc > > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > > UDP: 1024 -> 2001 > > > > > length 26, checksum 0x0000 > > > > > 00:02:23:366894: ip4-input-no-checksum > > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.45.188 -> 6.0.0.2 > > > > > version 0, header length 12 > > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > > 0xaf4d) > > > > > fragment id 0x4500 offset 368, flags > > > > > 00:02:23:366910: error-drop > > > > > ip4-input: ip4 length > l2 length > > > > > > > > > > -------------------------------------------------------------- > > > > > > > > > > > > > > > > > > > > On Tue, May 24, 2016 at 3:12 PM, John Lo (loj) <loj@cisco.com> > wrote: > > > > > Was a sub-interface created with the expected VLAN tag 900? An > > > > > example of creating a sub-interface and bring it up would be > something > > like: > > > > > > > > > > create sub GigabitEthernet1/0/0 900 set int state > > > > > GigabitEthernet1/0/0 up set int state > > > > > GigabitEthernet1/0/0.900 up > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > > > On Behalf Of Wiles, Keith > > > > > Sent: Tuesday, May 24, 2016 5:59 PM > > > > > To: Chandrasekar Kannan; vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I am starting to think the unknown VLAN is not accounted for in the > > processing of the input frame in the VPP DPDK interface code. > > > > > vnet/vnet/devices/dpdk/node.c > > > > > > > > > > The version and header length are wrong in the output below, which > > > > > leads me to believe the starting IPv4 header location is wrong. I > > > > > have not parsed all of the code in node.c:dpdk_device_input() yet, > > > > > but I do not think the l3_offset is being adjusted to the correct > > > > offset???? > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Chandrasekar > > > > > Kannan <ckannan@console.to> > > > > > Date: Tuesday, May 24, 2016 at 3:40 PM > > > > > To: vpp-dev <vpp-dev@lists.fd.io> > > > > > Subject: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > Hi, > > > > > > > > > > I was trying to setup VLAN subinterfaces with VPP but it appears > > > > > the packets are dropped at ingress. I'm tried with both > > > > > pktgen-dpdk and trex for traffic generation just to rule out any > > > > > corruption with > > > > pkt generation. But both are producing the same errors with vpp. > > > > > > > > > > yaml file from trex... > > > > > > > > > > ------------------------------ > > > > > [root@node04 scripts]# cat cap2/ipv4_vlan.yaml > > > > > - duration : 10 > > > > > generator : > > > > > distribution : "seq" > > > > > clients_start : "16.0.0.1" > > > > > clients_end : "16.0.1.255" > > > > > servers_start : "48.0.0.1" > > > > > servers_end : "48.0.0.255" > > > > > clients_per_gb : 201 > > > > > min_clients : 101 > > > > > dual_port_mask : "1.0.0.0" > > > > > tcp_aging : 1 > > > > > udp_aging : 1 > > > > > mac : [0x0,0x0,0x0,0x1,0x0,0x00] > > > > > vlan : { enable : 1 , vlan0 : 900 , vlan1 : 1100 } > > > > > cap_info : > > > > > - name: cap2/udp_64B.pcap > > > > > cps : 1000.0 > > > > > ipg : 10000 > > > > > rtt : 10000 > > > > > w : 4 > > > > > limit : 20 > > > > > [root@node04 scripts]# > > > > > --------------------------------------------------------------- > > > > > > > > > > vpp trace... > > > > > > > > > > 00:02:21:576810: dpdk-input > > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > > totlen-nifb 0, trace 0x1 > > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > > packet_type 0x291 > > > > > Packet Types > > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > > without extension headers > > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > > UDP: 16.0.1.0 -> 48.0.0.128 > > > > > tos 0x00, ttl 64, length 46, checksum 0xf93f > > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > > UDP: 1024 -> 2001 > > > > > length 26, checksum 0x0000 > > > > > 00:02:21:576817: ip4-input-no-checksum > > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.249.63 -> 16.0.1.0 > > > > > version 0, header length 12 > > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > > 0xaf4d) > > > > > fragment id 0x4500 offset 368, flags > > > > > 00:02:21:576828: error-drop > > > > > ip4-input: ip4 length > l2 length > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? 2016-06-01 18:44 ` Chandrasekar Kannan @ 2016-06-01 20:02 ` John Lo (loj) 2016-06-01 20:21 ` Chandrasekar Kannan 0 siblings, 1 reply; 6+ messages in thread From: John Lo (loj) @ 2016-06-01 20:02 UTC (permalink / raw) To: Chandrasekar Kannan, John Daley (johndale) Cc: Ananyev, Konstantin, Wiles, Keith, vpp-dev, Zhang, Helin, dev We will be adding a patch to the i40e driver that would check the packet for presence of VLAN tag set the required PKT_RX_VLAN_PKT flag. I hope to get it done by the end of this week. -John From: Chandrasekar Kannan [mailto:ckannan@console.to] Sent: Wednesday, June 01, 2016 2:45 PM To: John Daley (johndale) Cc: John Lo (loj); Ananyev, Konstantin; Wiles, Keith; vpp-dev; Zhang, Helin; dev@dpdk.org Subject: Re: [vpp-dev] VLAN packets dropped... ? Just checking back on this thread to see where things are. Are we saying this is a bug in dpdk or vpp ?. That part wasn't quite clear to me. -Chandra On Thu, May 26, 2016 at 3:56 PM, John Daley (johndale) <johndale@cisco.com<mailto:johndale@cisco.com>> wrote: John, As just discussed, what you suggest was considered but there are other apps depending a different behavior of the flag, so we thought the best thing to do is deprecate it. That is part of what Olivier's patch discussed in http://www.dpdk.org/ml/archives/dev/2016-May/038786.html does. Adding a new ptype for VLAN tagged packets after the patch is accepted would allow VPP to check if the ptype is supported by the PMD and if so, use it to determine if the delivered packet actually has a VLAN tag in it. No need to know if stripping is enabled/disabled. If the ptype is not supported, the app would have check the packet in SW. > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of John Lo (loj) > Sent: Thursday, May 26, 2016 11:11 AM > To: Ananyev, Konstantin <konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>>; Wiles, Keith > <keith.wiles@intel.com<mailto:keith.wiles@intel.com>>; Chandrasekar Kannan <ckannan@console.to<mailto:ckannan@console.to>> > Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>; Zhang, Helin <helin.zhang@intel.com<mailto:helin.zhang@intel.com>>; > dev@dpdk.org<mailto:dev@dpdk.org> > Subject: Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? > > Hi Konstantin, > > Thanks for the link to DPDK discussion wrt this VLAN offload flag > PKT_RX_VLAN_PKT. It seems the proposal was to deprecate > PKT_RX_VLAN_PKT and introduce two new flags PKT_RX_VLAN_STRIPPED > and PKT_RX_QINQ_STRIPPED. > > It would be a really good idea to keep PKT_RX_VLAN_PKT to indicate at least > one VLAN tag is present on the packet. For the case of i40e where its HW > cannot detect VLAN tag if strip is not enabled, it should be reasonable for the > i40e DPDK driver software to make a check and set this flag. I would think the > overhead to check the 1st ethertype field in the MAC header against dot1q > or dot1ad values in software be pretty minimal. > > Regards, > John > > > -----Original Message----- > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>] > Sent: Wednesday, May 25, 2016 3:23 PM > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > Cc: vpp-dev; Zhang, Helin; dev@dpdk.org<mailto:dev@dpdk.org> > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > I suppose this has to do with what is expected usage of the > > PKT_RX_VLAN_PKT offload flag. Is it set only for VLAN packets with the > > VLAN stripped or should always be set if VLAN is/was present in the > > received packet. It seems that different DPDK drivers are behaving > differently which will make it really hard for VPP to take advantage of NIC > and driver offload capability to provide better performance. > > Yes, that's true ixgbe/igb from one side and i40e do raise PKT_RX_VLAN_PKT > in a different manner. > There is an attempt to make it unified across all supported devices: > http://dpdk.org/dev/patchwork/patch/12938/ > > Though, I am not sure it will help you with your issue. > As I understand, for you the desired behaviour is: > If it is a vlan packet, keep the packet intact (don't strip the vlan) and raise > PKT_RX_VLAN_PK inside mbuf->ol_flags. > That's what ixgbe is doing with rte_eth_conf.rxmode.hw_vlan_strip==0. > Correct? > As far as I know, i40e HW doesn't provide such ability. > i40e Receive HW Descriptor can only flag was VLAN tag stripped from the > packet or not, but if stripping is disabled it wouldn't indicate in any way is > VLAN tag is present inside the packet or not. > I am CC-ing it to dpdk.org<http://dpdk.org> in case I am missing something here. > Probably someone knows a way to make it work in that way. > Konstantin > > > > > If VPP cannot rely on offload flags for VLAN so packets always have to go > through ethernet-input node, there is a performance cost. > > For the 10GE case, before the inverse patch of the ixgbe driver, 10GE > > Rx-vector path removed support of offload flag with DPDK 16.04 and so > > ethernet-input node is always used. The 10GE IPv4 throughput rate > > dropped from 6.17MPPSx2 to 4.92MPPSx2 for bidirectional traffic (2 > > streams each with a single IP address as destination) on a single core > > on my server. Konstantin suggested at that time to use scalar mode which > does support offload flags properly. The scalar mode did by-pass ethernet- > input and provided throughput of 5.72MPPS. We ended up inverse patched > the ixgbe driver to restore vector mode offload flag support as the original > restriction (the reason offload flag support was removed) would not affect > VPP. > > > > I think for 40GE driver to provide offload flag support in vector mode > > but not give indication of presence of VLAN tag is just wrong. This make the > offload flag support useless for VPP. > > > > Regards, > > John > > > > -----Original Message----- > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>] > > Sent: Wednesday, May 25, 2016 11:30 AM > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > Cc: vpp-dev; Zhang, Helin > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I see what you are getting at, Konstantin. The VPP init code does > > > not enable VLAN strip for Intel NICs as VLAN tag must be in the > > > packet for sub-interface lookup by ethernet-input node. I agree if > > > we enable VLAN tag strip for the i40e driver, we can get around this > > > problem > > but then all packets will be considered as received on the main interface. > > > > I see... > > As far as I know, when VLAN stripping is disabled, i40e RXD doesn't contain > information does that packet contain a VLAN or not. > > So, if enabling vlan stripping is not an option for you guys, then I > > don't see any other way on i40e to recognise is that VLAN packet or not, > except then parse the packet in SW. > > Helin, please correct me here, if I am missing something here. > > Thanks > > Konstantin > > > > > > > > Regards, > > > John > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>] > > > Sent: Wednesday, May 25, 2016 10:35 AM > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > Since this is the XL710 40GE NIC, I suppose the DPDK driver involved > would be the i40e driver and not ixgbe for 10GE NICs. > > > > > > Yes, I understand that you are facing problem on i40e, not ixgbe. > > > And the problem is that for i40e PKT_RX_VLAN_PKT flag is not set in > mbuf->ol_flags, correct? > > > That's why I asked: are you running it with > rte_eth_conf.rxmode.hw_vlan_strip==0 or not? > > > If yes, you can try with rte_eth_conf.rxmode.hw_vlan_strip=1 and see > would it help you. > > > > > > > > > > > As explained before, ixgbe driver had the inverse patch added. It > > > > does recognize VLAN tag in the packet and set PKT_RX_VLAN_PKT > offload flag properly: > > > > > > That patch has nothing to do with PKT_RX_VLAN_PKT and i40e. > > > So I don't think it is related to that problem at all. > > > Konstantin > > > > > > > > > > > 00:01:02:132370: dpdk-input > > > > TenGigabitEthernet5/0/0 rx queue 0 > > > > buffer 0x44cff: current data 0, length 96, free-list 0, totlen-nifb 0, trace > 0x0 > > > > PKT MBUF: port 3, nb_segs 1, pkt_len 96 > > > > buf_len 2176, data_len 96, ol_flags 0x1, > > > > packet_type 0x210 > > > > Packet Offload Flags > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > Packet Types > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > headers > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > UDP: 28.0.0.1 -> 29.0.0.254 > > > > tos 0x00, ttl 64, length 78, checksum 0x40a1 > > > > fragment id 0x0000 > > > > UDP: 63 -> 63 > > > > length 58, checksum 0x5b76 > > > > 00:01:02:132391: ethernet-input > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > 00:01:02:132408: error-drop > > > > ethernet-input: unknown vlan > > > > > > > > You can see from above the packet was passed to ethernet-input > > > > node and dropped because sub-interface for VLAN 1101 is not set up. > > > > > > > > Regards, > > > > John > > > > > > > > -----Original Message----- > > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>] > > > > Sent: Wednesday, May 25, 2016 4:46 AM > > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > Hi, > > > > I don’t think ptype recognition on ixgbe vRX is somehow related to that. > > > > Are you guys running FVL with > rte_eth_conf.rxmode.hw_vlan_strip==0? > > > > If so, can you set it to 1 and see would it help? > > > > Konstantin > > > > > > > > From: vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io> > > > > [mailto:vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io>] > > > > On Behalf Of John Lo (loj) > > > > Sent: Wednesday, May 25, 2016 1:56 AM > > > > To: Wiles, Keith; Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > I did put in a reverse patch, provided by Damjan, for this in fd.io<http://fd.io>: > > > > https://gerrit.fd.io/r/#/c/941/1/dpdk/dpdk-16.04_patches/0013-Reve > > > > rt -i xgbe-fix-packet-type-from-vector-Rx.patch > > > > > > > > I know this fixed the 10GE NIC behavior. Perhaps the 40GE NIC is using a > different DPDK driver? > > > > > > > > Regards, > > > > John > > > > > > > > From: Wiles, Keith [mailto:keith.wiles@intel.com<mailto:keith.wiles@intel.com>] > > > > Sent: Tuesday, May 24, 2016 8:30 PM > > > > To: John Lo (loj); Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > The commit ID is > > > > > > > > commit d9a2009a81089093645fea2e04b51dd37edf3e6f > > > > Author: Konstantin Ananyev <konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>> > > > > Date: Tue Mar 22 14:30:17 2016 +0000 > > > > > > > > ixgbe: fix packet type from vector Rx > > > > > > > > Current vector RX can't always set the packet_type properly. > > > > To be more specific: > > > > a) it never sets RTE_PTYPE_L2_ETHER > > > > b) it doesn't handle tunnel ipv4/ipv6 case correctly. > > > > c) it doesn't check is IXGBE_RXDADV_PKTTYPE_ETQF set or not. > > > > While a) is pretty easy to fix, b) and c) are not that > > > > straightforward > > > > in terms of SIMD ops (specially b). > > > > So far I wasn't able to make vRX support packet_type properly > > > > without > > > > noticeable performance loss. > > > > So for now, just remove that functionality from vector RX and > > > > update dev_supported_ptypes_get(). > > > > > > > > Fixes: 396254175854 ("mbuf: redefine packet type") > > > > > > > > Signed-off-by: Konstantin Ananyev > > > > <konstantin.ananyev@intel.com<mailto:konstantin.ananyev@intel.com>> > > > > Acked-by: Cunming Liang <cunming.liang@intel.com<mailto:cunming.liang@intel.com>> > > > > > > > > It appears the change was only down to the vector version in the RX > routine, could move to the none-vector version I guess. > > > > > > > > I would think adding a patch or adding the code to VPP DPDK init to use > the RX callback to setup things correctly. > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io>> on behalf of Keith Wiles > > > > <keith.wiles@intel.com<mailto:keith.wiles@intel.com>> > > > > Date: Tuesday, May 24, 2016 at 7:19 PM > > > > To: "John Lo (loj)" <loj@cisco.com<mailto:loj@cisco.com>>, Chandrasekar Kannan > > > > <ckannan@console.to<mailto:ckannan@console.to>> > > > > Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > Here is the starting thread: > > > > http://dpdk.org/ml/archives/dev/2016-April/037837.html > > > > > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io>> on behalf of Keith Wiles > > > > <keith.wiles@intel.com<mailto:keith.wiles@intel.com>> > > > > Date: Tuesday, May 24, 2016 at 7:16 PM > > > > To: "John Lo (loj)" <loj@cisco.com<mailto:loj@cisco.com>>, Chandrasekar Kannan > > > > <ckannan@console.to<mailto:ckannan@console.to>> > > > > Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > The flags in the rte_mbuf were changed a before 16.04, which > > > > removed the ptype testing and possible setting this flag. I thought a > patch to revert that change was done, did that not happen. > > > > > > > > The other solution is to add an option to DPDK to restore the > > > > previous behavior for this type of problem. The only other way is > > > > to have a installed in the PMD to setup the flags correctly for VPP. > > > > This would require adding a routine to VPP and setting the call > > > > back in the > > > driver. > > > > > > > > I really thought a patch was created, does anyone know? > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: "John Lo (loj)" <loj@cisco.com<mailto:loj@cisco.com>> > > > > Date: Tuesday, May 24, 2016 at 6:53 PM > > > > To: Chandrasekar Kannan <ckannan@console.to<mailto:ckannan@console.to>> > > > > Cc: Keith Wiles <keith.wiles@intel.com<mailto:keith.wiles@intel.com>>, vpp-dev > > > > <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > I believe the problem is due to 40GE DPDK driver not parsing VLAN > > > > packet properly and setting the packet offload flag > > > > PKT_RX_VLAN_PKT, causing dpdk-input node to send packet directly > > > > to ip4-input instead of ethernet-input. If this flag is set, then > > > > the packet will be passed to ethernet-input which will then parse > > > > the VLAN tag in the > > > packet to perform sub-interface lookup and find the start of IP > > > header in the packet properly. Following is an example trace of a correctly > marked packet: > > > > > > > > 23:11:09:855916: dpdk-input > > > > TenGigE0/0/0/0 rx queue 0 > > > > buffer 0x23f6c0: current data 0, length 118, free-list 0, > > > > totlen-nifb 0, trace 0x0 > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 118 > > > > buf_len 2176, data_len 118, ol_flags 0x1, > > > > packet_type 0x10 > > > > Packet Offload Flags > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > Packet Types > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > > > headers > > > > IP4: 74:a0:2f:e2:d5:14 -> 90:e2:ba:9f:8b:cc vlan 101 > > > > ICMP: 130.1.1.1 -> 130.1.1.2 > > > > tos 0x00, ttl 255, length 100, checksum 0xac4b > > > > fragment id 0x0948 > > > > ICMP echo_request checksum 0x900d > > > > > > > > Hoping someone familiar with the 40GE DPDK driver can comment > further. > > > > > > > > Regards, > > > > John > > > > > > > > From: Chandrasekar Kannan [mailto:ckannan@console.to<mailto:ckannan@console.to>] > > > > Sent: Tuesday, May 24, 2016 6:57 PM > > > > To: John Lo (loj) > > > > Cc: Wiles, Keith; vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > Yes. Subinterface has been configured like this ... > > > > > > > > vppctl set interface ip address del FortyGigabitEthernet83/0/0 all > > > > vppctl create sub FortyGigabitEthernet83/0/0 900 vppctl set > > > > interface ip address FortyGigabitEthernet83/0/0.900 6.0.0.1/24<http://6.0.0.1/24> > > > > vppctl set interface ip address FortyGigabitEthernet83/0/0.900 > > > > 7.0.0.1/24<http://7.0.0.1/24> vppctl set interface state FortyGigabitEthernet83/0/0 > > > > up vppctl set interface state FortyGigabitEthernet83/0/0.900 up > > > > vppctl set ip arp static > > > > FortyGigabitEthernet83/0/0.900 7.0.0.2 3c:fd:fe:9c:c3:88 > > > > > > > > -------------------------------------------------- > > > > Packet 4 > > > > > > > > 00:02:23:366879: dpdk-input > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > totlen-nifb 0, trace 0x3 > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > packet_type 0x291 > > > > Packet Types > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > without extension headers > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > UDP: 6.0.0.2 -> 7.0.0.2 > > > > tos 0x00, ttl 64, length 46, checksum 0x2dbc > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > UDP: 1024 -> 2001 > > > > length 26, checksum 0x0000 > > > > 00:02:23:366894: ip4-input-no-checksum > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.45.188 -> 6.0.0.2 > > > > version 0, header length 12 > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > 0xaf4d) > > > > fragment id 0x4500 offset 368, flags > > > > 00:02:23:366910: error-drop > > > > ip4-input: ip4 length > l2 length > > > > > > > > -------------------------------------------------------------- > > > > > > > > > > > > > > > > On Tue, May 24, 2016 at 3:12 PM, John Lo (loj) <loj@cisco.com<mailto:loj@cisco.com>> wrote: > > > > Was a sub-interface created with the expected VLAN tag 900? An > > > > example of creating a sub-interface and bring it up would be something > like: > > > > > > > > create sub GigabitEthernet1/0/0 900 set int state > > > > GigabitEthernet1/0/0 up set int state > > > > GigabitEthernet1/0/0.900 up > > > > > > > > Regards, > > > > John > > > > > > > > > > > > From: vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io> > > > > [mailto:vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io>] > > > > On Behalf Of Wiles, Keith > > > > Sent: Tuesday, May 24, 2016 5:59 PM > > > > To: Chandrasekar Kannan; vpp-dev > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > I am starting to think the unknown VLAN is not accounted for in the > processing of the input frame in the VPP DPDK interface code. > > > > vnet/vnet/devices/dpdk/node.c > > > > > > > > The version and header length are wrong in the output below, which > > > > leads me to believe the starting IPv4 header location is wrong. I > > > > have not parsed all of the code in node.c:dpdk_device_input() yet, > > > > but I do not think the l3_offset is being adjusted to the correct > > > offset???? > > > > > > > > Regards, > > > > Keith > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io<mailto:vpp-dev-bounces@lists.fd.io>> on behalf of Chandrasekar > > > > Kannan <ckannan@console.to<mailto:ckannan@console.to>> > > > > Date: Tuesday, May 24, 2016 at 3:40 PM > > > > To: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> > > > > Subject: [vpp-dev] VLAN packets dropped... ? > > > > > > > > Hi, > > > > > > > > I was trying to setup VLAN subinterfaces with VPP but it appears > > > > the packets are dropped at ingress. I'm tried with both > > > > pktgen-dpdk and trex for traffic generation just to rule out any > > > > corruption with > > > pkt generation. But both are producing the same errors with vpp. > > > > > > > > yaml file from trex... > > > > > > > > ------------------------------ > > > > [root@node04 scripts]# cat cap2/ipv4_vlan.yaml > > > > - duration : 10 > > > > generator : > > > > distribution : "seq" > > > > clients_start : "16.0.0.1" > > > > clients_end : "16.0.1.255" > > > > servers_start : "48.0.0.1" > > > > servers_end : "48.0.0.255" > > > > clients_per_gb : 201 > > > > min_clients : 101 > > > > dual_port_mask : "1.0.0.0" > > > > tcp_aging : 1 > > > > udp_aging : 1 > > > > mac : [0x0,0x0,0x0,0x1,0x0,0x00] > > > > vlan : { enable : 1 , vlan0 : 900 , vlan1 : 1100 } > > > > cap_info : > > > > - name: cap2/udp_64B.pcap > > > > cps : 1000.0 > > > > ipg : 10000 > > > > rtt : 10000 > > > > w : 4 > > > > limit : 20 > > > > [root@node04 scripts]# > > > > --------------------------------------------------------------- > > > > > > > > vpp trace... > > > > > > > > 00:02:21:576810: dpdk-input > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > totlen-nifb 0, trace 0x1 > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > packet_type 0x291 > > > > Packet Types > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > without extension headers > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > UDP: 16.0.1.0 -> 48.0.0.128 > > > > tos 0x00, ttl 64, length 46, checksum 0xf93f > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > UDP: 1024 -> 2001 > > > > length 26, checksum 0x0000 > > > > 00:02:21:576817: ip4-input-no-checksum > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.249.63 -> 16.0.1.0 > > > > version 0, header length 12 > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > 0xaf4d) > > > > fragment id 0x4500 offset 368, flags > > > > 00:02:21:576828: error-drop > > > > ip4-input: ip4 length > l2 length > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? 2016-06-01 20:02 ` John Lo (loj) @ 2016-06-01 20:21 ` Chandrasekar Kannan 0 siblings, 0 replies; 6+ messages in thread From: Chandrasekar Kannan @ 2016-06-01 20:21 UTC (permalink / raw) To: John Lo (loj) Cc: John Daley (johndale), Ananyev, Konstantin, Wiles, Keith, vpp-dev, Zhang, Helin, dev Awesome!. Thanks for the quick response John!. -Chandra On Wed, Jun 1, 2016 at 1:02 PM, John Lo (loj) <loj@cisco.com> wrote: > We will be adding a patch to the i40e driver that would check the packet > for presence of VLAN tag set the required PKT_RX_VLAN_PKT flag. I hope to > get it done by the end of this week. -John > > > > *From:* Chandrasekar Kannan [mailto:ckannan@console.to] > *Sent:* Wednesday, June 01, 2016 2:45 PM > *To:* John Daley (johndale) > *Cc:* John Lo (loj); Ananyev, Konstantin; Wiles, Keith; vpp-dev; Zhang, > Helin; dev@dpdk.org > *Subject:* Re: [vpp-dev] VLAN packets dropped... ? > > > > Just checking back on this thread to see where things are. Are we saying > this is a bug in dpdk or vpp ?. That part wasn't quite clear to me. > > > > -Chandra > > > > > > > > On Thu, May 26, 2016 at 3:56 PM, John Daley (johndale) <johndale@cisco.com> > wrote: > > John, > As just discussed, what you suggest was considered but there are other > apps depending a different behavior of the flag, so we thought the best > thing to do is deprecate it. That is part of what Olivier's patch discussed > in http://www.dpdk.org/ml/archives/dev/2016-May/038786.html does. Adding > a new ptype for VLAN tagged packets after the patch is accepted would allow > VPP to check if the ptype is supported by the PMD and if so, use it to > determine if the delivered packet actually has a VLAN tag in it. No need to > know if stripping is enabled/disabled. If the ptype is not supported, the > app would have check the packet in SW. > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of John Lo (loj) > > Sent: Thursday, May 26, 2016 11:11 AM > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Wiles, Keith > > <keith.wiles@intel.com>; Chandrasekar Kannan <ckannan@console.to> > > Cc: vpp-dev <vpp-dev@lists.fd.io>; Zhang, Helin <helin.zhang@intel.com>; > > dev@dpdk.org > > Subject: Re: [dpdk-dev] [vpp-dev] VLAN packets dropped... ? > > > > Hi Konstantin, > > > > Thanks for the link to DPDK discussion wrt this VLAN offload flag > > PKT_RX_VLAN_PKT. It seems the proposal was to deprecate > > PKT_RX_VLAN_PKT and introduce two new flags PKT_RX_VLAN_STRIPPED > > and PKT_RX_QINQ_STRIPPED. > > > > It would be a really good idea to keep PKT_RX_VLAN_PKT to indicate at > least > > one VLAN tag is present on the packet. For the case of i40e where its HW > > cannot detect VLAN tag if strip is not enabled, it should be reasonable > for the > > i40e DPDK driver software to make a check and set this flag. I would > think the > > overhead to check the 1st ethertype field in the MAC header against dot1q > > or dot1ad values in software be pretty minimal. > > > > Regards, > > John > > > > > > -----Original Message----- > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > Sent: Wednesday, May 25, 2016 3:23 PM > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > Cc: vpp-dev; Zhang, Helin; dev@dpdk.org > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > I suppose this has to do with what is expected usage of the > > > PKT_RX_VLAN_PKT offload flag. Is it set only for VLAN packets with the > > > VLAN stripped or should always be set if VLAN is/was present in the > > > received packet. It seems that different DPDK drivers are behaving > > differently which will make it really hard for VPP to take advantage of > NIC > > and driver offload capability to provide better performance. > > > > Yes, that's true ixgbe/igb from one side and i40e do raise > PKT_RX_VLAN_PKT > > in a different manner. > > There is an attempt to make it unified across all supported devices: > > http://dpdk.org/dev/patchwork/patch/12938/ > > > > Though, I am not sure it will help you with your issue. > > As I understand, for you the desired behaviour is: > > If it is a vlan packet, keep the packet intact (don't strip the vlan) > and raise > > PKT_RX_VLAN_PK inside mbuf->ol_flags. > > That's what ixgbe is doing with rte_eth_conf.rxmode.hw_vlan_strip==0. > > Correct? > > As far as I know, i40e HW doesn't provide such ability. > > i40e Receive HW Descriptor can only flag was VLAN tag stripped from the > > packet or not, but if stripping is disabled it wouldn't indicate in any > way is > > VLAN tag is present inside the packet or not. > > I am CC-ing it to dpdk.org in case I am missing something here. > > Probably someone knows a way to make it work in that way. > > Konstantin > > > > > > > > If VPP cannot rely on offload flags for VLAN so packets always have to > go > > through ethernet-input node, there is a performance cost. > > > For the 10GE case, before the inverse patch of the ixgbe driver, 10GE > > > Rx-vector path removed support of offload flag with DPDK 16.04 and so > > > ethernet-input node is always used. The 10GE IPv4 throughput rate > > > dropped from 6.17MPPSx2 to 4.92MPPSx2 for bidirectional traffic (2 > > > streams each with a single IP address as destination) on a single core > > > on my server. Konstantin suggested at that time to use scalar mode > which > > does support offload flags properly. The scalar mode did by-pass > ethernet- > > input and provided throughput of 5.72MPPS. We ended up inverse patched > > the ixgbe driver to restore vector mode offload flag support as the > original > > restriction (the reason offload flag support was removed) would not > affect > > VPP. > > > > > > I think for 40GE driver to provide offload flag support in vector mode > > > but not give indication of presence of VLAN tag is just wrong. This > make the > > offload flag support useless for VPP. > > > > > > Regards, > > > John > > > > > > -----Original Message----- > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > Sent: Wednesday, May 25, 2016 11:30 AM > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > Cc: vpp-dev; Zhang, Helin > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > I see what you are getting at, Konstantin. The VPP init code does > > > > not enable VLAN strip for Intel NICs as VLAN tag must be in the > > > > packet for sub-interface lookup by ethernet-input node. I agree if > > > > we enable VLAN tag strip for the i40e driver, we can get around this > > > > problem > > > but then all packets will be considered as received on the main > interface. > > > > > > I see... > > > As far as I know, when VLAN stripping is disabled, i40e RXD doesn't > contain > > information does that packet contain a VLAN or not. > > > So, if enabling vlan stripping is not an option for you guys, then I > > > don't see any other way on i40e to recognise is that VLAN packet or > not, > > except then parse the packet in SW. > > > Helin, please correct me here, if I am missing something here. > > > Thanks > > > Konstantin > > > > > > > > > > > Regards, > > > > John > > > > > > > > -----Original Message----- > > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > > Sent: Wednesday, May 25, 2016 10:35 AM > > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > > Cc: vpp-dev > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > > > > Since this is the XL710 40GE NIC, I suppose the DPDK driver > involved > > would be the i40e driver and not ixgbe for 10GE NICs. > > > > > > > > Yes, I understand that you are facing problem on i40e, not ixgbe. > > > > And the problem is that for i40e PKT_RX_VLAN_PKT flag is not set in > > mbuf->ol_flags, correct? > > > > That's why I asked: are you running it with > > rte_eth_conf.rxmode.hw_vlan_strip==0 or not? > > > > If yes, you can try with rte_eth_conf.rxmode.hw_vlan_strip=1 and see > > would it help you. > > > > > > > > > > > > > > As explained before, ixgbe driver had the inverse patch added. It > > > > > does recognize VLAN tag in the packet and set PKT_RX_VLAN_PKT > > offload flag properly: > > > > > > > > That patch has nothing to do with PKT_RX_VLAN_PKT and i40e. > > > > So I don't think it is related to that problem at all. > > > > Konstantin > > > > > > > > > > > > > > 00:01:02:132370: dpdk-input > > > > > TenGigabitEthernet5/0/0 rx queue 0 > > > > > buffer 0x44cff: current data 0, length 96, free-list 0, > totlen-nifb 0, trace > > 0x0 > > > > > PKT MBUF: port 3, nb_segs 1, pkt_len 96 > > > > > buf_len 2176, data_len 96, ol_flags 0x1, > > > > > packet_type 0x210 > > > > > Packet Offload Flags > > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > > Packet Types > > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > headers > > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > > UDP: 28.0.0.1 -> 29.0.0.254 > > > > > tos 0x00, ttl 64, length 78, checksum 0x40a1 > > > > > fragment id 0x0000 > > > > > UDP: 63 -> 63 > > > > > length 58, checksum 0x5b76 > > > > > 00:01:02:132391: ethernet-input > > > > > IP4: 00:00:00:00:00:aa -> 90:e2:ba:6e:e7:dc vlan 1101 > > > > > 00:01:02:132408: error-drop > > > > > ethernet-input: unknown vlan > > > > > > > > > > You can see from above the packet was passed to ethernet-input > > > > > node and dropped because sub-interface for VLAN 1101 is not set up. > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > -----Original Message----- > > > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com] > > > > > Sent: Wednesday, May 25, 2016 4:46 AM > > > > > To: John Lo (loj); Wiles, Keith; Chandrasekar Kannan > > > > > Cc: vpp-dev > > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > Hi, > > > > > I don’t think ptype recognition on ixgbe vRX is somehow related to > that. > > > > > Are you guys running FVL with > > rte_eth_conf.rxmode.hw_vlan_strip==0? > > > > > If so, can you set it to 1 and see would it help? > > > > > Konstantin > > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > > > On Behalf Of John Lo (loj) > > > > > Sent: Wednesday, May 25, 2016 1:56 AM > > > > > To: Wiles, Keith; Chandrasekar Kannan > > > > > Cc: vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I did put in a reverse patch, provided by Damjan, for this in > fd.io: > > > > > https://gerrit.fd.io/r/#/c/941/1/dpdk/dpdk-16.04_patches/0013-Reve > > > > > rt -i xgbe-fix-packet-type-from-vector-Rx.patch > > > > > > > > > > I know this fixed the 10GE NIC behavior. Perhaps the 40GE NIC is > using a > > different DPDK driver? > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > From: Wiles, Keith [mailto:keith.wiles@intel.com] > > > > > Sent: Tuesday, May 24, 2016 8:30 PM > > > > > To: John Lo (loj); Chandrasekar Kannan > > > > > Cc: vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > The commit ID is > > > > > > > > > > commit d9a2009a81089093645fea2e04b51dd37edf3e6f > > > > > Author: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > > > Date: Tue Mar 22 14:30:17 2016 +0000 > > > > > > > > > > ixgbe: fix packet type from vector Rx > > > > > > > > > > Current vector RX can't always set the packet_type properly. > > > > > To be more specific: > > > > > a) it never sets RTE_PTYPE_L2_ETHER > > > > > b) it doesn't handle tunnel ipv4/ipv6 case correctly. > > > > > c) it doesn't check is IXGBE_RXDADV_PKTTYPE_ETQF set or not. > > > > > While a) is pretty easy to fix, b) and c) are not that > > > > > straightforward > > > > > in terms of SIMD ops (specially b). > > > > > So far I wasn't able to make vRX support packet_type properly > > > > > without > > > > > noticeable performance loss. > > > > > So for now, just remove that functionality from vector RX and > > > > > update dev_supported_ptypes_get(). > > > > > > > > > > Fixes: 396254175854 ("mbuf: redefine packet type") > > > > > > > > > > Signed-off-by: Konstantin Ananyev > > > > > <konstantin.ananyev@intel.com> > > > > > Acked-by: Cunming Liang <cunming.liang@intel.com> > > > > > > > > > > It appears the change was only down to the vector version in the RX > > routine, could move to the none-vector version I guess. > > > > > > > > > > I would think adding a patch or adding the code to VPP DPDK init > to use > > the RX callback to setup things correctly. > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > > > <keith.wiles@intel.com> > > > > > Date: Tuesday, May 24, 2016 at 7:19 PM > > > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > > > <ckannan@console.to> > > > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > > > > > > Here is the starting thread: > > > > > http://dpdk.org/ml/archives/dev/2016-April/037837.html > > > > > > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Keith Wiles > > > > > <keith.wiles@intel.com> > > > > > Date: Tuesday, May 24, 2016 at 7:16 PM > > > > > To: "John Lo (loj)" <loj@cisco.com>, Chandrasekar Kannan > > > > > <ckannan@console.to> > > > > > Cc: vpp-dev <vpp-dev@lists.fd.io> > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > The flags in the rte_mbuf were changed a before 16.04, which > > > > > removed the ptype testing and possible setting this flag. I > thought a > > patch to revert that change was done, did that not happen. > > > > > > > > > > The other solution is to add an option to DPDK to restore the > > > > > previous behavior for this type of problem. The only other way is > > > > > to have a installed in the PMD to setup the flags correctly for > VPP. > > > > > This would require adding a routine to VPP and setting the call > > > > > back in the > > > > driver. > > > > > > > > > > I really thought a patch was created, does anyone know? > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: "John Lo (loj)" <loj@cisco.com> > > > > > Date: Tuesday, May 24, 2016 at 6:53 PM > > > > > To: Chandrasekar Kannan <ckannan@console.to> > > > > > Cc: Keith Wiles <keith.wiles@intel.com>, vpp-dev > > > > > <vpp-dev@lists.fd.io> > > > > > Subject: RE: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I believe the problem is due to 40GE DPDK driver not parsing VLAN > > > > > packet properly and setting the packet offload flag > > > > > PKT_RX_VLAN_PKT, causing dpdk-input node to send packet directly > > > > > to ip4-input instead of ethernet-input. If this flag is set, then > > > > > the packet will be passed to ethernet-input which will then parse > > > > > the VLAN tag in the > > > > packet to perform sub-interface lookup and find the start of IP > > > > header in the packet properly. Following is an example trace of a > correctly > > marked packet: > > > > > > > > > > 23:11:09:855916: dpdk-input > > > > > TenGigE0/0/0/0 rx queue 0 > > > > > buffer 0x23f6c0: current data 0, length 118, free-list 0, > > > > > totlen-nifb 0, trace 0x0 > > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 118 > > > > > buf_len 2176, data_len 118, ol_flags 0x1, > > > > > packet_type 0x10 > > > > > Packet Offload Flags > > > > > PKT_RX_VLAN_PKT (0x0001) RX packet is a 802.1q VLAN packet > > > > > Packet Types > > > > > RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension > > > > > headers > > > > > IP4: 74:a0:2f:e2:d5:14 -> 90:e2:ba:9f:8b:cc vlan 101 > > > > > ICMP: 130.1.1.1 -> 130.1.1.2 > > > > > tos 0x00, ttl 255, length 100, checksum 0xac4b > > > > > fragment id 0x0948 > > > > > ICMP echo_request checksum 0x900d > > > > > > > > > > Hoping someone familiar with the 40GE DPDK driver can comment > > further. > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > From: Chandrasekar Kannan [mailto:ckannan@console.to] > > > > > Sent: Tuesday, May 24, 2016 6:57 PM > > > > > To: John Lo (loj) > > > > > Cc: Wiles, Keith; vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > Yes. Subinterface has been configured like this ... > > > > > > > > > > vppctl set interface ip address del FortyGigabitEthernet83/0/0 all > > > > > vppctl create sub FortyGigabitEthernet83/0/0 900 vppctl set > > > > > interface ip address FortyGigabitEthernet83/0/0.900 6.0.0.1/24 > > > > > vppctl set interface ip address FortyGigabitEthernet83/0/0.900 > > > > > 7.0.0.1/24 vppctl set interface state FortyGigabitEthernet83/0/0 > > > > > up vppctl set interface state FortyGigabitEthernet83/0/0.900 up > > > > > vppctl set ip arp static > > > > > FortyGigabitEthernet83/0/0.900 7.0.0.2 3c:fd:fe:9c:c3:88 > > > > > > > > > > -------------------------------------------------- > > > > > Packet 4 > > > > > > > > > > 00:02:23:366879: dpdk-input > > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > > totlen-nifb 0, trace 0x3 > > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > > packet_type 0x291 > > > > > Packet Types > > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > > without extension headers > > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > > UDP: 6.0.0.2 -> 7.0.0.2 > > > > > tos 0x00, ttl 64, length 46, checksum 0x2dbc > > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > > UDP: 1024 -> 2001 > > > > > length 26, checksum 0x0000 > > > > > 00:02:23:366894: ip4-input-no-checksum > > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.45.188 -> 6.0.0.2 > > > > > version 0, header length 12 > > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > > 0xaf4d) > > > > > fragment id 0x4500 offset 368, flags > > > > > 00:02:23:366910: error-drop > > > > > ip4-input: ip4 length > l2 length > > > > > > > > > > -------------------------------------------------------------- > > > > > > > > > > > > > > > > > > > > On Tue, May 24, 2016 at 3:12 PM, John Lo (loj) <loj@cisco.com> > wrote: > > > > > Was a sub-interface created with the expected VLAN tag 900? An > > > > > example of creating a sub-interface and bring it up would be > something > > like: > > > > > > > > > > create sub GigabitEthernet1/0/0 900 set int state > > > > > GigabitEthernet1/0/0 up set int state > > > > > GigabitEthernet1/0/0.900 up > > > > > > > > > > Regards, > > > > > John > > > > > > > > > > > > > > > From: vpp-dev-bounces@lists.fd.io > > > > > [mailto:vpp-dev-bounces@lists.fd.io] > > > > > On Behalf Of Wiles, Keith > > > > > Sent: Tuesday, May 24, 2016 5:59 PM > > > > > To: Chandrasekar Kannan; vpp-dev > > > > > Subject: Re: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > I am starting to think the unknown VLAN is not accounted for in the > > processing of the input frame in the VPP DPDK interface code. > > > > > vnet/vnet/devices/dpdk/node.c > > > > > > > > > > The version and header length are wrong in the output below, which > > > > > leads me to believe the starting IPv4 header location is wrong. I > > > > > have not parsed all of the code in node.c:dpdk_device_input() yet, > > > > > but I do not think the l3_offset is being adjusted to the correct > > > > offset???? > > > > > > > > > > Regards, > > > > > Keith > > > > > > > > > > > > > > > From: <vpp-dev-bounces@lists.fd.io> on behalf of Chandrasekar > > > > > Kannan <ckannan@console.to> > > > > > Date: Tuesday, May 24, 2016 at 3:40 PM > > > > > To: vpp-dev <vpp-dev@lists.fd.io> > > > > > Subject: [vpp-dev] VLAN packets dropped... ? > > > > > > > > > > Hi, > > > > > > > > > > I was trying to setup VLAN subinterfaces with VPP but it appears > > > > > the packets are dropped at ingress. I'm tried with both > > > > > pktgen-dpdk and trex for traffic generation just to rule out any > > > > > corruption with > > > > pkt generation. But both are producing the same errors with vpp. > > > > > > > > > > yaml file from trex... > > > > > > > > > > ------------------------------ > > > > > [root@node04 scripts]# cat cap2/ipv4_vlan.yaml > > > > > - duration : 10 > > > > > generator : > > > > > distribution : "seq" > > > > > clients_start : "16.0.0.1" > > > > > clients_end : "16.0.1.255" > > > > > servers_start : "48.0.0.1" > > > > > servers_end : "48.0.0.255" > > > > > clients_per_gb : 201 > > > > > min_clients : 101 > > > > > dual_port_mask : "1.0.0.0" > > > > > tcp_aging : 1 > > > > > udp_aging : 1 > > > > > mac : [0x0,0x0,0x0,0x1,0x0,0x00] > > > > > vlan : { enable : 1 , vlan0 : 900 , vlan1 : 1100 } > > > > > cap_info : > > > > > - name: cap2/udp_64B.pcap > > > > > cps : 1000.0 > > > > > ipg : 10000 > > > > > rtt : 10000 > > > > > w : 4 > > > > > limit : 20 > > > > > [root@node04 scripts]# > > > > > --------------------------------------------------------------- > > > > > > > > > > vpp trace... > > > > > > > > > > 00:02:21:576810: dpdk-input > > > > > FortyGigabitEthernet83/0/0 rx queue 0 > > > > > buffer 0x10e6b: current data 14, length 50, free-list 0, > > > > > totlen-nifb 0, trace 0x1 > > > > > PKT MBUF: port 0, nb_segs 1, pkt_len 64 > > > > > buf_len 2176, data_len 64, ol_flags 0x0, > > > > > packet_type 0x291 > > > > > Packet Types > > > > > RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet > > > > > RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or > > > > > without extension headers > > > > > RTE_PTYPE_L4_UDP (0x0200) UDP packet > > > > > IP4: 3c:fd:fe:9c:c3:88 -> 3c:fd:fe:9d:a0:20 vlan 900 > > > > > UDP: 16.0.1.0 -> 48.0.0.128 > > > > > tos 0x00, ttl 64, length 46, checksum 0xf93f > > > > > fragment id 0x0000, flags DONT_FRAGMENT > > > > > UDP: 1024 -> 2001 > > > > > length 26, checksum 0x0000 > > > > > 00:02:21:576817: ip4-input-no-checksum > > > > > IP6_HOP_BY_HOP_OPTIONS: 64.17.249.63 -> 16.0.1.0 > > > > > version 0, header length 12 > > > > > tos 0x84, ttl 0, length 2048, checksum 0x4000 (should be > > > > > 0xaf4d) > > > > > fragment id 0x4500 offset 368, flags > > > > > 00:02:21:576828: error-drop > > > > > ip4-input: ip4 length > l2 length > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-06-01 20:21 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAF2CpC=BdtFMHD_kfRD8udJX_uM2R86wKjhUqoO3PtYcAdP96Q@mail.gmail.com>
[not found] ` <64B165E8-8209-4495-87CF-8DDD71598CBA@intel.com>
[not found] ` <85484e414fc440cf84b93f9136df715f@XCH-RCD-013.cisco.com>
[not found] ` <CAF2CpCn-My7TVjT0t0tGc7+x8iWuiJiyiMBvq_fWCvVFU64sVg@mail.gmail.com>
[not found] ` <83cc2c6c2012497babde2b8fd79e49b3@XCH-RCD-013.cisco.com>
[not found] ` <404492D5-F6DD-4E11-9007-A665ED15DE55@intel.com>
[not found] ` <CE812E88-4D86-4FF9-8AFD-B5BFA8134881@intel.com>
[not found] ` <3C1BF1A6-D122-42A1-B924-E7C1F73C6B66@intel.com>
[not found] ` <ffc68866981b4e0cb8ced4e78f18f51b@XCH-RCD-013.cisco.com>
[not found] ` <2601191342CEEE43887BDE71AB97725836B5EB20@irsmsx105.ger.corp.intel.com>
[not found] ` <0225ffe827bc4c89acfa71593e145ec9@XCH-RCD-013.cisco.com>
[not found] ` <2601191342CEEE43887BDE71AB97725836B5ED1E@irsmsx105.ger.corp.intel.com>
[not found] ` <f5e5352cdebf4d789a917994a5ce66d4@XCH-RCD-013.cisco.com>
[not found] ` <2601191342CEEE43887BDE71AB97725836B5EDA4@irsmsx105.ger.corp.intel.com>
[not found] ` <6a90755824944d70a5c239925eb25c73@XCH-RCD-013.cisco.com>
2016-05-25 19:23 ` [dpdk-dev] [vpp-dev] VLAN packets dropped... ? Ananyev, Konstantin
2016-05-26 18:11 ` John Lo (loj)
2016-05-26 22:56 ` John Daley (johndale)
2016-06-01 18:44 ` Chandrasekar Kannan
2016-06-01 20:02 ` John Lo (loj)
2016-06-01 20:21 ` Chandrasekar Kannan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).