Re: [dpdk-dev] [PATCH] net/ixgbe: Strip SR-IOV transparent VLANs in VF

[Robert Shearman <robertshearman@gmail.com>]

Hi Qi,

On 03/09/2018 12:45, Zhang, Qi Z wrote:
> Hi Robert:
> 
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of
>> robertshearman@gmail.com
>> Sent: Saturday, August 25, 2018 12:35 AM
>> To: dev@dpdk.org
>> Cc: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Ananyev, Konstantin
>> <konstantin.ananyev@intel.com>; Robert Shearman
>> <robert.shearman@att.com>
>> Subject: [dpdk-dev] [PATCH] net/ixgbe: Strip SR-IOV transparent VLANs in VF
>>
>> From: Robert Shearman <robert.shearman@att.com>
>>
>> SR-IOV VFs support "transparent" VLANs. Traffic from/to a VM associated with
>> a VF has a VLAN tag inserted/stripped in a manner intended to be totally
>> transparent to the VM.  On a Linux hypervisor the vlan can be specified by "ip
>> link set <device> vf <n> vlan <v>".
>> The VM VF driver is not configured to use any VLAN and the VM should never
>> see the transparent VLAN for that reason.  However, in practice these VLAN
>> headers are being received by the VM which discards the packets as that VLAN
>> is unknown to it.  The Linux kernel ixbge driver explicitly removes the VLAN in
>> this case (presumably due to the hardware not being able to do this) but the
>> DPDK driver does not.
> 
> I'm not quite understand this part.
> What does explicitly remove the VLAN means?,
> DPDK also discard unmatched VLAN and strip vlan if vlan_strip is enabled what is the gap?
> It will be better if you can give same examples

Sure. Typical use case for this is a hypervisor where it is necessary to 
provide L2 access into the guests, but there are insufficient, and so 
the hypervisor is using the PF and VFs are assigned to guests. In order 
to avoid having to configure each guest to use the VLAN and to not send 
any untagged traffic it is desirable to use transparent VLANs. For example:
Guest 1 = VLAN 10
Guest 2 = VLAN 20

ip link set eth0 vf 1 vlan 10
ip link set eth0 vf 2 vlan 20

Now this means that packets arriving tagged on the physical port should 
be delivered to the guest and arrive in the guest untagged. Similarly, 
packets transmitted untagged by the guest should gain a tag before they 
go out of the physical port. What you get when using the Linux VF ixgbe 
driver inside the VMs is exactly this since the driver knows that for 
this hardware the transparent stripping isn't done in hardware and is 
done inside the driver. What you get currently when using the DPDK VF 
ixgbe driver inside the VMs is that packets arrive tagged (e.g. with 
VLAN tag 10) and these are then dropped because the VM doesn't know 
about VLAN 10.

Transparent VLAN insertion works currently with both Linux and DPDK VF 
drivers.

> 
>>
>> This patch mirrors the kernel driver behaviour by removing the VLAN on the VF
>> side. This is done by checking the VLAN in the VFTA, where the hypervisor will
>> have set the bit in the VFTA corresponding to the VLAN if transparent VLANs
>> were being used for the VF. If the VLAN is set in the VFTA then it is known that
>> it's a transparent VLAN case and so the VLAN is stripped from the mbuf.
> 
> This is missing leading.
>  From your code, I only saw vlan flag in ol_flag is stripped, but not VLAN is stripped.
> I think vlan is always be stripped if we enable vlan strip on queue.

I think you're saying that the VLAN isn't removed if hardware RX VLAN 
stripping isn't configured. This is true, but might cost performance to 
cover this case too. If you're happy with that, then I can issue a V2 
with that addressed.

If you're suggesting that m->vlan_tci needs to be set to 0 when 
PKT_RX_VLAN is cleared from m->ol_flags, then I don't think that is 
necessary since my understanding is an application should only be 
looking at m->vlan_tci if m->ol_flags has PKT_RX_VLAN set.

> 
>> To
>> limit any potential performance impact on the PF data path, the RX path is split
>> into PF and VF versions with the transparent VLAN stripping only done in the
>> VF path. Measurements with our application show ~2% performance hit for
>> the VF case and none for the PF case.
>>
> 
> ...
> 
>> +/*
>> + * Filter out unknown vlans resulting from use of transparent vlan.
>> + *
>> + * When a VF is configured to use transparent vlans then the VF can
>> + * see this VLAN being set in the packet, meaning that the transparent
>> + * property isn't preserved. Furthermore, when the VF is used in a
>> + * guest VM then there's no way of knowing for sure that transparent
>> + * VLAN is in use and what tag value has been configured. So work
>> + * around this by removing the VLAN flag if the VF isn't interested in
>> + * the VLAN tag.
>> + */
>> +static inline void
>> +ixgbevf_trans_vlan_sw_filter_hdr(struct rte_mbuf *m,
>> +				 const struct ixgbe_vfta *vfta)
>> +{
>> +	if (m->ol_flags & PKT_RX_VLAN) {
>> +		uint16_t vlan = m->vlan_tci & 0xFFF;
>> +
>> +		if (!ixgbe_vfta_is_vlan_set(vfta, vlan))
>> +			m->ol_flags &= ~PKT_RX_VLAN;
>> +	}
>> +}
>> +
> 
> Ideally all driver's behavior should be consistent with the same configure.
> if "transparent vlan" looks like a general feature, it may not only bind to VF or even just ixgbevf.  (what about i40evf?)
> Otherwise, it should be handled in application , but not the driver.

It's a general feature, but the implementation is specific to a driver. 
I believe that this is handled in hardware on i40e, but this is just 
based on the there being no special handling of this case in the RX path 
in the Linux i40e VF driver.

Furthermore, transparent VLANs implemented in the application would just 
be called "VLANs" :-) More specifically, the application running in the 
guest cannot know what has been configured for the VF in the hypervisor 
in a driver-independent manner, or whether the hardware has in fact 
transparently removed the VLAN already (as may be the case for i40e).

> 
> ...
> 
>> +		ixgbe_unknown_vlan_sw_filter_hdr(rx_pkts[pos + 3], vfta, rxq);
> 
> Where is ixgbe_unknown_vlan_sw_filter_hdr be defined? I saw it is only be used in ixgbe_rxtx_vec_neon.c, so assume there will be a compile error on that platform?

Good catch. I don't have the ability to compile for that platform, and 
missed the rename I did during development. Will fix in V2.

Thanks,
Rob