[RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
not very well documented. Even the documentation that does exist appears
contradictory.

For example, the doxygen docs for the mbuf flag
RTE_MBUF_F_RX_QINQ_STRIPPED says:

  "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
  is unset, only the outer VLAN is removed from packet data,..."

but the docs for RTE_MBUF_F_RX_QINQ says:

  "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
  headers have been stripped from mbuf data, ..."

Without a good definition of what the correct behaviour is, it's not
possible to assess and ensure conformance across drivers. Update the
documentation for NIC features, ethdev and mbuf library to all report
the same information: that VLAN strip feature is stripping one flag, and
QinQ strip feature is removing two.

Summary of VLAN and QinQ stripping behaviour as reported in docs after
this patch:

+-------------------+----------------------+----------------------------+
| Input Traffic     | VLAN-strip on        | QinQ strip on              |
+===================+======================+============================+
| Single VLAN pkts  | Tag in vlan_tci      | Tag in vlan_tci            |
+-------------------+----------------------+----------------------------+
| Double VLAN pkts  | Outer tag in vlan_tci| Outer tag in vlan_tci_outer|
|                   |                      | Inner tag in vlan_tci      |
+-------------------+----------------------+----------------------------+

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 doc/guides/nics/features.rst | 21 +++++++++++++++++++++
 lib/ethdev/rte_ethdev.h      | 20 ++++++++++++++++++++
 lib/mbuf/rte_mbuf_core.h     | 36 ++++++++++++++++++++----------------
 3 files changed, 61 insertions(+), 16 deletions(-)

diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
index a075c057ec..c57ea8e08f 100644
--- a/doc/guides/nics/features.rst
+++ b/doc/guides/nics/features.rst
@@ -483,6 +483,11 @@ VLAN offload
 ------------
 
 Supports VLAN offload to hardware.
+This includes both VLAN stripping on Rx and VLAN insertion on Tx.
+
+On Rx VLAN strip always strips one VLAN tag if available.
+If multiple VLAN tags are present, it strips the outer tag.
+The stripped VLAN TCI is saved in mbuf->vlan_tci.
 
 * **[uses]       rte_eth_rxconf,rte_eth_rxmode**: ``offloads:RTE_ETH_RX_OFFLOAD_VLAN_STRIP,RTE_ETH_RX_OFFLOAD_VLAN_FILTER,RTE_ETH_RX_OFFLOAD_VLAN_EXTEND``.
 * **[uses]       rte_eth_txconf,rte_eth_txmode**: ``offloads:RTE_ETH_TX_OFFLOAD_VLAN_INSERT``.
@@ -501,6 +506,22 @@ QinQ offload
 ------------
 
 Supports QinQ (queue in queue) offload.
+This includes both QinQ stripping on Rx and QinQ insertion on Tx.
+
+On Rx, QinQ strip strips two VLAN tags if present.
+If only one tag is present, it behaves as VLAN strip.
+Specifying both VLAN strip and QinQ strip is equivalent to QinQ strip alone.
+
+Summary of VLAN and QinQ stripping behavior:
+
++----------------------+----------------------+------------------------------+
+| Input Traffic        | VLAN-strip on        | QinQ strip on                |
++======================+======================+==============================+
+| Single VLAN packets  | Tag in vlan_tci      | Tag in vlan_tci              |
++----------------------+----------------------+------------------------------+
+| Double VLAN packets  | Outer tag in vlan_tci| Outer tag in vlan_tci_outer  |
+|                      |                      | Inner tag in vlan_tci        |
++----------------------+----------------------+------------------------------+
 
 * **[uses]     rte_eth_rxconf,rte_eth_rxmode**: ``offloads:RTE_ETH_RX_OFFLOAD_QINQ_STRIP``.
 * **[uses]     rte_eth_txconf,rte_eth_txmode**: ``offloads:RTE_ETH_TX_OFFLOAD_QINQ_INSERT``.
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index f9fb6ae549..ccf0cf5e30 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -1552,11 +1552,31 @@ struct rte_eth_conf {
 /**
  * Rx offload capabilities of a device.
  */
+/**
+ * VLAN strip offload.
+ *
+ * When enabled, strips one VLAN tag if available.
+ * If multiple VLAN tags are present, it strips the outer tag.
+ * The stripped VLAN TCI is saved in mbuf->vlan_tci and RTE_MBUF_F_RX_VLAN_STRIPPED flag is set.
+ */
 #define RTE_ETH_RX_OFFLOAD_VLAN_STRIP       RTE_BIT64(0)
 #define RTE_ETH_RX_OFFLOAD_IPV4_CKSUM       RTE_BIT64(1)
 #define RTE_ETH_RX_OFFLOAD_UDP_CKSUM        RTE_BIT64(2)
 #define RTE_ETH_RX_OFFLOAD_TCP_CKSUM        RTE_BIT64(3)
 #define RTE_ETH_RX_OFFLOAD_TCP_LRO          RTE_BIT64(4)
+/**
+ * QinQ strip offload.
+ *
+ * When enabled, strips two VLAN tags if present.
+ * If only one tag is present, it behaves as VLAN strip.
+ * The stripped VLAN TCIs are saved in mbuf fields and appropriate RTE_MBUF_F_RX_* flags are set.
+ *
+ * For single VLAN packets: Tag is saved in mbuf->vlan_tci (same as VLAN strip)
+ * For double VLAN packets: Outer tag is saved in mbuf->vlan_tci_outer,
+ *                          Inner tag is saved in mbuf->vlan_tci
+ *
+ * Note: Specifying both VLAN strip and QinQ strip is equivalent to QinQ strip alone.
+ */
 #define RTE_ETH_RX_OFFLOAD_QINQ_STRIP       RTE_BIT64(5)
 #define RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM RTE_BIT64(6)
 #define RTE_ETH_RX_OFFLOAD_MACSEC_STRIP     RTE_BIT64(7)
diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h
index a0df265b5d..23aed8ec69 100644
--- a/lib/mbuf/rte_mbuf_core.h
+++ b/lib/mbuf/rte_mbuf_core.h
@@ -63,10 +63,17 @@ extern "C" {
 #define RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD (1ULL << 5)
 
 /**
- * A vlan has been stripped by the hardware and its tci is saved in
- * mbuf->vlan_tci. This can only happen if vlan stripping is enabled
- * in the RX configuration of the PMD.
- * When RTE_MBUF_F_RX_VLAN_STRIPPED is set, RTE_MBUF_F_RX_VLAN must also be set.
+ * A vlan has been stripped by the hardware and its tci is saved in mbuf->vlan_tci.
+ * This can only happen if vlan or QinQ stripping is enabled in the RX configuration of the PMD.
+ *
+ * NOTE:
+ * - If VLAN stripping is enabled, but not QinQ, the tag stripped will be the outer
+ *   VLAN tag of a QinQ packet.
+ * - If QinQ stripping is enabled, then the outer VLAN tag is stripped and saved in
+ *   mbuf->vlan_tci_outer (indicated by the presence of flag @ref RTE_MBUF_F_RX_QINQ_STRIPPED),
+ *   while the inner VLAN tag is stripped and saved in mbuf->vlan_tci.
+ *
+ * When @ref RTE_MBUF_F_RX_VLAN_STRIPPED is set, @ref RTE_MBUF_F_RX_VLAN must also be set.
  */
 #define RTE_MBUF_F_RX_VLAN_STRIPPED (1ULL << 6)
 
@@ -113,19 +120,16 @@ extern "C" {
 #define RTE_MBUF_F_RX_FDIR_FLX      (1ULL << 14)
 
 /**
- * The outer VLAN has been stripped by the hardware and its TCI is
- * saved in mbuf->vlan_tci_outer.
- * This can only happen if VLAN stripping is enabled in the Rx
- * configuration of the PMD.
- * When RTE_MBUF_F_RX_QINQ_STRIPPED is set, the flags RTE_MBUF_F_RX_VLAN
- * and RTE_MBUF_F_RX_QINQ must also be set.
+ * Two VLANs have been stripped from the packet by hardware and are
+ * reported in the vlan_tci and vlan_tci_outer fields.
+ *
+ * When this flag is set:
+ * - The outer VLAN has been stripped by the hardware and it is saved in mbuf->vlan_tci_outer.
+ * - The inner VLAN has also been stripped by the hardware and it is saved in mbuf->vlan_tci.
  *
- * - If both RTE_MBUF_F_RX_QINQ_STRIPPED and RTE_MBUF_F_RX_VLAN_STRIPPED are
- *   set, the 2 VLANs have been stripped by the hardware and their TCIs are
- *   saved in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
- * - If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
- *   is unset, only the outer VLAN is removed from packet data, but both tci
- *   are saved in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
+ * When @ref RTE_MBUF_F_RX_QINQ_STRIPPED is set, the flags @ref RTE_MBUF_F_RX_VLAN,
+ * @ref RTE_MBUF_F_RX_VLAN_STRIPPED,
+ * and @ref RTE_MBUF_F_RX_QINQ must also be set.
  */
 #define RTE_MBUF_F_RX_QINQ_STRIPPED (1ULL << 15)
 
-- 
2.48.1

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Stephen Hemminger]

On Mon, 14 Jul 2025 14:30:14 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> not very well documented. Even the documentation that does exist appears
> contradictory.
> 
> For example, the doxygen docs for the mbuf flag
> RTE_MBUF_F_RX_QINQ_STRIPPED says:
> 
>   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
>   is unset, only the outer VLAN is removed from packet data,..."
> 
> but the docs for RTE_MBUF_F_RX_QINQ says:
> 
>   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
>   headers have been stripped from mbuf data, ..."
> 
> Without a good definition of what the correct behaviour is, it's not
> possible to assess and ensure conformance across drivers. Update the
> documentation for NIC features, ethdev and mbuf library to all report
> the same information: that VLAN strip feature is stripping one flag, and
> QinQ strip feature is removing two.
> 
> Summary of VLAN and QinQ stripping behaviour as reported in docs after
> this patch:
> 
> +-------------------+----------------------+----------------------------+
> | Input Traffic     | VLAN-strip on        | QinQ strip on              |
> +===================+======================+============================+
> | Single VLAN pkts  | Tag in vlan_tci      | Tag in vlan_tci            |
> +-------------------+----------------------+----------------------------+
> | Double VLAN pkts  | Outer tag in vlan_tci| Outer tag in vlan_tci_outer|
> |                   |                      | Inner tag in vlan_tci      |
> +-------------------+----------------------+----------------------------+
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
>  doc/guides/nics/features.rst | 21 +++++++++++++++++++++
>  lib/ethdev/rte_ethdev.h      | 20 ++++++++++++++++++++
>  lib/mbuf/rte_mbuf_core.h     | 36 ++++++++++++++++++++----------------
>  3 files changed, 61 insertions(+), 16 deletions(-)
> 
> diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
> index a075c057ec..c57ea8e08f 100644
> --- a/doc/guides/nics/features.rst
> +++ b/doc/guides/nics/features.rst
> @@ -483,6 +483,11 @@ VLAN offload
>  ------------
>  
>  Supports VLAN offload to hardware.
> +This includes both VLAN stripping on Rx and VLAN insertion on Tx.
> +
> +On Rx VLAN strip always strips one VLAN tag if available.
> +If multiple VLAN tags are present, it strips the outer tag.
> +The stripped VLAN TCI is saved in mbuf->vlan_tci.
>  
>  * **[uses]       rte_eth_rxconf,rte_eth_rxmode**: ``offloads:RTE_ETH_RX_OFFLOAD_VLAN_STRIP,RTE_ETH_RX_OFFLOAD_VLAN_FILTER,RTE_ETH_RX_OFFLOAD_VLAN_EXTEND``.
>  * **[uses]       rte_eth_txconf,rte_eth_txmode**: ``offloads:RTE_ETH_TX_OFFLOAD_VLAN_INSERT``.
> @@ -501,6 +506,22 @@ QinQ offload
>  ------------
>  
>  Supports QinQ (queue in queue) offload.
> +This includes both QinQ stripping on Rx and QinQ insertion on Tx.
> +
> +On Rx, QinQ strip strips two VLAN tags if present.
> +If only one tag is present, it behaves as VLAN strip.
> +Specifying both VLAN strip and QinQ strip is equivalent to QinQ strip alone.
> +
> +Summary of VLAN and QinQ stripping behavior:
> +
> ++----------------------+----------------------+------------------------------+
> +| Input Traffic        | VLAN-strip on        | QinQ strip on                |
> ++======================+======================+==============================+
> +| Single VLAN packets  | Tag in vlan_tci      | Tag in vlan_tci              |
> ++----------------------+----------------------+------------------------------+
> +| Double VLAN packets  | Outer tag in vlan_tci| Outer tag in vlan_tci_outer  |
> +|                      |                      | Inner tag in vlan_tci        |
> ++----------------------+----------------------+------------------------------+
>  
>  * **[uses]     rte_eth_rxconf,rte_eth_rxmode**: ``offloads:RTE_ETH_RX_OFFLOAD_QINQ_STRIP``.
>  * **[uses]     rte_eth_txconf,rte_eth_txmode**: ``offloads:RTE_ETH_TX_OFFLOAD_QINQ_INSERT``.
> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> index f9fb6ae549..ccf0cf5e30 100644
> --- a/lib/ethdev/rte_ethdev.h
> +++ b/lib/ethdev/rte_ethdev.h
> @@ -1552,11 +1552,31 @@ struct rte_eth_conf {
>  /**
>   * Rx offload capabilities of a device.
>   */
> +/**
> + * VLAN strip offload.
> + *
> + * When enabled, strips one VLAN tag if available.
> + * If multiple VLAN tags are present, it strips the outer tag.
> + * The stripped VLAN TCI is saved in mbuf->vlan_tci and RTE_MBUF_F_RX_VLAN_STRIPPED flag is set.
> + */
>  #define RTE_ETH_RX_OFFLOAD_VLAN_STRIP       RTE_BIT64(0)
>  #define RTE_ETH_RX_OFFLOAD_IPV4_CKSUM       RTE_BIT64(1)
>  #define RTE_ETH_RX_OFFLOAD_UDP_CKSUM        RTE_BIT64(2)
>  #define RTE_ETH_RX_OFFLOAD_TCP_CKSUM        RTE_BIT64(3)
>  #define RTE_ETH_RX_OFFLOAD_TCP_LRO          RTE_BIT64(4)
> +/**
> + * QinQ strip offload.
> + *
> + * When enabled, strips two VLAN tags if present.
> + * If only one tag is present, it behaves as VLAN strip.
> + * The stripped VLAN TCIs are saved in mbuf fields and appropriate RTE_MBUF_F_RX_* flags are set.
> + *
> + * For single VLAN packets: Tag is saved in mbuf->vlan_tci (same as VLAN strip)
> + * For double VLAN packets: Outer tag is saved in mbuf->vlan_tci_outer,
> + *                          Inner tag is saved in mbuf->vlan_tci
> + *
> + * Note: Specifying both VLAN strip and QinQ strip is equivalent to QinQ strip alone.
> + */
>  #define RTE_ETH_RX_OFFLOAD_QINQ_STRIP       RTE_BIT64(5)
>  #define RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM RTE_BIT64(6)
>  #define RTE_ETH_RX_OFFLOAD_MACSEC_STRIP     RTE_BIT64(7)
> diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h
> index a0df265b5d..23aed8ec69 100644
> --- a/lib/mbuf/rte_mbuf_core.h
> +++ b/lib/mbuf/rte_mbuf_core.h
> @@ -63,10 +63,17 @@ extern "C" {
>  #define RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD (1ULL << 5)
>  
>  /**
> - * A vlan has been stripped by the hardware and its tci is saved in
> - * mbuf->vlan_tci. This can only happen if vlan stripping is enabled
> - * in the RX configuration of the PMD.
> - * When RTE_MBUF_F_RX_VLAN_STRIPPED is set, RTE_MBUF_F_RX_VLAN must also be set.
> + * A vlan has been stripped by the hardware and its tci is saved in mbuf->vlan_tci.
> + * This can only happen if vlan or QinQ stripping is enabled in the RX configuration of the PMD.
> + *
> + * NOTE:
> + * - If VLAN stripping is enabled, but not QinQ, the tag stripped will be the outer
> + *   VLAN tag of a QinQ packet.
> + * - If QinQ stripping is enabled, then the outer VLAN tag is stripped and saved in
> + *   mbuf->vlan_tci_outer (indicated by the presence of flag @ref RTE_MBUF_F_RX_QINQ_STRIPPED),
> + *   while the inner VLAN tag is stripped and saved in mbuf->vlan_tci.
> + *
> + * When @ref RTE_MBUF_F_RX_VLAN_STRIPPED is set, @ref RTE_MBUF_F_RX_VLAN must also be set.
>   */
>  #define RTE_MBUF_F_RX_VLAN_STRIPPED (1ULL << 6)
>  
> @@ -113,19 +120,16 @@ extern "C" {
>  #define RTE_MBUF_F_RX_FDIR_FLX      (1ULL << 14)
>  
>  /**
> - * The outer VLAN has been stripped by the hardware and its TCI is
> - * saved in mbuf->vlan_tci_outer.
> - * This can only happen if VLAN stripping is enabled in the Rx
> - * configuration of the PMD.
> - * When RTE_MBUF_F_RX_QINQ_STRIPPED is set, the flags RTE_MBUF_F_RX_VLAN
> - * and RTE_MBUF_F_RX_QINQ must also be set.
> + * Two VLANs have been stripped from the packet by hardware and are
> + * reported in the vlan_tci and vlan_tci_outer fields.
> + *
> + * When this flag is set:
> + * - The outer VLAN has been stripped by the hardware and it is saved in mbuf->vlan_tci_outer.
> + * - The inner VLAN has also been stripped by the hardware and it is saved in mbuf->vlan_tci.
>   *
> - * - If both RTE_MBUF_F_RX_QINQ_STRIPPED and RTE_MBUF_F_RX_VLAN_STRIPPED are
> - *   set, the 2 VLANs have been stripped by the hardware and their TCIs are
> - *   saved in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
> - * - If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> - *   is unset, only the outer VLAN is removed from packet data, but both tci
> - *   are saved in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
> + * When @ref RTE_MBUF_F_RX_QINQ_STRIPPED is set, the flags @ref RTE_MBUF_F_RX_VLAN,
> + * @ref RTE_MBUF_F_RX_VLAN_STRIPPED,
> + * and @ref RTE_MBUF_F_RX_QINQ must also be set.
>   */
>  #define RTE_MBUF_F_RX_QINQ_STRIPPED (1ULL << 15)
>  

Since QINQ strip without VLAN strip would be undefined, it should be blocked
at ethdev layer. Something like:

diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index dd7c00bc94..0d51c7cf82 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -4542,6 +4542,18 @@ rte_eth_dev_set_vlan_offload(uint16_t port_id, int offload_mask)
 	if (mask == 0)
 		return ret;
 
+	/*
+	 * QinQ offloading dtoes no make sense without also stripping
+	 * outer tag.
+	 */
+	if ((dev_offloads & RTE_ETH_QINQ_STRIP_OFFLOAD) &&
+	    !(dev_offloads & RTE_ETH_VLAN_STRIP_OFFLOAD)) {
+		RTE_ETHDEV_LOG_LINE(ERR,
+			    "Ethdev port_id=%u requested QinQ strip without VLAN strip",
+			    port_id);
+		return -EINVAL;
+	}
+
 	ret = rte_eth_dev_info_get(port_id, &dev_info);
 	if (ret != 0)
 		return ret;

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Mon, Jul 14, 2025 at 08:06:54AM -0700, Stephen Hemminger wrote:
> On Mon, 14 Jul 2025 14:30:14 +0100
> Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> > The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> > not very well documented. Even the documentation that does exist appears
> > contradictory.
> > 
> > For example, the doxygen docs for the mbuf flag
> > RTE_MBUF_F_RX_QINQ_STRIPPED says:
> > 
> >   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> >   is unset, only the outer VLAN is removed from packet data,..."
> > 
> > but the docs for RTE_MBUF_F_RX_QINQ says:
> > 
> >   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
> >   headers have been stripped from mbuf data, ..."
> > 
> > Without a good definition of what the correct behaviour is, it's not
> > possible to assess and ensure conformance across drivers. Update the
> > documentation for NIC features, ethdev and mbuf library to all report
> > the same information: that VLAN strip feature is stripping one flag, and
> > QinQ strip feature is removing two.
> > 
> > Summary of VLAN and QinQ stripping behaviour as reported in docs after
> > this patch:
> > 
> > +-------------------+----------------------+----------------------------+
> > | Input Traffic     | VLAN-strip on        | QinQ strip on              |
> > +===================+======================+============================+
> > | Single VLAN pkts  | Tag in vlan_tci      | Tag in vlan_tci            |
> > +-------------------+----------------------+----------------------------+
> > | Double VLAN pkts  | Outer tag in vlan_tci| Outer tag in vlan_tci_outer|
> > |                   |                      | Inner tag in vlan_tci      |
> > +-------------------+----------------------+----------------------------+
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> >  doc/guides/nics/features.rst | 21 +++++++++++++++++++++
> >  lib/ethdev/rte_ethdev.h      | 20 ++++++++++++++++++++
> >  lib/mbuf/rte_mbuf_core.h     | 36 ++++++++++++++++++++----------------
> >  3 files changed, 61 insertions(+), 16 deletions(-)
> > 
> > diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
> > index a075c057ec..c57ea8e08f 100644
> > --- a/doc/guides/nics/features.rst
> > +++ b/doc/guides/nics/features.rst
> > @@ -483,6 +483,11 @@ VLAN offload
> >  ------------
> >  
> >  Supports VLAN offload to hardware.
> > +This includes both VLAN stripping on Rx and VLAN insertion on Tx.
> > +
> > +On Rx VLAN strip always strips one VLAN tag if available.
> > +If multiple VLAN tags are present, it strips the outer tag.
> > +The stripped VLAN TCI is saved in mbuf->vlan_tci.
> >  
> >  * **[uses]       rte_eth_rxconf,rte_eth_rxmode**: ``offloads:RTE_ETH_RX_OFFLOAD_VLAN_STRIP,RTE_ETH_RX_OFFLOAD_VLAN_FILTER,RTE_ETH_RX_OFFLOAD_VLAN_EXTEND``.
> >  * **[uses]       rte_eth_txconf,rte_eth_txmode**: ``offloads:RTE_ETH_TX_OFFLOAD_VLAN_INSERT``.
> > @@ -501,6 +506,22 @@ QinQ offload
> >  ------------
> >  
> >  Supports QinQ (queue in queue) offload.
> > +This includes both QinQ stripping on Rx and QinQ insertion on Tx.
> > +
> > +On Rx, QinQ strip strips two VLAN tags if present.
> > +If only one tag is present, it behaves as VLAN strip.
> > +Specifying both VLAN strip and QinQ strip is equivalent to QinQ strip alone.
> > +
> > +Summary of VLAN and QinQ stripping behavior:
> > +
> > ++----------------------+----------------------+------------------------------+
> > +| Input Traffic        | VLAN-strip on        | QinQ strip on                |
> > ++======================+======================+==============================+
> > +| Single VLAN packets  | Tag in vlan_tci      | Tag in vlan_tci              |
> > ++----------------------+----------------------+------------------------------+
> > +| Double VLAN packets  | Outer tag in vlan_tci| Outer tag in vlan_tci_outer  |
> > +|                      |                      | Inner tag in vlan_tci        |
> > ++----------------------+----------------------+------------------------------+
> >  
> >  * **[uses]     rte_eth_rxconf,rte_eth_rxmode**: ``offloads:RTE_ETH_RX_OFFLOAD_QINQ_STRIP``.
> >  * **[uses]     rte_eth_txconf,rte_eth_txmode**: ``offloads:RTE_ETH_TX_OFFLOAD_QINQ_INSERT``.
> > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> > index f9fb6ae549..ccf0cf5e30 100644
> > --- a/lib/ethdev/rte_ethdev.h
> > +++ b/lib/ethdev/rte_ethdev.h
> > @@ -1552,11 +1552,31 @@ struct rte_eth_conf {
> >  /**
> >   * Rx offload capabilities of a device.
> >   */
> > +/**
> > + * VLAN strip offload.
> > + *
> > + * When enabled, strips one VLAN tag if available.
> > + * If multiple VLAN tags are present, it strips the outer tag.
> > + * The stripped VLAN TCI is saved in mbuf->vlan_tci and RTE_MBUF_F_RX_VLAN_STRIPPED flag is set.
> > + */
> >  #define RTE_ETH_RX_OFFLOAD_VLAN_STRIP       RTE_BIT64(0)
> >  #define RTE_ETH_RX_OFFLOAD_IPV4_CKSUM       RTE_BIT64(1)
> >  #define RTE_ETH_RX_OFFLOAD_UDP_CKSUM        RTE_BIT64(2)
> >  #define RTE_ETH_RX_OFFLOAD_TCP_CKSUM        RTE_BIT64(3)
> >  #define RTE_ETH_RX_OFFLOAD_TCP_LRO          RTE_BIT64(4)
> > +/**
> > + * QinQ strip offload.
> > + *
> > + * When enabled, strips two VLAN tags if present.
> > + * If only one tag is present, it behaves as VLAN strip.
> > + * The stripped VLAN TCIs are saved in mbuf fields and appropriate RTE_MBUF_F_RX_* flags are set.
> > + *
> > + * For single VLAN packets: Tag is saved in mbuf->vlan_tci (same as VLAN strip)
> > + * For double VLAN packets: Outer tag is saved in mbuf->vlan_tci_outer,
> > + *                          Inner tag is saved in mbuf->vlan_tci
> > + *
> > + * Note: Specifying both VLAN strip and QinQ strip is equivalent to QinQ strip alone.
> > + */
> >  #define RTE_ETH_RX_OFFLOAD_QINQ_STRIP       RTE_BIT64(5)
> >  #define RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM RTE_BIT64(6)
> >  #define RTE_ETH_RX_OFFLOAD_MACSEC_STRIP     RTE_BIT64(7)
> > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h
> > index a0df265b5d..23aed8ec69 100644
> > --- a/lib/mbuf/rte_mbuf_core.h
> > +++ b/lib/mbuf/rte_mbuf_core.h
> > @@ -63,10 +63,17 @@ extern "C" {
> >  #define RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD (1ULL << 5)
> >  
> >  /**
> > - * A vlan has been stripped by the hardware and its tci is saved in
> > - * mbuf->vlan_tci. This can only happen if vlan stripping is enabled
> > - * in the RX configuration of the PMD.
> > - * When RTE_MBUF_F_RX_VLAN_STRIPPED is set, RTE_MBUF_F_RX_VLAN must also be set.
> > + * A vlan has been stripped by the hardware and its tci is saved in mbuf->vlan_tci.
> > + * This can only happen if vlan or QinQ stripping is enabled in the RX configuration of the PMD.
> > + *
> > + * NOTE:
> > + * - If VLAN stripping is enabled, but not QinQ, the tag stripped will be the outer
> > + *   VLAN tag of a QinQ packet.
> > + * - If QinQ stripping is enabled, then the outer VLAN tag is stripped and saved in
> > + *   mbuf->vlan_tci_outer (indicated by the presence of flag @ref RTE_MBUF_F_RX_QINQ_STRIPPED),
> > + *   while the inner VLAN tag is stripped and saved in mbuf->vlan_tci.
> > + *
> > + * When @ref RTE_MBUF_F_RX_VLAN_STRIPPED is set, @ref RTE_MBUF_F_RX_VLAN must also be set.
> >   */
> >  #define RTE_MBUF_F_RX_VLAN_STRIPPED (1ULL << 6)
> >  
> > @@ -113,19 +120,16 @@ extern "C" {
> >  #define RTE_MBUF_F_RX_FDIR_FLX      (1ULL << 14)
> >  
> >  /**
> > - * The outer VLAN has been stripped by the hardware and its TCI is
> > - * saved in mbuf->vlan_tci_outer.
> > - * This can only happen if VLAN stripping is enabled in the Rx
> > - * configuration of the PMD.
> > - * When RTE_MBUF_F_RX_QINQ_STRIPPED is set, the flags RTE_MBUF_F_RX_VLAN
> > - * and RTE_MBUF_F_RX_QINQ must also be set.
> > + * Two VLANs have been stripped from the packet by hardware and are
> > + * reported in the vlan_tci and vlan_tci_outer fields.
> > + *
> > + * When this flag is set:
> > + * - The outer VLAN has been stripped by the hardware and it is saved in mbuf->vlan_tci_outer.
> > + * - The inner VLAN has also been stripped by the hardware and it is saved in mbuf->vlan_tci.
> >   *
> > - * - If both RTE_MBUF_F_RX_QINQ_STRIPPED and RTE_MBUF_F_RX_VLAN_STRIPPED are
> > - *   set, the 2 VLANs have been stripped by the hardware and their TCIs are
> > - *   saved in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
> > - * - If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> > - *   is unset, only the outer VLAN is removed from packet data, but both tci
> > - *   are saved in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
> > + * When @ref RTE_MBUF_F_RX_QINQ_STRIPPED is set, the flags @ref RTE_MBUF_F_RX_VLAN,
> > + * @ref RTE_MBUF_F_RX_VLAN_STRIPPED,
> > + * and @ref RTE_MBUF_F_RX_QINQ must also be set.
> >   */
> >  #define RTE_MBUF_F_RX_QINQ_STRIPPED (1ULL << 15)
> >  
> 
> Since QINQ strip without VLAN strip would be undefined, it should be blocked
> at ethdev layer. Something like:
> 
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> index dd7c00bc94..0d51c7cf82 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -4542,6 +4542,18 @@ rte_eth_dev_set_vlan_offload(uint16_t port_id, int offload_mask)
>  	if (mask == 0)
>  		return ret;
>  
> +	/*
> +	 * QinQ offloading dtoes no make sense without also stripping
> +	 * outer tag.
> +	 */
> +	if ((dev_offloads & RTE_ETH_QINQ_STRIP_OFFLOAD) &&
> +	    !(dev_offloads & RTE_ETH_VLAN_STRIP_OFFLOAD)) {
> +		RTE_ETHDEV_LOG_LINE(ERR,
> +			    "Ethdev port_id=%u requested QinQ strip without VLAN strip",
> +			    port_id);
> +		return -EINVAL;
> +	}
> +
>  	ret = rte_eth_dev_info_get(port_id, &dev_info);
>  	if (ret != 0)
>  		return ret;

Ok, that seems reasonable enough to enforce.

/Bruce

RE: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Monday, 14 July 2025 15.30
> 
> The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> not very well documented. Even the documentation that does exist appears
> contradictory.
> 
> For example, the doxygen docs for the mbuf flag
> RTE_MBUF_F_RX_QINQ_STRIPPED says:
> 
>   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
>   is unset, only the outer VLAN is removed from packet data,..."
> 
> but the docs for RTE_MBUF_F_RX_QINQ says:
> 
>   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
>   headers have been stripped from mbuf data, ..."
> 
> Without a good definition of what the correct behaviour is, it's not
> possible to assess and ensure conformance across drivers. Update the
> documentation for NIC features, ethdev and mbuf library to all report
> the same information: that VLAN strip feature is stripping one flag, and
> QinQ strip feature is removing two.
> 
> Summary of VLAN and QinQ stripping behaviour as reported in docs after
> this patch:
> 
> +-------------------+----------------------+----------------------------
> +
> | Input Traffic     | VLAN-strip on        | QinQ strip on
> |
> +===================+======================+============================
> +
> | Single VLAN pkts  | Tag in vlan_tci      | Tag in vlan_tci
> |
> +-------------------+----------------------+----------------------------
> +
> | Double VLAN pkts  | Outer tag in vlan_tci| Outer tag in
> vlan_tci_outer|
> |                   |                      | Inner tag in vlan_tci
> |
> +-------------------+----------------------+----------------------------
> +
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

I think your RFC is not a description of originally intended behavior.
However, I think your last thought in the previous discussion, speculating about the original intention, was correct:

The QINQ flag and the VLAN flag are completely independent.

The QINQ flag refers to EtherType 0x88a8 (QinQ) tags, and vlan_tci_outer holds the ID of such a tag.
It can be the outer tag of a double-tagged packet (i.e. the S-TAG of a packet with a C-TAG (C-TAG = Customer's VLAN tag)), or
the only tag of a single EtherType 0x88a8 tagged packet (i.e. the S-TAG of a customer packet with no VLAN tag).

The VLAN flag refers to EtherType 0x8100 (VLAN) tags, and vlan_tci holds the ID of such a tag.
It can be the only tag of a single EtherType 0x8100 tagged packet (i.e. a normal VLAN tag), or
the inner tag (i.e. the C-TAG) of a double-tagged packet with an outer EtherType 0x88a8 tag (the S-TAG).

On RX, RTE_MBUF_F_RX_QINQ (and vlan_tci_outer) should be set if the packet has an EtherType 0x88a8 tag (as the only tag, or as the outer tag).
If it was stripped, RTE_MBUF_F_RX_QINQ_STRIPPED should also be set.
Similarly on RX, RTE_MBUF_F_RX_VLAN (and vlan_tci) should be set if the packet has an EtherType 0x8100 tag (as the only tag, or after the QinQ tag).
If it was stripped, RTE_MBUF_F_RX_VLAN_STRIPPED should also be set.

Same goes for TX: If RTE_MBUF_F_TX_VLAN_INSERT is set, an EtherType 0x8100 tag should be inserted with the ID coming from vlan_tci.
Similarly for TX, if RTE_MBUF_F_TX_QINQ_INSERT is set, an EtherType 0x88a8 tag should be inserted with the ID coming from vlan_tci_outer.
Any HW inserted tags are inserted as the outermost tag, i.e. after the MAC addresses in the Ethernet header.
And the VLAN tag insertion (if any) happens before the QinQ tag insertion (if any).

Note:
With this behavior, VLAN Stacking (i.e. double-tagged packets using EtherType 0x8100 for both inner and outer tag) can only be partially hardware offloaded.
On RX, HW VLAN stripping will strip the outer VLAN tag to vlan_tci, and the application must move vlan_tci to vlan_tci_outer and manually strip the inner VLAN tag to vlan_tci.
On TX, the application must manually insert the inner tag from vlan_tci, and move vlan_tci_outer to vlan_tci, and HW VLAN insertion will insert the outer VLAN tag from vlan_tci.


Support for VLAN Stacking would be nice, but it should be added as a new feature, not through a doc update.

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Mon, Jul 14, 2025 at 06:33:43PM +0200, Morten Brørup wrote:
> > From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> > Sent: Monday, 14 July 2025 15.30
> > 
> > The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> > not very well documented. Even the documentation that does exist appears
> > contradictory.
> > 
> > For example, the doxygen docs for the mbuf flag
> > RTE_MBUF_F_RX_QINQ_STRIPPED says:
> > 
> >   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> >   is unset, only the outer VLAN is removed from packet data,..."
> > 
> > but the docs for RTE_MBUF_F_RX_QINQ says:
> > 
> >   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
> >   headers have been stripped from mbuf data, ..."
> > 
> > Without a good definition of what the correct behaviour is, it's not
> > possible to assess and ensure conformance across drivers. Update the
> > documentation for NIC features, ethdev and mbuf library to all report
> > the same information: that VLAN strip feature is stripping one flag, and
> > QinQ strip feature is removing two.
> > 
> > Summary of VLAN and QinQ stripping behaviour as reported in docs after
> > this patch:
> > 
> > +-------------------+----------------------+----------------------------
> > +
> > | Input Traffic     | VLAN-strip on        | QinQ strip on
> > |
> > +===================+======================+============================
> > +
> > | Single VLAN pkts  | Tag in vlan_tci      | Tag in vlan_tci
> > |
> > +-------------------+----------------------+----------------------------
> > +
> > | Double VLAN pkts  | Outer tag in vlan_tci| Outer tag in
> > vlan_tci_outer|
> > |                   |                      | Inner tag in vlan_tci
> > |
> > +-------------------+----------------------+----------------------------
> > +
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> 
> I think your RFC is not a description of originally intended behavior.
> However, I think your last thought in the previous discussion, speculating about the original intention, was correct:
> 
> The QINQ flag and the VLAN flag are completely independent.
> 
> The QINQ flag refers to EtherType 0x88a8 (QinQ) tags, and vlan_tci_outer holds the ID of such a tag.
> It can be the outer tag of a double-tagged packet (i.e. the S-TAG of a packet with a C-TAG (C-TAG = Customer's VLAN tag)), or
> the only tag of a single EtherType 0x88a8 tagged packet (i.e. the S-TAG of a customer packet with no VLAN tag).
> 
> The VLAN flag refers to EtherType 0x8100 (VLAN) tags, and vlan_tci holds the ID of such a tag.
> It can be the only tag of a single EtherType 0x8100 tagged packet (i.e. a normal VLAN tag), or
> the inner tag (i.e. the C-TAG) of a double-tagged packet with an outer EtherType 0x88a8 tag (the S-TAG).
> 
> On RX, RTE_MBUF_F_RX_QINQ (and vlan_tci_outer) should be set if the packet has an EtherType 0x88a8 tag (as the only tag, or as the outer tag).
> If it was stripped, RTE_MBUF_F_RX_QINQ_STRIPPED should also be set.
> Similarly on RX, RTE_MBUF_F_RX_VLAN (and vlan_tci) should be set if the packet has an EtherType 0x8100 tag (as the only tag, or after the QinQ tag).
> If it was stripped, RTE_MBUF_F_RX_VLAN_STRIPPED should also be set.
> 
> Same goes for TX: If RTE_MBUF_F_TX_VLAN_INSERT is set, an EtherType 0x8100 tag should be inserted with the ID coming from vlan_tci.
> Similarly for TX, if RTE_MBUF_F_TX_QINQ_INSERT is set, an EtherType 0x88a8 tag should be inserted with the ID coming from vlan_tci_outer.
> Any HW inserted tags are inserted as the outermost tag, i.e. after the MAC addresses in the Ethernet header.
> And the VLAN tag insertion (if any) happens before the QinQ tag insertion (if any).
> 
> Note:
> With this behavior, VLAN Stacking (i.e. double-tagged packets using EtherType 0x8100 for both inner and outer tag) can only be partially hardware offloaded.
> On RX, HW VLAN stripping will strip the outer VLAN tag to vlan_tci, and the application must move vlan_tci to vlan_tci_outer and manually strip the inner VLAN tag to vlan_tci.
> On TX, the application must manually insert the inner tag from vlan_tci, and move vlan_tci_outer to vlan_tci, and HW VLAN insertion will insert the outer VLAN tag from vlan_tci.
> 
> 
> Support for VLAN Stacking would be nice, but it should be added as a new feature, not through a doc update.
>
Yes, I suspect that some of that was the original intention, but I also
suspect it may not have been tied to tag ids. The documentation I
quoted earlier on the QinQ (non-strip) mbuf flag suggests the alternate
way of working as described above - which further confuses things.

However, the main concern for me now is what way to current drivers
actually implement this.

From chatting to Patrick on slack, it looks like the IOL is currently
working on testing the VLAN and QinQ features, so hopefully we will have
some idea shortly. To avoid major breakage, I would rather standardize on
what we currently have implemented, rather than shooting for an
"original idea" or a "best case" implementation.

/Bruce

RE: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Morten Brørup]

FYI:
Last time I looked, it seemed like the VLAN/QINQ "tag present" mbuf RX flags were only set along with the RX _STRIPPED flags, i.e. not set when stripping was not enabled.
Although setting the "tag present" flags (and the vlan_tci/vlan_tci_outer fields) without stripping would be nice to have, I think we should initially focus on stripping behavior.

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Tue, Jul 15, 2025 at 05:04:42PM +0200, Morten Brørup wrote:
> FYI:
> Last time I looked, it seemed like the VLAN/QINQ "tag present" mbuf RX flags were only set along with the RX _STRIPPED flags, i.e. not set when stripping was not enabled.
> Although setting the "tag present" flags (and the vlan_tci/vlan_tci_outer fields) without stripping would be nice to have, I think we should initially focus on stripping behavior.
> 
Yep. Again, it may be that HW doesn't support reporting a tag unless it is
stripped. :-(

/Bruce

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Dean Marx]

On Mon, Jul 14, 2025 at 9:30 AM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> not very well documented. Even the documentation that does exist appears
> contradictory.
>
> For example, the doxygen docs for the mbuf flag
> RTE_MBUF_F_RX_QINQ_STRIPPED says:
>
>   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
>   is unset, only the outer VLAN is removed from packet data,..."
>
> but the docs for RTE_MBUF_F_RX_QINQ says:
>
>   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
>   headers have been stripped from mbuf data, ..."
>
> Without a good definition of what the correct behaviour is, it's not
> possible to assess and ensure conformance across drivers. Update the
> documentation for NIC features, ethdev and mbuf library to all report
> the same information: that VLAN strip feature is stripping one flag, and
> QinQ strip feature is removing two.

I'm working on testing QinQ/VLAN stripping features across PMDs, and
so far I've found that our Intel devices are capable of QinQ
stripping, while our Mellanox/Broadcom devices are not. When QinQ
stripping is enabled on an Intel PMD, the test packet is received with
its outer VLAN layer stripped, but the inner VLAN layer remains
intact. Thus, the doxygen example is more accurate for what is
currently supported. I'm also running some tests on VLAN stripping
behavior, I'll update this thread with the results once these are
finished.

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 3047 bytes --]

On Mon, Jul 14, 2025 at 4:09 PM Dean Marx <dmarx@iol.unh.edu> wrote:

> On Mon, Jul 14, 2025 at 9:30 AM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> > not very well documented. Even the documentation that does exist appears
> > contradictory.
> >
> > For example, the doxygen docs for the mbuf flag
> > RTE_MBUF_F_RX_QINQ_STRIPPED says:
> >
> >   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> >   is unset, only the outer VLAN is removed from packet data,..."
> >
> > but the docs for RTE_MBUF_F_RX_QINQ says:
> >
> >   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
> >   headers have been stripped from mbuf data, ..."
> >
> > Without a good definition of what the correct behaviour is, it's not
> > possible to assess and ensure conformance across drivers. Update the
> > documentation for NIC features, ethdev and mbuf library to all report
> > the same information: that VLAN strip feature is stripping one flag, and
> > QinQ strip feature is removing two.
>
> I'm working on testing QinQ/VLAN stripping features across PMDs, and
> so far I've found that our Intel devices are capable of QinQ
> stripping, while our Mellanox/Broadcom devices are not. When QinQ
> stripping is enabled on an Intel PMD, the test packet is received with
> its outer VLAN layer stripped, but the inner VLAN layer remains
> intact. Thus, the doxygen example is more accurate for what is
> currently supported. I'm also running some tests on VLAN stripping
> behavior, I'll update this thread with the results once these are
> finished.
>

Thanks for checking the test results on the Intel NICs. That's interesting
that the behavior (only outer tag stripped) differs from the description in
the patch. But, I guess this question will become irrelevant if QinQ strip
without VLAN strip will become disallowed at the ethdev layer. So problem
solved there I guess!

Otherwise, it sounds like there are basically 3 valid "strip" configs a
port can be set to.

1. No strip
2. VLAN Strip
3. VLAN Strip + QinQ strip

And then, as Morten points out, there are various packet profiles which we
can send through these 3 configurations for your testcases.

Single VLAN: Ether() / Dot1Q(vlan=100, type=0x8100) / IP(dst="1.2.3.4") /
UDP(dport=1234, sport=4321)
Double VLAN: Ether() / Dot1Q(vlan=100, type=0x8100) / Dot1Q(vlan=200,
type=0x8100) / IP(dst="1.2.3.4") / UDP(dport=1234, sport=4321)
Single 0x88a8 tag only: Ether() / Dot1Q(vlan=100,
type=0x88a8 / IP(dst="1.2.3.4") / UDP(dport=1234, sport=4321)
Single VLAN + 1 0x88a8 tag: Ether() / Dot1Q(vlan=100, type=0x88a8)
/ Dot1Q(vlan=100, type=0x8100) / IP(dst="1.2.3.4") / UDP(dport=1234,
sport=4321)

Dean I gotta run for now but tomorrow let's see if we can define the
expected behavior for the intersection of these various stripping option
and packet profiles and pseudo code out some testcases based on that.

[-- Attachment #2: Type: text/html, Size: 3880 bytes --]

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Mon, Jul 14, 2025 at 04:09:11PM -0400, Dean Marx wrote:
> On Mon, Jul 14, 2025 at 9:30 AM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> > not very well documented. Even the documentation that does exist appears
> > contradictory.
> >
> > For example, the doxygen docs for the mbuf flag
> > RTE_MBUF_F_RX_QINQ_STRIPPED says:
> >
> >   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> >   is unset, only the outer VLAN is removed from packet data,..."
> >
> > but the docs for RTE_MBUF_F_RX_QINQ says:
> >
> >   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
> >   headers have been stripped from mbuf data, ..."
> >
> > Without a good definition of what the correct behaviour is, it's not
> > possible to assess and ensure conformance across drivers. Update the
> > documentation for NIC features, ethdev and mbuf library to all report
> > the same information: that VLAN strip feature is stripping one flag, and
> > QinQ strip feature is removing two.
> 
> I'm working on testing QinQ/VLAN stripping features across PMDs, and
> so far I've found that our Intel devices are capable of QinQ
> stripping, while our Mellanox/Broadcom devices are not. When QinQ
> stripping is enabled on an Intel PMD, the test packet is received with
> its outer VLAN layer stripped, but the inner VLAN layer remains
> intact. Thus, the doxygen example is more accurate for what is
> currently supported. I'm also running some tests on VLAN stripping
> behavior, I'll update this thread with the results once these are
> finished.

Thanks. Let me know how the testing otherwise goes. If only Intel NICs are
supporting QinQ, then it can't be that widely used a feature yet, so we may
be able to tweak things a bit if it diverges from what is expected.

I also wonder if the definition of expected behaviour is preventing other
NICs from implementing the feature?

Can you also check with VLAN stripping enabled? My biggest issue with the
behaviour description of QinQ strip only removing outer tag is that it
implies for QinQ traffoc that VLAN strip alone should strip inner tag without
removing outer. That doesn't make sense to me - and I'm not sure if NIC
hardware supports such features of removing inner headers while leaving
outer.

/Bruce

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 1718 bytes --]

On Tue, Jul 15, 2025 at 3:47 AM Bruce Richardson <bruce.richardson@intel.com>
wrote:

>
>
> Thanks. Let me know how the testing otherwise goes. If only Intel NICs are
> supporting QinQ, then it can't be that widely used a feature yet, so we may
> be able to tweak things a bit if it diverges from what is expected.
>

Yeah, there seems to be fairly broad support for VLAN strip, but much less
for QinQ strip. So, we can verify "correct" VLAN stripping behavior given
the various packet profiles we discussed previously like double vlan,
vlan + qinq, single qinq etc.

For QinQ strip like Dean said, most PMDs dont support this currently, with
Intel PMDs being an exception among the NICs we have at UNH. We took a look
at the DPDK networking drivers support table which reflects this (see the
QinQ offload row). https://doc.dpdk.org/guides/nics/overview.html

So, it looks like there are a few (ZTE DXDH is an example) that are also
supporting QinQ Offload, but not many right now.



> I also wonder if the definition of expected behaviour is preventing other
> NICs from implementing the feature?
>
> Can you also check with VLAN stripping enabled? My biggest issue with the
> behaviour description of QinQ strip only removing outer tag is that it
> implies for QinQ traffoc that VLAN strip alone should strip inner tag
> without
> removing outer.


Sure, we can check this case. across the NICs at UNH. I have a sync
wth Dean tomorrow morning for this, at the end of which we'll share back to
you what we have.


> That doesn't make sense to me - and I'm not sure if NIC
> hardware supports such features of removing inner headers while leaving
> outer.
>
> /Bruce
>

[-- Attachment #2: Type: text/html, Size: 2550 bytes --]

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Mon, Jul 14, 2025 at 04:09:11PM -0400, Dean Marx wrote:
> On Mon, Jul 14, 2025 at 9:30 AM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > The behaviour of VLAN tag stripping Rx offloads is unclear in DPDK, and
> > not very well documented. Even the documentation that does exist appears
> > contradictory.
> >
> > For example, the doxygen docs for the mbuf flag
> > RTE_MBUF_F_RX_QINQ_STRIPPED says:
> >
> >   "If RTE_MBUF_F_RX_QINQ_STRIPPED is set and RTE_MBUF_F_RX_VLAN_STRIPPED
> >   is unset, only the outer VLAN is removed from packet data,..."
> >
> > but the docs for RTE_MBUF_F_RX_QINQ says:
> >
> >   "If the flag RTE_MBUF_F_RX_QINQ_STRIPPED is also present, both VLANs
> >   headers have been stripped from mbuf data, ..."
> >
> > Without a good definition of what the correct behaviour is, it's not
> > possible to assess and ensure conformance across drivers. Update the
> > documentation for NIC features, ethdev and mbuf library to all report
> > the same information: that VLAN strip feature is stripping one flag, and
> > QinQ strip feature is removing two.
> 
> I'm working on testing QinQ/VLAN stripping features across PMDs, and
> so far I've found that our Intel devices are capable of QinQ
> stripping, while our Mellanox/Broadcom devices are not. When QinQ
> stripping is enabled on an Intel PMD, the test packet is received with
> its outer VLAN layer stripped, but the inner VLAN layer remains
> intact. Thus, the doxygen example is more accurate for what is
> currently supported. I'm also running some tests on VLAN stripping
> behavior, I'll update this thread with the results once these are
> finished.

I may follow up with you directly on this testing, because from what I've
found so far there are some issues with our current VLAN/QinQ support, which
requires multiple fixes - some of which aren't even merged yet and will
only be in the next release. For example, for correct reporting of a
stripped single VLAN tag, I believe patch [1] is necessary.

/Bruce

[1] https://patches.dpdk.org/project/dpdk/patch/20250714161050.289375-1-bruce.richardson@intel.com/

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Dean Marx]

I've created a list of test cases which I'm planning on implementing
in future QinQ testing. After talking with Bruce about some Intel test
results earlier, I've got a solid idea of what the expected behaviors
are in five of these, but I'm unsure about the other three. If anyone
is able to provide some insight as to what they believe these expected
behaviors should be, please let me know as it would be immensely
helpful. Thanks!

Vlan Strip ON and QinQ Strip OFF:
1. Single VLAN:
    Expected result: VLAN Stripped
2. Double VLAN:
    Expected result: Should strip outer VLAN
3. Single 0x88a8 tag only:
    Expected result: 0x88a8 tag stripped
4. Single VLAN + 1 0x88a8 tag:
    Expected result: 0x88a8 tag stripped, inner VLAN untouched

Vlan Strip ON and QinQ Strip ON:
1. Single VLAN:
    Expected result: VLAN stripped
2. Double VLAN:
    Expected result: Unknown (Outer VLAN stripped?)
3. Single 0x88a8 tag only:
    Expected result: Unknown (0x88a8 tag stripped?)
4. Single VLAN + 1 0x88a8 tag:
    Expected result: Unknown (0x88a8 tag stripped and VLAN tag stripped?)

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Wed, Jul 16, 2025 at 03:24:55PM -0400, Dean Marx wrote:
> I've created a list of test cases which I'm planning on implementing
> in future QinQ testing. After talking with Bruce about some Intel test
> results earlier, I've got a solid idea of what the expected behaviors
> are in five of these, but I'm unsure about the other three. If anyone
> is able to provide some insight as to what they believe these expected
> behaviors should be, please let me know as it would be immensely
> helpful. Thanks!
> 
> Vlan Strip ON and QinQ Strip OFF:
> 1. Single VLAN:
>     Expected result: VLAN Stripped
> 2. Double VLAN:
>     Expected result: Should strip outer VLAN
> 3. Single 0x88a8 tag only:
>     Expected result: 0x88a8 tag stripped
> 4. Single VLAN + 1 0x88a8 tag:
>     Expected result: 0x88a8 tag stripped, inner VLAN untouched

Technically, the result goes for the test case of "0x88a8 tag +
Single VLAN", since the 88a8 tag is first. :-)
The "Single VLAN + 0x88a8 Tag" would be a 5th test case if you
want to try it. Expected result should be again that outer tag is stripped.

As for the expected results you quote, those may or may not be expected
depending on whether hardware supports multiple VLAN tag eth-types or
configurable VLAN tag eth-types. From my testing, in at least some of our
NICs by default, they ONLY strip VLAN tags if they have a 0x8100 tag.
However, I would be surprised if that is not configurable.

> 
> Vlan Strip ON and QinQ Strip ON:
> 1. Single VLAN:
>     Expected result: VLAN stripped
> 2. Double VLAN:
>     Expected result: Unknown (Outer VLAN stripped?)
> 3. Single 0x88a8 tag only:
>     Expected result: Unknown (0x88a8 tag stripped?)
> 4. Single VLAN + 1 0x88a8 tag:
>     Expected result: Unknown (0x88a8 tag stripped and VLAN tag stripped?)

Again, the expected result would likely depend on what are configured
ethtypes for recognised VLANs.

/Bruce

RE: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Wednesday, 16 July 2025 21.47
> 
> On Wed, Jul 16, 2025 at 03:24:55PM -0400, Dean Marx wrote:
> > I've created a list of test cases which I'm planning on implementing
> > in future QinQ testing. After talking with Bruce about some Intel test
> > results earlier, I've got a solid idea of what the expected behaviors
> > are in five of these, but I'm unsure about the other three. If anyone
> > is able to provide some insight as to what they believe these expected
> > behaviors should be, please let me know as it would be immensely
> > helpful. Thanks!
> >
> > Vlan Strip ON and QinQ Strip OFF:
> > 1. Single VLAN:
> >     Expected result: VLAN Stripped
> > 2. Double VLAN:
> >     Expected result: Should strip outer VLAN
> > 3. Single 0x88a8 tag only:
> >     Expected result: 0x88a8 tag stripped
> > 4. Single VLAN + 1 0x88a8 tag:
> >     Expected result: 0x88a8 tag stripped, inner VLAN untouched
> 
> Technically, the result goes for the test case of "0x88a8 tag +
> Single VLAN", since the 88a8 tag is first. :-)

Agree.

> The "Single VLAN + 0x88a8 Tag" would be a 5th test case if you
> want to try it. Expected result should be again that outer tag is
> stripped.

NB: The 5th case would be an extremely weird packet, either due to a bug swapping the two tags, or it could be a packet in a VLAN tunnel carrying a QinQ packet with no S-TAG. In the latter case, triple-tagged packets (VLAN + S-TAG (0x88a8) + C-TAG (VLAN)) could occur on such a VLAN tunnel too.

Assuming we treat EtherType 0x8100 and 0x88a8 as being the same on RX, then yes to the above.

However, if we treat them differently, the results would be:

QinQ Strip OFF, Vlan Strip ON:
1. Single VLAN:
    Expected result: VLAN Stripped
2. Double VLAN:
    Expected result: Should strip outer VLAN
3. Single 0x88a8 tag only (S-TAG, no C-TAG):
    Expected result: Nothing stripped
4 (corrected). Outer 0x88a8 tag + inner VLAN (S-TAG + C-TAG):
    Expected result: 0x88a8 tag untouched! inner VLAN untouched or stripped?
5. Outer VLAN + inner 0x88a8 tag (weird packet):
    Expected result: Outer VLAN stripped, inner 0x88a8 tag untouched

> 
> As for the expected results you quote, those may or may not be expected
> depending on whether hardware supports multiple VLAN tag eth-types or
> configurable VLAN tag eth-types. From my testing, in at least some of
> our
> NICs by default, they ONLY strip VLAN tags if they have a 0x8100 tag.
> However, I would be surprised if that is not configurable.
> 
> >
> > Vlan Strip ON and QinQ Strip ON:
> > 1. Single VLAN:
> >     Expected result: VLAN stripped
> > 2. Double VLAN:
> >     Expected result: Unknown (Outer VLAN stripped?)
> > 3. Single 0x88a8 tag only:
> >     Expected result: Unknown (0x88a8 tag stripped?)
> > 4. Single VLAN + 1 0x88a8 tag:
> >     Expected result: Unknown (0x88a8 tag stripped and VLAN tag
> stripped?)
> 
> Again, the expected result would likely depend on what are configured
> ethtypes for recognised VLANs.

If we are considering application breakage as a decision parameter for this doc update, it would be better to log the results of the various test cases with the various NICs, to compare their behaviors, rather than pass/fail some "expected result".

> 
> /Bruce

I think we need to discuss VLAN/QinQ stripping in a broader context.

QinQ and VLAN stripping/insertion is very simple to implement in software, so the value of this feature itself (in an isolated context) is relatively low.

Hardware based QinQ and VLAN stripping/insertion has higher value when combined with other hardware offloads, such as GRO/TSO and IP defragmentation/fragmentation.

E.g. for GRO, the NIC might only be able to merge a bunch of received QinQ tagged TCP packets into one larger TCP packet if the NIC hardware has the ability to parse (and possibly strip) the QinQ tags.

Similarly, for TSO, the NIC must add the correct tag(s) to all the smaller TCP packets it transmits when splitting a large TCP packet. (And "correct" includes the EtherType of those tags. Inserting a VLAN tag instead of an S-TAG (0x88a8) and no C-TAG (VLAN) is not correct.)

The same goes for IP fragmentation/defragmentation. I don't know how well the IP fragmentation library supports VLAN/QinQ tagged packets.

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Dean Marx]

I've created a v1 of a QinQ test suite around the set of test cases
discussed earlier (which is not set in stone, and I expect it to
change significantly across many future versions.) The PASS/FAIL
values can be mostly disregarded in the context of this conversation,
but I've added logging to explain which packets are sent, and what
happened upon reception, which I hope will be more informative. After
running on mlx5/i40e drivers, I got the following results:

test_vlan_strip: QinQ strip OFF and VLAN strip ON
test_qinq_strip: QinQ strip ON and VLAN strip ON

i40e:
    test_qinq_strip (sent packet: Single VLAN): FAIL
      reason: VLAN tags found in packet when should have been
stripped: Ether / Dot1Q / 802.1q (0x1c) vlan 1280 / LLC / Raw /
Padding
    test_qinq_strip (sent packet: Stacked VLAN): FAIL
      reason: Expected one VLAN tag but found 2: Ether / Dot1Q / Dot1Q
/ 802.1q (0x1c) vlan 1280 / LLC / Raw / Padding
    test_qinq_strip (sent packet: Single S-VLAN): FAIL
      reason: VLAN tags found in packet when should have been
stripped: Ether / Dot1Q / 802.1q (??) vlan ?? / LLC / Raw / Padding
    test_qinq_strip (sent packet: QinQ): FAIL
      reason: VLAN tags found in packet when should have been
stripped: Ether / Dot1Q / Dot1AD / 802.1q (0x1c) vlan 1280 / LLC / Raw
/ Padding
    test_vlan_strip (sent packet: Single VLAN): PASS
      reason: VLAN tag stripped from packet
    test_vlan_strip (sent packet: Stacked VLAN): PASS
      reason: Received packet had outer VLAN stripped, with inner VLAN intact
    test_vlan_strip (sent packet: Single S-VLAN): PASS
      reason: S-VLAN tag stripped from packet
    test_vlan_strip (sent packet: QinQ): FAIL
      reason: Neither tag stripped

mlx5_core:
    test_qinq_strip: SKIP
      reason: Required capability
'{NicCapability.RX_OFFLOAD_QINQ_STRIP}' not found.
    test_vlan_strip (sent packet: Single VLAN): PASS
      reason: VLAN tags found in packet when should have been
stripped: Ether / 802.1q 24:8a:07:aa:83:ee > 0c:42:a1:54:25:86 (0x1c)
vlan 1280 / LLC / Raw / Padding
    test_vlan_strip (sent packet: Stacked VLAN): PASS
    test_vlan_strip (sent packet: Single S-VLAN): PASS
      reason: S-VLAN tag stripped from packet
    test_vlan_strip (sent packet: QinQ): FAIL
      reason: VLAN tag 0x8100 not found in packet: Ether / Dot1Q /
802.1q (??) vlan ?? / LLC / Raw / Padding

Hopefully this can give a clearer picture of the current QinQ/VLAN
strip behavior. If anyone wants to take a look at the suite I'll link
it here:

https://patchwork.dpdk.org/project/dpdk/patch/20250717205718.108826-2-dmarx@iol.unh.edu/

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Thu, Jul 17, 2025 at 05:03:13PM -0400, Dean Marx wrote:
> I've created a v1 of a QinQ test suite around the set of test cases
> discussed earlier (which is not set in stone, and I expect it to
> change significantly across many future versions.) The PASS/FAIL
> values can be mostly disregarded in the context of this conversation,
> but I've added logging to explain which packets are sent, and what
> happened upon reception, which I hope will be more informative. After
> running on mlx5/i40e drivers, I got the following results:
> 
> test_vlan_strip: QinQ strip OFF and VLAN strip ON
> test_qinq_strip: QinQ strip ON and VLAN strip ON
> 
> i40e:
>     test_qinq_strip (sent packet: Single VLAN): FAIL
>       reason: VLAN tags found in packet when should have been
> stripped: Ether / Dot1Q / 802.1q (0x1c) vlan 1280 / LLC / Raw /
> Padding
>     test_qinq_strip (sent packet: Stacked VLAN): FAIL
>       reason: Expected one VLAN tag but found 2: Ether / Dot1Q / Dot1Q
> / 802.1q (0x1c) vlan 1280 / LLC / Raw / Padding
>     test_qinq_strip (sent packet: Single S-VLAN): FAIL
>       reason: VLAN tags found in packet when should have been
> stripped: Ether / Dot1Q / 802.1q (??) vlan ?? / LLC / Raw / Padding
>     test_qinq_strip (sent packet: QinQ): FAIL
>       reason: VLAN tags found in packet when should have been
> stripped: Ether / Dot1Q / Dot1AD / 802.1q (0x1c) vlan 1280 / LLC / Raw
> / Padding
>     test_vlan_strip (sent packet: Single VLAN): PASS
>       reason: VLAN tag stripped from packet
>     test_vlan_strip (sent packet: Stacked VLAN): PASS
>       reason: Received packet had outer VLAN stripped, with inner VLAN intact
>     test_vlan_strip (sent packet: Single S-VLAN): PASS
>       reason: S-VLAN tag stripped from packet
>     test_vlan_strip (sent packet: QinQ): FAIL
>       reason: Neither tag stripped
>

Can you confirm exactly what is being sent in each case for the ethertype
of the VLAN tag? When you say single and stacked VLANs, that is VLANs with
0x8100 type, correct? Is single S-VLAN a tag with ethertype 0x88a8, and
QinQ packet a packet with one 0x88a8 and one 0x8100? No other type options,
e.g. 0x9100 were checked, right?

/Bruce

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Dean Marx]

On Fri, Jul 18, 2025 at 4:23 AM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Thu, Jul 17, 2025 at 05:03:13PM -0400, Dean Marx wrote:
> > I've created a v1 of a QinQ test suite around the set of test cases
> > discussed earlier (which is not set in stone, and I expect it to
> > change significantly across many future versions.) The PASS/FAIL
> > values can be mostly disregarded in the context of this conversation,
> > but I've added logging to explain which packets are sent, and what
> > happened upon reception, which I hope will be more informative. After
> > running on mlx5/i40e drivers, I got the following results:
> >
> > test_vlan_strip: QinQ strip OFF and VLAN strip ON
> > test_qinq_strip: QinQ strip ON and VLAN strip ON
> >
> > i40e:
> >     test_qinq_strip (sent packet: Single VLAN): FAIL
> >       reason: VLAN tags found in packet when should have been
> > stripped: Ether / Dot1Q / 802.1q (0x1c) vlan 1280 / LLC / Raw /
> > Padding
> >     test_qinq_strip (sent packet: Stacked VLAN): FAIL
> >       reason: Expected one VLAN tag but found 2: Ether / Dot1Q / Dot1Q
> > / 802.1q (0x1c) vlan 1280 / LLC / Raw / Padding
> >     test_qinq_strip (sent packet: Single S-VLAN): FAIL
> >       reason: VLAN tags found in packet when should have been
> > stripped: Ether / Dot1Q / 802.1q (??) vlan ?? / LLC / Raw / Padding
> >     test_qinq_strip (sent packet: QinQ): FAIL
> >       reason: VLAN tags found in packet when should have been
> > stripped: Ether / Dot1Q / Dot1AD / 802.1q (0x1c) vlan 1280 / LLC / Raw
> > / Padding
> >     test_vlan_strip (sent packet: Single VLAN): PASS
> >       reason: VLAN tag stripped from packet
> >     test_vlan_strip (sent packet: Stacked VLAN): PASS
> >       reason: Received packet had outer VLAN stripped, with inner VLAN intact
> >     test_vlan_strip (sent packet: Single S-VLAN): PASS
> >       reason: S-VLAN tag stripped from packet
> >     test_vlan_strip (sent packet: QinQ): FAIL
> >       reason: Neither tag stripped
> >
>
> Can you confirm exactly what is being sent in each case for the ethertype
> of the VLAN tag? When you say single and stacked VLANs, that is VLANs with
> 0x8100 type, correct? Is single S-VLAN a tag with ethertype 0x88a8, and
> QinQ packet a packet with one 0x88a8 and one 0x8100? No other type options,
> e.g. 0x9100 were checked, right?
>
> /Bruce

That's correct, single VLAN is one 0x8100 tag, stacked is two, single
S-VLAN is one 0x88a8, and QinQ is 0x88a8 and 0x8100. No other types
were tested in the stripping case

RE: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Morten Brørup]

> From: Dean Marx [mailto:dmarx@iol.unh.edu]
> Sent: Friday, 18 July 2025 15.18
> 
> On Fri, Jul 18, 2025 at 4:23 AM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Thu, Jul 17, 2025 at 05:03:13PM -0400, Dean Marx wrote:
> > > I've created a v1 of a QinQ test suite around the set of test cases
> > > discussed earlier (which is not set in stone, and I expect it to
> > > change significantly across many future versions.) The PASS/FAIL
> > > values can be mostly disregarded in the context of this conversation,
> > > but I've added logging to explain which packets are sent, and what
> > > happened upon reception, which I hope will be more informative. After
> > > running on mlx5/i40e drivers, I got the following results:
> > >
> > > test_vlan_strip: QinQ strip OFF and VLAN strip ON
> > > test_qinq_strip: QinQ strip ON and VLAN strip ON
> > >
> > > i40e:
> > >     test_qinq_strip (sent packet: Single VLAN): FAIL
> > >       reason: VLAN tags found in packet when should have been
> > > stripped: Ether / Dot1Q / 802.1q (0x1c) vlan 1280 / LLC / Raw /
> > > Padding
> > >     test_qinq_strip (sent packet: Stacked VLAN): FAIL
> > >       reason: Expected one VLAN tag but found 2: Ether / Dot1Q / Dot1Q
> > > / 802.1q (0x1c) vlan 1280 / LLC / Raw / Padding
> > >     test_qinq_strip (sent packet: Single S-VLAN): FAIL
> > >       reason: VLAN tags found in packet when should have been
> > > stripped: Ether / Dot1Q / 802.1q (??) vlan ?? / LLC / Raw / Padding
> > >     test_qinq_strip (sent packet: QinQ): FAIL
> > >       reason: VLAN tags found in packet when should have been
> > > stripped: Ether / Dot1Q / Dot1AD / 802.1q (0x1c) vlan 1280 / LLC / Raw
> > > / Padding
> > >     test_vlan_strip (sent packet: Single VLAN): PASS
> > >       reason: VLAN tag stripped from packet
> > >     test_vlan_strip (sent packet: Stacked VLAN): PASS
> > >       reason: Received packet had outer VLAN stripped, with inner VLAN
> intact
> > >     test_vlan_strip (sent packet: Single S-VLAN): PASS
> > >       reason: S-VLAN tag stripped from packet
> > >     test_vlan_strip (sent packet: QinQ): FAIL
> > >       reason: Neither tag stripped
> > >
> >
> > Can you confirm exactly what is being sent in each case for the ethertype
> > of the VLAN tag? When you say single and stacked VLANs, that is VLANs with
> > 0x8100 type, correct? Is single S-VLAN a tag with ethertype 0x88a8, and
> > QinQ packet a packet with one 0x88a8 and one 0x8100? No other type options,
> > e.g. 0x9100 were checked, right?
> >
> > /Bruce
> 
> That's correct, single VLAN is one 0x8100 tag, stacked is two, single
> S-VLAN is one 0x88a8, and QinQ is 0x88a8 and 0x8100. No other types
> were tested in the stripping case


Bruce,

It seems the drivers have the ability to set the EtherType of the Outer (and sometimes Inner) tag:
https://elixir.bootlin.com/dpdk/v25.07/source/lib/ethdev/rte_ethdev.h#L3752
https://elixir.bootlin.com/dpdk/v25.07/source/drivers/net/intel/e1000/igb_ethdev.c#L2739
https://elixir.bootlin.com/dpdk/v25.07/source/drivers/net/intel/i40e/i40e_ethdev.c#L4038

-Morten

Re: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Bruce Richardson]

On Mon, Jul 28, 2025 at 04:51:30PM +0200, Morten Brørup wrote:
> > From: Dean Marx [mailto:dmarx@iol.unh.edu]
> > Sent: Friday, 18 July 2025 15.18
> > 
> > On Fri, Jul 18, 2025 at 4:23 AM Bruce Richardson
> > <bruce.richardson@intel.com> wrote:
> > >
> > > On Thu, Jul 17, 2025 at 05:03:13PM -0400, Dean Marx wrote:
> > > > I've created a v1 of a QinQ test suite around the set of test cases
> > > > discussed earlier (which is not set in stone, and I expect it to
> > > > change significantly across many future versions.) The PASS/FAIL
> > > > values can be mostly disregarded in the context of this conversation,
> > > > but I've added logging to explain which packets are sent, and what
> > > > happened upon reception, which I hope will be more informative. After
> > > > running on mlx5/i40e drivers, I got the following results:
> > > >
> > > > test_vlan_strip: QinQ strip OFF and VLAN strip ON
> > > > test_qinq_strip: QinQ strip ON and VLAN strip ON
> > > >
> > > > i40e:
> > > >     test_qinq_strip (sent packet: Single VLAN): FAIL
> > > >       reason: VLAN tags found in packet when should have been
> > > > stripped: Ether / Dot1Q / 802.1q (0x1c) vlan 1280 / LLC / Raw /
> > > > Padding
> > > >     test_qinq_strip (sent packet: Stacked VLAN): FAIL
> > > >       reason: Expected one VLAN tag but found 2: Ether / Dot1Q / Dot1Q
> > > > / 802.1q (0x1c) vlan 1280 / LLC / Raw / Padding
> > > >     test_qinq_strip (sent packet: Single S-VLAN): FAIL
> > > >       reason: VLAN tags found in packet when should have been
> > > > stripped: Ether / Dot1Q / 802.1q (??) vlan ?? / LLC / Raw / Padding
> > > >     test_qinq_strip (sent packet: QinQ): FAIL
> > > >       reason: VLAN tags found in packet when should have been
> > > > stripped: Ether / Dot1Q / Dot1AD / 802.1q (0x1c) vlan 1280 / LLC / Raw
> > > > / Padding
> > > >     test_vlan_strip (sent packet: Single VLAN): PASS
> > > >       reason: VLAN tag stripped from packet
> > > >     test_vlan_strip (sent packet: Stacked VLAN): PASS
> > > >       reason: Received packet had outer VLAN stripped, with inner VLAN
> > intact
> > > >     test_vlan_strip (sent packet: Single S-VLAN): PASS
> > > >       reason: S-VLAN tag stripped from packet
> > > >     test_vlan_strip (sent packet: QinQ): FAIL
> > > >       reason: Neither tag stripped
> > > >
> > >
> > > Can you confirm exactly what is being sent in each case for the ethertype
> > > of the VLAN tag? When you say single and stacked VLANs, that is VLANs with
> > > 0x8100 type, correct? Is single S-VLAN a tag with ethertype 0x88a8, and
> > > QinQ packet a packet with one 0x88a8 and one 0x8100? No other type options,
> > > e.g. 0x9100 were checked, right?
> > >
> > > /Bruce
> > 
> > That's correct, single VLAN is one 0x8100 tag, stacked is two, single
> > S-VLAN is one 0x88a8, and QinQ is 0x88a8 and 0x8100. No other types
> > were tested in the stripping case
> 
> 
> Bruce,
> 
> It seems the drivers have the ability to set the EtherType of the Outer (and sometimes Inner) tag:
> https://elixir.bootlin.com/dpdk/v25.07/source/lib/ethdev/rte_ethdev.h#L3752
> https://elixir.bootlin.com/dpdk/v25.07/source/drivers/net/intel/e1000/igb_ethdev.c#L2739
> https://elixir.bootlin.com/dpdk/v25.07/source/drivers/net/intel/i40e/i40e_ethdev.c#L4038
> 
> -Morten
>
Thanks. Question is, does this help us to clarify the behaviour for these
tags? Based on the fact that the comment for the tag_type says that the
outer is the same as single for vlan types, then should the behaviour be:

* VLAN strip - strip at most one tag as defined by the "outer/single" VLAN
  type.
* QinQ strip:
  - if outer/single VLAN tag matches the outer tag type, strip it
  - if outer tag has been stripped, and inner tag matches the tag type,
    strip that also.
  - if a single VLAN tag is present, it gets stripped only if it's tag type
    matches outer type - it is left alone if it matches the inner type
  - if two VLAN tags are present, and the inner tag matches, it is not
    stripped if the outer tag does not match/has not been stripped.

Also, should we specify for DPDK what the default tags should be for the
two cases. It seems for the Intel NICs that I tried, that both inner and
outer tags always start with 0x8100. That's probably a good default for
VLAN strip, but for QinQ strip, we probably want hardware to default to
0x88a8 and 0x8100. Alternatively, we could/should mandate that drivers
explicitly set the required tags before starting the port.

/Bruce

RE: [RFC PATCH] doc: clarify VLAN and QinQ stripping behaviour

[Morten Brørup]

> From: Bruce Richardson [mailto:bruce.richardson@intel.com]
> Sent: Monday, 28 July 2025 17.11
> 
> On Mon, Jul 28, 2025 at 04:51:30PM +0200, Morten Brørup wrote:
> > > From: Dean Marx [mailto:dmarx@iol.unh.edu]
> > > Sent: Friday, 18 July 2025 15.18
> > >
> > > On Fri, Jul 18, 2025 at 4:23 AM Bruce Richardson
> > > <bruce.richardson@intel.com> wrote:
> > > >
> > > > On Thu, Jul 17, 2025 at 05:03:13PM -0400, Dean Marx wrote:
> > > > > I've created a v1 of a QinQ test suite around the set of test cases
> > > > > discussed earlier (which is not set in stone, and I expect it to
> > > > > change significantly across many future versions.) The PASS/FAIL
> > > > > values can be mostly disregarded in the context of this conversation,
> > > > > but I've added logging to explain which packets are sent, and what
> > > > > happened upon reception, which I hope will be more informative. After
> > > > > running on mlx5/i40e drivers, I got the following results:
> > > > >
> > > > > test_vlan_strip: QinQ strip OFF and VLAN strip ON
> > > > > test_qinq_strip: QinQ strip ON and VLAN strip ON
> > > > >
> > > > > i40e:
> > > > >     test_qinq_strip (sent packet: Single VLAN): FAIL
> > > > >       reason: VLAN tags found in packet when should have been
> > > > > stripped: Ether / Dot1Q / 802.1q (0x1c) vlan 1280 / LLC / Raw /
> > > > > Padding
> > > > >     test_qinq_strip (sent packet: Stacked VLAN): FAIL
> > > > >       reason: Expected one VLAN tag but found 2: Ether / Dot1Q / Dot1Q
> > > > > / 802.1q (0x1c) vlan 1280 / LLC / Raw / Padding
> > > > >     test_qinq_strip (sent packet: Single S-VLAN): FAIL
> > > > >       reason: VLAN tags found in packet when should have been
> > > > > stripped: Ether / Dot1Q / 802.1q (??) vlan ?? / LLC / Raw / Padding
> > > > >     test_qinq_strip (sent packet: QinQ): FAIL
> > > > >       reason: VLAN tags found in packet when should have been
> > > > > stripped: Ether / Dot1Q / Dot1AD / 802.1q (0x1c) vlan 1280 / LLC / Raw
> > > > > / Padding
> > > > >     test_vlan_strip (sent packet: Single VLAN): PASS
> > > > >       reason: VLAN tag stripped from packet
> > > > >     test_vlan_strip (sent packet: Stacked VLAN): PASS
> > > > >       reason: Received packet had outer VLAN stripped, with inner VLAN
> > > intact
> > > > >     test_vlan_strip (sent packet: Single S-VLAN): PASS
> > > > >       reason: S-VLAN tag stripped from packet
> > > > >     test_vlan_strip (sent packet: QinQ): FAIL
> > > > >       reason: Neither tag stripped
> > > > >
> > > >
> > > > Can you confirm exactly what is being sent in each case for the
> ethertype
> > > > of the VLAN tag? When you say single and stacked VLANs, that is VLANs
> with
> > > > 0x8100 type, correct? Is single S-VLAN a tag with ethertype 0x88a8, and
> > > > QinQ packet a packet with one 0x88a8 and one 0x8100? No other type
> options,
> > > > e.g. 0x9100 were checked, right?
> > > >
> > > > /Bruce
> > >
> > > That's correct, single VLAN is one 0x8100 tag, stacked is two, single
> > > S-VLAN is one 0x88a8, and QinQ is 0x88a8 and 0x8100. No other types
> > > were tested in the stripping case
> >
> >
> > Bruce,
> >
> > It seems the drivers have the ability to set the EtherType of the Outer (and
> sometimes Inner) tag:
> > https://elixir.bootlin.com/dpdk/v25.07/source/lib/ethdev/rte_ethdev.h#L3752
> >
> https://elixir.bootlin.com/dpdk/v25.07/source/drivers/net/intel/e1000/igb_ethd
> ev.c#L2739
> >
> https://elixir.bootlin.com/dpdk/v25.07/source/drivers/net/intel/i40e/i40e_ethd
> ev.c#L4038
> >
> > -Morten
> >
> Thanks. Question is, does this help us to clarify the behaviour for these
> tags? Based on the fact that the comment for the tag_type says that the
> outer is the same as single for vlan types, then should the behaviour be:
> 
> * VLAN strip - strip at most one tag as defined by the "outer/single" VLAN
>   type.
> * QinQ strip:
>   - if outer/single VLAN tag matches the outer tag type, strip it
>   - if outer tag has been stripped, and inner tag matches the tag type,
>     strip that also.
>   - if a single VLAN tag is present, it gets stripped only if it's tag type
>     matches outer type - it is left alone if it matches the inner type
>   - if two VLAN tags are present, and the inner tag matches, it is not
>     stripped if the outer tag does not match/has not been stripped.
> 
> Also, should we specify for DPDK what the default tags should be for the
> two cases. It seems for the Intel NICs that I tried, that both inner and
> outer tags always start with 0x8100. That's probably a good default for
> VLAN strip, but for QinQ strip, we probably want hardware to default to
> 0x88a8 and 0x8100. Alternatively, we could/should mandate that drivers
> explicitly set the required tags before starting the port.
> 
> /Bruce

My train of thought is:

1. When not configured for QinQ, maximum one tag is considered. I think we all agree on this.
VLAN is EtherType 0x8100, and (if present) is associated with the first (outermost) tag in the packet, and kept in mbuf->vlan_tci, regardless if any more (inner) tags are present or not.
This defines placement of the VLAN tag ID in the mbuf and the default EtherType used in the packet headers when parsing/stripping/inserting the tag.
It also defines the purpose of VLAN capability/present/strip/insert flags in the ethdev and mbuf offload flags.
Feature creep: The EtherType of this (outermost) tag might be configurable, if the hardware and driver supports it. I would consider such a feature very exotic.

2. For more tags, start by strictly conforming to the IEEE 802.3 standard.
QinQ is EtherType 0x88a8 and is associated with the outer tag (S-TAG), mbuf->vlan_tci_outer, regardless if an inner tag (C-TAG) is present or not.
VLAN is EtherType 0x8100 and is associated with the inner tag (C-TAG) or simple tag (VLAN tag), mbuf->vlan_tci, regardless if an outer tag is present or not. (Note: When disregarding the outer tag here, this is exactly the same behavior as for VLAN in bullet 1 above.)
This defines placements of the tag IDs in the mbuf and the default EtherTypes used in the packet headers when parsing/stripping/inserting tags.
It also defines the purpose of VLAN and QINQ capability/present/strip/insert flags in the ethdev and mbuf offload flags.

3. Then add support for VLAN Stacking, i.e. using EtherType 0x8100 for the outer tag.
This mode of operation is only relevant when configured for QinQ. It does not support "super hybrid links" (mixing QinQ tagged packets with simple VLAN tagged packets), because a packet with one VLAN tag is considered an S-tagged packet with no C-TAG, and such a packet would normally be considered a simple VLAN tagged packet.
Now, QinQ means EtherType 0x8100. It requires that the hardware and driver supports changing the EtherType of the outer tag.
QinQ is EtherType 0x8100 and is associated with the outer tag (S-TAG), mbuf->vlan_tci_outer, regardless if an inner tag (C-TAG) is present or not.
VLAN is EtherType 0x8100 and is associated with the inner tag (C-TAG), mbuf->vlan_tci, and can only exist when an outer tag is present.
Note: rte_net_get_ptype() cannot support this mode of operation without adding a parameter containing information about the EtherType of the outer tag!

This describes the behavior I have now come to expect, after getting wiser through this discussion.
Drivers might do something else, and applications might have adopted what drivers actually do.

-Morten