Re: [dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure for QinQ support

[Bruce Richardson <bruce.richardson@intel.com>]

On Wed, May 06, 2015 at 04:06:17AM +0000, Zhang, Helin wrote:
> 
> 
> > -----Original Message-----
> > From: Ananyev, Konstantin
> > Sent: Tuesday, May 5, 2015 7:05 PM
> > To: Zhang, Helin; dev@dpdk.org
> > Subject: RE: [dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure for
> > QinQ support
> > 
> > Hi Helin,
> > 
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Helin Zhang
> > > Sent: Tuesday, May 05, 2015 3:32 AM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure for
> > > QinQ support
> > >
> > > To support QinQ, 'vlan_tci' should be replaced by 'vlan_tci0' and
> > > 'vlan_tci1'. Also new offload flags of 'PKT_RX_QINQ_PKT' and
> > > 'PKT_TX_QINQ_PKT' should be added.
> > >
> > > Signed-off-by: Helin Zhang <helin.zhang@intel.com>
> > > ---
> > >  app/test-pmd/flowgen.c                |  2 +-
> > >  app/test-pmd/macfwd.c                 |  2 +-
> > >  app/test-pmd/macswap.c                |  2 +-
> > >  app/test-pmd/rxonly.c                 |  2 +-
> > >  app/test-pmd/txonly.c                 |  2 +-
> > >  app/test/packet_burst_generator.c     |  4 ++--
> > >  lib/librte_ether/rte_ether.h          |  4 ++--
> > >  lib/librte_mbuf/rte_mbuf.h            | 22
> > +++++++++++++++++++---
> > >  lib/librte_pmd_e1000/em_rxtx.c        |  8 ++++----
> > >  lib/librte_pmd_e1000/igb_rxtx.c       |  8 ++++----
> > >  lib/librte_pmd_enic/enic_ethdev.c     |  2 +-
> > >  lib/librte_pmd_enic/enic_main.c       |  2 +-
> > >  lib/librte_pmd_fm10k/fm10k_rxtx.c     |  2 +-
> > >  lib/librte_pmd_i40e/i40e_rxtx.c       |  8 ++++----
> > >  lib/librte_pmd_ixgbe/ixgbe_rxtx.c     | 11 +++++------
> > >  lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c |  6 +++---
> > >  16 files changed, 51 insertions(+), 36 deletions(-)
> > >
<snip>
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > @@ -101,11 +101,17 @@ extern "C" {
> > >  #define PKT_RX_TUNNEL_IPV6_HDR (1ULL << 12) /**< RX tunnel
> > packet with IPv6 header. */
> > >  #define PKT_RX_FDIR_ID       (1ULL << 13) /**< FD id reported if
> > FDIR match. */
> > >  #define PKT_RX_FDIR_FLX      (1ULL << 14) /**< Flexible bytes
> > reported if FDIR match. */
> > > +#define PKT_RX_QINQ_PKT     (1ULL << 15)  /**< RX packet with
> > double VLAN stripped. */
> > >  /* add new RX flags here */
> > >
> > >  /* add new TX flags here */
> > >
> > >  /**
> > > + * Second VLAN insertion (QinQ) flag.
> > > + */
> > > +#define PKT_TX_QINQ_PKT    (1ULL << 49)
> > > +
> > > +/**
> > >   * TCP segmentation offload. To enable this offload feature for a
> > >   * packet to be transmitted on hardware supporting TSO:
> > >   *  - set the PKT_TX_TCP_SEG flag in mbuf->ol_flags (this flag
> > > implies @@ -268,7 +274,6 @@ struct rte_mbuf {
> > >
> > >  	uint16_t data_len;        /**< Amount of data in segment buffer. */
> > >  	uint32_t pkt_len;         /**< Total pkt len: sum of all segments. */
> > > -	uint16_t vlan_tci;        /**< VLAN Tag Control Identifier (CPU
> > order) */
> > >  	uint16_t reserved;
> > 
> > Now here is an implicit 2-bytes whole between 'reserved' and 'rss'.
> > Probably better to make it explicit - make 'reserved' uint32_t.
> Yes, the layout will be changed according to the demands of Vector PMD.
> The vlan structure will be kept the same, but the mbuf structure layout will
> be re-organized a bit.

Why not just put the extra vlan tag into the reserved space. In the original
work to restructure the mbuf, that was what the reserved space was put there
for [it was marked as reserved as it was requested that fields not be fully
dedicated until used, and we did not have double-vlan support at that time].
However, it seems more sensible to put the vlans there now, unless there is
a good reason to move them to the new location in the mbuf that you propose
below.

/Bruce

> 
> > 
> > Another thing - it looks like your change will break  ixgbe vector RX.
> Yes, in the cover-letter, I noted that the vector PMD will be updated soon
> together with the code changes.
> 
> > 
> > >  	union {
> > >  		uint32_t rss;     /**< RSS hash result if RSS enabled */
> > > @@ -289,6 +294,15 @@ struct rte_mbuf {
> > >  		uint32_t usr;	  /**< User defined tags. See
> > rte_distributor_process() */
> > >  	} hash;                   /**< hash information */
> > >
> > > +	/* VLAN tags */
> > > +	union {
> > > +		uint32_t vlan_tags;
> > > +		struct {
> > > +			uint16_t vlan_tci0;
> > > +			uint16_t vlan_tci1;
> > 
> > Do you really need to change vlan_tci to vlan_tci0?
> > Can't you keep 'vlan_tci' for first vlan tag, and add something like
> > 'vlan_tci_ext', or 'vlan_tci_next' for second one?
> > Would save you a lot of changes, again users who use single vlan wouldn't
> > need to update their code for 2.1.
> Yes, good point! The names came from the original mbuf definition done by
> Bruce long long ago. If more guys suggest keeping th old one, and just add a new
> one, I will do like that in the next version of patch set.
> Thank you all!
> 
> > 
> > > +		};
> > > +	};
> > > +
> > >  	uint32_t seqn; /**< Sequence number. See also rte_reorder_insert()
> > > */
> > >
> > >  	/* second cache line - fields only used in slow path or on TX */ @@
> > > -766,7 +780,8 @@ static inline void rte_pktmbuf_reset(struct rte_mbuf
> > *m)
> > >  	m->next = NULL;
> > >  	m->pkt_len = 0;
> > >  	m->tx_offload = 0;
> > > -	m->vlan_tci = 0;
> > > +	m->vlan_tci0 = 0;
> > > +	m->vlan_tci1 = 0;
> > 
> > Why just not:
> > m-> vlan_tags = 0;
> > ?
> Accepted. Good point!
> 
> > 
> > >  	m->nb_segs = 1;
> > >  	m->port = 0xff;
> > >
> > > @@ -838,7 +853,8 @@ static inline void rte_pktmbuf_attach(struct
> > rte_mbuf *mi, struct rte_mbuf *m)
> > >  	mi->data_off = m->data_off;
> > >  	mi->data_len = m->data_len;
> > >  	mi->port = m->port;
> > > -	mi->vlan_tci = m->vlan_tci;
> > > +	mi->vlan_tci0 = m->vlan_tci0;
> > > +	mi->vlan_tci1 = m->vlan_tci1;
> > 
> > Same thing, why not:
> > mi-> vlan_tags = m-> vlan_tags;
> > ?
> Accepted. Good point!
> 
> Regards,
> Helin

<snip>