From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <konstantin.ananyev@intel.com>
Received: from mga14.intel.com (mga14.intel.com [192.55.52.115])
 by dpdk.org (Postfix) with ESMTP id 532552952
 for <dev@dpdk.org>; Thu,  7 Jul 2016 14:24:47 +0200 (CEST)
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by fmsmga103.fm.intel.com with ESMTP; 07 Jul 2016 05:24:46 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.28,324,1464678000"; d="scan'208";a="135280087"
Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159])
 by fmsmga004.fm.intel.com with ESMTP; 07 Jul 2016 05:24:45 -0700
Received: from irsmsx105.ger.corp.intel.com ([169.254.7.51]) by
 IRSMSX104.ger.corp.intel.com ([163.33.3.159]) with mapi id 14.03.0248.002;
 Thu, 7 Jul 2016 13:24:44 +0100
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>, "Tao, Zhe"
 <zhe.tao@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
CC: "Tao, Zhe" <zhe.tao@intel.com>, "Wu, Jingjing" <jingjing.wu@intel.com>
Thread-Topic: [dpdk-dev] [PATCH v2] i40: fix the VXLAN TSO issue
Thread-Index: AQHR2AfwwMIE602mcEayyOmuapEZ3KAMuq7ggAACbKCAAB/eoA==
Date: Thu, 7 Jul 2016 12:24:43 +0000
Message-ID: <2601191342CEEE43887BDE71AB97725836B7B3ED@irsmsx105.ger.corp.intel.com>
References: <1467752375-25984-1-git-send-email-zhe.tao@intel.com>
 <1467865627-23524-1-git-send-email-zhe.tao@intel.com>
 <2601191342CEEE43887BDE71AB97725836B7B36E@irsmsx105.ger.corp.intel.com>
In-Reply-To: <2601191342CEEE43887BDE71AB97725836B7B36E@irsmsx105.ger.corp.intel.com>
Accept-Language: en-IE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [163.33.239.182]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [dpdk-dev] [PATCH v2] i40: fix the VXLAN TSO issue
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jul 2016 12:24:48 -0000



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ananyev, Konstantin
> Sent: Thursday, July 07, 2016 11:51 AM
> To: Tao, Zhe; dev@dpdk.org
> Cc: Tao, Zhe; Wu, Jingjing
> Subject: Re: [dpdk-dev] [PATCH v2] i40: fix the VXLAN TSO issue
>=20
>=20
> Hi Tao,
>=20
> Sorry hit send button too early by accident :)
>=20
> > >
> > > Problem:
> > > When using the TSO + VXLAN feature in i40e, the outer UDP length fiel=
ds in
> > > the multiple UDP segments which are TSOed by the i40e will have the
> > > wrong value.
> > >
> > > Fix this problem by adding the tunnel type field in the i40e descript=
or
> > > which was missed before.
> > >
> > > Fixes: 77b8301733c3 ("i40e: VXLAN Tx checksum offload")
> > >
> > > Signed-off-by: Zhe Tao <zhe.tao@intel.com>
> > > ---
> > > V2: Edited some comments for mbuf structure and i40e driver.
> > >
> > >  app/test-pmd/csumonly.c      | 26 +++++++++++++++++++-------
> > >  drivers/net/i40e/i40e_rxtx.c | 12 +++++++++---
> > >  lib/librte_mbuf/rte_mbuf.h   | 16 +++++++++++++++-
> > >  3 files changed, 43 insertions(+), 11 deletions(-)
> > >
> > > diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c
> > > index ac4bd8f..d423c20 100644
> > > --- a/app/test-pmd/csumonly.c
> > > +++ b/app/test-pmd/csumonly.c
> > > @@ -204,7 +204,8 @@ parse_ethernet(struct ether_hdr *eth_hdr, struct =
testpmd_offload_info *info)
> > >  static void
> > >  parse_vxlan(struct udp_hdr *udp_hdr,
> > >  	    struct testpmd_offload_info *info,
> > > -	    uint32_t pkt_type)
> > > +	    uint32_t pkt_type,
> > > +	    uint64_t *ol_flags)
> > >  {
> > >  	struct ether_hdr *eth_hdr;
> > >
> > > @@ -215,6 +216,7 @@ parse_vxlan(struct udp_hdr *udp_hdr,
> > >  		RTE_ETH_IS_TUNNEL_PKT(pkt_type) =3D=3D 0)
> > >  		return;
> > >
> > > +	*ol_flags |=3D PKT_TX_TUNNEL_VXLAN;

Do we always have to setup tunnelling flags here?
Obviously it would mean an extra CTD per packet and might slowdown things.
In fact, I think current patch wouldn't work correctly if
TESTPMD_TX_OFFLOAD_OUTER_IP_CKSUM is not set.
So, can we do it only when TSO is enabled or outer IP checksum is enabled?

> > >  	info->is_tunnel =3D 1;
> > >  	info->outer_ethertype =3D info->ethertype;
> > >  	info->outer_l2_len =3D info->l2_len;
> > > @@ -231,7 +233,9 @@ parse_vxlan(struct udp_hdr *udp_hdr,
> > >
> > >  /* Parse a gre header */
> > >  static void
> > > -parse_gre(struct simple_gre_hdr *gre_hdr, struct testpmd_offload_inf=
o *info)
> > > +parse_gre(struct simple_gre_hdr *gre_hdr,
> > > +	  struct testpmd_offload_info *info,
> > > +	  uint64_t *ol_flags)
> > >  {
> > >  	struct ether_hdr *eth_hdr;
> > >  	struct ipv4_hdr *ipv4_hdr;
> > > @@ -242,6 +246,8 @@ parse_gre(struct simple_gre_hdr *gre_hdr, struct =
testpmd_offload_info *info)
> > >  	if ((gre_hdr->flags & _htons(~GRE_SUPPORTED_FIELDS)) !=3D 0)
> > >  		return;
> > >
> > > +	*ol_flags |=3D PKT_TX_TUNNEL_GRE;
> > > +
> > >  	gre_len +=3D sizeof(struct simple_gre_hdr);
> > >
> > >  	if (gre_hdr->flags & _htons(GRE_KEY_PRESENT))
> > > @@ -417,7 +423,7 @@ process_inner_cksums(void *l3_hdr, const struct t=
estpmd_offload_info *info,
> > >   * packet */
> > >  static uint64_t
> > >  process_outer_cksums(void *outer_l3_hdr, struct testpmd_offload_info=
 *info,
> > > -	uint16_t testpmd_ol_flags)
> > > +	uint16_t testpmd_ol_flags, uint64_t orig_ol_flags)
> > >  {
> > >  	struct ipv4_hdr *ipv4_hdr =3D outer_l3_hdr;
> > >  	struct ipv6_hdr *ipv6_hdr =3D outer_l3_hdr;
> > > @@ -442,6 +448,9 @@ process_outer_cksums(void *outer_l3_hdr, struct t=
estpmd_offload_info *info,
> > >  	 * hardware supporting it today, and no API for it. */
> > >
> > >  	udp_hdr =3D (struct udp_hdr *)((char *)outer_l3_hdr + info->outer_l=
3_len);
> > > +	if ((orig_ol_flags & PKT_TX_TCP_SEG) &&
> > > +	    ((orig_ol_flags & PKT_TX_TUNNEL_MASK) =3D=3D PKT_TX_TUNNEL_VXLA=
N))
> > > +		udp_hdr->dgram_cksum =3D 0;
> > >  	/* do not recalculate udp cksum if it was 0 */
> > >  	if (udp_hdr->dgram_cksum !=3D 0) {
> > >  		udp_hdr->dgram_cksum =3D 0;
> > > @@ -705,15 +714,18 @@ pkt_burst_checksum_forward(struct fwd_stream *f=
s)
> > >  			if (info.l4_proto =3D=3D IPPROTO_UDP) {
> > >  				struct udp_hdr *udp_hdr;
> > >  				udp_hdr =3D (struct udp_hdr *)((char *)l3_hdr +
> > > -					info.l3_len);
> > > -				parse_vxlan(udp_hdr, &info, m->packet_type);
> > > +					   info.l3_len);
> > > +				parse_vxlan(udp_hdr, &info, m->packet_type,
> > > +					    &ol_flags);
> > >  			} else if (info.l4_proto =3D=3D IPPROTO_GRE) {
> > >  				struct simple_gre_hdr *gre_hdr;
> > >  				gre_hdr =3D (struct simple_gre_hdr *)
> > >  					((char *)l3_hdr + info.l3_len);
> > > -				parse_gre(gre_hdr, &info);
> > > +				parse_gre(gre_hdr, &info, &ol_flags);
> > >  			} else if (info.l4_proto =3D=3D IPPROTO_IPIP) {
> > >  				void *encap_ip_hdr;
> > > +
> > > +				ol_flags |=3D PKT_TX_TUNNEL_IPIP;
> > >  				encap_ip_hdr =3D (char *)l3_hdr + info.l3_len;
> > >  				parse_encap_ip(encap_ip_hdr, &info);
> > >  			}
> > > @@ -745,7 +757,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
> > >  		 * processed in hardware. */
> > >  		if (info.is_tunnel =3D=3D 1) {
> > >  			ol_flags |=3D process_outer_cksums(outer_l3_hdr, &info,
> > > -				testpmd_ol_flags);
> > > +				testpmd_ol_flags, ol_flags);
> > >  		}
> > >
> > >  		/* step 4: fill the mbuf meta data (flags and header lengths) */
> > > diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxt=
x.c
> > > index 049a813..4c987f2 100644
> > > --- a/drivers/net/i40e/i40e_rxtx.c
> > > +++ b/drivers/net/i40e/i40e_rxtx.c
> > > @@ -801,6 +801,12 @@ i40e_txd_enable_checksum(uint64_t ol_flags,
> > >  			union i40e_tx_offload tx_offload,
> > >  			uint32_t *cd_tunneling)
> > >  {
> > > +	/* Tx pkts tunnel type*/
> > > +	if ((ol_flags & PKT_TX_TUNNEL_MASK) =3D=3D PKT_TX_TUNNEL_VXLAN)
> > > +		*cd_tunneling |=3D I40E_TXD_CTX_UDP_TUNNELING;
> > > +	else if ((ol_flags & PKT_TX_TUNNEL_MASK) =3D=3D PKT_TX_TUNNEL_GRE)
> > > +		*cd_tunneling |=3D I40E_TXD_CTX_GRE_TUNNELING;
> > > +
>=20
> As I understand that fix is needed to enable TSO for tunnelling packets, =
correct?
> For that case, should we setup EIPLEN also, no matter is PKT_TX_OUTER_IP_=
CKSUM
> is on/off?

In fact, it seems we have always to setup all 3: EIPLEN, MACLEN and L4TUNLE=
N,
when L4TUNT !=3D 0.

>=20
> > >  	/* UDP tunneling packet TX checksum offload */
> > >  	if (ol_flags & PKT_TX_OUTER_IP_CKSUM) {
> > >
> > > @@ -1510,7 +1516,8 @@ i40e_calc_context_desc(uint64_t flags)
> > >
> > >  /* set i40e TSO context descriptor */
> > >  static inline uint64_t
> > > -i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_off=
load)
> > > +i40e_set_tso_ctx(struct rte_mbuf *mbuf,
> > > +		 union i40e_tx_offload tx_offload)
> > >  {
> > >  	uint64_t ctx_desc =3D 0;
> > >  	uint32_t cd_cmd, hdr_len, cd_tso_len;
> > > @@ -1521,7 +1528,7 @@ i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i=
40e_tx_offload tx_offload)
> > >  	}
> > >
> > >  	/**
> > > -	 * in case of tunneling packet, the outer_l2_len and
> > > +	 * in case of non tunneling packet, the outer_l2_len and
> > >  	 * outer_l3_len must be 0.
> > >  	 */
> > >  	hdr_len =3D tx_offload.outer_l2_len +
> > > @@ -1537,7 +1544,6 @@ i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i=
40e_tx_offload tx_offload)
> > >  		 I40E_TXD_CTX_QW1_TSO_LEN_SHIFT) |
> > >  		((uint64_t)mbuf->tso_segsz <<
> > >  		 I40E_TXD_CTX_QW1_MSS_SHIFT);
> > > -
> > >  	return ctx_desc;
> > >  }
> > >
> > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > index 15e3a10..8eb0d33 100644
> > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > @@ -133,6 +133,17 @@ extern "C" {
> > >  /* add new TX flags here */
> > >
> > >  /**
> > > + * Bits 45:48 used for the tunnel type.
> > > + * When doing Tx offload like TSO or checksum, the HW needs to confi=
gure the
> > > + * tunnel type into the HW descriptors.
> > > + */
> > > +#define PKT_TX_TUNNEL_VXLAN   (1ULL << 45)
> > > +#define PKT_TX_TUNNEL_GRE   (2ULL << 45)
> > > +#define PKT_TX_TUNNEL_IPIP    (3ULL << 45)
> > > +/* add new TX TUNNEL type here */
> > > +#define PKT_TX_TUNNEL_MASK    (0xFULL << 45)
> > > +
> > > +/**
> > >   * Second VLAN insertion (QinQ) flag.
> > >   */
> > >  #define PKT_TX_QINQ_PKT    (1ULL << 49)   /**< TX packet with double=
 VLAN inserted. */
> > > @@ -867,7 +878,10 @@ struct rte_mbuf {
> > >  	union {
> > >  		uint64_t tx_offload;       /**< combined for easy fetch */
> > >  		struct {
> > > -			uint64_t l2_len:7; /**< L2 (MAC) Header Length. */
> > > +			/* L2 (MAC) Header Length if it is not a tunneling pkt.
> > > +			 * for tunnel it is outer L4 len+tunnel len+inner L2 len
> > > +			 */
>=20
> As a nit: that doesn't look like doxygen style comment to me.
> Konstantin
>=20
> > > +			uint64_t l2_len:7;
> > >  			uint64_t l3_len:9; /**< L3 (IP) Header Length. */
> > >  			uint64_t l4_len:8; /**< L4 (TCP/UDP) Header Length. */
> > >  			uint64_t tso_segsz:16; /**< TCP TSO segment size */
> > > --
> > > 2.1.4