DPDK patches and discussions
 help / color / mirror / Atom feed
From: Olivier Matz <olivier.matz@6wind.com>
To: "Xueming(Steven) Li" <xuemingl@mellanox.com>
Cc: Yongseok Koh <yskoh@mellanox.com>,
	Wenzhuo Lu <wenzhuo.lu@intel.com>,
	Jingjing Wu <jingjing.wu@intel.com>,
	Thomas Monjalon <thomas@monjalon.net>,
	Shahaf Shuler <shahafs@mellanox.com>,
	Ferruh Yigit <ferruh.yigit@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 1/7] ethdev: introduce Tx generic tunnel L3/L4 offload
Date: Wed, 28 Mar 2018 14:52:31 +0200	[thread overview]
Message-ID: <20180328125231.5djcb64fdy3c6pzk@platinum> (raw)
In-Reply-To: <VI1PR05MB1678315ECD5E9B85FD936670ACA90@VI1PR05MB1678.eurprd05.prod.outlook.com>

Hi Xueming, Yongseok,

On Thu, Mar 22, 2018 at 01:55:05PM +0000, Xueming(Steven) Li wrote:
> 
> 
> > -----Original Message-----
> > From: Yongseok Koh
> > Sent: Wednesday, March 21, 2018 9:41 AM
> > To: Xueming(Steven) Li <xuemingl@mellanox.com>
> > Cc: Wenzhuo Lu <wenzhuo.lu@intel.com>; Jingjing Wu <jingjing.wu@intel.com>;
> > Thomas Monjalon <thomas@monjalon.net>; Olivier MATZ
> > <olivier.matz@6wind.com>; Shahaf Shuler <shahafs@mellanox.com>; Ferruh
> > Yigit <ferruh.yigit@intel.com>; dev@dpdk.org
> > Subject: Re: [PATCH v3 1/7] ethdev: introduce Tx generic tunnel L3/L4
> > offload
> > 
> > On Mon, Mar 05, 2018 at 10:51:15PM +0800, Xueming Li wrote:
> > > This patch introduce new TX offload flags for device that supports
> > > tunnel agnostic L3/L4 checksum and TSO offload.
> > >
> > > The support from the device is for inner and outer checksums on
> > > IPV4/TCP/UDP and TSO for *any packet with the following format*:
> > >
> > > < some headers > / [optional IPv4/IPv6] / [optional TCP/UDP] / <some
> > > headers> / [optional inner IPv4/IPv6] / [optional TCP/UDP]
> > >
> > > For example the following packets can use this feature:
> > >
> > > 1. eth / ipv4 / udp / VXLAN / ip / tcp 2. eth / ipv4 / GRE / MPLS /
> > > ipv4 / udp
> > >
> > > Signed-off-by: Xueming Li <xuemingl@mellanox.com>
> > > ---
> > >  lib/librte_ether/rte_ethdev.h | 24 ++++++++++++++++++++++++
> > >  lib/librte_mbuf/rte_mbuf.c    |  5 +++++
> > >  lib/librte_mbuf/rte_mbuf.h    | 18 ++++++++++++++++--
> > >  3 files changed, 45 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/lib/librte_ether/rte_ethdev.h
> > > b/lib/librte_ether/rte_ethdev.h index 036153306..66d12d3e0 100644
> > > --- a/lib/librte_ether/rte_ethdev.h
> > > +++ b/lib/librte_ether/rte_ethdev.h
> > > @@ -980,6 +980,30 @@ struct rte_eth_conf {
> > >   *   the same mempool and has refcnt = 1.
> > >   */
> > >  #define DEV_TX_OFFLOAD_SECURITY         0x00020000
> > > +/**< Generic tunnel L3/L4 checksum offload. To enable this offload
> > > +feature
> > > + * for a packet to be transmitted on hardware supporting generic
> > > +tunnel L3/L4
> > > + * checksum offload:
> > > + *  - fill outer_l2_len and outer_l3_len in mbuf
> > > + *  - fill l2_len and l3_len in mbuf
> > > + *  - set the flags PKT_TX_TUNNEL_xxx (use PKT_TX_TUNNEL_UNKNOWN if
> > > +undefined)
> > > + *  - set the flags PKT_TX_OUTER_IP_CKSUM
> > > + *  - set the flags PKT_TX_IP_CKSUM
> > > + *  - set the flags PKT_TX_TCP_CKSUM, PKT_TX_SCTP_CKSUM or
> > > +PKT_TX_UDP_CKSUM  */
> > > +#define DEV_TX_OFFLOAD_GENERIC_TNL_CKSUM	0x00040000
> > 
> > It looks redundant to me. Isn't it same as having DEV_TX_OFFLOAD_*_CKSUM?
> > According to the API document, when PKT_TX_*_CKSUM is set, all the
> > necessary length fields should be filled in (e.g. mbuf->outer_l?_len and
> > mbuf->l?_len). It doesn't need to specify any tunnel type for checksum.
> > For example, in order to request checksum offload for an unknown tunnel
> > type which is similar to VxLAN, what should app do? Even without defining
> > this new offload flag, it is currently possible by setting
> > 	(PKT_TX_OUTER_IP_CKSUM | PKT_TX_OUTER_IPV4 | PKT_TX_IP_CKSUM |
> > 	 PKT_TX_IPV4 | PKT_TX_UDP_CKSUM)
> > along with various length fields.
> 
> Agree it almost same as existing checksum offloading requirement, besides:
> 1. it does not demand pseudo checksum
> 2. no need to clear IP checksum
> I guess the requirement on API doc is a super set and differences could be 
> documented on PMD doc.

Since commit 4fb7e803eb1a ("ethdev: add Tx preparation"), setting the
IP checksum to 0 and the pseudo header checksum in TCP is not required.
The user just need to set the flags, then before transmitting the packet,
call tx_prepare() which will do the proper work, according to the hw
requirements.

It looks that the API documentation of PKT_TX_TCP_SEG should be updated.
I'll submit a patch for this.


> For mlx5 original chksum offloading, hw will parse everything w/o demanding
> mbuf headers, a little better performance than SWP. The question is how to 
> choose between SWP and HW checksum offloading - assuming PKT_TX_TUNNEL_UNKNOWN
> is the key. Take L3 VXLAN(no inner L2 header) as example, only SWP could 
> offload it correctly and PKT_TX_TUNNEL_UNKOWN should be used here, not 
> PKT_TX_TUNNEL_VXLAN, make sense?
> 
> Another question, how does user app know the NIC capability of generic tunnel 
> checksum offload?

After reading the patchset, I still don't see what you want to achieve.
In patch 1, we talk about these 2 packets:

1. eth / ipv4 / udp / VXLAN / ip / tcp     (L3 VXLAN as you said above)
2. eth / ipv4 / GRE / MPLS / ipv4 / udp

What is not doable with current API?

I didn't try, but I think the following would work currently
with tx_prepare() + tx_burst().

- inner TSO on 1.

  flags = PKT_TX_TUNNEL_VXLAN | PKT_TX_IPV4 | PKT_TX_IP_CKSUM |
          PKT_TX_TCP_CKSUM | PKT_TX_TCP_SEG
  outer_l2_len = size(eth)
  outer_l3_len = size(ipv4)
  l2_len = size(udp + vxlan)
  l3_len = size(ip)

- inner TCP checksum on 1.

  flags = PKT_TX_TUNNEL_VXLAN | PKT_TX_IPV4 | PKT_TX_IP_CKSUM |
          PKT_TX_TCP_CKSUM
  outer_l2_len = size(eth)
  outer_l3_len = size(ipv4)
  l2_len = size(udp + vxlan)
  l3_len = size(ip)

- inner UDP checksum on 2.

  flags = PKT_TX_TUNNEL_GRE | PKT_TX_IPV4 | PKT_TX_IP_CKSUM |
          PKT_TX_UDP_CKSUM
  outer_l2_len = size(eth)
  outer_l3_len = size(ipv4)
  l2_len = size(gre + mpls)
  l3_len = size(ipv4)

Yes, writing the tunnel len in l2_len is a bit strange, but it
should do the trick.

It's still unclear for me why you need to add a "generic" flag.
Is there something you can't do today with the current API?


> 
> > 
> > > +/**< Generic tunnel segmentation offload. To enable it, the user needs
> > to:
> > > + *  - fill outer_l2_len and outer_l3_len in mbuf
> > > + *  - fill l2_len and l3_len in mbuf
> > > + *  - set the flags PKT_TX_TUNNEL_xxx (use PKT_TX_TUNNEL_UNKNOWN if
> > > +undefined)
> > > + *  - set the flags PKT_TX_OUTER_IPV4 or PKT_TX_OUTER_IPV6
> > > + *  - if it's UDP tunnel, set the flags PKT_TX_OUTER_UDP
> > > + *  - set the flags PKT_TX_IPV4 or PKT_TX_IPV6
> > > + *  - set the PKT_TX_TCP_SEG flag in mbuf->ol_flags (this flag implies
> > > + *    PKT_TX_OUTER_IP_CKSUM, PKT_TX_IP_CKSUM and PKT_TX_TCP_CKSUM)
> > > + * Hardware that supports generic tunnel TSO offload only update
> > > +outer/inner
> > > + * L3/L4 fields, tunnel fields are not touched.
> > > + */
> > > +#define DEV_TX_OFFLOAD_GENERIC_TNL_TSO		0x00080000
> > 
> > Practically, for the existing TSO offloads for tunneled packets, tunnel
> > type
> > (PKT_TX_TUNNEL_*) is needed for knowing if transport is UDP-based or IP-
> > based because the length field of UDP header should be updated for TSO.
> > AFAIK, there's no TCP-based tunnel. For PKT_TX_TUNNEL_UNKNOWN, the only
> > thing _unknown_ seems to be the type of transport and by defining
> > PKT_TX_OUTER_UDP, you seem to recognize the type of transport between UDP
> > and IP (it's not non-UDP). Given that, the new flags still look redundant
> > and ambiguous. If PKT_TX_TUNNEL_VXLAN is set, there's no need to set
> > PKT_TX_OUTER_UDP redundantly.
> > 
> > Instead, I think defining PKT_TX_TUNNEL_UDP_UNKNOWN and
> > PKT_TX_TUNNEL_IP_UNKNOWN could be a good alternative. It is only my humble
> > two cents and I'd like to hear from more people regarding this.
> > 
> 

I agree with Yongseok here. The flag PKT_TX_TUNNEL_VXLAN implies that
the protocol is UDP. Having a flag PKT_TX_TUNNEL_UDP or PKT_TX_TUNNEL_IP
could make sense if we find a use case that cannot be done currently with
the existing API (for instance, another tunnel type over UDP).

> This flag looks more like a feature notification. Current tunnel offloading 
> capability like "DEV_TX_OFFLOAD_VXLAN_TNL_TSO" ties to VXLAN tunnel type,
> DEV_TX_OFFLOAD_GENERIC_TNL_TSO should be able to let users know that NIC's
> capability of offloading generic tunnel type like MPLS-in-UDP that not defined
> yet. PKT_TX_OUTER_UDP is something must to have to flag an exists of outer UDP
> in such case.

So, maybe, instead of DEV_TX_OFFLOAD_GENERIC_TNL_TSO it could be
DEV_TX_OFFLOAD_UDP_TUNNEL_TSO. Because it is not possible for a hardware
to support TSO on top of any kind of tunnel type.

For instance, if the tunnel header has a checksum or a sequence id, the
hardware must be aware of it.

  reply	other threads:[~2018-03-28 12:52 UTC|newest]

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-09 14:11 [dpdk-dev] [PATCH 0/6] Support generic tunnel TX csum and TSO Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 1/6] net/mlx5: support tx swp tunnel offloading Xueming Li
2018-01-29 15:08   ` [dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx generic tunnel offloads Xueming Li
2018-01-29 16:49     ` Ananyev, Konstantin
2018-01-30  3:01       ` Xueming(Steven) Li
2018-01-30 13:28         ` Ananyev, Konstantin
2018-01-30 15:27           ` Xueming(Steven) Li
2018-01-30 15:33             ` Ananyev, Konstantin
2018-01-30 15:47               ` Xueming(Steven) Li
2018-01-30 16:02                 ` Ananyev, Konstantin
2018-01-30 16:10                   ` Xueming(Steven) Li
2018-01-30 17:04                     ` Ananyev, Konstantin
2018-01-30 17:54                       ` Xueming(Steven) Li
2018-01-30 20:21                         ` Thomas Monjalon
2018-01-31 15:20                           ` Xueming(Steven) Li
2018-01-31 15:17                         ` Xueming(Steven) Li
2018-01-29 15:08   ` [dpdk-dev] [PATCH v2 2/5] app/testpmd: testpmd support " Xueming Li
2018-01-29 15:08   ` [dpdk-dev] [PATCH v2 3/5] net/mlx5: separate TSO function in Tx data path Xueming Li
2018-01-29 15:08   ` [dpdk-dev] [PATCH v2 4/5] net/mlx5: support generic tunnel offloading Xueming Li
2018-01-29 15:08   ` [dpdk-dev] [PATCH v2 5/5] net/mlx5: allow max 192B TSO inline header length Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 0/7] support generic tunnel Tx checksum and TSO Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 1/7] ethdev: introduce Tx generic tunnel L3/L4 offload Xueming Li
2018-03-21  1:40     ` Yongseok Koh
2018-03-22 13:55       ` Xueming(Steven) Li
2018-03-28 12:52         ` Olivier Matz [this message]
2018-04-04  8:20           ` Xueming(Steven) Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 2/7] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 3/7] app/testpmd: add more GRE extension to csum engine Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 4/7] app/testpmd: introduce VXLAN GPE to csum forwarding engine Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 5/7] net/mlx5: separate TSO function in Tx data path Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 6/7] net/mlx5: support generic tunnel offloading Xueming Li
2018-03-05 14:51   ` [dpdk-dev] [PATCH v3 7/7] net/mlx5: allow max 192B TSO inline header length Xueming Li
2018-04-08 12:32   ` [dpdk-dev] [PATCH v4 0/4] support Tx generic tunnel checksum and TSO Xueming Li
2018-04-17 14:43     ` [dpdk-dev] [PATCH v5 0/2] " Xueming Li
2018-04-17 14:47     ` [dpdk-dev] [PATCH v5 1/2] ethdev: introduce generic IP/UDP " Xueming Li
2018-04-17 21:21       ` Thomas Monjalon
2018-04-17 14:49     ` [dpdk-dev] [PATCH v5 2/2] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-18 13:38     ` [dpdk-dev] [PATCH v6 0/2] support Tx generic tunnel checksum and TSO Xueming Li
2018-04-18 13:58     ` [dpdk-dev] [PATCH v6 1/2] ethdev: introduce generic IP/UDP " Xueming Li
2018-04-18 14:28       ` Thomas Monjalon
2018-04-18 16:45         ` Ananyev, Konstantin
2018-04-18 18:02           ` Thomas Monjalon
2018-04-23  9:55             ` Olivier Matz
2018-04-20 12:48       ` [dpdk-dev] [PATCH v7 0/2] support Tx generic " Xueming Li
2018-04-23 11:36         ` [dpdk-dev] [PATCH v8 " Xueming Li
2018-04-23 16:17           ` Ferruh Yigit
2018-04-23 11:36         ` [dpdk-dev] [PATCH v8 1/2] ethdev: introduce generic IP/UDP " Xueming Li
2018-04-23 11:49           ` Xueming Li
2018-04-23 11:36         ` [dpdk-dev] [PATCH v8 2/2] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-20 12:48       ` [dpdk-dev] [PATCH v7 1/2] ethdev: introduce generic IP/UDP tunnel checksum and TSO Xueming Li
2018-04-23  9:59         ` Olivier Matz
2018-04-20 12:48       ` [dpdk-dev] [PATCH v7 2/2] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-18 13:59     ` [dpdk-dev] [PATCH v6 " Xueming Li
2018-04-08 12:32   ` [dpdk-dev] [PATCH v4 1/4] ethdev: introduce generic IP/UDP tunnel checksum and TSO Xueming Li
2018-04-16 22:42     ` Thomas Monjalon
2018-04-17  7:53       ` Xueming(Steven) Li
2018-04-17  8:10         ` Thomas Monjalon
2018-04-08 12:32   ` [dpdk-dev] [PATCH v4 2/4] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-17 14:24     ` Iremonger, Bernard
2018-04-17 15:44       ` Xueming(Steven) Li
2018-04-08 12:32   ` [dpdk-dev] [PATCH v4 3/4] app/testpmd: add more GRE extension to csum engine Xueming Li
2018-04-16 22:45     ` Thomas Monjalon
2018-04-17  5:19       ` Xueming(Steven) Li
2018-04-08 12:32   ` [dpdk-dev] [PATCH v4 4/4] app/testpmd: introduce VXLAN GPE to csum forwarding engine Xueming Li
2018-04-16 22:46     ` Thomas Monjalon
2018-04-17 13:56       ` Iremonger, Bernard
2018-04-17 14:12         ` Xueming(Steven) Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 2/6] net/mlx5: allow max 192B WQE TSO inline header length Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 3/6] net/mlx5: add SWP PCI parameter for TX common tunnel offloads Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 4/6] ethdev: introduce " Xueming Li
2018-01-11 18:38   ` Ferruh Yigit
2018-01-16 17:10   ` Olivier Matz
2018-01-16 17:28     ` Xueming(Steven) Li
2018-01-16 19:06       ` Shahaf Shuler
2018-01-22 12:46         ` Olivier Matz
2018-01-22 20:06           ` Shahaf Shuler
2018-01-17  0:50   ` Yongseok Koh
2018-01-09 14:11 ` [dpdk-dev] [PATCH 5/6] net/mlx5: support " Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 6/6] app/testpmd: testpmd " Xueming Li
2018-01-16  3:09   ` Lu, Wenzhuo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180328125231.5djcb64fdy3c6pzk@platinum \
    --to=olivier.matz@6wind.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jingjing.wu@intel.com \
    --cc=shahafs@mellanox.com \
    --cc=thomas@monjalon.net \
    --cc=wenzhuo.lu@intel.com \
    --cc=xuemingl@mellanox.com \
    --cc=yskoh@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).