From: "Xueming(Steven) Li" <xuemingl@mellanox.com>
To: Olivier Matz <olivier.matz@6wind.com>
Cc: Yongseok Koh <yskoh@mellanox.com>,
Wenzhuo Lu <wenzhuo.lu@intel.com>,
Jingjing Wu <jingjing.wu@intel.com>,
Thomas Monjalon <thomas@monjalon.net>,
Shahaf Shuler <shahafs@mellanox.com>,
Ferruh Yigit <ferruh.yigit@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 1/7] ethdev: introduce Tx generic tunnel L3/L4 offload
Date: Wed, 4 Apr 2018 08:20:49 +0000 [thread overview]
Message-ID: <VI1PR05MB1678175E6641A7AD008D0683ACA40@VI1PR05MB1678.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <20180328125231.5djcb64fdy3c6pzk@platinum>
Hi Olivier,
> -----Original Message-----
> From: Olivier Matz <olivier.matz@6wind.com>
> Sent: Wednesday, March 28, 2018 8:53 PM
> To: Xueming(Steven) Li <xuemingl@mellanox.com>
> Cc: Yongseok Koh <yskoh@mellanox.com>; Wenzhuo Lu <wenzhuo.lu@intel.com>;
> Jingjing Wu <jingjing.wu@intel.com>; Thomas Monjalon <thomas@monjalon.net>;
> Shahaf Shuler <shahafs@mellanox.com>; Ferruh Yigit
> <ferruh.yigit@intel.com>; dev@dpdk.org
> Subject: Re: [PATCH v3 1/7] ethdev: introduce Tx generic tunnel L3/L4
> offload
>
> Hi Xueming, Yongseok,
>
> On Thu, Mar 22, 2018 at 01:55:05PM +0000, Xueming(Steven) Li wrote:
> >
> >
> > > -----Original Message-----
> > > From: Yongseok Koh
> > > Sent: Wednesday, March 21, 2018 9:41 AM
> > > To: Xueming(Steven) Li <xuemingl@mellanox.com>
> > > Cc: Wenzhuo Lu <wenzhuo.lu@intel.com>; Jingjing Wu
> > > <jingjing.wu@intel.com>; Thomas Monjalon <thomas@monjalon.net>;
> > > Olivier MATZ <olivier.matz@6wind.com>; Shahaf Shuler
> > > <shahafs@mellanox.com>; Ferruh Yigit <ferruh.yigit@intel.com>;
> > > dev@dpdk.org
> > > Subject: Re: [PATCH v3 1/7] ethdev: introduce Tx generic tunnel
> > > L3/L4 offload
> > >
> > > On Mon, Mar 05, 2018 at 10:51:15PM +0800, Xueming Li wrote:
> > > > This patch introduce new TX offload flags for device that supports
> > > > tunnel agnostic L3/L4 checksum and TSO offload.
> > > >
> > > > The support from the device is for inner and outer checksums on
> > > > IPV4/TCP/UDP and TSO for *any packet with the following format*:
> > > >
> > > > < some headers > / [optional IPv4/IPv6] / [optional TCP/UDP] /
> > > > <some
> > > > headers> / [optional inner IPv4/IPv6] / [optional TCP/UDP]
> > > >
> > > > For example the following packets can use this feature:
> > > >
> > > > 1. eth / ipv4 / udp / VXLAN / ip / tcp 2. eth / ipv4 / GRE / MPLS
> > > > /
> > > > ipv4 / udp
> > > >
> > > > Signed-off-by: Xueming Li <xuemingl@mellanox.com>
> > > > ---
> > > > lib/librte_ether/rte_ethdev.h | 24 ++++++++++++++++++++++++
> > > > lib/librte_mbuf/rte_mbuf.c | 5 +++++
> > > > lib/librte_mbuf/rte_mbuf.h | 18 ++++++++++++++++--
> > > > 3 files changed, 45 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/lib/librte_ether/rte_ethdev.h
> > > > b/lib/librte_ether/rte_ethdev.h index 036153306..66d12d3e0 100644
> > > > --- a/lib/librte_ether/rte_ethdev.h
> > > > +++ b/lib/librte_ether/rte_ethdev.h
> > > > @@ -980,6 +980,30 @@ struct rte_eth_conf {
> > > > * the same mempool and has refcnt = 1.
> > > > */
> > > > #define DEV_TX_OFFLOAD_SECURITY 0x00020000
> > > > +/**< Generic tunnel L3/L4 checksum offload. To enable this
> > > > +offload feature
> > > > + * for a packet to be transmitted on hardware supporting generic
> > > > +tunnel L3/L4
> > > > + * checksum offload:
> > > > + * - fill outer_l2_len and outer_l3_len in mbuf
> > > > + * - fill l2_len and l3_len in mbuf
> > > > + * - set the flags PKT_TX_TUNNEL_xxx (use PKT_TX_TUNNEL_UNKNOWN
> > > > +if
> > > > +undefined)
> > > > + * - set the flags PKT_TX_OUTER_IP_CKSUM
> > > > + * - set the flags PKT_TX_IP_CKSUM
> > > > + * - set the flags PKT_TX_TCP_CKSUM, PKT_TX_SCTP_CKSUM or
> > > > +PKT_TX_UDP_CKSUM */
> > > > +#define DEV_TX_OFFLOAD_GENERIC_TNL_CKSUM 0x00040000
> > >
> > > It looks redundant to me. Isn't it same as having
> DEV_TX_OFFLOAD_*_CKSUM?
> > > According to the API document, when PKT_TX_*_CKSUM is set, all the
> > > necessary length fields should be filled in (e.g. mbuf->outer_l?_len
> > > and
> > > mbuf->l?_len). It doesn't need to specify any tunnel type for checksum.
> > > For example, in order to request checksum offload for an unknown
> > > tunnel type which is similar to VxLAN, what should app do? Even
> > > without defining this new offload flag, it is currently possible by
> setting
> > > (PKT_TX_OUTER_IP_CKSUM | PKT_TX_OUTER_IPV4 | PKT_TX_IP_CKSUM |
> > > PKT_TX_IPV4 | PKT_TX_UDP_CKSUM)
> > > along with various length fields.
> >
> > Agree it almost same as existing checksum offloading requirement,
> besides:
> > 1. it does not demand pseudo checksum
> > 2. no need to clear IP checksum
> > I guess the requirement on API doc is a super set and differences
> > could be documented on PMD doc.
>
> Since commit 4fb7e803eb1a ("ethdev: add Tx preparation"), setting the IP
> checksum to 0 and the pseudo header checksum in TCP is not required.
> The user just need to set the flags, then before transmitting the packet,
> call tx_prepare() which will do the proper work, according to the hw
> requirements.
>
> It looks that the API documentation of PKT_TX_TCP_SEG should be updated.
> I'll submit a patch for this.
>
Nice to hide intel specific requirement and now it's much clear, thanks.
>
> > For mlx5 original chksum offloading, hw will parse everything w/o
> > demanding mbuf headers, a little better performance than SWP. The
> > question is how to choose between SWP and HW checksum offloading -
> > assuming PKT_TX_TUNNEL_UNKNOWN is the key. Take L3 VXLAN(no inner L2
> > header) as example, only SWP could offload it correctly and
> > PKT_TX_TUNNEL_UNKOWN should be used here, not PKT_TX_TUNNEL_VXLAN, make
> sense?
> >
> > Another question, how does user app know the NIC capability of generic
> > tunnel checksum offload?
>
> After reading the patchset, I still don't see what you want to achieve.
> In patch 1, we talk about these 2 packets:
>
> 1. eth / ipv4 / udp / VXLAN / ip / tcp (L3 VXLAN as you said above)
> 2. eth / ipv4 / GRE / MPLS / ipv4 / udp
>
> What is not doable with current API?
>
> I didn't try, but I think the following would work currently with
> tx_prepare() + tx_burst().
>
> - inner TSO on 1.
>
> flags = PKT_TX_TUNNEL_VXLAN | PKT_TX_IPV4 | PKT_TX_IP_CKSUM |
> PKT_TX_TCP_CKSUM | PKT_TX_TCP_SEG
> outer_l2_len = size(eth)
> outer_l3_len = size(ipv4)
> l2_len = size(udp + vxlan)
> l3_len = size(ip)
>
> - inner TCP checksum on 1.
>
> flags = PKT_TX_TUNNEL_VXLAN | PKT_TX_IPV4 | PKT_TX_IP_CKSUM |
> PKT_TX_TCP_CKSUM
> outer_l2_len = size(eth)
> outer_l3_len = size(ipv4)
> l2_len = size(udp + vxlan)
> l3_len = size(ip)
>
> - inner UDP checksum on 2.
>
> flags = PKT_TX_TUNNEL_GRE | PKT_TX_IPV4 | PKT_TX_IP_CKSUM |
> PKT_TX_UDP_CKSUM
> outer_l2_len = size(eth)
> outer_l3_len = size(ipv4)
> l2_len = size(gre + mpls)
> l3_len = size(ipv4)
>
> Yes, writing the tunnel len in l2_len is a bit strange, but it should do
> the trick.
>
> It's still unclear for me why you need to add a "generic" flag.
> Is there something you can't do today with the current API?
>
I think the problem is mlx5 current hw so smart that not generic enough,
that's why this patchset tried to introduce new hw feature and new generic flag.
Since current checksum API already covered, no reason to add new, thanks.
>
> >
> > >
> > > > +/**< Generic tunnel segmentation offload. To enable it, the user
> > > > +needs
> > > to:
> > > > + * - fill outer_l2_len and outer_l3_len in mbuf
> > > > + * - fill l2_len and l3_len in mbuf
> > > > + * - set the flags PKT_TX_TUNNEL_xxx (use PKT_TX_TUNNEL_UNKNOWN
> > > > +if
> > > > +undefined)
> > > > + * - set the flags PKT_TX_OUTER_IPV4 or PKT_TX_OUTER_IPV6
> > > > + * - if it's UDP tunnel, set the flags PKT_TX_OUTER_UDP
> > > > + * - set the flags PKT_TX_IPV4 or PKT_TX_IPV6
> > > > + * - set the PKT_TX_TCP_SEG flag in mbuf->ol_flags (this flag
> implies
> > > > + * PKT_TX_OUTER_IP_CKSUM, PKT_TX_IP_CKSUM and PKT_TX_TCP_CKSUM)
> > > > + * Hardware that supports generic tunnel TSO offload only update
> > > > +outer/inner
> > > > + * L3/L4 fields, tunnel fields are not touched.
> > > > + */
> > > > +#define DEV_TX_OFFLOAD_GENERIC_TNL_TSO 0x00080000
> > >
> > > Practically, for the existing TSO offloads for tunneled packets,
> > > tunnel type
> > > (PKT_TX_TUNNEL_*) is needed for knowing if transport is UDP-based or
> > > IP- based because the length field of UDP header should be updated for
> TSO.
> > > AFAIK, there's no TCP-based tunnel. For PKT_TX_TUNNEL_UNKNOWN, the
> > > only thing _unknown_ seems to be the type of transport and by
> > > defining PKT_TX_OUTER_UDP, you seem to recognize the type of
> > > transport between UDP and IP (it's not non-UDP). Given that, the new
> > > flags still look redundant and ambiguous. If PKT_TX_TUNNEL_VXLAN is
> > > set, there's no need to set PKT_TX_OUTER_UDP redundantly.
> > >
> > > Instead, I think defining PKT_TX_TUNNEL_UDP_UNKNOWN and
> > > PKT_TX_TUNNEL_IP_UNKNOWN could be a good alternative. It is only my
> > > humble two cents and I'd like to hear from more people regarding this.
> > >
> >
>
> I agree with Yongseok here. The flag PKT_TX_TUNNEL_VXLAN implies that the
> protocol is UDP. Having a flag PKT_TX_TUNNEL_UDP or PKT_TX_TUNNEL_IP could
> make sense if we find a use case that cannot be done currently with the
> existing API (for instance, another tunnel type over UDP).
PKT_TX_TUNNEL_UDP + PKT_TX_TUNNEL_IP looks similar to PKT_TX_TUNNEL_UNKOWN +
PKT_TX_OUTER_UDP, I'm fine with this proposal.
>
> > This flag looks more like a feature notification. Current tunnel
> > offloading capability like "DEV_TX_OFFLOAD_VXLAN_TNL_TSO" ties to
> > VXLAN tunnel type, DEV_TX_OFFLOAD_GENERIC_TNL_TSO should be able to
> > let users know that NIC's capability of offloading generic tunnel type
> > like MPLS-in-UDP that not defined yet. PKT_TX_OUTER_UDP is something
> > must to have to flag an exists of outer UDP in such case.
>
> So, maybe, instead of DEV_TX_OFFLOAD_GENERIC_TNL_TSO it could be
> DEV_TX_OFFLOAD_UDP_TUNNEL_TSO. Because it is not possible for a hardware
> to support TSO on top of any kind of tunnel type.
>
> For instance, if the tunnel header has a checksum or a sequence id, the
> hardware must be aware of it.
Similarly, I'll split DEV_TX_OFFLOAD_GENERIC_TNL_TSO into DEV_TX_OFFLOAD_UDP_TNL_TSO
and DEV_TX_OFFLOAD_IP_TNL_TSO to cover tunnel types that no checksum and seq id.
Overall speaking, existing app using tunnel offloading should work w/o any change.
I'm working on a new version to reflect this change, and split this patch set into
public and mlx5.
next prev parent reply other threads:[~2018-04-04 8:20 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-09 14:11 [dpdk-dev] [PATCH 0/6] Support generic tunnel TX csum and TSO Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 1/6] net/mlx5: support tx swp tunnel offloading Xueming Li
2018-01-29 15:08 ` [dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx generic tunnel offloads Xueming Li
2018-01-29 16:49 ` Ananyev, Konstantin
2018-01-30 3:01 ` Xueming(Steven) Li
2018-01-30 13:28 ` Ananyev, Konstantin
2018-01-30 15:27 ` Xueming(Steven) Li
2018-01-30 15:33 ` Ananyev, Konstantin
2018-01-30 15:47 ` Xueming(Steven) Li
2018-01-30 16:02 ` Ananyev, Konstantin
2018-01-30 16:10 ` Xueming(Steven) Li
2018-01-30 17:04 ` Ananyev, Konstantin
2018-01-30 17:54 ` Xueming(Steven) Li
2018-01-30 20:21 ` Thomas Monjalon
2018-01-31 15:20 ` Xueming(Steven) Li
2018-01-31 15:17 ` Xueming(Steven) Li
2018-01-29 15:08 ` [dpdk-dev] [PATCH v2 2/5] app/testpmd: testpmd support " Xueming Li
2018-01-29 15:08 ` [dpdk-dev] [PATCH v2 3/5] net/mlx5: separate TSO function in Tx data path Xueming Li
2018-01-29 15:08 ` [dpdk-dev] [PATCH v2 4/5] net/mlx5: support generic tunnel offloading Xueming Li
2018-01-29 15:08 ` [dpdk-dev] [PATCH v2 5/5] net/mlx5: allow max 192B TSO inline header length Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 0/7] support generic tunnel Tx checksum and TSO Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 1/7] ethdev: introduce Tx generic tunnel L3/L4 offload Xueming Li
2018-03-21 1:40 ` Yongseok Koh
2018-03-22 13:55 ` Xueming(Steven) Li
2018-03-28 12:52 ` Olivier Matz
2018-04-04 8:20 ` Xueming(Steven) Li [this message]
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 2/7] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 3/7] app/testpmd: add more GRE extension to csum engine Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 4/7] app/testpmd: introduce VXLAN GPE to csum forwarding engine Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 5/7] net/mlx5: separate TSO function in Tx data path Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 6/7] net/mlx5: support generic tunnel offloading Xueming Li
2018-03-05 14:51 ` [dpdk-dev] [PATCH v3 7/7] net/mlx5: allow max 192B TSO inline header length Xueming Li
2018-04-08 12:32 ` [dpdk-dev] [PATCH v4 0/4] support Tx generic tunnel checksum and TSO Xueming Li
2018-04-17 14:43 ` [dpdk-dev] [PATCH v5 0/2] " Xueming Li
2018-04-17 14:47 ` [dpdk-dev] [PATCH v5 1/2] ethdev: introduce generic IP/UDP " Xueming Li
2018-04-17 21:21 ` Thomas Monjalon
2018-04-17 14:49 ` [dpdk-dev] [PATCH v5 2/2] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-18 13:38 ` [dpdk-dev] [PATCH v6 0/2] support Tx generic tunnel checksum and TSO Xueming Li
2018-04-18 13:58 ` [dpdk-dev] [PATCH v6 1/2] ethdev: introduce generic IP/UDP " Xueming Li
2018-04-18 14:28 ` Thomas Monjalon
2018-04-18 16:45 ` Ananyev, Konstantin
2018-04-18 18:02 ` Thomas Monjalon
2018-04-23 9:55 ` Olivier Matz
2018-04-20 12:48 ` [dpdk-dev] [PATCH v7 0/2] support Tx generic " Xueming Li
2018-04-23 11:36 ` [dpdk-dev] [PATCH v8 " Xueming Li
2018-04-23 16:17 ` Ferruh Yigit
2018-04-23 11:36 ` [dpdk-dev] [PATCH v8 1/2] ethdev: introduce generic IP/UDP " Xueming Li
2018-04-23 11:49 ` Xueming Li
2018-04-23 11:36 ` [dpdk-dev] [PATCH v8 2/2] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-20 12:48 ` [dpdk-dev] [PATCH v7 1/2] ethdev: introduce generic IP/UDP tunnel checksum and TSO Xueming Li
2018-04-23 9:59 ` Olivier Matz
2018-04-20 12:48 ` [dpdk-dev] [PATCH v7 2/2] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-18 13:59 ` [dpdk-dev] [PATCH v6 " Xueming Li
2018-04-08 12:32 ` [dpdk-dev] [PATCH v4 1/4] ethdev: introduce generic IP/UDP tunnel checksum and TSO Xueming Li
2018-04-16 22:42 ` Thomas Monjalon
2018-04-17 7:53 ` Xueming(Steven) Li
2018-04-17 8:10 ` Thomas Monjalon
2018-04-08 12:32 ` [dpdk-dev] [PATCH v4 2/4] app/testpmd: testpmd support Tx generic tunnel offloads Xueming Li
2018-04-17 14:24 ` Iremonger, Bernard
2018-04-17 15:44 ` Xueming(Steven) Li
2018-04-08 12:32 ` [dpdk-dev] [PATCH v4 3/4] app/testpmd: add more GRE extension to csum engine Xueming Li
2018-04-16 22:45 ` Thomas Monjalon
2018-04-17 5:19 ` Xueming(Steven) Li
2018-04-08 12:32 ` [dpdk-dev] [PATCH v4 4/4] app/testpmd: introduce VXLAN GPE to csum forwarding engine Xueming Li
2018-04-16 22:46 ` Thomas Monjalon
2018-04-17 13:56 ` Iremonger, Bernard
2018-04-17 14:12 ` Xueming(Steven) Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 2/6] net/mlx5: allow max 192B WQE TSO inline header length Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 3/6] net/mlx5: add SWP PCI parameter for TX common tunnel offloads Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 4/6] ethdev: introduce " Xueming Li
2018-01-11 18:38 ` Ferruh Yigit
2018-01-16 17:10 ` Olivier Matz
2018-01-16 17:28 ` Xueming(Steven) Li
2018-01-16 19:06 ` Shahaf Shuler
2018-01-22 12:46 ` Olivier Matz
2018-01-22 20:06 ` Shahaf Shuler
2018-01-17 0:50 ` Yongseok Koh
2018-01-09 14:11 ` [dpdk-dev] [PATCH 5/6] net/mlx5: support " Xueming Li
2018-01-09 14:11 ` [dpdk-dev] [PATCH 6/6] app/testpmd: testpmd " Xueming Li
2018-01-16 3:09 ` Lu, Wenzhuo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=VI1PR05MB1678175E6641A7AD008D0683ACA40@VI1PR05MB1678.eurprd05.prod.outlook.com \
--to=xuemingl@mellanox.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=jingjing.wu@intel.com \
--cc=olivier.matz@6wind.com \
--cc=shahafs@mellanox.com \
--cc=thomas@monjalon.net \
--cc=wenzhuo.lu@intel.com \
--cc=yskoh@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).