DPDK patches and discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
	Ivan Malov <Ivan.Malov@oktetlabs.ru>
Cc: dev@dpdk.org, Andy Moreton <amoreton@xilinx.com>,
	orika@nvidia.com, ferruh.yigit@intel.com, olivier.matz@6wind.com
Subject: Re: [dpdk-dev] [PATCH v3 0/5] A means to negotiate delivery of Rx meta data
Date: Fri, 01 Oct 2021 11:48:52 +0200	[thread overview]
Message-ID: <2522405.PTVv94qZMn@thomas> (raw)
In-Reply-To: <daf72ff3-e0d1-89c8-a597-5d60fab5152e@oktetlabs.ru>

01/10/2021 10:55, Ivan Malov:
> On 01/10/2021 11:11, Thomas Monjalon wrote:
> > 01/10/2021 08:47, Andrew Rybchenko:
> >> On 9/30/21 10:30 PM, Ivan Malov wrote:
> >>> On 30/09/2021 19:18, Thomas Monjalon wrote:
> >>>> 23/09/2021 13:20, Ivan Malov:
> >>>>> In 2019, commit [1] announced changes in DEV_RX_OFFLOAD namespace
> >>>>> intending to add new flags, RSS_HASH and FLOW_MARK. Since then,
> >>>>> only the former has been added. The problem hasn't been solved.
> >>>>> Applications still assume that no efforts are needed to enable
> >>>>> flow mark and similar meta data delivery.
> >>>>>
> >>>>> The team behind net/sfc driver has to take over the efforts since
> >>>>> the problem has started impacting us. Riverhead, a cutting edge
> >>>>> Xilinx smart NIC family, has two Rx prefix types. Rx meta data
> >>>>> is available only from long Rx prefix. Switching between the
> >>>>> prefix formats can't happen in started state. Hence, we run
> >>>>> into the same problem which [1] was aiming to solve.
> >>>>
> >>>> Sorry I don't understand what is Rx prefix?
> >>>
> >>> A small chunk of per-packet metadata in Rx packet buffer preceding the
> >>> actual packet data. In terms of mbuf, this could be something lying
> >>> before m->data_off.
> > 
> > I've never seen the word "Rx prefix".
> > In general we talk about mbuf headroom and mbuf metadata,
> > the rest being the mbuf payload and mbuf tailroom.
> > I guess you mean mbuf metadata in the space of the struct rte_mbuf?
> 
> In this paragraph I describe the two ways how the NIC itself can provide 
> metadata buffers of different sizes. Hence the term "Rx prefix". As you 
> understand, the NIC HW is unaware of DPDK, mbufs and whatever else SW 
> concepts. To NIC, this is "Rx prefix", that is, a chunk of per-packet 
> metadata *preceding* the actual packet data. It's responsibility of the 
> PMD to treat this the right way, care about headroom, payload and 
> tailroom. I describe the two Rx prefix formats in NIC terminology just 
> to provide the gist of the problem.

OK but it is confusing as it is vendor-specific.
Please stick with DPDK terms if possible.

> >>>>> Rx meta data (mark, flag, tunnel ID) delivery is not an offload
> >>>>> on its own since the corresponding flows must be active to set
> >>>>> the data in the first place. Hence, adding offload flags
> >>>>> similar to RSS_HASH is not a good idea.
> >>>>
> >>>> What means "active" here?
> >>>
> >>> Active = inserted and functional. What this paragraph is trying to say
> >>> is that when you enable, say, RSS_HASH, that implies both computation of
> >>> the hash and the driver's ability to extract in from packets
> >>> ("delivery"). But when it comes to MARK, it's just "delivery". No
> >>> "offload" here: the NIC won't set any mark in packets unless you create
> >>> a flow rule to make it do so. That's the gist of it.
> > 
> > OK
> > Yes I agree RTE_FLOW_ACTION_TYPE_MARK doesn't need any offload flag.
> > Same for RTE_FLOW_ACTION_TYPE_SET_META.
> > 
> >>>>> Patch [1/5] of this series adds a generic API to let applications
> >>>>> negotiate delivery of Rx meta data during initialisation period.
> > 
> > What is a metadata?
> > Do you mean RTE_FLOW_ITEM_TYPE_META and RTE_FLOW_ITEM_TYPE_MARK?
> > Metadata word could cover any field in the mbuf struct so it is vague.
> 
> Metadata here is *any* additional information provided by the NIC for 
> each received packet. For example, Rx flag, Rx mark, RSS hash, packet 
> classification info, you name it. I'd like to stress out that the 
> suggested API comes with flags each of which is crystal clear on what 
> concrete kind of metadata it covers, eg. Rx mark.

I missed the flags.
You mean these 3 flags?

+/** The ethdev sees flagged packets if there are flows with action FLAG. */
+#define RTE_ETH_RX_META_USER_FLAG (UINT64_C(1) << 0)
+
+/** The ethdev sees mark IDs in packets if there are flows with action MARK. */
+#define RTE_ETH_RX_META_USER_MARK (UINT64_C(1) << 1)
+
+/** The ethdev detects missed packets if there are "tunnel_set" flows in use. */
+#define RTE_ETH_RX_META_TUNNEL_ID (UINT64_C(1) << 2)

It is not crystal clear because it does not reference the API,
like RTE_FLOW_ACTION_TYPE_MARK.
And it covers a limited set of metadata.
Do you intend to extend to all mbuf metadata?

> >>>>> This way, an application knows right from the start which parts
> >>>>> of Rx meta data won't be delivered. Hence, no necessity to try
> >>>>> inserting flows requesting such data and handle the failures.
> >>>>
> >>>> Sorry I don't understand the problem you want to solve.
> >>>> And sorry for not noticing earlier.
> >>>
> >>> No worries. *Some* PMDs do not enable delivery of, say, Rx mark with the
> >>> packets by default (for performance reasons). If the application tries
> >>> to insert a flow with action MARK, the PMD may not be able to enable
> >>> delivery of Rx mark without the need to re-start Rx sub-system. And
> >>> that's fraught with traffic disruption and similar bad consequences. In
> >>> order to address it, we need to let the application express its interest
> >>> in receiving mark with packets as early as possible. This way, the PMD
> >>> can enable Rx mark delivery in advance. And, as an additional benefit,
> >>> the application can learn *from the very beginning* whether it will be
> >>> possible to use the feature or not. If this API tells the application
> >>> that no mark delivery will be enabled, then the application can just
> >>> skip many unnecessary attempts to insert wittingly unsupported flows
> >>> during runtime.
> > 
> > I'm puzzled, because we could have the same reasoning for any offload.
> 
> We're not discussing *offloads*. An offload is when NIC *computes 
> something* and *delivers* it. We are discussing precisely *delivery*.

OK but still, there are a lot more mbuf metadata delivered.

> > I don't understand why we are focusing on mark only
> 
> We are not focusing on mark on purpose. It's just how our discussion 
> goes. I chose mark (could've chosen flag or anything else) just to show 
> you an example.
> 
> > I would prefer we find a generic solution using the rte_flow API. > Can we make rte_flow_validate() working before port start?
> > If validating a fake rule doesn't make sense,
> > why not having a new function accepting a single action as parameter?
> 
> A noble idea, but if we feed the entire flow rule to the driver for 
> validation, then the driver must not look specifically for actions FLAG 
> or MARK in it (to enable or disable metadata delivery). This way, the 
> driver is obliged to also validate match criteria, attributes, etc. And, 
> if something is unsupported (say, some specific item), the driver will 
> have to reject the rule as a whole thus leaving the application to join 
> the dots itself.
>
> Say, you ask the driver to validate the following rule:
> pattern blah-blah-1 / blah-blah-2 / end action flag / end
> intending to check support for FLAG delivery. Suppose, the driver 
> doesn't support pattern item "blah-blah-1". It will throw an error right 
> after seeing this unsupported item and won't even go further to see the 
> action FLAG. How can application know whether its request for FLAG was 
> heard or not?

No, I'm proposing a new function to validate the action alone,
without any match etc.
Example:
	rte_flow_action_request(RTE_FLOW_ACTION_TYPE_MARK)


> And I'd not bind delivery of metadata to flow API. Consider the 
> following example. We have a DPDK application sitting at the *host* and 
> we have a *guest* with its *own* DPDK instance. The guest DPDK has asked 
> the NIC (by virtue of flow API) to mark all outgoing packets. This 
> packets reach the *host* DPDK. Say, the host application just wants to 
> see the marked packets from the guest. Its own, (the host's) use of flow 
> API is a don't care here. The host doesn't want to mark packets itself, 
> it wants to see packets marked by the guest.

It does not make sense to me. We are talking about a DPDK API.
My concern is to avoid redefining new flags
while we already have rte_flow actions.




  reply	other threads:[~2021-10-01  9:48 UTC|newest]

Thread overview: 97+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-02 14:23 [dpdk-dev] [PATCH 0/5] A means to negotiate support for Rx meta information Ivan Malov
2021-09-02 14:23 ` [dpdk-dev] [PATCH 1/5] ethdev: add API " Ivan Malov
2021-09-02 14:47   ` Jerin Jacob
2021-09-02 16:14   ` Kinsella, Ray
2021-09-03  9:34   ` Jerin Jacob
2021-09-02 14:23 ` [dpdk-dev] [PATCH 2/5] net/sfc: provide API to negotiate supported Rx meta features Ivan Malov
2021-09-02 14:23 ` [dpdk-dev] [PATCH 3/5] net/sfc: allow to use EF100 native datapath Rx mark in flows Ivan Malov
2021-09-02 14:23 ` [dpdk-dev] [PATCH 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-09-02 14:23 ` [dpdk-dev] [PATCH 5/5] net/sfc: allow to discern user flag on EF100 native datapath Ivan Malov
2021-09-03  0:15 ` [dpdk-dev] [PATCH v2 0/5] A means to negotiate support for Rx meta information Ivan Malov
2021-09-03  0:15   ` [dpdk-dev] [PATCH v2 1/5] ethdev: add API " Ivan Malov
2021-09-03  0:15   ` [dpdk-dev] [PATCH v2 2/5] net/sfc: provide API to negotiate supported Rx meta features Ivan Malov
2021-09-03  0:15   ` [dpdk-dev] [PATCH v2 3/5] net/sfc: allow to use EF100 native datapath Rx mark in flows Ivan Malov
2021-09-03  0:15   ` [dpdk-dev] [PATCH v2 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-09-03  0:15   ` [dpdk-dev] [PATCH v2 5/5] net/sfc: allow to discern user flag on EF100 native datapath Ivan Malov
2021-09-23 11:20 ` [dpdk-dev] [PATCH v3 0/5] A means to negotiate delivery of Rx meta data Ivan Malov
2021-09-23 11:20   ` [dpdk-dev] [PATCH v3 1/5] ethdev: add API " Ivan Malov
2021-09-30 14:59     ` Ori Kam
2021-09-30 15:07       ` Andrew Rybchenko
2021-09-30 19:07       ` Ivan Malov
2021-10-01  6:50         ` Andrew Rybchenko
2021-10-03  7:42           ` Ori Kam
2021-10-03  9:30             ` Ivan Malov
2021-10-03 11:01               ` Ori Kam
2021-10-03 17:30                 ` Ivan Malov
2021-10-03 21:04                   ` Ori Kam
2021-10-03 23:50                     ` Ivan Malov
2021-10-04  6:56                       ` Ori Kam
2021-10-04 11:39                         ` Ivan Malov
2021-10-04 13:53                           ` Andrew Rybchenko
2021-10-05  6:30                             ` Ori Kam
2021-10-05  7:27                               ` Andrew Rybchenko
2021-10-05  8:17                                 ` Ori Kam
2021-10-05  8:38                                   ` Andrew Rybchenko
2021-10-05  9:41                                     ` Ori Kam
2021-10-05 10:01                                       ` Andrew Rybchenko
2021-10-05 10:10                                         ` Ori Kam
2021-10-05 11:11                                           ` Andrew Rybchenko
2021-10-06  8:30                                             ` Thomas Monjalon
2021-10-06  8:38                                               ` Andrew Rybchenko
2021-10-06  9:14                                                 ` Ori Kam
2021-09-30 21:48     ` Ajit Khaparde
2021-09-30 22:00       ` Ivan Malov
2021-09-30 22:12         ` Ajit Khaparde
2021-09-30 22:22           ` Ivan Malov
2021-10-03  7:05             ` Ori Kam
2021-09-23 11:20   ` [dpdk-dev] [PATCH v3 2/5] net/sfc: support " Ivan Malov
2021-09-23 11:20   ` [dpdk-dev] [PATCH v3 3/5] net/sfc: support flow mark delivery on EF100 native datapath Ivan Malov
2021-09-23 11:20   ` [dpdk-dev] [PATCH v3 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-09-23 11:20   ` [dpdk-dev] [PATCH v3 5/5] net/sfc: report user flag on EF100 native datapath Ivan Malov
2021-09-30 16:18   ` [dpdk-dev] [PATCH v3 0/5] A means to negotiate delivery of Rx meta data Thomas Monjalon
2021-09-30 19:30     ` Ivan Malov
2021-10-01  6:47       ` Andrew Rybchenko
2021-10-01  8:11         ` Thomas Monjalon
2021-10-01  8:54           ` Andrew Rybchenko
2021-10-01  9:32             ` Thomas Monjalon
2021-10-01  9:41               ` Andrew Rybchenko
2021-10-01  8:55           ` Ivan Malov
2021-10-01  9:48             ` Thomas Monjalon [this message]
2021-10-01 10:15               ` Andrew Rybchenko
2021-10-01 12:10                 ` Thomas Monjalon
2021-10-04  9:17                   ` Andrew Rybchenko
2021-10-04 23:50   ` [dpdk-dev] [PATCH v4 0/5] Negotiate the NIC's ability to deliver Rx metadata to the PMD Ivan Malov
2021-10-04 23:50     ` [dpdk-dev] [PATCH v4 1/5] ethdev: negotiate delivery of packet metadata from HW to PMD Ivan Malov
2021-10-05 12:03       ` Ori Kam
2021-10-05 12:50         ` Ivan Malov
2021-10-05 13:17           ` Andrew Rybchenko
2021-10-04 23:50     ` [dpdk-dev] [PATCH v4 2/5] net/sfc: support API to negotiate delivery of Rx metadata Ivan Malov
2021-10-04 23:50     ` [dpdk-dev] [PATCH v4 3/5] net/sfc: support flow mark delivery on EF100 native datapath Ivan Malov
2021-10-04 23:50     ` [dpdk-dev] [PATCH v4 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-10-04 23:50     ` [dpdk-dev] [PATCH v4 5/5] net/sfc: report user flag on EF100 native datapath Ivan Malov
2021-10-05 15:56   ` [dpdk-dev] [PATCH v5 0/5] ethdev: negotiate the NIC's ability to deliver Rx metadata to the PMD Ivan Malov
2021-10-05 15:56     ` [dpdk-dev] [PATCH v5 1/5] ethdev: negotiate delivery of packet metadata from HW to PMD Ivan Malov
2021-10-05 21:40       ` Ajit Khaparde
2021-10-06  6:04         ` Somnath Kotur
2021-10-06  6:10           ` Ori Kam
2021-10-06  7:22             ` Wisam Monther
2021-10-05 15:56     ` [dpdk-dev] [PATCH v5 2/5] net/sfc: support API to negotiate delivery of Rx metadata Ivan Malov
2021-10-05 15:56     ` [dpdk-dev] [PATCH v5 3/5] net/sfc: support flow mark delivery on EF100 native datapath Ivan Malov
2021-10-05 15:56     ` [dpdk-dev] [PATCH v5 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-10-05 15:56     ` [dpdk-dev] [PATCH v5 5/5] net/sfc: report user flag on EF100 native datapath Ivan Malov
2021-10-12 18:08       ` Ferruh Yigit
2021-10-12 19:39         ` Ivan Malov
2021-10-12 19:48         ` Ivan Malov
2021-10-12 19:38   ` [dpdk-dev] [PATCH v6 0/5] ethdev: negotiate the NIC's ability to deliver Rx metadata to the PMD Ivan Malov
2021-10-12 19:38     ` [dpdk-dev] [PATCH v6 1/5] ethdev: negotiate delivery of packet metadata from HW to PMD Ivan Malov
2021-10-12 19:38     ` [dpdk-dev] [PATCH v6 2/5] net/sfc: support API to negotiate delivery of Rx metadata Ivan Malov
2021-10-12 19:38     ` [dpdk-dev] [PATCH v6 3/5] net/sfc: support flow mark delivery on EF100 native datapath Ivan Malov
2021-10-12 19:38     ` [dpdk-dev] [PATCH v6 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-10-12 19:38     ` [dpdk-dev] [PATCH v6 5/5] net/sfc: report user flag on EF100 native datapath Ivan Malov
2021-10-12 19:46   ` [dpdk-dev] [PATCH v7 0/5] ethdev: negotiate the NIC's ability to deliver Rx metadata to the PMD Ivan Malov
2021-10-12 19:46     ` [dpdk-dev] [PATCH v7 1/5] ethdev: negotiate delivery of packet metadata from HW to PMD Ivan Malov
2021-10-12 19:46     ` [dpdk-dev] [PATCH v7 2/5] net/sfc: support API to negotiate delivery of Rx metadata Ivan Malov
2021-10-12 19:46     ` [dpdk-dev] [PATCH v7 3/5] net/sfc: support flow mark delivery on EF100 native datapath Ivan Malov
2021-10-12 19:46     ` [dpdk-dev] [PATCH v7 4/5] common/sfc_efx/base: add RxQ flag to use Rx prefix user flag Ivan Malov
2021-10-12 19:46     ` [dpdk-dev] [PATCH v7 5/5] net/sfc: report user flag on EF100 native datapath Ivan Malov
2021-10-12 23:25     ` [dpdk-dev] [PATCH v7 0/5] ethdev: negotiate the NIC's ability to deliver Rx metadata to the PMD Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2522405.PTVv94qZMn@thomas \
    --to=thomas@monjalon.net \
    --cc=Ivan.Malov@oktetlabs.ru \
    --cc=amoreton@xilinx.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=olivier.matz@6wind.com \
    --cc=orika@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).