Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

> -----Original Message-----
> From: Yongseok Koh [mailto:yskoh@mellanox.com]
> Sent: Thursday, August 23, 2018 10:32 PM
> To: Andrew Rybchenko <arybchenko@solarflare.com>
> Cc: Dekel Peled <dekelp@mellanox.com>; dev@dpdk.org; orika@mellanox.com; shahafs@mellanox.com; Thomas Monjalon
> <thomas@monjalon.net>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Adrien
> Mazarguil <adrien.mazarguil@6wind.com>; Olivier Matz <olivier.matz@6wind.com>
> Subject: Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria
> 
> On Wed, Aug 22, 2018 at 04:31:14PM +0300, Andrew Rybchenko wrote:
> > On 13.08.2018 10:46, Dekel Peled wrote:
> > > Current implementation of rte_flow allows match pattern of flow rule,
> > > based on packet data or header fields.
> > > This limits the application use of match patterns.
> > >
> > > For example, consider a vswitch application which controls a set of VMs,
> > > connected with virtio, in a fabric with overlay of VXLAN.
> > > Several VMs can have the same inner tuple, while the outer tuple is
> > > different and controlled by the vswitch (encap action).
> > > For the vswtich to be able to offload the rule to the NIC, it must use a
> > > unique match criteria, independent from the inner tuple, to perform the
> > > encap action.
> > >
> > > This RFC adds support for additional metadata to use as match pattern.
> > > The metadata is an opaque item, fully controlled by the application.
> > >
> > > The use of metadata is relevant for egress rules only.
> > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > >
> > > Application should set the packet metdata in the mbuf->metadata field,
> > > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > The NIC will use the packet metadata as match criteria for relevant flow
> > > rules.
> > >
> > > For example, to do an encap action depending on the VM id, the
> > > application needs to configure 'match on metadata' rte_flow rule with
> > > VM id as metadata, along with desired encap action.
> > > When preparing an egress data packet, application will set VM id data in
> > > mbuf metadata field and set PKT_TX_METADATA flag.
> > >
> > > PMD will send data packets to NIC, with VM id as metadata.
> > > Egress flow on NIC will match metadata as done with other criteria.
> > > Upon match on metadata (VM id) the appropriate encap action will be
> > > performed.
> > >
> > > This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
> > > along with corresponding struct rte_flow_item_meta and ol_flag
> > > PKT_TX_METADATA.
> > > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > >
> > > Comments are welcome.
> > >
> > > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > > ---
> > >   doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > >   lib/librte_ethdev/rte_flow.c       |  1 +
> > >   lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > >   lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> > >   4 files changed, 58 insertions(+)
> > >
> > > diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> > > index b305a72..b6e35f1 100644
> > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > >   - `Item: ICMP6_ND_NS`_
> > >   - `Item: ICMP6_ND_OPT`_
> > > +Item: ``META``
> > > +^^^^^^^^^^^^^^
> > > +
> > > +Matches an application specific 64 bit metadata item.
> > > +
> > > +- Default ``mask`` matches any 64 bit value.
> > > +
> > > +.. _table_rte_flow_item_meta:
> > > +
> > > +.. table:: META
> > > +
> > > +   +----------+----------+---------------------------+
> > > +   | Field    | Subfield | Value                     |
> > > +   +==========+==========+===========================+
> > > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > > +   +----------+--------------------------------------+
> > > +   | ``last`` | ``data`` | upper range value         |
> > > +   +----------+----------+---------------------------+
> > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > +   +----------+----------+---------------------------+
> > > +
> > >   Actions
> > >   ~~~~~~~
> > > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
> > > index cff4b52..54e5ef8 100644
> > > --- a/lib/librte_ethdev/rte_flow.c
> > > +++ b/lib/librte_ethdev/rte_flow.c
> > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > >   		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > >   	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > >   		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > >   };
> > >   /** Generate flow_action[] entry. */
> > > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> > > index f8ba71c..b81c816 100644
> > > --- a/lib/librte_ethdev/rte_flow.h
> > > +++ b/lib/librte_ethdev/rte_flow.h
> > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > >   	 * See struct rte_flow_item_mark.
> > >   	 */
> > >   	RTE_FLOW_ITEM_TYPE_MARK,
> > > +
> > > +	/**
> > > +	 * [META]
> > > +	 *
> > > +	 * Matches a metadata value specified in mbuf metadata field.
> > > +	 *
> > > +	 * See struct rte_flow_item_meta.
> > > +	 */
> > > +	RTE_FLOW_ITEM_TYPE_META,
> > >   };
> > >   /**
> > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {
> > >   #endif
> > >   /**
> > > + * RTE_FLOW_ITEM_TYPE_META.
> > > + *
> > > + * Matches a specified metadata value.
> > > + */
> > > +struct rte_flow_item_meta {
> > > +	uint64_t data;
> > > +};
> > > +
> > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
> > > +#ifndef __cplusplus
> > > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > > +	.data = RTE_BE64(UINT64_MAX),
> > > +};
> > > +#endif
> > > +
> > > +/**
> > >    * RTE_FLOW_ITEM_TYPE_FUZZY
> > >    *
> > >    * Fuzzy pattern match, expect faster than default.
> > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > index 9ce5d76..8f06a78 100644
> > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > @@ -182,6 +182,11 @@
> > >   /* add new TX flags here */
> > >   /**
> > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > + */
> > > +#define PKT_TX_METADATA		(1ULL << 41)
> > > +
> > > +/**
> > >    * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > >    * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> > >    * to store the MSS of UDP fragments.
> > > @@ -593,6 +598,12 @@ struct rte_mbuf {
> > >   	 */
> > >   	struct rte_mbuf_ext_shared_info *shinfo;
> > > +	/**
> > > +	 * Application specific metadata value for flow rule match.
> > > +	 * Valid if PKT_TX_METADATA is set.
> > > +	 */
> > > +	uint64_t metadata;
> > > +
> >
> > I don't see the difference from hash union which is 64-bit wide as well.
> > hash.fdir.hi is used by flow mark action and mark match item (but just
> > 32-bit).
> 
> Rx metadata would be different from flow mark ID. Mark ID is set when the flow
> is created (it is a kind of marking classification result) but metadata could be
> sent by other entity, e.g. VM-to-VM traffic or VM-to-HV traffic.

Ok, but it could be either rss OR flow id OR metdata (based on ol_flags) -
hash is a union after all.
Konstantin

> 
> Thanks,
> Yongseok
> 
> >
> > >   } __rte_cache_aligned;
> > >   /**
> >