[dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Dekel Peled]

Current implementation of rte_flow allows match pattern of flow rule,
based on packet data or header fields.
This limits the application use of match patterns.

For example, consider a vswitch application which controls a set of VMs,
connected with virtio, in a fabric with overlay of VXLAN.
Several VMs can have the same inner tuple, while the outer tuple is
different and controlled by the vswitch (encap action).
For the vswtich to be able to offload the rule to the NIC, it must use a
unique match criteria, independent from the inner tuple, to perform the
encap action.

This RFC adds support for additional metadata to use as match pattern.
The metadata is an opaque item, fully controlled by the application.

The use of metadata is relevant for egress rules only.
It can be set in the flow rule using the RTE_FLOW_ITEM_META.

Application should set the packet metdata in the mbuf->metadata field,
and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
The NIC will use the packet metadata as match criteria for relevant flow
rules.

For example, to do an encap action depending on the VM id, the
application needs to configure 'match on metadata' rte_flow rule with
VM id as metadata, along with desired encap action.
When preparing an egress data packet, application will set VM id data in
mbuf metadata field and set PKT_TX_METADATA flag.

PMD will send data packets to NIC, with VM id as metadata.
Egress flow on NIC will match metadata as done with other criteria.
Upon match on metadata (VM id) the appropriate encap action will be
performed.

This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
along with corresponding struct rte_flow_item_meta and ol_flag
PKT_TX_METADATA.
It also enhances struct rte_mbuf with new data item, uint64_t metadata.

Comments are welcome.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
---
 doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
 lib/librte_ethdev/rte_flow.c       |  1 +
 lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
 lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
 4 files changed, 58 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index b305a72..b6e35f1 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1191,6 +1191,27 @@ Normally preceded by any of:
 - `Item: ICMP6_ND_NS`_
 - `Item: ICMP6_ND_OPT`_
 
+Item: ``META``
+^^^^^^^^^^^^^^
+
+Matches an application specific 64 bit metadata item.
+
+- Default ``mask`` matches any 64 bit value.
+
+.. _table_rte_flow_item_meta:
+
+.. table:: META
+
+   +----------+----------+---------------------------+
+   | Field    | Subfield | Value                     |
+   +==========+==========+===========================+
+   | ``spec`` | ``data`` | 64 bit metadata value     |
+   +----------+--------------------------------------+
+   | ``last`` | ``data`` | upper range value         |
+   +----------+----------+---------------------------+
+   | ``mask`` | ``data`` | zeroed to match any value |
+   +----------+----------+---------------------------+
+
 Actions
 ~~~~~~~
 
diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
index cff4b52..54e5ef8 100644
--- a/lib/librte_ethdev/rte_flow.c
+++ b/lib/librte_ethdev/rte_flow.c
@@ -66,6 +66,7 @@ struct rte_flow_desc_data {
 		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
 	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
 		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
+	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
index f8ba71c..b81c816 100644
--- a/lib/librte_ethdev/rte_flow.h
+++ b/lib/librte_ethdev/rte_flow.h
@@ -413,6 +413,15 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_mark.
 	 */
 	RTE_FLOW_ITEM_TYPE_MARK,
+
+	/**
+	 * [META]
+	 *
+	 * Matches a metadata value specified in mbuf metadata field.
+	 *
+	 * See struct rte_flow_item_meta.
+	 */
+	RTE_FLOW_ITEM_TYPE_META,
 };
 
 /**
@@ -849,6 +858,22 @@ struct rte_flow_item_gre {
 #endif
 
 /**
+ * RTE_FLOW_ITEM_TYPE_META.
+ *
+ * Matches a specified metadata value.
+ */
+struct rte_flow_item_meta {
+	uint64_t data;
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
+	.data = RTE_BE64(UINT64_MAX),
+};
+#endif
+
+/**
  * RTE_FLOW_ITEM_TYPE_FUZZY
  *
  * Fuzzy pattern match, expect faster than default.
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 9ce5d76..8f06a78 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -182,6 +182,11 @@
 /* add new TX flags here */
 
 /**
+ * This flag indicates that the metadata field in the mbuf is in use.
+ */
+#define PKT_TX_METADATA		(1ULL << 41)
+
+/**
  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
  * to store the MSS of UDP fragments.
@@ -593,6 +598,12 @@ struct rte_mbuf {
 	 */
 	struct rte_mbuf_ext_shared_info *shinfo;
 
+	/**
+	 * Application specific metadata value for flow rule match.
+	 * Valid if PKT_TX_METADATA is set.
+	 */
+	uint64_t metadata;
+
 } __rte_cache_aligned;
 
 /**
-- 
1.8.3.1

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Dekel Peled]

Adding relevant maintainers.
 
> -----Original Message-----
> From: Dekel Peled [mailto:dekelp@mellanox.com]
> Sent: Monday, August 13, 2018 10:47 AM
> To: dev@dpdk.org
> Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
> <shahafs@mellanox.com>
> Subject: [RFC] ethdev: support metadata as flow rule criteria
> 
> Current implementation of rte_flow allows match pattern of flow rule, based
> on packet data or header fields.
> This limits the application use of match patterns.
> 
> For example, consider a vswitch application which controls a set of VMs,
> connected with virtio, in a fabric with overlay of VXLAN.
> Several VMs can have the same inner tuple, while the outer tuple is different
> and controlled by the vswitch (encap action).
> For the vswtich to be able to offload the rule to the NIC, it must use a unique
> match criteria, independent from the inner tuple, to perform the encap
> action.
> 
> This RFC adds support for additional metadata to use as match pattern.
> The metadata is an opaque item, fully controlled by the application.
> 
> The use of metadata is relevant for egress rules only.
> It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> 
> Application should set the packet metdata in the mbuf->metadata field, and
> set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> The NIC will use the packet metadata as match criteria for relevant flow rules.
> 
> For example, to do an encap action depending on the VM id, the application
> needs to configure 'match on metadata' rte_flow rule with VM id as
> metadata, along with desired encap action.
> When preparing an egress data packet, application will set VM id data in
> mbuf metadata field and set PKT_TX_METADATA flag.
> 
> PMD will send data packets to NIC, with VM id as metadata.
> Egress flow on NIC will match metadata as done with other criteria.
> Upon match on metadata (VM id) the appropriate encap action will be
> performed.
> 
> This RFC introduces metadata item type for rte_flow
> RTE_FLOW_ITEM_META, along with corresponding struct
> rte_flow_item_meta and ol_flag PKT_TX_METADATA.
> It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> 
> Comments are welcome.
> 
> Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
>  lib/librte_ethdev/rte_flow.c       |  1 +
>  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
>  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
>  4 files changed, 58 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst
> b/doc/guides/prog_guide/rte_flow.rst
> index b305a72..b6e35f1 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1191,6 +1191,27 @@ Normally preceded by any of:
>  - `Item: ICMP6_ND_NS`_
>  - `Item: ICMP6_ND_OPT`_
> 
> +Item: ``META``
> +^^^^^^^^^^^^^^
> +
> +Matches an application specific 64 bit metadata item.
> +
> +- Default ``mask`` matches any 64 bit value.
> +
> +.. _table_rte_flow_item_meta:
> +
> +.. table:: META
> +
> +   +----------+----------+---------------------------+
> +   | Field    | Subfield | Value                     |
> +   +==========+==========+===========================+
> +   | ``spec`` | ``data`` | 64 bit metadata value     |
> +   +----------+--------------------------------------+
> +   | ``last`` | ``data`` | upper range value         |
> +   +----------+----------+---------------------------+
> +   | ``mask`` | ``data`` | zeroed to match any value |
> +   +----------+----------+---------------------------+
> +
>  Actions
>  ~~~~~~~
> 
> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c index
> cff4b52..54e5ef8 100644
> --- a/lib/librte_ethdev/rte_flow.c
> +++ b/lib/librte_ethdev/rte_flow.c
> @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
>  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
>  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
>  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
>  };
> 
>  /** Generate flow_action[] entry. */
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> f8ba71c..b81c816 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -413,6 +413,15 @@ enum rte_flow_item_type {
>  	 * See struct rte_flow_item_mark.
>  	 */
>  	RTE_FLOW_ITEM_TYPE_MARK,
> +
> +	/**
> +	 * [META]
> +	 *
> +	 * Matches a metadata value specified in mbuf metadata field.
> +	 *
> +	 * See struct rte_flow_item_meta.
> +	 */
> +	RTE_FLOW_ITEM_TYPE_META,
>  };
> 
>  /**
> @@ -849,6 +858,22 @@ struct rte_flow_item_gre {  #endif
> 
>  /**
> + * RTE_FLOW_ITEM_TYPE_META.
> + *
> + * Matches a specified metadata value.
> + */
> +struct rte_flow_item_meta {
> +	uint64_t data;
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef __cplusplus
> +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> +	.data = RTE_BE64(UINT64_MAX),
> +};
> +#endif
> +
> +/**
>   * RTE_FLOW_ITEM_TYPE_FUZZY
>   *
>   * Fuzzy pattern match, expect faster than default.
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index
> 9ce5d76..8f06a78 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -182,6 +182,11 @@
>  /* add new TX flags here */
> 
>  /**
> + * This flag indicates that the metadata field in the mbuf is in use.
> + */
> +#define PKT_TX_METADATA		(1ULL << 41)
> +
> +/**
>   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
>   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
>   * to store the MSS of UDP fragments.
> @@ -593,6 +598,12 @@ struct rte_mbuf {
>  	 */
>  	struct rte_mbuf_ext_shared_info *shinfo;
> 
> +	/**
> +	 * Application specific metadata value for flow rule match.
> +	 * Valid if PKT_TX_METADATA is set.
> +	 */
> +	uint64_t metadata;
> +
>  } __rte_cache_aligned;
> 
>  /**
> --
> 1.8.3.1

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Ananyev, Konstantin]

> 
> > -----Original Message-----
> > From: Dekel Peled [mailto:dekelp@mellanox.com]
> > Sent: Monday, August 13, 2018 10:47 AM
> > To: dev@dpdk.org
> > Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
> > <shahafs@mellanox.com>
> > Subject: [RFC] ethdev: support metadata as flow rule criteria
> >
> > Current implementation of rte_flow allows match pattern of flow rule, based
> > on packet data or header fields.
> > This limits the application use of match patterns.
> >
> > For example, consider a vswitch application which controls a set of VMs,
> > connected with virtio, in a fabric with overlay of VXLAN.
> > Several VMs can have the same inner tuple, while the outer tuple is different
> > and controlled by the vswitch (encap action).
> > For the vswtich to be able to offload the rule to the NIC, it must use a unique
> > match criteria, independent from the inner tuple, to perform the encap
> > action.
> >
> > This RFC adds support for additional metadata to use as match pattern.
> > The metadata is an opaque item, fully controlled by the application.
> >
> > The use of metadata is relevant for egress rules only.
> > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> >
> > Application should set the packet metdata in the mbuf->metadata field, and
> > set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > The NIC will use the packet metadata as match criteria for relevant flow rules.
> >
> > For example, to do an encap action depending on the VM id, the application
> > needs to configure 'match on metadata' rte_flow rule with VM id as
> > metadata, along with desired encap action.
> > When preparing an egress data packet, application will set VM id data in
> > mbuf metadata field and set PKT_TX_METADATA flag.
> >
> > PMD will send data packets to NIC, with VM id as metadata.
> > Egress flow on NIC will match metadata as done with other criteria.
> > Upon match on metadata (VM id) the appropriate encap action will be
> > performed.
> >
> > This RFC introduces metadata item type for rte_flow
> > RTE_FLOW_ITEM_META, along with corresponding struct
> > rte_flow_item_meta and ol_flag PKT_TX_METADATA.
> > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> >
> > Comments are welcome.
> >
> > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > ---
> >  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> >  lib/librte_ethdev/rte_flow.c       |  1 +
> >  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> >  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> >  4 files changed, 58 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index b305a72..b6e35f1 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> >  - `Item: ICMP6_ND_NS`_
> >  - `Item: ICMP6_ND_OPT`_
> >
> > +Item: ``META``
> > +^^^^^^^^^^^^^^
> > +
> > +Matches an application specific 64 bit metadata item.
> > +
> > +- Default ``mask`` matches any 64 bit value.
> > +
> > +.. _table_rte_flow_item_meta:
> > +
> > +.. table:: META
> > +
> > +   +----------+----------+---------------------------+
> > +   | Field    | Subfield | Value                     |
> > +   +==========+==========+===========================+
> > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > +   +----------+--------------------------------------+
> > +   | ``last`` | ``data`` | upper range value         |
> > +   +----------+----------+---------------------------+
> > +   | ``mask`` | ``data`` | zeroed to match any value |
> > +   +----------+----------+---------------------------+
> > +
> >  Actions
> >  ~~~~~~~
> >
> > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c index
> > cff4b52..54e5ef8 100644
> > --- a/lib/librte_ethdev/rte_flow.c
> > +++ b/lib/librte_ethdev/rte_flow.c
> > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> >  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> >  };
> >
> >  /** Generate flow_action[] entry. */
> > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> > f8ba71c..b81c816 100644
> > --- a/lib/librte_ethdev/rte_flow.h
> > +++ b/lib/librte_ethdev/rte_flow.h
> > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> >  	 * See struct rte_flow_item_mark.
> >  	 */
> >  	RTE_FLOW_ITEM_TYPE_MARK,
> > +
> > +	/**
> > +	 * [META]
> > +	 *
> > +	 * Matches a metadata value specified in mbuf metadata field.
> > +	 *
> > +	 * See struct rte_flow_item_meta.
> > +	 */
> > +	RTE_FLOW_ITEM_TYPE_META,
> >  };
> >
> >  /**
> > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {  #endif
> >
> >  /**
> > + * RTE_FLOW_ITEM_TYPE_META.
> > + *
> > + * Matches a specified metadata value.
> > + */
> > +struct rte_flow_item_meta {
> > +	uint64_t data;
> > +};
> > +
> > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef __cplusplus
> > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > +	.data = RTE_BE64(UINT64_MAX),
> > +};
> > +#endif
> > +
> > +/**
> >   * RTE_FLOW_ITEM_TYPE_FUZZY
> >   *
> >   * Fuzzy pattern match, expect faster than default.
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index
> > 9ce5d76..8f06a78 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -182,6 +182,11 @@
> >  /* add new TX flags here */
> >
> >  /**
> > + * This flag indicates that the metadata field in the mbuf is in use.
> > + */
> > +#define PKT_TX_METADATA		(1ULL << 41)
> > +
> > +/**
> >   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> >   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> >   * to store the MSS of UDP fragments.
> > @@ -593,6 +598,12 @@ struct rte_mbuf {
> >  	 */
> >  	struct rte_mbuf_ext_shared_info *shinfo;
> >
> > +	/**
> > +	 * Application specific metadata value for flow rule match.
> > +	 * Valid if PKT_TX_METADATA is set.
> > +	 */
> > +	uint64_t metadata;
> > +

Just one thought - with that change we'll have only 8 free bytes left inside rte_mbuf.
Wonder tan this metadata field be combined within tx_offload or probably hash fields?
Konstantin


> >  } __rte_cache_aligned;
> >
> >  /**
> > --
> > 1.8.3.1

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Dekel Peled]

Thanks, PSB.

> -----Original Message-----
> From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com]
> Sent: Tuesday, August 21, 2018 4:08 PM
> To: Dekel Peled <dekelp@mellanox.com>; dev@dpdk.org; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; olivier.matz@6wind.com
> Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
> <shahafs@mellanox.com>
> Subject: RE: [RFC] ethdev: support metadata as flow rule criteria
> 
> 
> 
> >
> > > -----Original Message-----
> > > From: Dekel Peled [mailto:dekelp@mellanox.com]
> > > Sent: Monday, August 13, 2018 10:47 AM
> > > To: dev@dpdk.org
> > > Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
> > > <shahafs@mellanox.com>
> > > Subject: [RFC] ethdev: support metadata as flow rule criteria
> > >
> > > Current implementation of rte_flow allows match pattern of flow
> > > rule, based on packet data or header fields.
> > > This limits the application use of match patterns.
> > >
> > > For example, consider a vswitch application which controls a set of
> > > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > > Several VMs can have the same inner tuple, while the outer tuple is
> > > different and controlled by the vswitch (encap action).
> > > For the vswtich to be able to offload the rule to the NIC, it must
> > > use a unique match criteria, independent from the inner tuple, to
> > > perform the encap action.
> > >
> > > This RFC adds support for additional metadata to use as match pattern.
> > > The metadata is an opaque item, fully controlled by the application.
> > >
> > > The use of metadata is relevant for egress rules only.
> > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > >
> > > Application should set the packet metdata in the mbuf->metadata
> > > field, and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > The NIC will use the packet metadata as match criteria for relevant flow
> rules.
> > >
> > > For example, to do an encap action depending on the VM id, the
> > > application needs to configure 'match on metadata' rte_flow rule
> > > with VM id as metadata, along with desired encap action.
> > > When preparing an egress data packet, application will set VM id
> > > data in mbuf metadata field and set PKT_TX_METADATA flag.
> > >
> > > PMD will send data packets to NIC, with VM id as metadata.
> > > Egress flow on NIC will match metadata as done with other criteria.
> > > Upon match on metadata (VM id) the appropriate encap action will be
> > > performed.
> > >
> > > This RFC introduces metadata item type for rte_flow
> > > RTE_FLOW_ITEM_META, along with corresponding struct
> > > rte_flow_item_meta and ol_flag PKT_TX_METADATA.
> > > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > >
> > > Comments are welcome.
> > >
> > > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > > ---
> > >  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > >  lib/librte_ethdev/rte_flow.c       |  1 +
> > >  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > >  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> > >  4 files changed, 58 insertions(+)
> > >
> > > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > > b/doc/guides/prog_guide/rte_flow.rst
> > > index b305a72..b6e35f1 100644
> > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > >  - `Item: ICMP6_ND_NS`_
> > >  - `Item: ICMP6_ND_OPT`_
> > >
> > > +Item: ``META``
> > > +^^^^^^^^^^^^^^
> > > +
> > > +Matches an application specific 64 bit metadata item.
> > > +
> > > +- Default ``mask`` matches any 64 bit value.
> > > +
> > > +.. _table_rte_flow_item_meta:
> > > +
> > > +.. table:: META
> > > +
> > > +   +----------+----------+---------------------------+
> > > +   | Field    | Subfield | Value                     |
> > > +   +==========+==========+===========================+
> > > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > > +   +----------+--------------------------------------+
> > > +   | ``last`` | ``data`` | upper range value         |
> > > +   +----------+----------+---------------------------+
> > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > +   +----------+----------+---------------------------+
> > > +
> > >  Actions
> > >  ~~~~~~~
> > >
> > > diff --git a/lib/librte_ethdev/rte_flow.c
> > > b/lib/librte_ethdev/rte_flow.c index
> > > cff4b52..54e5ef8 100644
> > > --- a/lib/librte_ethdev/rte_flow.c
> > > +++ b/lib/librte_ethdev/rte_flow.c
> > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > >  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > >  };
> > >
> > >  /** Generate flow_action[] entry. */ diff --git
> > > a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> > > f8ba71c..b81c816 100644
> > > --- a/lib/librte_ethdev/rte_flow.h
> > > +++ b/lib/librte_ethdev/rte_flow.h
> > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > >  	 * See struct rte_flow_item_mark.
> > >  	 */
> > >  	RTE_FLOW_ITEM_TYPE_MARK,
> > > +
> > > +	/**
> > > +	 * [META]
> > > +	 *
> > > +	 * Matches a metadata value specified in mbuf metadata field.
> > > +	 *
> > > +	 * See struct rte_flow_item_meta.
> > > +	 */
> > > +	RTE_FLOW_ITEM_TYPE_META,
> > >  };
> > >
> > >  /**
> > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {  #endif
> > >
> > >  /**
> > > + * RTE_FLOW_ITEM_TYPE_META.
> > > + *
> > > + * Matches a specified metadata value.
> > > + */
> > > +struct rte_flow_item_meta {
> > > +	uint64_t data;
> > > +};
> > > +
> > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> > > +__cplusplus static const struct rte_flow_item_meta
> rte_flow_item_meta_mask = {
> > > +	.data = RTE_BE64(UINT64_MAX),
> > > +};
> > > +#endif
> > > +
> > > +/**
> > >   * RTE_FLOW_ITEM_TYPE_FUZZY
> > >   *
> > >   * Fuzzy pattern match, expect faster than default.
> > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > index
> > > 9ce5d76..8f06a78 100644
> > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > @@ -182,6 +182,11 @@
> > >  /* add new TX flags here */
> > >
> > >  /**
> > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > + */
> > > +#define PKT_TX_METADATA		(1ULL << 41)
> > > +
> > > +/**
> > >   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > >   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is
> used
> > >   * to store the MSS of UDP fragments.
> > > @@ -593,6 +598,12 @@ struct rte_mbuf {
> > >  	 */
> > >  	struct rte_mbuf_ext_shared_info *shinfo;
> > >
> > > +	/**
> > > +	 * Application specific metadata value for flow rule match.
> > > +	 * Valid if PKT_TX_METADATA is set.
> > > +	 */
> > > +	uint64_t metadata;
> > > +
> 
> Just one thought - with that change we'll have only 8 free bytes left inside
> rte_mbuf.
> Wonder tan this metadata field be combined within tx_offload or probably
> hash fields?
> Konstantin

The match on metadata feature is currently implemented for egress, but is planned to be extended for ingress use in the future.
Hence the need for dedicated field, detached from Tx specific or Rx specific fields.
Dekel

> 
> 
> > >  } __rte_cache_aligned;
> > >
> > >  /**
> > > --
> > > 1.8.3.1

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Ananyev, Konstantin]

Hi Dekel,

> >
> >
> > >
> > > > -----Original Message-----
> > > > From: Dekel Peled [mailto:dekelp@mellanox.com]
> > > > Sent: Monday, August 13, 2018 10:47 AM
> > > > To: dev@dpdk.org
> > > > Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
> > > > <shahafs@mellanox.com>
> > > > Subject: [RFC] ethdev: support metadata as flow rule criteria
> > > >
> > > > Current implementation of rte_flow allows match pattern of flow
> > > > rule, based on packet data or header fields.
> > > > This limits the application use of match patterns.
> > > >
> > > > For example, consider a vswitch application which controls a set of
> > > > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > > > Several VMs can have the same inner tuple, while the outer tuple is
> > > > different and controlled by the vswitch (encap action).
> > > > For the vswtich to be able to offload the rule to the NIC, it must
> > > > use a unique match criteria, independent from the inner tuple, to
> > > > perform the encap action.
> > > >
> > > > This RFC adds support for additional metadata to use as match pattern.
> > > > The metadata is an opaque item, fully controlled by the application.
> > > >
> > > > The use of metadata is relevant for egress rules only.
> > > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > > >
> > > > Application should set the packet metdata in the mbuf->metadata
> > > > field, and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > > The NIC will use the packet metadata as match criteria for relevant flow
> > rules.
> > > >
> > > > For example, to do an encap action depending on the VM id, the
> > > > application needs to configure 'match on metadata' rte_flow rule
> > > > with VM id as metadata, along with desired encap action.
> > > > When preparing an egress data packet, application will set VM id
> > > > data in mbuf metadata field and set PKT_TX_METADATA flag.
> > > >
> > > > PMD will send data packets to NIC, with VM id as metadata.
> > > > Egress flow on NIC will match metadata as done with other criteria.
> > > > Upon match on metadata (VM id) the appropriate encap action will be
> > > > performed.
> > > >
> > > > This RFC introduces metadata item type for rte_flow
> > > > RTE_FLOW_ITEM_META, along with corresponding struct
> > > > rte_flow_item_meta and ol_flag PKT_TX_METADATA.
> > > > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > > >
> > > > Comments are welcome.
> > > >
> > > > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > > > ---
> > > >  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > > >  lib/librte_ethdev/rte_flow.c       |  1 +
> > > >  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > > >  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> > > >  4 files changed, 58 insertions(+)
> > > >
> > > > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > > > b/doc/guides/prog_guide/rte_flow.rst
> > > > index b305a72..b6e35f1 100644
> > > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > > >  - `Item: ICMP6_ND_NS`_
> > > >  - `Item: ICMP6_ND_OPT`_
> > > >
> > > > +Item: ``META``
> > > > +^^^^^^^^^^^^^^
> > > > +
> > > > +Matches an application specific 64 bit metadata item.
> > > > +
> > > > +- Default ``mask`` matches any 64 bit value.
> > > > +
> > > > +.. _table_rte_flow_item_meta:
> > > > +
> > > > +.. table:: META
> > > > +
> > > > +   +----------+----------+---------------------------+
> > > > +   | Field    | Subfield | Value                     |
> > > > +   +==========+==========+===========================+
> > > > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > > > +   +----------+--------------------------------------+
> > > > +   | ``last`` | ``data`` | upper range value         |
> > > > +   +----------+----------+---------------------------+
> > > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > > +   +----------+----------+---------------------------+
> > > > +
> > > >  Actions
> > > >  ~~~~~~~
> > > >
> > > > diff --git a/lib/librte_ethdev/rte_flow.c
> > > > b/lib/librte_ethdev/rte_flow.c index
> > > > cff4b52..54e5ef8 100644
> > > > --- a/lib/librte_ethdev/rte_flow.c
> > > > +++ b/lib/librte_ethdev/rte_flow.c
> > > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > > >  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > > >  };
> > > >
> > > >  /** Generate flow_action[] entry. */ diff --git
> > > > a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> > > > f8ba71c..b81c816 100644
> > > > --- a/lib/librte_ethdev/rte_flow.h
> > > > +++ b/lib/librte_ethdev/rte_flow.h
> > > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > > >  	 * See struct rte_flow_item_mark.
> > > >  	 */
> > > >  	RTE_FLOW_ITEM_TYPE_MARK,
> > > > +
> > > > +	/**
> > > > +	 * [META]
> > > > +	 *
> > > > +	 * Matches a metadata value specified in mbuf metadata field.
> > > > +	 *
> > > > +	 * See struct rte_flow_item_meta.
> > > > +	 */
> > > > +	RTE_FLOW_ITEM_TYPE_META,
> > > >  };
> > > >
> > > >  /**
> > > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {  #endif
> > > >
> > > >  /**
> > > > + * RTE_FLOW_ITEM_TYPE_META.
> > > > + *
> > > > + * Matches a specified metadata value.
> > > > + */
> > > > +struct rte_flow_item_meta {
> > > > +	uint64_t data;
> > > > +};
> > > > +
> > > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> > > > +__cplusplus static const struct rte_flow_item_meta
> > rte_flow_item_meta_mask = {
> > > > +	.data = RTE_BE64(UINT64_MAX),
> > > > +};
> > > > +#endif
> > > > +
> > > > +/**
> > > >   * RTE_FLOW_ITEM_TYPE_FUZZY
> > > >   *
> > > >   * Fuzzy pattern match, expect faster than default.
> > > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > > index
> > > > 9ce5d76..8f06a78 100644
> > > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > > @@ -182,6 +182,11 @@
> > > >  /* add new TX flags here */
> > > >
> > > >  /**
> > > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > > + */
> > > > +#define PKT_TX_METADATA		(1ULL << 41)
> > > > +
> > > > +/**
> > > >   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > > >   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is
> > used
> > > >   * to store the MSS of UDP fragments.
> > > > @@ -593,6 +598,12 @@ struct rte_mbuf {
> > > >  	 */
> > > >  	struct rte_mbuf_ext_shared_info *shinfo;
> > > >
> > > > +	/**
> > > > +	 * Application specific metadata value for flow rule match.
> > > > +	 * Valid if PKT_TX_METADATA is set.
> > > > +	 */
> > > > +	uint64_t metadata;
> > > > +
> >
> > Just one thought - with that change we'll have only 8 free bytes left inside
> > rte_mbuf.
> > Wonder tan this metadata field be combined within tx_offload or probably
> > hash fields?
> > Konstantin
> 
> The match on metadata feature is currently implemented for egress, but is planned to be extended for ingress use in the future.
> Hence the need for dedicated field, detached from Tx specific or Rx specific fields.

Could you probably explain a bit more how it will be used for ingress?
As I understand it would be some user defined value associated with particular HW filter.
Right now mbuf's hash might be used for similar purposes - it can contain flow filter ID.
Do you expect HW to provide both rss/flow and this new metadata info simultaneously
for the same packet?
Konstantin

> Dekel
> 
> >
> >
> > > >  } __rte_cache_aligned;
> > > >
> > > >  /**
> > > > --
> > > > 1.8.3.1

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Andrew Rybchenko]

On 13.08.2018 10:46, Dekel Peled wrote:
> Current implementation of rte_flow allows match pattern of flow rule,
> based on packet data or header fields.
> This limits the application use of match patterns.
>
> For example, consider a vswitch application which controls a set of VMs,
> connected with virtio, in a fabric with overlay of VXLAN.
> Several VMs can have the same inner tuple, while the outer tuple is
> different and controlled by the vswitch (encap action).
> For the vswtich to be able to offload the rule to the NIC, it must use a
> unique match criteria, independent from the inner tuple, to perform the
> encap action.
>
> This RFC adds support for additional metadata to use as match pattern.
> The metadata is an opaque item, fully controlled by the application.
>
> The use of metadata is relevant for egress rules only.
> It can be set in the flow rule using the RTE_FLOW_ITEM_META.
>
> Application should set the packet metdata in the mbuf->metadata field,
> and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> The NIC will use the packet metadata as match criteria for relevant flow
> rules.
>
> For example, to do an encap action depending on the VM id, the
> application needs to configure 'match on metadata' rte_flow rule with
> VM id as metadata, along with desired encap action.
> When preparing an egress data packet, application will set VM id data in
> mbuf metadata field and set PKT_TX_METADATA flag.
>
> PMD will send data packets to NIC, with VM id as metadata.
> Egress flow on NIC will match metadata as done with other criteria.
> Upon match on metadata (VM id) the appropriate encap action will be
> performed.
>
> This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
> along with corresponding struct rte_flow_item_meta and ol_flag
> PKT_TX_METADATA.
> It also enhances struct rte_mbuf with new data item, uint64_t metadata.
>
> Comments are welcome.
>
> Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> ---
>   doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
>   lib/librte_ethdev/rte_flow.c       |  1 +
>   lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
>   lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
>   4 files changed, 58 insertions(+)
>
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index b305a72..b6e35f1 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1191,6 +1191,27 @@ Normally preceded by any of:
>   - `Item: ICMP6_ND_NS`_
>   - `Item: ICMP6_ND_OPT`_
>   
> +Item: ``META``
> +^^^^^^^^^^^^^^
> +
> +Matches an application specific 64 bit metadata item.
> +
> +- Default ``mask`` matches any 64 bit value.
> +
> +.. _table_rte_flow_item_meta:
> +
> +.. table:: META
> +
> +   +----------+----------+---------------------------+
> +   | Field    | Subfield | Value                     |
> +   +==========+==========+===========================+
> +   | ``spec`` | ``data`` | 64 bit metadata value     |
> +   +----------+--------------------------------------+
> +   | ``last`` | ``data`` | upper range value         |
> +   +----------+----------+---------------------------+
> +   | ``mask`` | ``data`` | zeroed to match any value |
> +   +----------+----------+---------------------------+
> +
>   Actions
>   ~~~~~~~
>   
> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
> index cff4b52..54e5ef8 100644
> --- a/lib/librte_ethdev/rte_flow.c
> +++ b/lib/librte_ethdev/rte_flow.c
> @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
>   		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
>   	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
>   		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
>   };
>   
>   /** Generate flow_action[] entry. */
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> index f8ba71c..b81c816 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -413,6 +413,15 @@ enum rte_flow_item_type {
>   	 * See struct rte_flow_item_mark.
>   	 */
>   	RTE_FLOW_ITEM_TYPE_MARK,
> +
> +	/**
> +	 * [META]
> +	 *
> +	 * Matches a metadata value specified in mbuf metadata field.
> +	 *
> +	 * See struct rte_flow_item_meta.
> +	 */
> +	RTE_FLOW_ITEM_TYPE_META,
>   };
>   
>   /**
> @@ -849,6 +858,22 @@ struct rte_flow_item_gre {
>   #endif
>   
>   /**
> + * RTE_FLOW_ITEM_TYPE_META.
> + *
> + * Matches a specified metadata value.
> + */
> +struct rte_flow_item_meta {
> +	uint64_t data;
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
> +#ifndef __cplusplus
> +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> +	.data = RTE_BE64(UINT64_MAX),
> +};
> +#endif
> +
> +/**
>    * RTE_FLOW_ITEM_TYPE_FUZZY
>    *
>    * Fuzzy pattern match, expect faster than default.
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index 9ce5d76..8f06a78 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -182,6 +182,11 @@
>   /* add new TX flags here */
>   
>   /**
> + * This flag indicates that the metadata field in the mbuf is in use.
> + */
> +#define PKT_TX_METADATA		(1ULL << 41)
> +
> +/**
>    * UDP Fragmentation Offload flag. This flag is used for enabling UDP
>    * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
>    * to store the MSS of UDP fragments.
> @@ -593,6 +598,12 @@ struct rte_mbuf {
>   	 */
>   	struct rte_mbuf_ext_shared_info *shinfo;
>   
> +	/**
> +	 * Application specific metadata value for flow rule match.
> +	 * Valid if PKT_TX_METADATA is set.
> +	 */
> +	uint64_t metadata;
> +

I don't see the difference from hash union which is 64-bit wide as well.
hash.fdir.hi is used by flow mark action and mark match item (but just 
32-bit).

>   } __rte_cache_aligned;
>   
>   /**

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Ferruh Yigit]

On 8/13/2018 9:03 AM, Dekel Peled wrote:
> Adding relevant maintainers.
>  
>> -----Original Message-----
>> From: Dekel Peled [mailto:dekelp@mellanox.com]
>> Sent: Monday, August 13, 2018 10:47 AM
>> To: dev@dpdk.org
>> Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
>> <shahafs@mellanox.com>
>> Subject: [RFC] ethdev: support metadata as flow rule criteria
>>
>> Current implementation of rte_flow allows match pattern of flow rule, based
>> on packet data or header fields.
>> This limits the application use of match patterns.
>>
>> For example, consider a vswitch application which controls a set of VMs,
>> connected with virtio, in a fabric with overlay of VXLAN.
>> Several VMs can have the same inner tuple, while the outer tuple is different
>> and controlled by the vswitch (encap action).
>> For the vswtich to be able to offload the rule to the NIC, it must use a unique
>> match criteria, independent from the inner tuple, to perform the encap
>> action.
>>
>> This RFC adds support for additional metadata to use as match pattern.
>> The metadata is an opaque item, fully controlled by the application.
>>
>> The use of metadata is relevant for egress rules only.
>> It can be set in the flow rule using the RTE_FLOW_ITEM_META.
>>
>> Application should set the packet metdata in the mbuf->metadata field, and
>> set the PKT_TX_METADATA flag in the mbuf->ol_flags.
>> The NIC will use the packet metadata as match criteria for relevant flow rules.
>>
>> For example, to do an encap action depending on the VM id, the application
>> needs to configure 'match on metadata' rte_flow rule with VM id as
>> metadata, along with desired encap action.
>> When preparing an egress data packet, application will set VM id data in
>> mbuf metadata field and set PKT_TX_METADATA flag.
>>
>> PMD will send data packets to NIC, with VM id as metadata.
>> Egress flow on NIC will match metadata as done with other criteria.
>> Upon match on metadata (VM id) the appropriate encap action will be
>> performed.
>>
>> This RFC introduces metadata item type for rte_flow
>> RTE_FLOW_ITEM_META, along with corresponding struct
>> rte_flow_item_meta and ol_flag PKT_TX_METADATA.
>> It also enhances struct rte_mbuf with new data item, uint64_t metadata.
>>
>> Comments are welcome.
>>
>> Signed-off-by: Dekel Peled <dekelp@mellanox.com>

Why not use mbuf->udata64 but add a new field?


In your sample, the information that "metadata" contains the VM id needs to be
implemented into PMD so that it can program HW accordingly, this is not
flexible. What if app needs to provide another custom id, how PMD can know if
provided metadata is VM id or custom id?
Won't be better to provide an id_type and id_value, PMD can be implemented to
behave different for id_type and use the id_value in the context of that type?

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Yongseok Koh]

On Wed, Aug 22, 2018 at 04:31:14PM +0300, Andrew Rybchenko wrote:
> On 13.08.2018 10:46, Dekel Peled wrote:
> > Current implementation of rte_flow allows match pattern of flow rule,
> > based on packet data or header fields.
> > This limits the application use of match patterns.
> > 
> > For example, consider a vswitch application which controls a set of VMs,
> > connected with virtio, in a fabric with overlay of VXLAN.
> > Several VMs can have the same inner tuple, while the outer tuple is
> > different and controlled by the vswitch (encap action).
> > For the vswtich to be able to offload the rule to the NIC, it must use a
> > unique match criteria, independent from the inner tuple, to perform the
> > encap action.
> > 
> > This RFC adds support for additional metadata to use as match pattern.
> > The metadata is an opaque item, fully controlled by the application.
> > 
> > The use of metadata is relevant for egress rules only.
> > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > 
> > Application should set the packet metdata in the mbuf->metadata field,
> > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > The NIC will use the packet metadata as match criteria for relevant flow
> > rules.
> > 
> > For example, to do an encap action depending on the VM id, the
> > application needs to configure 'match on metadata' rte_flow rule with
> > VM id as metadata, along with desired encap action.
> > When preparing an egress data packet, application will set VM id data in
> > mbuf metadata field and set PKT_TX_METADATA flag.
> > 
> > PMD will send data packets to NIC, with VM id as metadata.
> > Egress flow on NIC will match metadata as done with other criteria.
> > Upon match on metadata (VM id) the appropriate encap action will be
> > performed.
> > 
> > This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
> > along with corresponding struct rte_flow_item_meta and ol_flag
> > PKT_TX_METADATA.
> > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > 
> > Comments are welcome.
> > 
> > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > ---
> >   doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> >   lib/librte_ethdev/rte_flow.c       |  1 +
> >   lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> >   lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> >   4 files changed, 58 insertions(+)
> > 
> > diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> > index b305a72..b6e35f1 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> >   - `Item: ICMP6_ND_NS`_
> >   - `Item: ICMP6_ND_OPT`_
> > +Item: ``META``
> > +^^^^^^^^^^^^^^
> > +
> > +Matches an application specific 64 bit metadata item.
> > +
> > +- Default ``mask`` matches any 64 bit value.
> > +
> > +.. _table_rte_flow_item_meta:
> > +
> > +.. table:: META
> > +
> > +   +----------+----------+---------------------------+
> > +   | Field    | Subfield | Value                     |
> > +   +==========+==========+===========================+
> > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > +   +----------+--------------------------------------+
> > +   | ``last`` | ``data`` | upper range value         |
> > +   +----------+----------+---------------------------+
> > +   | ``mask`` | ``data`` | zeroed to match any value |
> > +   +----------+----------+---------------------------+
> > +
> >   Actions
> >   ~~~~~~~
> > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
> > index cff4b52..54e5ef8 100644
> > --- a/lib/librte_ethdev/rte_flow.c
> > +++ b/lib/librte_ethdev/rte_flow.c
> > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> >   		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> >   	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> >   		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> >   };
> >   /** Generate flow_action[] entry. */
> > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> > index f8ba71c..b81c816 100644
> > --- a/lib/librte_ethdev/rte_flow.h
> > +++ b/lib/librte_ethdev/rte_flow.h
> > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> >   	 * See struct rte_flow_item_mark.
> >   	 */
> >   	RTE_FLOW_ITEM_TYPE_MARK,
> > +
> > +	/**
> > +	 * [META]
> > +	 *
> > +	 * Matches a metadata value specified in mbuf metadata field.
> > +	 *
> > +	 * See struct rte_flow_item_meta.
> > +	 */
> > +	RTE_FLOW_ITEM_TYPE_META,
> >   };
> >   /**
> > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {
> >   #endif
> >   /**
> > + * RTE_FLOW_ITEM_TYPE_META.
> > + *
> > + * Matches a specified metadata value.
> > + */
> > +struct rte_flow_item_meta {
> > +	uint64_t data;
> > +};
> > +
> > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
> > +#ifndef __cplusplus
> > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > +	.data = RTE_BE64(UINT64_MAX),
> > +};
> > +#endif
> > +
> > +/**
> >    * RTE_FLOW_ITEM_TYPE_FUZZY
> >    *
> >    * Fuzzy pattern match, expect faster than default.
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index 9ce5d76..8f06a78 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -182,6 +182,11 @@
> >   /* add new TX flags here */
> >   /**
> > + * This flag indicates that the metadata field in the mbuf is in use.
> > + */
> > +#define PKT_TX_METADATA		(1ULL << 41)
> > +
> > +/**
> >    * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> >    * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> >    * to store the MSS of UDP fragments.
> > @@ -593,6 +598,12 @@ struct rte_mbuf {
> >   	 */
> >   	struct rte_mbuf_ext_shared_info *shinfo;
> > +	/**
> > +	 * Application specific metadata value for flow rule match.
> > +	 * Valid if PKT_TX_METADATA is set.
> > +	 */
> > +	uint64_t metadata;
> > +
> 
> I don't see the difference from hash union which is 64-bit wide as well.
> hash.fdir.hi is used by flow mark action and mark match item (but just
> 32-bit).

Rx metadata would be different from flow mark ID. Mark ID is set when the flow
is created (it is a kind of marking classification result) but metadata could be
sent by other entity, e.g. VM-to-VM traffic or VM-to-HV traffic.

Thanks,
Yongseok

> 
> >   } __rte_cache_aligned;
> >   /**
> 

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Yongseok Koh]

On Wed, Aug 22, 2018 at 12:13:19PM +0000, Ananyev, Konstantin wrote:
> Hi Dekel,
> 
> > >
> > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Dekel Peled [mailto:dekelp@mellanox.com]
> > > > > Sent: Monday, August 13, 2018 10:47 AM
> > > > > To: dev@dpdk.org
> > > > > Cc: Ori Kam <orika@mellanox.com>; Shahaf Shuler
> > > > > <shahafs@mellanox.com>
> > > > > Subject: [RFC] ethdev: support metadata as flow rule criteria
> > > > >
> > > > > Current implementation of rte_flow allows match pattern of flow
> > > > > rule, based on packet data or header fields.
> > > > > This limits the application use of match patterns.
> > > > >
> > > > > For example, consider a vswitch application which controls a set of
> > > > > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > > > > Several VMs can have the same inner tuple, while the outer tuple is
> > > > > different and controlled by the vswitch (encap action).
> > > > > For the vswtich to be able to offload the rule to the NIC, it must
> > > > > use a unique match criteria, independent from the inner tuple, to
> > > > > perform the encap action.
> > > > >
> > > > > This RFC adds support for additional metadata to use as match pattern.
> > > > > The metadata is an opaque item, fully controlled by the application.
> > > > >
> > > > > The use of metadata is relevant for egress rules only.
> > > > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > > > >
> > > > > Application should set the packet metdata in the mbuf->metadata
> > > > > field, and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > > > The NIC will use the packet metadata as match criteria for relevant flow
> > > rules.
> > > > >
> > > > > For example, to do an encap action depending on the VM id, the
> > > > > application needs to configure 'match on metadata' rte_flow rule
> > > > > with VM id as metadata, along with desired encap action.
> > > > > When preparing an egress data packet, application will set VM id
> > > > > data in mbuf metadata field and set PKT_TX_METADATA flag.
> > > > >
> > > > > PMD will send data packets to NIC, with VM id as metadata.
> > > > > Egress flow on NIC will match metadata as done with other criteria.
> > > > > Upon match on metadata (VM id) the appropriate encap action will be
> > > > > performed.
> > > > >
> > > > > This RFC introduces metadata item type for rte_flow
> > > > > RTE_FLOW_ITEM_META, along with corresponding struct
> > > > > rte_flow_item_meta and ol_flag PKT_TX_METADATA.
> > > > > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > > > >
> > > > > Comments are welcome.
> > > > >
> > > > > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > > > > ---
> > > > >  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > > > >  lib/librte_ethdev/rte_flow.c       |  1 +
> > > > >  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > > > >  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> > > > >  4 files changed, 58 insertions(+)
> > > > >
> > > > > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > > > > b/doc/guides/prog_guide/rte_flow.rst
> > > > > index b305a72..b6e35f1 100644
> > > > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > > > >  - `Item: ICMP6_ND_NS`_
> > > > >  - `Item: ICMP6_ND_OPT`_
> > > > >
> > > > > +Item: ``META``
> > > > > +^^^^^^^^^^^^^^
> > > > > +
> > > > > +Matches an application specific 64 bit metadata item.
> > > > > +
> > > > > +- Default ``mask`` matches any 64 bit value.
> > > > > +
> > > > > +.. _table_rte_flow_item_meta:
> > > > > +
> > > > > +.. table:: META
> > > > > +
> > > > > +   +----------+----------+---------------------------+
> > > > > +   | Field    | Subfield | Value                     |
> > > > > +   +==========+==========+===========================+
> > > > > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > > > > +   +----------+--------------------------------------+
> > > > > +   | ``last`` | ``data`` | upper range value         |
> > > > > +   +----------+----------+---------------------------+
> > > > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > > > +   +----------+----------+---------------------------+
> > > > > +
> > > > >  Actions
> > > > >  ~~~~~~~
> > > > >
> > > > > diff --git a/lib/librte_ethdev/rte_flow.c
> > > > > b/lib/librte_ethdev/rte_flow.c index
> > > > > cff4b52..54e5ef8 100644
> > > > > --- a/lib/librte_ethdev/rte_flow.c
> > > > > +++ b/lib/librte_ethdev/rte_flow.c
> > > > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > > > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > > > >  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > > > >  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > > > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > > > >  };
> > > > >
> > > > >  /** Generate flow_action[] entry. */ diff --git
> > > > > a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h index
> > > > > f8ba71c..b81c816 100644
> > > > > --- a/lib/librte_ethdev/rte_flow.h
> > > > > +++ b/lib/librte_ethdev/rte_flow.h
> > > > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > > > >  	 * See struct rte_flow_item_mark.
> > > > >  	 */
> > > > >  	RTE_FLOW_ITEM_TYPE_MARK,
> > > > > +
> > > > > +	/**
> > > > > +	 * [META]
> > > > > +	 *
> > > > > +	 * Matches a metadata value specified in mbuf metadata field.
> > > > > +	 *
> > > > > +	 * See struct rte_flow_item_meta.
> > > > > +	 */
> > > > > +	RTE_FLOW_ITEM_TYPE_META,
> > > > >  };
> > > > >
> > > > >  /**
> > > > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {  #endif
> > > > >
> > > > >  /**
> > > > > + * RTE_FLOW_ITEM_TYPE_META.
> > > > > + *
> > > > > + * Matches a specified metadata value.
> > > > > + */
> > > > > +struct rte_flow_item_meta {
> > > > > +	uint64_t data;
> > > > > +};
> > > > > +
> > > > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> > > > > +__cplusplus static const struct rte_flow_item_meta
> > > rte_flow_item_meta_mask = {
> > > > > +	.data = RTE_BE64(UINT64_MAX),
> > > > > +};
> > > > > +#endif
> > > > > +
> > > > > +/**
> > > > >   * RTE_FLOW_ITEM_TYPE_FUZZY
> > > > >   *
> > > > >   * Fuzzy pattern match, expect faster than default.
> > > > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > > > index
> > > > > 9ce5d76..8f06a78 100644
> > > > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > > > @@ -182,6 +182,11 @@
> > > > >  /* add new TX flags here */
> > > > >
> > > > >  /**
> > > > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > > > + */
> > > > > +#define PKT_TX_METADATA		(1ULL << 41)
> > > > > +
> > > > > +/**
> > > > >   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > > > >   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is
> > > used
> > > > >   * to store the MSS of UDP fragments.
> > > > > @@ -593,6 +598,12 @@ struct rte_mbuf {
> > > > >  	 */
> > > > >  	struct rte_mbuf_ext_shared_info *shinfo;
> > > > >
> > > > > +	/**
> > > > > +	 * Application specific metadata value for flow rule match.
> > > > > +	 * Valid if PKT_TX_METADATA is set.
> > > > > +	 */
> > > > > +	uint64_t metadata;
> > > > > +
> > >
> > > Just one thought - with that change we'll have only 8 free bytes left inside
> > > rte_mbuf.
> > > Wonder tan this metadata field be combined within tx_offload or probably
> > > hash fields?
> > > Konstantin
> > 
> > The match on metadata feature is currently implemented for egress, but is planned to be extended for ingress use in the future.
> > Hence the need for dedicated field, detached from Tx specific or Rx specific fields.
> 
> Could you probably explain a bit more how it will be used for ingress?
> As I understand it would be some user defined value associated with particular HW filter.
> Right now mbuf's hash might be used for similar purposes - it can contain flow filter ID.
> Do you expect HW to provide both rss/flow and this new metadata info simultaneously
> for the same packet?

Like I replied to Andrew, it would be possible. And metadata can even be used
for flow match. Flow ID is the classification result but metadata has meaning by
itself, could be coming from other entity.

Yongseok

> Konstantin
> 
> > Dekel
> > 
> > >
> > >
> > > > >  } __rte_cache_aligned;
> > > > >
> > > > >  /**
> > > > > --
> > > > > 1.8.3.1
> 

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Ananyev, Konstantin]

> -----Original Message-----
> From: Yongseok Koh [mailto:yskoh@mellanox.com]
> Sent: Thursday, August 23, 2018 10:32 PM
> To: Andrew Rybchenko <arybchenko@solarflare.com>
> Cc: Dekel Peled <dekelp@mellanox.com>; dev@dpdk.org; orika@mellanox.com; shahafs@mellanox.com; Thomas Monjalon
> <thomas@monjalon.net>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Adrien
> Mazarguil <adrien.mazarguil@6wind.com>; Olivier Matz <olivier.matz@6wind.com>
> Subject: Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria
> 
> On Wed, Aug 22, 2018 at 04:31:14PM +0300, Andrew Rybchenko wrote:
> > On 13.08.2018 10:46, Dekel Peled wrote:
> > > Current implementation of rte_flow allows match pattern of flow rule,
> > > based on packet data or header fields.
> > > This limits the application use of match patterns.
> > >
> > > For example, consider a vswitch application which controls a set of VMs,
> > > connected with virtio, in a fabric with overlay of VXLAN.
> > > Several VMs can have the same inner tuple, while the outer tuple is
> > > different and controlled by the vswitch (encap action).
> > > For the vswtich to be able to offload the rule to the NIC, it must use a
> > > unique match criteria, independent from the inner tuple, to perform the
> > > encap action.
> > >
> > > This RFC adds support for additional metadata to use as match pattern.
> > > The metadata is an opaque item, fully controlled by the application.
> > >
> > > The use of metadata is relevant for egress rules only.
> > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > >
> > > Application should set the packet metdata in the mbuf->metadata field,
> > > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > The NIC will use the packet metadata as match criteria for relevant flow
> > > rules.
> > >
> > > For example, to do an encap action depending on the VM id, the
> > > application needs to configure 'match on metadata' rte_flow rule with
> > > VM id as metadata, along with desired encap action.
> > > When preparing an egress data packet, application will set VM id data in
> > > mbuf metadata field and set PKT_TX_METADATA flag.
> > >
> > > PMD will send data packets to NIC, with VM id as metadata.
> > > Egress flow on NIC will match metadata as done with other criteria.
> > > Upon match on metadata (VM id) the appropriate encap action will be
> > > performed.
> > >
> > > This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
> > > along with corresponding struct rte_flow_item_meta and ol_flag
> > > PKT_TX_METADATA.
> > > It also enhances struct rte_mbuf with new data item, uint64_t metadata.
> > >
> > > Comments are welcome.
> > >
> > > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > > ---
> > >   doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > >   lib/librte_ethdev/rte_flow.c       |  1 +
> > >   lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > >   lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
> > >   4 files changed, 58 insertions(+)
> > >
> > > diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> > > index b305a72..b6e35f1 100644
> > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > >   - `Item: ICMP6_ND_NS`_
> > >   - `Item: ICMP6_ND_OPT`_
> > > +Item: ``META``
> > > +^^^^^^^^^^^^^^
> > > +
> > > +Matches an application specific 64 bit metadata item.
> > > +
> > > +- Default ``mask`` matches any 64 bit value.
> > > +
> > > +.. _table_rte_flow_item_meta:
> > > +
> > > +.. table:: META
> > > +
> > > +   +----------+----------+---------------------------+
> > > +   | Field    | Subfield | Value                     |
> > > +   +==========+==========+===========================+
> > > +   | ``spec`` | ``data`` | 64 bit metadata value     |
> > > +   +----------+--------------------------------------+
> > > +   | ``last`` | ``data`` | upper range value         |
> > > +   +----------+----------+---------------------------+
> > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > +   +----------+----------+---------------------------+
> > > +
> > >   Actions
> > >   ~~~~~~~
> > > diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
> > > index cff4b52..54e5ef8 100644
> > > --- a/lib/librte_ethdev/rte_flow.c
> > > +++ b/lib/librte_ethdev/rte_flow.c
> > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > >   		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > >   	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > >   		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > >   };
> > >   /** Generate flow_action[] entry. */
> > > diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> > > index f8ba71c..b81c816 100644
> > > --- a/lib/librte_ethdev/rte_flow.h
> > > +++ b/lib/librte_ethdev/rte_flow.h
> > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > >   	 * See struct rte_flow_item_mark.
> > >   	 */
> > >   	RTE_FLOW_ITEM_TYPE_MARK,
> > > +
> > > +	/**
> > > +	 * [META]
> > > +	 *
> > > +	 * Matches a metadata value specified in mbuf metadata field.
> > > +	 *
> > > +	 * See struct rte_flow_item_meta.
> > > +	 */
> > > +	RTE_FLOW_ITEM_TYPE_META,
> > >   };
> > >   /**
> > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre {
> > >   #endif
> > >   /**
> > > + * RTE_FLOW_ITEM_TYPE_META.
> > > + *
> > > + * Matches a specified metadata value.
> > > + */
> > > +struct rte_flow_item_meta {
> > > +	uint64_t data;
> > > +};
> > > +
> > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
> > > +#ifndef __cplusplus
> > > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > > +	.data = RTE_BE64(UINT64_MAX),
> > > +};
> > > +#endif
> > > +
> > > +/**
> > >    * RTE_FLOW_ITEM_TYPE_FUZZY
> > >    *
> > >    * Fuzzy pattern match, expect faster than default.
> > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > index 9ce5d76..8f06a78 100644
> > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > @@ -182,6 +182,11 @@
> > >   /* add new TX flags here */
> > >   /**
> > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > + */
> > > +#define PKT_TX_METADATA		(1ULL << 41)
> > > +
> > > +/**
> > >    * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > >    * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> > >    * to store the MSS of UDP fragments.
> > > @@ -593,6 +598,12 @@ struct rte_mbuf {
> > >   	 */
> > >   	struct rte_mbuf_ext_shared_info *shinfo;
> > > +	/**
> > > +	 * Application specific metadata value for flow rule match.
> > > +	 * Valid if PKT_TX_METADATA is set.
> > > +	 */
> > > +	uint64_t metadata;
> > > +
> >
> > I don't see the difference from hash union which is 64-bit wide as well.
> > hash.fdir.hi is used by flow mark action and mark match item (but just
> > 32-bit).
> 
> Rx metadata would be different from flow mark ID. Mark ID is set when the flow
> is created (it is a kind of marking classification result) but metadata could be
> sent by other entity, e.g. VM-to-VM traffic or VM-to-HV traffic.

Ok, but it could be either rss OR flow id OR metdata (based on ol_flags) -
hash is a union after all.
Konstantin

> 
> Thanks,
> Yongseok
> 
> >
> > >   } __rte_cache_aligned;
> > >   /**
> >

[dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

[Dekel Peled]

Current implementation of rte_flow allows match pattern of flow rule,
based on packet data or header fields.
This limits the application use of match patterns.

For example, consider a vswitch application which controls a set of VMs,
connected with virtio, in a fabric with overlay of VXLAN.
Several VMs can have the same inner tuple, while the outer tuple is
different and controlled by the vswitch (encap action).
For the vswtich to be able to offload the rule to the NIC, it must use a
unique match criteria, independent from the inner tuple, to perform the
encap action.

This RFC adds support for additional metadata to use as match pattern.
The metadata is an opaque item, fully controlled by the application.

The use of metadata is relevant for egress rules only.
It can be set in the flow rule using the RTE_FLOW_ITEM_META.

In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
will be used to carry the metadata item. This field is used only in
ingress packets, so using it for egress metadata will not cause conflicts.

Application should set the packet metdata in the mbuf dedicated field,
and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
The NIC will use the packet metadata as match criteria for relevant flow
rules.

For example, to do an encap action depending on the VM id, the
application needs to configure 'match on metadata' rte_flow rule with
VM id as metadata, along with desired encap action.
When preparing an egress data packet, application will set VM id data in
mbuf dedicated field, and set PKT_TX_METADATA flag.

PMD will send data packets to NIC, with VM id as metadata.
Egress flow on NIC will match metadata as done with other criteria.
Upon match on metadata (VM id) the appropriate encap action will be
performed.

This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
along with corresponding struct rte_flow_item_meta and ol_flag
PKT_TX_METADATA.

Comments are welcome.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
---
v2: Use existing field in mbuf for metadata item, as suggested, instead 
    of adding a new field.
    Metadata item size adjusted to 32 bits.
---
 doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
 lib/librte_ethdev/rte_flow.c       |  1 +
 lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
 lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
 4 files changed, 60 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index b305a72..560e45a 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1191,6 +1191,27 @@ Normally preceded by any of:
 - `Item: ICMP6_ND_NS`_
 - `Item: ICMP6_ND_OPT`_
 
+Item: ``META``
+^^^^^^^^^^^^^^
+
+Matches an application specific 32 bit metadata item.
+
+- Default ``mask`` matches any 32 bit value.
+
+.. _table_rte_flow_item_meta:
+
+.. table:: META
+
+   +----------+----------+---------------------------+
+   | Field    | Subfield | Value                     |
+   +==========+==========+===========================+
+   | ``spec`` | ``data`` | 32 bit metadata value     |
+   +----------+--------------------------------------+
+   | ``last`` | ``data`` | upper range value         |
+   +----------+----------+---------------------------+
+   | ``mask`` | ``data`` | zeroed to match any value |
+   +----------+----------+---------------------------+
+
 Actions
 ~~~~~~~
 
diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
index cff4b52..54e5ef8 100644
--- a/lib/librte_ethdev/rte_flow.c
+++ b/lib/librte_ethdev/rte_flow.c
@@ -66,6 +66,7 @@ struct rte_flow_desc_data {
 		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
 	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
 		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
+	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
index f8ba71c..eba3cc4 100644
--- a/lib/librte_ethdev/rte_flow.h
+++ b/lib/librte_ethdev/rte_flow.h
@@ -413,6 +413,15 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_mark.
 	 */
 	RTE_FLOW_ITEM_TYPE_MARK,
+
+	/**
+	 * [META]
+	 *
+	 * Matches a metadata value specified in mbuf metadata field.
+	 *
+	 * See struct rte_flow_item_meta.
+	 */
+	RTE_FLOW_ITEM_TYPE_META,
 };
 
 /**
@@ -849,6 +858,22 @@ struct rte_flow_item_gre {
 #endif
 
 /**
+ * RTE_FLOW_ITEM_TYPE_META.
+ *
+ * Matches a specified metadata value.
+ */
+struct rte_flow_item_meta {
+	uint32_t data;
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
+	.data = RTE_BE32(UINT32_MAX),
+};
+#endif
+
+/**
  * RTE_FLOW_ITEM_TYPE_FUZZY
  *
  * Fuzzy pattern match, expect faster than default.
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 9ce5d76..77c1552 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -182,6 +182,11 @@
 /* add new TX flags here */
 
 /**
+ * This flag indicates that the metadata field in the mbuf is in use.
+ */
+#define PKT_TX_METADATA		(1ULL << 41)
+
+/**
  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
  * to store the MSS of UDP fragments.
@@ -526,6 +531,14 @@ struct rte_mbuf {
 			uint32_t hi;
 			/**< First 4 flexible bytes or FD ID, dependent on
 			     PKT_RX_FDIR_* flag in ol_flags. */
+
+			/**
+			 * Above item has optional use on egress:
+			 * Application specific metadata value
+			 * for flow rule match.
+			 * Valid if PKT_TX_METADATA is set.
+			 */
+
 		} fdir;           /**< Filter identifier if FDIR enabled */
 		struct {
 			uint32_t lo;
-- 
1.8.3.1

Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria

[Yongseok Koh]

> On Aug 24, 2018, at 3:11 AM, Ananyev, Konstantin <konstantin.ananyev@intel.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Yongseok Koh [mailto:yskoh@mellanox.com]
>> Sent: Thursday, August 23, 2018 10:32 PM
>> To: Andrew Rybchenko <arybchenko@solarflare.com>
>> Cc: Dekel Peled <dekelp@mellanox.com>; dev@dpdk.org; orika@mellanox.com; shahafs@mellanox.com; Thomas Monjalon
>> <thomas@monjalon.net>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Adrien
>> Mazarguil <adrien.mazarguil@6wind.com>; Olivier Matz <olivier.matz@6wind.com>
>> Subject: Re: [dpdk-dev] [RFC] ethdev: support metadata as flow rule criteria
>> 
>> On Wed, Aug 22, 2018 at 04:31:14PM +0300, Andrew Rybchenko wrote:
>>> On 13.08.2018 10:46, Dekel Peled wrote:
>>>> Current implementation of rte_flow allows match pattern of flow rule,
>>>> based on packet data or header fields.
>>>> This limits the application use of match patterns.
>>>> 
>>>> For example, consider a vswitch application which controls a set of VMs,
>>>> connected with virtio, in a fabric with overlay of VXLAN.
>>>> Several VMs can have the same inner tuple, while the outer tuple is
>>>> different and controlled by the vswitch (encap action).
>>>> For the vswtich to be able to offload the rule to the NIC, it must use a
>>>> unique match criteria, independent from the inner tuple, to perform the
>>>> encap action.
>>>> 
>>>> This RFC adds support for additional metadata to use as match pattern.
>>>> The metadata is an opaque item, fully controlled by the application.
>>>> 
>>>> The use of metadata is relevant for egress rules only.
>>>> It can be set in the flow rule using the RTE_FLOW_ITEM_META.
>>>> 
>>>> Application should set the packet metdata in the mbuf->metadata field,
>>>> and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
>>>> The NIC will use the packet metadata as match criteria for relevant flow
>>>> rules.
>>>> 
>>>> For example, to do an encap action depending on the VM id, the
>>>> application needs to configure 'match on metadata' rte_flow rule with
>>>> VM id as metadata, along with desired encap action.
>>>> When preparing an egress data packet, application will set VM id data in
>>>> mbuf metadata field and set PKT_TX_METADATA flag.
>>>> 
>>>> PMD will send data packets to NIC, with VM id as metadata.
>>>> Egress flow on NIC will match metadata as done with other criteria.
>>>> Upon match on metadata (VM id) the appropriate encap action will be
>>>> performed.
>>>> 
>>>> This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
>>>> along with corresponding struct rte_flow_item_meta and ol_flag
>>>> PKT_TX_METADATA.
>>>> It also enhances struct rte_mbuf with new data item, uint64_t metadata.
>>>> 
>>>> Comments are welcome.
>>>> 
>>>> Signed-off-by: Dekel Peled <dekelp@mellanox.com>
>>>> ---
>>>>  doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
>>>>  lib/librte_ethdev/rte_flow.c       |  1 +
>>>>  lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
>>>>  lib/librte_mbuf/rte_mbuf.h         | 11 +++++++++++
>>>>  4 files changed, 58 insertions(+)
>>>> 
>>>> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
>>>> index b305a72..b6e35f1 100644
>>>> --- a/doc/guides/prog_guide/rte_flow.rst
>>>> +++ b/doc/guides/prog_guide/rte_flow.rst
>>>> @@ -1191,6 +1191,27 @@ Normally preceded by any of:
>>>>  - `Item: ICMP6_ND_NS`_
>>>>  - `Item: ICMP6_ND_OPT`_
>>>> +Item: ``META``
>>>> +^^^^^^^^^^^^^^
>>>> +
>>>> +Matches an application specific 64 bit metadata item.
>>>> +
>>>> +- Default ``mask`` matches any 64 bit value.
>>>> +
>>>> +.. _table_rte_flow_item_meta:
>>>> +
>>>> +.. table:: META
>>>> +
>>>> +   +----------+----------+---------------------------+
>>>> +   | Field    | Subfield | Value                     |
>>>> +   +==========+==========+===========================+
>>>> +   | ``spec`` | ``data`` | 64 bit metadata value     |
>>>> +   +----------+--------------------------------------+
>>>> +   | ``last`` | ``data`` | upper range value         |
>>>> +   +----------+----------+---------------------------+
>>>> +   | ``mask`` | ``data`` | zeroed to match any value |
>>>> +   +----------+----------+---------------------------+
>>>> +
>>>>  Actions
>>>>  ~~~~~~~
>>>> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
>>>> index cff4b52..54e5ef8 100644
>>>> --- a/lib/librte_ethdev/rte_flow.c
>>>> +++ b/lib/librte_ethdev/rte_flow.c
>>>> @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
>>>>  		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
>>>>  	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
>>>>  		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
>>>> +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
>>>>  };
>>>>  /** Generate flow_action[] entry. */
>>>> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
>>>> index f8ba71c..b81c816 100644
>>>> --- a/lib/librte_ethdev/rte_flow.h
>>>> +++ b/lib/librte_ethdev/rte_flow.h
>>>> @@ -413,6 +413,15 @@ enum rte_flow_item_type {
>>>>  	 * See struct rte_flow_item_mark.
>>>>  	 */
>>>>  	RTE_FLOW_ITEM_TYPE_MARK,
>>>> +
>>>> +	/**
>>>> +	 * [META]
>>>> +	 *
>>>> +	 * Matches a metadata value specified in mbuf metadata field.
>>>> +	 *
>>>> +	 * See struct rte_flow_item_meta.
>>>> +	 */
>>>> +	RTE_FLOW_ITEM_TYPE_META,
>>>>  };
>>>>  /**
>>>> @@ -849,6 +858,22 @@ struct rte_flow_item_gre {
>>>>  #endif
>>>>  /**
>>>> + * RTE_FLOW_ITEM_TYPE_META.
>>>> + *
>>>> + * Matches a specified metadata value.
>>>> + */
>>>> +struct rte_flow_item_meta {
>>>> +	uint64_t data;
>>>> +};
>>>> +
>>>> +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
>>>> +#ifndef __cplusplus
>>>> +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
>>>> +	.data = RTE_BE64(UINT64_MAX),
>>>> +};
>>>> +#endif
>>>> +
>>>> +/**
>>>>   * RTE_FLOW_ITEM_TYPE_FUZZY
>>>>   *
>>>>   * Fuzzy pattern match, expect faster than default.
>>>> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
>>>> index 9ce5d76..8f06a78 100644
>>>> --- a/lib/librte_mbuf/rte_mbuf.h
>>>> +++ b/lib/librte_mbuf/rte_mbuf.h
>>>> @@ -182,6 +182,11 @@
>>>>  /* add new TX flags here */
>>>>  /**
>>>> + * This flag indicates that the metadata field in the mbuf is in use.
>>>> + */
>>>> +#define PKT_TX_METADATA		(1ULL << 41)
>>>> +
>>>> +/**
>>>>   * UDP Fragmentation Offload flag. This flag is used for enabling UDP
>>>>   * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
>>>>   * to store the MSS of UDP fragments.
>>>> @@ -593,6 +598,12 @@ struct rte_mbuf {
>>>>  	 */
>>>>  	struct rte_mbuf_ext_shared_info *shinfo;
>>>> +	/**
>>>> +	 * Application specific metadata value for flow rule match.
>>>> +	 * Valid if PKT_TX_METADATA is set.
>>>> +	 */
>>>> +	uint64_t metadata;
>>>> +
>>> 
>>> I don't see the difference from hash union which is 64-bit wide as well.
>>> hash.fdir.hi is used by flow mark action and mark match item (but just
>>> 32-bit).
>> 
>> Rx metadata would be different from flow mark ID. Mark ID is set when the flow
>> is created (it is a kind of marking classification result) but metadata could be
>> sent by other entity, e.g. VM-to-VM traffic or VM-to-HV traffic.
> 
> Ok, but it could be either rss OR flow id OR metdata (based on ol_flags) -
> hash is a union after all.
> Konstantin

Not sure, why can't both (flow ID and metadata) be set in a mbuf?
Why do you think it has to be exclusive?

Thanks,
Yongseok

Re: [dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

[Yongseok Koh]

> On Aug 26, 2018, at 7:09 AM, Dekel Peled <dekelp@mellanox.com> wrote:
> 
> Current implementation of rte_flow allows match pattern of flow rule,
> based on packet data or header fields.
> This limits the application use of match patterns.
> 
> For example, consider a vswitch application which controls a set of VMs,
> connected with virtio, in a fabric with overlay of VXLAN.
> Several VMs can have the same inner tuple, while the outer tuple is
> different and controlled by the vswitch (encap action).
> For the vswtich to be able to offload the rule to the NIC, it must use a
> unique match criteria, independent from the inner tuple, to perform the
> encap action.
> 
> This RFC adds support for additional metadata to use as match pattern.
> The metadata is an opaque item, fully controlled by the application.
> 
> The use of metadata is relevant for egress rules only.
> It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> 
> In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
> will be used to carry the metadata item. This field is used only in
> ingress packets, so using it for egress metadata will not cause conflicts.
> 
> Application should set the packet metdata in the mbuf dedicated field,
> and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> The NIC will use the packet metadata as match criteria for relevant flow
> rules.
> 
> For example, to do an encap action depending on the VM id, the
> application needs to configure 'match on metadata' rte_flow rule with
> VM id as metadata, along with desired encap action.
> When preparing an egress data packet, application will set VM id data in
> mbuf dedicated field, and set PKT_TX_METADATA flag.
> 
> PMD will send data packets to NIC, with VM id as metadata.
> Egress flow on NIC will match metadata as done with other criteria.
> Upon match on metadata (VM id) the appropriate encap action will be
> performed.
> 
> This RFC introduces metadata item type for rte_flow RTE_FLOW_ITEM_META,
> along with corresponding struct rte_flow_item_meta and ol_flag
> PKT_TX_METADATA.
> 
> Comments are welcome.
> 
> Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> ---
> v2: Use existing field in mbuf for metadata item, as suggested, instead 
>    of adding a new field.
>    Metadata item size adjusted to 32 bits.
> ---
> doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> lib/librte_ethdev/rte_flow.c       |  1 +
> lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
> 4 files changed, 60 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index b305a72..560e45a 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> - `Item: ICMP6_ND_NS`_
> - `Item: ICMP6_ND_OPT`_
> 
> +Item: ``META``
> +^^^^^^^^^^^^^^
> +
> +Matches an application specific 32 bit metadata item.
> +
> +- Default ``mask`` matches any 32 bit value.
> +
> +.. _table_rte_flow_item_meta:
> +
> +.. table:: META
> +
> +   +----------+----------+---------------------------+
> +   | Field    | Subfield | Value                     |
> +   +==========+==========+===========================+
> +   | ``spec`` | ``data`` | 32 bit metadata value     |
> +   +----------+--------------------------------------+
> +   | ``last`` | ``data`` | upper range value         |
> +   +----------+----------+---------------------------+
> +   | ``mask`` | ``data`` | zeroed to match any value |
> +   +----------+----------+---------------------------+
> +
> Actions
> ~~~~~~~
> 
> diff --git a/lib/librte_ethdev/rte_flow.c b/lib/librte_ethdev/rte_flow.c
> index cff4b52..54e5ef8 100644
> --- a/lib/librte_ethdev/rte_flow.c
> +++ b/lib/librte_ethdev/rte_flow.c
> @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> 		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> 	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> 		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> };
> 
> /** Generate flow_action[] entry. */
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> index f8ba71c..eba3cc4 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> 	 * See struct rte_flow_item_mark.
> 	 */
> 	RTE_FLOW_ITEM_TYPE_MARK,
> +
> +	/**
> +	 * [META]
> +	 *
> +	 * Matches a metadata value specified in mbuf metadata field.
> +	 *
> +	 * See struct rte_flow_item_meta.
> +	 */
> +	RTE_FLOW_ITEM_TYPE_META,
> };
> 
> /**
> @@ -849,6 +858,22 @@ struct rte_flow_item_gre {
> #endif
> 
> /**
> + * RTE_FLOW_ITEM_TYPE_META.
> + *
> + * Matches a specified metadata value.
> + */
> +struct rte_flow_item_meta {
> +	uint32_t data;
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */
> +#ifndef __cplusplus
> +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> +	.data = RTE_BE32(UINT32_MAX),
> +};
> +#endif
> +
> +/**
>  * RTE_FLOW_ITEM_TYPE_FUZZY
>  *
>  * Fuzzy pattern match, expect faster than default.
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index 9ce5d76..77c1552 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -182,6 +182,11 @@
> /* add new TX flags here */
> 
> /**
> + * This flag indicates that the metadata field in the mbuf is in use.
> + */
> +#define PKT_TX_METADATA		(1ULL << 41)
> +
> +/**
>  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
>  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
>  * to store the MSS of UDP fragments.
> @@ -526,6 +531,14 @@ struct rte_mbuf {
> 			uint32_t hi;
> 			/**< First 4 flexible bytes or FD ID, dependent on
> 			     PKT_RX_FDIR_* flag in ol_flags. */
> +
> +			/**
> +			 * Above item has optional use on egress:
> +			 * Application specific metadata value
> +			 * for flow rule match.
> +			 * Valid if PKT_TX_METADATA is set.
> +			 */
> +

Hi Dekel,

I don't think we have reached to a conclusion?? I remember there were three
options.
1) add a new 64bit field
2) use userdata/udata64
3) use hash

I still prefer 1) but if people here think that more fields will have to be
added in the near feature then 2) would be my next preference. But, if we just
have some unclear anxiety (like the depletion of IPv4 address :-), 1) would
still be good.

But, 3) is my least preference as a Rx mbuf still can have both flow ID and
metadata.

We still need more input/discussion.

Thanks,
Yongseok

> 		} fdir;           /**< Filter identifier if FDIR enabled */
> 		struct {
> 			uint32_t lo;
> -- 
> 1.8.3.1
> 

Re: [dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

[Dekel Peled]

> -----Original Message-----
> From: Yongseok Koh
> Sent: Tuesday, August 28, 2018 10:44 PM
> To: Dekel Peled <dekelp@mellanox.com>
> Cc: dev <dev@dpdk.org>; Shahaf Shuler <shahafs@mellanox.com>; Ori Kam
> <orika@mellanox.com>; Andrew Rybchenko <arybchenko@solarflare.com>;
> Yigit, Ferruh <ferruh.yigit@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; Olivier Matz <olivier.matz@6wind.com>;
> Alex Rosenbaum <alexr@mellanox.com>
> Subject: Re: [RFC v2] ethdev: support metadata as flow rule criteria
> 
> > On Aug 26, 2018, at 7:09 AM, Dekel Peled <dekelp@mellanox.com> wrote:
> >
> > Current implementation of rte_flow allows match pattern of flow rule,
> > based on packet data or header fields.
> > This limits the application use of match patterns.
> >
> > For example, consider a vswitch application which controls a set of
> > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > Several VMs can have the same inner tuple, while the outer tuple is
> > different and controlled by the vswitch (encap action).
> > For the vswtich to be able to offload the rule to the NIC, it must use
> > a unique match criteria, independent from the inner tuple, to perform
> > the encap action.
> >
> > This RFC adds support for additional metadata to use as match pattern.
> > The metadata is an opaque item, fully controlled by the application.
> >
> > The use of metadata is relevant for egress rules only.
> > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> >
> > In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
> > will be used to carry the metadata item. This field is used only in
> > ingress packets, so using it for egress metadata will not cause conflicts.
> >
> > Application should set the packet metdata in the mbuf dedicated field,
> > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > The NIC will use the packet metadata as match criteria for relevant
> > flow rules.
> >
> > For example, to do an encap action depending on the VM id, the
> > application needs to configure 'match on metadata' rte_flow rule with
> > VM id as metadata, along with desired encap action.
> > When preparing an egress data packet, application will set VM id data
> > in mbuf dedicated field, and set PKT_TX_METADATA flag.
> >
> > PMD will send data packets to NIC, with VM id as metadata.
> > Egress flow on NIC will match metadata as done with other criteria.
> > Upon match on metadata (VM id) the appropriate encap action will be
> > performed.
> >
> > This RFC introduces metadata item type for rte_flow
> > RTE_FLOW_ITEM_META, along with corresponding struct
> rte_flow_item_meta
> > and ol_flag PKT_TX_METADATA.
> >
> > Comments are welcome.
> >
> > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > ---
> > v2: Use existing field in mbuf for metadata item, as suggested, instead
> >    of adding a new field.
> >    Metadata item size adjusted to 32 bits.
> > ---
> > doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > lib/librte_ethdev/rte_flow.c       |  1 +
> > lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
> > 4 files changed, 60 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index b305a72..560e45a 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > - `Item: ICMP6_ND_NS`_
> > - `Item: ICMP6_ND_OPT`_
> >
> > +Item: ``META``
> > +^^^^^^^^^^^^^^
> > +
> > +Matches an application specific 32 bit metadata item.
> > +
> > +- Default ``mask`` matches any 32 bit value.
> > +
> > +.. _table_rte_flow_item_meta:
> > +
> > +.. table:: META
> > +
> > +   +----------+----------+---------------------------+
> > +   | Field    | Subfield | Value                     |
> > +   +==========+==========+===========================+
> > +   | ``spec`` | ``data`` | 32 bit metadata value     |
> > +   +----------+--------------------------------------+
> > +   | ``last`` | ``data`` | upper range value         |
> > +   +----------+----------+---------------------------+
> > +   | ``mask`` | ``data`` | zeroed to match any value |
> > +   +----------+----------+---------------------------+
> > +
> > Actions
> > ~~~~~~~
> >
> > diff --git a/lib/librte_ethdev/rte_flow.c
> > b/lib/librte_ethdev/rte_flow.c index cff4b52..54e5ef8 100644
> > --- a/lib/librte_ethdev/rte_flow.c
> > +++ b/lib/librte_ethdev/rte_flow.c
> > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > 		     sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > 	MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > 		     sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > +	MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > };
> >
> > /** Generate flow_action[] entry. */
> > diff --git a/lib/librte_ethdev/rte_flow.h
> > b/lib/librte_ethdev/rte_flow.h index f8ba71c..eba3cc4 100644
> > --- a/lib/librte_ethdev/rte_flow.h
> > +++ b/lib/librte_ethdev/rte_flow.h
> > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > 	 * See struct rte_flow_item_mark.
> > 	 */
> > 	RTE_FLOW_ITEM_TYPE_MARK,
> > +
> > +	/**
> > +	 * [META]
> > +	 *
> > +	 * Matches a metadata value specified in mbuf metadata field.
> > +	 *
> > +	 * See struct rte_flow_item_meta.
> > +	 */
> > +	RTE_FLOW_ITEM_TYPE_META,
> > };
> >
> > /**
> > @@ -849,6 +858,22 @@ struct rte_flow_item_gre { #endif
> >
> > /**
> > + * RTE_FLOW_ITEM_TYPE_META.
> > + *
> > + * Matches a specified metadata value.
> > + */
> > +struct rte_flow_item_meta {
> > +	uint32_t data;
> > +};
> > +
> > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> __cplusplus
> > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > +	.data = RTE_BE32(UINT32_MAX),
> > +};
> > +#endif
> > +
> > +/**
> >  * RTE_FLOW_ITEM_TYPE_FUZZY
> >  *
> >  * Fuzzy pattern match, expect faster than default.
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index 9ce5d76..77c1552 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -182,6 +182,11 @@
> > /* add new TX flags here */
> >
> > /**
> > + * This flag indicates that the metadata field in the mbuf is in use.
> > + */
> > +#define PKT_TX_METADATA		(1ULL << 41)
> > +
> > +/**
> >  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> >  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> >  * to store the MSS of UDP fragments.
> > @@ -526,6 +531,14 @@ struct rte_mbuf {
> > 			uint32_t hi;
> > 			/**< First 4 flexible bytes or FD ID, dependent on
> > 			     PKT_RX_FDIR_* flag in ol_flags. */
> > +
> > +			/**
> > +			 * Above item has optional use on egress:
> > +			 * Application specific metadata value
> > +			 * for flow rule match.
> > +			 * Valid if PKT_TX_METADATA is set.
> > +			 */
> > +
> 
> Hi Dekel,
> 
> I don't think we have reached to a conclusion?? I remember there were
> three options.
> 1) add a new 64bit field
> 2) use userdata/udata64
> 3) use hash
> 
> I still prefer 1) but if people here think that more fields will have to be added
> in the near feature then 2) would be my next preference. But, if we just
> have some unclear anxiety (like the depletion of IPv4 address :-), 1) would
> still be good.
> 
> But, 3) is my least preference as a Rx mbuf still can have both flow ID and
> metadata.
> 
> We still need more input/discussion.

Option 1 was not favored in discussions so far, see RFC email chain.
Option 2 is unwanted since there may be applications using userdata/udata64.
Currently we see use of metadata in tx only, hence option 3 is preferred.

> 
> Thanks,
> Yongseok
> 
> > 		} fdir;           /**< Filter identifier if FDIR enabled */
> > 		struct {
> > 			uint32_t lo;
> > --
> > 1.8.3.1
> >

Re: [dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

[Somnath Kotur]

Hi Dekel,
        Could you please show with an example i.e how the corresponding
'flow create' cmd will look like in testpmd?
Also I'm guessing you would need to change the cmdline_parser logic in
testpmd application as well to recognize this new rte_flow_item?

Thanks
Som

On Wed, Aug 29, 2018 at 12:03 PM, Dekel Peled <dekelp@mellanox.com> wrote:

>
>
> > -----Original Message-----
> > From: Yongseok Koh
> > Sent: Tuesday, August 28, 2018 10:44 PM
> > To: Dekel Peled <dekelp@mellanox.com>
> > Cc: dev <dev@dpdk.org>; Shahaf Shuler <shahafs@mellanox.com>; Ori Kam
> > <orika@mellanox.com>; Andrew Rybchenko <arybchenko@solarflare.com>;
> > Yigit, Ferruh <ferruh.yigit@intel.com>; Thomas Monjalon
> > <thomas@monjalon.net>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>; Adrien Mazarguil
> > <adrien.mazarguil@6wind.com>; Olivier Matz <olivier.matz@6wind.com>;
> > Alex Rosenbaum <alexr@mellanox.com>
> > Subject: Re: [RFC v2] ethdev: support metadata as flow rule criteria
> >
> > > On Aug 26, 2018, at 7:09 AM, Dekel Peled <dekelp@mellanox.com> wrote:
> > >
> > > Current implementation of rte_flow allows match pattern of flow rule,
> > > based on packet data or header fields.
> > > This limits the application use of match patterns.
> > >
> > > For example, consider a vswitch application which controls a set of
> > > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > > Several VMs can have the same inner tuple, while the outer tuple is
> > > different and controlled by the vswitch (encap action).
> > > For the vswtich to be able to offload the rule to the NIC, it must use
> > > a unique match criteria, independent from the inner tuple, to perform
> > > the encap action.
> > >
> > > This RFC adds support for additional metadata to use as match pattern.
> > > The metadata is an opaque item, fully controlled by the application.
> > >
> > > The use of metadata is relevant for egress rules only.
> > > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> > >
> > > In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
> > > will be used to carry the metadata item. This field is used only in
> > > ingress packets, so using it for egress metadata will not cause
> conflicts.
> > >
> > > Application should set the packet metdata in the mbuf dedicated field,
> > > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > > The NIC will use the packet metadata as match criteria for relevant
> > > flow rules.
> > >
> > > For example, to do an encap action depending on the VM id, the
> > > application needs to configure 'match on metadata' rte_flow rule with
> > > VM id as metadata, along with desired encap action.
> > > When preparing an egress data packet, application will set VM id data
> > > in mbuf dedicated field, and set PKT_TX_METADATA flag.
> > >
> > > PMD will send data packets to NIC, with VM id as metadata.
> > > Egress flow on NIC will match metadata as done with other criteria.
> > > Upon match on metadata (VM id) the appropriate encap action will be
> > > performed.
> > >
> > > This RFC introduces metadata item type for rte_flow
> > > RTE_FLOW_ITEM_META, along with corresponding struct
> > rte_flow_item_meta
> > > and ol_flag PKT_TX_METADATA.
> > >
> > > Comments are welcome.
> > >
> > > Signed-off-by: Dekel Peled <dekelp@mellanox.com>
> > > ---
> > > v2: Use existing field in mbuf for metadata item, as suggested, instead
> > >    of adding a new field.
> > >    Metadata item size adjusted to 32 bits.
> > > ---
> > > doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > > lib/librte_ethdev/rte_flow.c       |  1 +
> > > lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > > lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
> > > 4 files changed, 60 insertions(+)
> > >
> > > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > > b/doc/guides/prog_guide/rte_flow.rst
> > > index b305a72..560e45a 100644
> > > --- a/doc/guides/prog_guide/rte_flow.rst
> > > +++ b/doc/guides/prog_guide/rte_flow.rst
> > > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > > - `Item: ICMP6_ND_NS`_
> > > - `Item: ICMP6_ND_OPT`_
> > >
> > > +Item: ``META``
> > > +^^^^^^^^^^^^^^
> > > +
> > > +Matches an application specific 32 bit metadata item.
> > > +
> > > +- Default ``mask`` matches any 32 bit value.
> > > +
> > > +.. _table_rte_flow_item_meta:
> > > +
> > > +.. table:: META
> > > +
> > > +   +----------+----------+---------------------------+
> > > +   | Field    | Subfield | Value                     |
> > > +   +==========+==========+===========================+
> > > +   | ``spec`` | ``data`` | 32 bit metadata value     |
> > > +   +----------+--------------------------------------+
> > > +   | ``last`` | ``data`` | upper range value         |
> > > +   +----------+----------+---------------------------+
> > > +   | ``mask`` | ``data`` | zeroed to match any value |
> > > +   +----------+----------+---------------------------+
> > > +
> > > Actions
> > > ~~~~~~~
> > >
> > > diff --git a/lib/librte_ethdev/rte_flow.c
> > > b/lib/librte_ethdev/rte_flow.c index cff4b52..54e5ef8 100644
> > > --- a/lib/librte_ethdev/rte_flow.c
> > > +++ b/lib/librte_ethdev/rte_flow.c
> > > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> > >                  sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> > >     MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> > >                  sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > > +   MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > > };
> > >
> > > /** Generate flow_action[] entry. */
> > > diff --git a/lib/librte_ethdev/rte_flow.h
> > > b/lib/librte_ethdev/rte_flow.h index f8ba71c..eba3cc4 100644
> > > --- a/lib/librte_ethdev/rte_flow.h
> > > +++ b/lib/librte_ethdev/rte_flow.h
> > > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> > >      * See struct rte_flow_item_mark.
> > >      */
> > >     RTE_FLOW_ITEM_TYPE_MARK,
> > > +
> > > +   /**
> > > +    * [META]
> > > +    *
> > > +    * Matches a metadata value specified in mbuf metadata field.
> > > +    *
> > > +    * See struct rte_flow_item_meta.
> > > +    */
> > > +   RTE_FLOW_ITEM_TYPE_META,
> > > };
> > >
> > > /**
> > > @@ -849,6 +858,22 @@ struct rte_flow_item_gre { #endif
> > >
> > > /**
> > > + * RTE_FLOW_ITEM_TYPE_META.
> > > + *
> > > + * Matches a specified metadata value.
> > > + */
> > > +struct rte_flow_item_meta {
> > > +   uint32_t data;
> > > +};
> > > +
> > > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> > __cplusplus
> > > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > > +   .data = RTE_BE32(UINT32_MAX),
> > > +};
> > > +#endif
> > > +
> > > +/**
> > >  * RTE_FLOW_ITEM_TYPE_FUZZY
> > >  *
> > >  * Fuzzy pattern match, expect faster than default.
> > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > > index 9ce5d76..77c1552 100644
> > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > @@ -182,6 +182,11 @@
> > > /* add new TX flags here */
> > >
> > > /**
> > > + * This flag indicates that the metadata field in the mbuf is in use.
> > > + */
> > > +#define PKT_TX_METADATA            (1ULL << 41)
> > > +
> > > +/**
> > >  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> > >  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> > >  * to store the MSS of UDP fragments.
> > > @@ -526,6 +531,14 @@ struct rte_mbuf {
> > >                     uint32_t hi;
> > >                     /**< First 4 flexible bytes or FD ID, dependent on
> > >                          PKT_RX_FDIR_* flag in ol_flags. */
> > > +
> > > +                   /**
> > > +                    * Above item has optional use on egress:
> > > +                    * Application specific metadata value
> > > +                    * for flow rule match.
> > > +                    * Valid if PKT_TX_METADATA is set.
> > > +                    */
> > > +
> >
> > Hi Dekel,
> >
> > I don't think we have reached to a conclusion?? I remember there were
> > three options.
> > 1) add a new 64bit field
> > 2) use userdata/udata64
> > 3) use hash
> >
> > I still prefer 1) but if people here think that more fields will have to
> be added
> > in the near feature then 2) would be my next preference. But, if we just
> > have some unclear anxiety (like the depletion of IPv4 address :-), 1)
> would
> > still be good.
> >
> > But, 3) is my least preference as a Rx mbuf still can have both flow ID
> and
> > metadata.
> >
> > We still need more input/discussion.
>
> Option 1 was not favored in discussions so far, see RFC email chain.
> Option 2 is unwanted since there may be applications using
> userdata/udata64.
> Currently we see use of metadata in tx only, hence option 3 is preferred.
>
> >
> > Thanks,
> > Yongseok
> >
> > >             } fdir;           /**< Filter identifier if FDIR enabled */
> > >             struct {
> > >                     uint32_t lo;
> > > --
> > > 1.8.3.1
> > >
>
>

Re: [dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

[Dekel Peled]

Hi,

Example command, including just the metadata item:
flow create 0 egress pattern meta data is 666 / end actions vxlan_encap / end

I will update the cmdline parser as part of the complete feature implementation.

Regards,
Dekel

From: Somnath Kotur [mailto:somnath.kotur@broadcom.com] 
Sent: Wednesday, August 29, 2018 3:07 PM
To: Dekel Peled <dekelp@mellanox.com>
Cc: Yongseok Koh <yskoh@mellanox.com>; dev <dev@dpdk.org>; Shahaf Shuler <shahafs@mellanox.com>; Ori Kam <orika@mellanox.com>; Andrew Rybchenko <arybchenko@solarflare.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Thomas Monjalon <thomas@monjalon.net>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Adrien Mazarguil <adrien.mazarguil@6wind.com>; Olivier Matz <olivier.matz@6wind.com>; Alex Rosenbaum <alexr@mellanox.com>
Subject: Re: [dpdk-dev] [RFC v2] ethdev: support metadata as flow rule criteria

Hi Dekel,
        Could you please show with an example i.e how the corresponding 'flow create' cmd will look like in testpmd? 
Also I'm guessing you would need to change the cmdline_parser logic in testpmd application as well to recognize this new rte_flow_item?

Thanks
Som

On Wed, Aug 29, 2018 at 12:03 PM, Dekel Peled <mailto:dekelp@mellanox.com> wrote:


> -----Original Message-----
> From: Yongseok Koh
> Sent: Tuesday, August 28, 2018 10:44 PM
> To: Dekel Peled <mailto:dekelp@mellanox.com>
> Cc: dev <mailto:dev@dpdk.org>; Shahaf Shuler <mailto:shahafs@mellanox.com>; Ori Kam
> <mailto:orika@mellanox.com>; Andrew Rybchenko <mailto:arybchenko@solarflare.com>;
> Yigit, Ferruh <mailto:ferruh.yigit@intel.com>; Thomas Monjalon
> <mailto:thomas@monjalon.net>; Ananyev, Konstantin
> <mailto:konstantin.ananyev@intel.com>; Adrien Mazarguil
> <mailto:adrien.mazarguil@6wind.com>; Olivier Matz <mailto:olivier.matz@6wind.com>;
> Alex Rosenbaum <mailto:alexr@mellanox.com>
> Subject: Re: [RFC v2] ethdev: support metadata as flow rule criteria
> 
> > On Aug 26, 2018, at 7:09 AM, Dekel Peled <mailto:dekelp@mellanox.com> wrote:
> >
> > Current implementation of rte_flow allows match pattern of flow rule,
> > based on packet data or header fields.
> > This limits the application use of match patterns.
> >
> > For example, consider a vswitch application which controls a set of
> > VMs, connected with virtio, in a fabric with overlay of VXLAN.
> > Several VMs can have the same inner tuple, while the outer tuple is
> > different and controlled by the vswitch (encap action).
> > For the vswtich to be able to offload the rule to the NIC, it must use
> > a unique match criteria, independent from the inner tuple, to perform
> > the encap action.
> >
> > This RFC adds support for additional metadata to use as match pattern.
> > The metadata is an opaque item, fully controlled by the application.
> >
> > The use of metadata is relevant for egress rules only.
> > It can be set in the flow rule using the RTE_FLOW_ITEM_META.
> >
> > In order to avoid change in mbuf API, exisitng field mbuf.hash.fdir.hi
> > will be used to carry the metadata item. This field is used only in
> > ingress packets, so using it for egress metadata will not cause conflicts.
> >
> > Application should set the packet metdata in the mbuf dedicated field,
> > and set the PKT_TX_METADATA flag in the mbuf->ol_flags.
> > The NIC will use the packet metadata as match criteria for relevant
> > flow rules.
> >
> > For example, to do an encap action depending on the VM id, the
> > application needs to configure 'match on metadata' rte_flow rule with
> > VM id as metadata, along with desired encap action.
> > When preparing an egress data packet, application will set VM id data
> > in mbuf dedicated field, and set PKT_TX_METADATA flag.
> >
> > PMD will send data packets to NIC, with VM id as metadata.
> > Egress flow on NIC will match metadata as done with other criteria.
> > Upon match on metadata (VM id) the appropriate encap action will be
> > performed.
> >
> > This RFC introduces metadata item type for rte_flow
> > RTE_FLOW_ITEM_META, along with corresponding struct
> rte_flow_item_meta
> > and ol_flag PKT_TX_METADATA.
> >
> > Comments are welcome.
> >
> > Signed-off-by: Dekel Peled <mailto:dekelp@mellanox.com>
> > ---
> > v2: Use existing field in mbuf for metadata item, as suggested, instead
> >    of adding a new field.
> >    Metadata item size adjusted to 32 bits.
> > ---
> > doc/guides/prog_guide/rte_flow.rst | 21 +++++++++++++++++++++
> > lib/librte_ethdev/rte_flow.c       |  1 +
> > lib/librte_ethdev/rte_flow.h       | 25 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf.h         | 13 +++++++++++++
> > 4 files changed, 60 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/rte_flow.rst
> > b/doc/guides/prog_guide/rte_flow.rst
> > index b305a72..560e45a 100644
> > --- a/doc/guides/prog_guide/rte_flow.rst
> > +++ b/doc/guides/prog_guide/rte_flow.rst
> > @@ -1191,6 +1191,27 @@ Normally preceded by any of:
> > - `Item: ICMP6_ND_NS`_
> > - `Item: ICMP6_ND_OPT`_
> >
> > +Item: ``META``
> > +^^^^^^^^^^^^^^
> > +
> > +Matches an application specific 32 bit metadata item.
> > +
> > +- Default ``mask`` matches any 32 bit value.
> > +
> > +.. _table_rte_flow_item_meta:
> > +
> > +.. table:: META
> > +
> > +   +----------+----------+---------------------------+
> > +   | Field    | Subfield | Value                     |
> > +   +==========+==========+===========================+
> > +   | ``spec`` | ``data`` | 32 bit metadata value     |
> > +   +----------+--------------------------------------+
> > +   | ``last`` | ``data`` | upper range value         |
> > +   +----------+----------+---------------------------+
> > +   | ``mask`` | ``data`` | zeroed to match any value |
> > +   +----------+----------+---------------------------+
> > +
> > Actions
> > ~~~~~~~
> >
> > diff --git a/lib/librte_ethdev/rte_flow.c
> > b/lib/librte_ethdev/rte_flow.c index cff4b52..54e5ef8 100644
> > --- a/lib/librte_ethdev/rte_flow.c
> > +++ b/lib/librte_ethdev/rte_flow.c
> > @@ -66,6 +66,7 @@ struct rte_flow_desc_data {
> >                  sizeof(struct rte_flow_item_icmp6_nd_opt_sla_eth)),
> >     MK_FLOW_ITEM(ICMP6_ND_OPT_TLA_ETH,
> >                  sizeof(struct rte_flow_item_icmp6_nd_opt_tla_eth)),
> > +   MK_FLOW_ITEM(META, sizeof(struct rte_flow_item_meta)),
> > };
> >
> > /** Generate flow_action[] entry. */
> > diff --git a/lib/librte_ethdev/rte_flow.h
> > b/lib/librte_ethdev/rte_flow.h index f8ba71c..eba3cc4 100644
> > --- a/lib/librte_ethdev/rte_flow.h
> > +++ b/lib/librte_ethdev/rte_flow.h
> > @@ -413,6 +413,15 @@ enum rte_flow_item_type {
> >      * See struct rte_flow_item_mark.
> >      */
> >     RTE_FLOW_ITEM_TYPE_MARK,
> > +
> > +   /**
> > +    * [META]
> > +    *
> > +    * Matches a metadata value specified in mbuf metadata field.
> > +    *
> > +    * See struct rte_flow_item_meta.
> > +    */
> > +   RTE_FLOW_ITEM_TYPE_META,
> > };
> >
> > /**
> > @@ -849,6 +858,22 @@ struct rte_flow_item_gre { #endif
> >
> > /**
> > + * RTE_FLOW_ITEM_TYPE_META.
> > + *
> > + * Matches a specified metadata value.
> > + */
> > +struct rte_flow_item_meta {
> > +   uint32_t data;
> > +};
> > +
> > +/** Default mask for RTE_FLOW_ITEM_TYPE_META. */ #ifndef
> __cplusplus
> > +static const struct rte_flow_item_meta rte_flow_item_meta_mask = {
> > +   .data = RTE_BE32(UINT32_MAX),
> > +};
> > +#endif
> > +
> > +/**
> >  * RTE_FLOW_ITEM_TYPE_FUZZY
> >  *
> >  * Fuzzy pattern match, expect faster than default.
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index 9ce5d76..77c1552 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -182,6 +182,11 @@
> > /* add new TX flags here */
> >
> > /**
> > + * This flag indicates that the metadata field in the mbuf is in use.
> > + */
> > +#define PKT_TX_METADATA            (1ULL << 41)
> > +
> > +/**
> >  * UDP Fragmentation Offload flag. This flag is used for enabling UDP
> >  * fragmentation in SW or in HW. When use UFO, mbuf->tso_segsz is used
> >  * to store the MSS of UDP fragments.
> > @@ -526,6 +531,14 @@ struct rte_mbuf {
> >                     uint32_t hi;
> >                     /**< First 4 flexible bytes or FD ID, dependent on
> >                          PKT_RX_FDIR_* flag in ol_flags. */
> > +
> > +                   /**
> > +                    * Above item has optional use on egress:
> > +                    * Application specific metadata value
> > +                    * for flow rule match.
> > +                    * Valid if PKT_TX_METADATA is set.
> > +                    */
> > +
> 
> Hi Dekel,
> 
> I don't think we have reached to a conclusion?? I remember there were
> three options.
> 1) add a new 64bit field
> 2) use userdata/udata64
> 3) use hash
> 
> I still prefer 1) but if people here think that more fields will have to be added
> in the near feature then 2) would be my next preference. But, if we just
> have some unclear anxiety (like the depletion of IPv4 address :-), 1) would
> still be good.
> 
> But, 3) is my least preference as a Rx mbuf still can have both flow ID and
> metadata.
> 
> We still need more input/discussion.
Option 1 was not favored in discussions so far, see RFC email chain.
Option 2 is unwanted since there may be applications using userdata/udata64.
Currently we see use of metadata in tx only, hence option 3 is preferred.

> 
> Thanks,
> Yongseok
> 
> >             } fdir;           /**< Filter identifier if FDIR enabled */
> >             struct {
> >                     uint32_t lo;
> > --
> > 1.8.3.1
> >