[RFC] ethdev: datapath-focused meter actions

[RFC] ethdev: datapath-focused meter actions

[Alexander Kozyrev]

The introduction of asynchronous flow rules operations allowed users
to create/destroy flow rules as part of the datapath without blocking
on Flow API and slowing the packet processing down.

That applies to every possible action that has no preparation steps.
Unfortunately, one notable exception is the meter action.
There is a separate API to prepare a meter profile and a meter policy
before any meter object can be used as a flow rule action.

The application logic is the following:
1. rte_mtr_meter_profile_add() is called to create the meter profile
first to define how to classify incoming packets and to assign an
appropriate color to them.
2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
based on its color (practically creating flow rules, matching colors).
3. rte_mtr_create() is then needed to search (with locks) for previously
created profile and policy in order to create the meter object.
4. rte_flow_create() is now finally can be used to specify the created
meter as an action.

This approach doesn't fit into the asynchronous rule creation model
and can be improved with the following proposal:
1. Creating a policy may be replaced with the creation of a group with
up to 3 different rules for every color using asynchronous Flow API.
That requires the introduction of a new pattern item - meter color.
Then creation a flow rule with the meter means a simple jump to a group:
rte_flow_async_create(group=1, pattern=color, actions=...);
rte_flow_async_create(group=0, pattern=5-tuple,
		      actions=meter,jump group 1);
This allows to classify packets and act upon their color classifications.
The Meter action assigns a color to a packet and an appropriate action
is selected based on the Meter color in group 1.

2. Preparing a meter object should be the part of flow rule creation
and use the same flow queue to benefit from asynchronous operations:
rte_flow_async_create(group=0, pattern=5-tuple,
		      actions=meter id 1 profile rfc2697, jump group 1);
Creation of the meter object takes time and flow creation must wait
until it is ready before inserting the rule. Using the same queue allows
ensuring that. There is no need to create a meter object outside of the
Flow API, but this approach won't affect the old Meter API in any way.

3. Another point of optimization is to prepare all the resources needed
in advance in rte_flow_configure(). All the policy rules can be created
during the initialization stage easily and put into several groups.
These groups can be used by many meter objects by simple jump action to
an appropriate group. Meter objects can be preallocated as well and
configured with required profile parameters later at the flow rule
creation stage. The number of pre-allocated profiles/policies is
specified in the Flow engine resources settings.

These optimizations alongside already existing pattern/actions templates
can improve the insertion rate significantly and allow meter usage as
part of the datapath. The introduction of the new API is intended to be
used with the asynchronous Flow API. Deprecation of the old Meter API
is not planned at this point.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 lib/ethdev/rte_flow.h | 71 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d8827dd184..aec36a9f0a 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -33,6 +33,7 @@
 #include <rte_bitops.h>
 #include <rte_mbuf.h>
 #include <rte_mbuf_dyn.h>
+#include <rte_mtr.h>
 #include <rte_meter.h>
 #include <rte_gtp.h>
 #include <rte_l2tpv2.h>
@@ -671,6 +672,13 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+
+	/**
+	 * Matches Meter Color.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -1990,6 +1998,26 @@ static const struct rte_flow_item_ppp rte_flow_item_ppp_mask = {
 };
 #endif
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice
+ *
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR
+ *
+ * Matches a meter color set in the packet meta-data
+ * (i.e. struct rte_mbuf::sched::color).
+ */
+struct rte_flow_item_meter_color {
+	enum rte_color color; /**< Packet color. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+	.color = 0x3,
+};
+#endif
+
 /**
  * Matching pattern item definition.
  *
@@ -2376,6 +2404,14 @@ enum rte_flow_action_type {
 	 */
 	RTE_FLOW_ACTION_TYPE_METER,
 
+	/**
+	 * Extended Traffic metering and policing (MTR).
+	 *
+	 * See struct rte_flow_action_meter_ext.
+	 * See file rte_mtr.h for MTR object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_EXT,
+
 	/**
 	 * Redirects packets to security engine of current device for security
 	 * processing as specified by security session.
@@ -3128,6 +3164,19 @@ struct rte_flow_action_meter {
 	uint32_t mtr_id; /**< MTR object ID created with rte_mtr_create(). */
 };
 
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_EXT
+ *
+ * Extended Traffic metering and policing (MTR).
+ *
+ * Packets matched by items of this type can be either dropped or passed to the
+ * next item with their color set by the MTR object.
+ */
+struct rte_flow_action_meter_ext {
+	uint32_t mtr_id; /**< MTR object ID. */
+	struct rte_meter_profile *profile; /**< MTR profile. */
+};
+
 /**
  * RTE_FLOW_ACTION_TYPE_SECURITY
  *
@@ -4899,10 +4948,20 @@ struct rte_flow_port_info {
 	 */
 	uint32_t max_nb_aging_objects;
 	/**
-	 * Maximum number traffic meters.
+	 * Maximum number of traffic meters.
 	 * @see RTE_FLOW_ACTION_TYPE_METER
 	 */
 	uint32_t max_nb_meters;
+	/**
+	 * Maximum number of traffic meter profiles.
+	 * @see RTE_FLOW_ACTION_TYPE_METER
+	 */
+	uint32_t max_nb_meter_profiles;
+	/**
+	 * Maximum number of traffic meters policices.
+	 * @see RTE_FLOW_ACTION_TYPE_METER
+	 */
+	uint32_t max_nb_meter_policies;
 };
 
 /**
@@ -4972,6 +5031,16 @@ struct rte_flow_port_attr {
 	 * @see RTE_FLOW_ACTION_TYPE_METER
 	 */
 	uint32_t nb_meters;
+	/**
+	 * Number of meter profiles to configure.
+	 * @see RTE_FLOW_ACTION_TYPE_METER
+	 */
+	uint32_t nb_meter_profiles;
+	/**
+	 * Number of meter policies to configure.
+	 * @see RTE_FLOW_ACTION_TYPE_METER
+	 */
+	uint32_t nb_meter_policies;
 };
 
 /**
-- 
2.18.2

Re: [RFC] ethdev: datapath-focused meter actions

[Jerin Jacob]

+ @Cristian Dumitrescu meter maintainer.


On Fri, Apr 8, 2022 at 8:17 AM Alexander Kozyrev <akozyrev@nvidia.com> wrote:
>
> The introduction of asynchronous flow rules operations allowed users
> to create/destroy flow rules as part of the datapath without blocking
> on Flow API and slowing the packet processing down.
>
> That applies to every possible action that has no preparation steps.
> Unfortunately, one notable exception is the meter action.
> There is a separate API to prepare a meter profile and a meter policy
> before any meter object can be used as a flow rule action.
>
> The application logic is the following:
> 1. rte_mtr_meter_profile_add() is called to create the meter profile
> first to define how to classify incoming packets and to assign an
> appropriate color to them.
> 2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
> based on its color (practically creating flow rules, matching colors).
> 3. rte_mtr_create() is then needed to search (with locks) for previously
> created profile and policy in order to create the meter object.
> 4. rte_flow_create() is now finally can be used to specify the created
> meter as an action.
>
> This approach doesn't fit into the asynchronous rule creation model
> and can be improved with the following proposal:
> 1. Creating a policy may be replaced with the creation of a group with
> up to 3 different rules for every color using asynchronous Flow API.
> That requires the introduction of a new pattern item - meter color.
> Then creation a flow rule with the meter means a simple jump to a group:
> rte_flow_async_create(group=1, pattern=color, actions=...);
> rte_flow_async_create(group=0, pattern=5-tuple,
>                       actions=meter,jump group 1);
> This allows to classify packets and act upon their color classifications.
> The Meter action assigns a color to a packet and an appropriate action
> is selected based on the Meter color in group 1.
>
> 2. Preparing a meter object should be the part of flow rule creation
> and use the same flow queue to benefit from asynchronous operations:
> rte_flow_async_create(group=0, pattern=5-tuple,
>                       actions=meter id 1 profile rfc2697, jump group 1);
> Creation of the meter object takes time and flow creation must wait
> until it is ready before inserting the rule. Using the same queue allows
> ensuring that. There is no need to create a meter object outside of the
> Flow API, but this approach won't affect the old Meter API in any way.
>
> 3. Another point of optimization is to prepare all the resources needed
> in advance in rte_flow_configure(). All the policy rules can be created
> during the initialization stage easily and put into several groups.
> These groups can be used by many meter objects by simple jump action to
> an appropriate group. Meter objects can be preallocated as well and
> configured with required profile parameters later at the flow rule
> creation stage. The number of pre-allocated profiles/policies is
> specified in the Flow engine resources settings.
>
> These optimizations alongside already existing pattern/actions templates
> can improve the insertion rate significantly and allow meter usage as
> part of the datapath. The introduction of the new API is intended to be
> used with the asynchronous Flow API. Deprecation of the old Meter API
> is not planned at this point.
>
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  lib/ethdev/rte_flow.h | 71 ++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 70 insertions(+), 1 deletion(-)
>
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index d8827dd184..aec36a9f0a 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -33,6 +33,7 @@
>  #include <rte_bitops.h>
>  #include <rte_mbuf.h>
>  #include <rte_mbuf_dyn.h>
> +#include <rte_mtr.h>
>  #include <rte_meter.h>
>  #include <rte_gtp.h>
>  #include <rte_l2tpv2.h>
> @@ -671,6 +672,13 @@ enum rte_flow_item_type {
>          * See struct rte_flow_item_gre_opt.
>          */
>         RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> +
> +       /**
> +        * Matches Meter Color.
> +        *
> +        * See struct rte_flow_item_meter_color.
> +        */
> +       RTE_FLOW_ITEM_TYPE_METER_COLOR,
>  };
>
>  /**
> @@ -1990,6 +1998,26 @@ static const struct rte_flow_item_ppp rte_flow_item_ppp_mask = {
>  };
>  #endif
>
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this structure may change without prior notice
> + *
> + * RTE_FLOW_ITEM_TYPE_METER_COLOR
> + *
> + * Matches a meter color set in the packet meta-data
> + * (i.e. struct rte_mbuf::sched::color).
> + */
> +struct rte_flow_item_meter_color {
> +       enum rte_color color; /**< Packet color. */
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> +#ifndef __cplusplus
> +static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
> +       .color = 0x3,
> +};
> +#endif
> +
>  /**
>   * Matching pattern item definition.
>   *
> @@ -2376,6 +2404,14 @@ enum rte_flow_action_type {
>          */
>         RTE_FLOW_ACTION_TYPE_METER,
>
> +       /**
> +        * Extended Traffic metering and policing (MTR).
> +        *
> +        * See struct rte_flow_action_meter_ext.
> +        * See file rte_mtr.h for MTR object configuration.
> +        */
> +       RTE_FLOW_ACTION_TYPE_METER_EXT,
> +
>         /**
>          * Redirects packets to security engine of current device for security
>          * processing as specified by security session.
> @@ -3128,6 +3164,19 @@ struct rte_flow_action_meter {
>         uint32_t mtr_id; /**< MTR object ID created with rte_mtr_create(). */
>  };
>
> +/**
> + * RTE_FLOW_ACTION_TYPE_METER_EXT
> + *
> + * Extended Traffic metering and policing (MTR).
> + *
> + * Packets matched by items of this type can be either dropped or passed to the
> + * next item with their color set by the MTR object.
> + */
> +struct rte_flow_action_meter_ext {
> +       uint32_t mtr_id; /**< MTR object ID. */
> +       struct rte_meter_profile *profile; /**< MTR profile. */
> +};
> +
>  /**
>   * RTE_FLOW_ACTION_TYPE_SECURITY
>   *
> @@ -4899,10 +4948,20 @@ struct rte_flow_port_info {
>          */
>         uint32_t max_nb_aging_objects;
>         /**
> -        * Maximum number traffic meters.
> +        * Maximum number of traffic meters.
>          * @see RTE_FLOW_ACTION_TYPE_METER
>          */
>         uint32_t max_nb_meters;
> +       /**
> +        * Maximum number of traffic meter profiles.
> +        * @see RTE_FLOW_ACTION_TYPE_METER
> +        */
> +       uint32_t max_nb_meter_profiles;
> +       /**
> +        * Maximum number of traffic meters policices.
> +        * @see RTE_FLOW_ACTION_TYPE_METER
> +        */
> +       uint32_t max_nb_meter_policies;
>  };
>
>  /**
> @@ -4972,6 +5031,16 @@ struct rte_flow_port_attr {
>          * @see RTE_FLOW_ACTION_TYPE_METER
>          */
>         uint32_t nb_meters;
> +       /**
> +        * Number of meter profiles to configure.
> +        * @see RTE_FLOW_ACTION_TYPE_METER
> +        */
> +       uint32_t nb_meter_profiles;
> +       /**
> +        * Number of meter policies to configure.
> +        * @see RTE_FLOW_ACTION_TYPE_METER
> +        */
> +       uint32_t nb_meter_policies;
>  };
>
>  /**
> --
> 2.18.2
>

RE: [RFC] ethdev: datapath-focused meter actions

[Dumitrescu, Cristian]

Hi Alexander,

After reviewing this RFC, I have to say that your proposal is very unclear to me. I don't understand what is the problem you're trying to solve and what exactly is that you cannot do with the current meter and flow APIs.

I suggest we get together for a community call with all the interested folks invited in order to get more clarity on your proposal, thank you!

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Friday, April 8, 2022 9:21 AM
> To: Alexander Kozyrev <akozyrev@nvidia.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>
> Cc: dpdk-dev <dev@dpdk.org>; Ori Kam <orika@nvidia.com>; Thomas
> Monjalon <thomas@monjalon.net>; Ivan Malov <ivan.malov@oktetlabs.ru>;
> Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Yigit, Ferruh
> <ferruh.yigit@intel.com>; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Jerin Jacob <jerinj@marvell.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: Re: [RFC] ethdev: datapath-focused meter actions
> 
> + @Cristian Dumitrescu meter maintainer.
> 
> 
> On Fri, Apr 8, 2022 at 8:17 AM Alexander Kozyrev <akozyrev@nvidia.com>
> wrote:
> >
> > The introduction of asynchronous flow rules operations allowed users
> > to create/destroy flow rules as part of the datapath without blocking
> > on Flow API and slowing the packet processing down.
> >
> > That applies to every possible action that has no preparation steps.
> > Unfortunately, one notable exception is the meter action.
> > There is a separate API to prepare a meter profile and a meter policy
> > before any meter object can be used as a flow rule action.

I disagree. Creation of meter policies and meter objects is decoupled from the flow creation. Meter policies and meter objects can all be created at initialization or on-the-fly, and their creation does not directly require the data plane to be stopped.

Please explain what problem are you trying to fix here. I suggest you provide the sequence diagram and tell us where the problem is.

> >
> > The application logic is the following:
> > 1. rte_mtr_meter_profile_add() is called to create the meter profile
> > first to define how to classify incoming packets and to assign an
> > appropriate color to them.
> > 2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
> > based on its color (practically creating flow rules, matching colors).

Nope, the policy add does not create any flows. In fact, it does not create any meter objects either. It simply defines a configuration pattern that can be reused many times when meter objects are created afterwards.

> > 3. rte_mtr_create() is then needed to search (with locks) for previously
> > created profile and policy in order to create the meter object.

The rte_mtr_create() is not created at the time the flow is created, but at a prior decoupled moment. I don't see any issue here.

> > 4. rte_flow_create() is now finally can be used to specify the created
> > meter as an action.
> >
> > This approach doesn't fit into the asynchronous rule creation model
> > and can be improved with the following proposal:

Again, the creation of meter policies and objects is decoupled from the flow creation; in fact, the meter policies and objects must be created before the flows using them are created.

> > 1. Creating a policy may be replaced with the creation of a group with
> > up to 3 different rules for every color using asynchronous Flow API.
> > That requires the introduction of a new pattern item - meter color.
> > Then creation a flow rule with the meter means a simple jump to a group:
> > rte_flow_async_create(group=1, pattern=color, actions=...);
> > rte_flow_async_create(group=0, pattern=5-tuple,
> >                       actions=meter,jump group 1);
> > This allows to classify packets and act upon their color classifications.
> > The Meter action assigns a color to a packet and an appropriate action
> > is selected based on the Meter color in group 1.
> >

The meter objects requires a relatively complex configuration procedure. This is one of the reasons meters have their own API, so we can keep that complexity away from the flow API.

You seem to indicate that your desired behavior is to create the meter objects when the flow is created rather than in advance. Did I get it correctly? This is possible with the current API as well by simply creating the meter object immediately before the flow gets created.

Stitching the creation of new meter object to the flow creation (if I understand your approach right) doe not allow for some important features, such as:
-reusing meter objects that were previously created by reassigning them to a different flow
-having multiple flows use the same shared meter.

> > 2. Preparing a meter object should be the part of flow rule creation

Why?? Please take some time to clearly explain this, your entire proposal seems to be predicated on this assertion being true.

> > and use the same flow queue to benefit from asynchronous operations:
> > rte_flow_async_create(group=0, pattern=5-tuple,
> >                       actions=meter id 1 profile rfc2697, jump group 1);
> > Creation of the meter object takes time and flow creation must wait
> > until it is ready before inserting the rule. Using the same queue allows
> > ensuring that. There is no need to create a meter object outside of the
> > Flow API, but this approach won't affect the old Meter API in any way.
> >
> > 3. Another point of optimization is to prepare all the resources needed
> > in advance in rte_flow_configure(). 

This seems to directly contradict you previous statement that meter objects need to be created at the same time when the flow is created (exact quote of your statement from above: " Preparing a meter object should be the part of flow rule creation").

All the policy rules can be created
> > during the initialization stage easily and put into several groups.
> > These groups can be used by many meter objects by simple jump action to
> > an appropriate group. Meter objects can be preallocated as well and
> > configured with required profile parameters later at the flow rule
> > creation stage. The number of pre-allocated profiles/policies is
> > specified in the Flow engine resources settings.
> >
> > These optimizations alongside already existing pattern/actions templates
> > can improve the insertion rate significantly and allow meter usage as
> > part of the datapath. The introduction of the new API is intended to be
> > used with the asynchronous Flow API. Deprecation of the old Meter API
> > is not planned at this point.
> >
> > Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> > ---
> >  lib/ethdev/rte_flow.h | 71
> ++++++++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 70 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> > index d8827dd184..aec36a9f0a 100644
> > --- a/lib/ethdev/rte_flow.h
> > +++ b/lib/ethdev/rte_flow.h
> > @@ -33,6 +33,7 @@
> >  #include <rte_bitops.h>
> >  #include <rte_mbuf.h>
> >  #include <rte_mbuf_dyn.h>
> > +#include <rte_mtr.h>
> >  #include <rte_meter.h>
> >  #include <rte_gtp.h>
> >  #include <rte_l2tpv2.h>
> > @@ -671,6 +672,13 @@ enum rte_flow_item_type {
> >          * See struct rte_flow_item_gre_opt.
> >          */
> >         RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> > +
> > +       /**
> > +        * Matches Meter Color.
> > +        *
> > +        * See struct rte_flow_item_meter_color.
> > +        */
> > +       RTE_FLOW_ITEM_TYPE_METER_COLOR,

As discussed in the previous community call on meters, it makes perfect sense to me to be able to use the meter color as one of the flow match fields.

We just need to make sure that when this is needed, it is guaranteed that the packet has a color, i.e. there is a meter action previously in this chain that got executed, or there is a default packet color if not. How do we make sure of this?

> >  };
> >
> >  /**
> > @@ -1990,6 +1998,26 @@ static const struct rte_flow_item_ppp
> rte_flow_item_ppp_mask = {
> >  };
> >  #endif
> >
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this structure may change without prior notice
> > + *
> > + * RTE_FLOW_ITEM_TYPE_METER_COLOR
> > + *
> > + * Matches a meter color set in the packet meta-data
> > + * (i.e. struct rte_mbuf::sched::color).
> > + */
> > +struct rte_flow_item_meter_color {
> > +       enum rte_color color; /**< Packet color. */
> > +};
> > +
> > +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> > +#ifndef __cplusplus
> > +static const struct rte_flow_item_meter_color
> rte_flow_item_meter_color_mask = {
> > +       .color = 0x3,
> > +};
> > +#endif
> > +
> >  /**
> >   * Matching pattern item definition.
> >   *
> > @@ -2376,6 +2404,14 @@ enum rte_flow_action_type {
> >          */
> >         RTE_FLOW_ACTION_TYPE_METER,
> >
> > +       /**
> > +        * Extended Traffic metering and policing (MTR).
> > +        *
> > +        * See struct rte_flow_action_meter_ext.
> > +        * See file rte_mtr.h for MTR object configuration.
> > +        */
> > +       RTE_FLOW_ACTION_TYPE_METER_EXT,
> > +
> >         /**
> >          * Redirects packets to security engine of current device for security
> >          * processing as specified by security session.
> > @@ -3128,6 +3164,19 @@ struct rte_flow_action_meter {
> >         uint32_t mtr_id; /**< MTR object ID created with rte_mtr_create(). */
> >  };
> >
> > +/**
> > + * RTE_FLOW_ACTION_TYPE_METER_EXT
> > + *
> > + * Extended Traffic metering and policing (MTR).
> > + *
> > + * Packets matched by items of this type can be either dropped or passed to
> the
> > + * next item with their color set by the MTR object.
> > + */
> > +struct rte_flow_action_meter_ext {
> > +       uint32_t mtr_id; /**< MTR object ID. */
> > +       struct rte_meter_profile *profile; /**< MTR profile. */
> > +};
> > +

How is this proposed meter extended action different from the existing meter action? This is not explained at all here, please explain.

The comment seems to indicate a copy & paste error, as "packets matched by items of this type ..." indicates a match item, and this is an action item.

> >  /**
> >   * RTE_FLOW_ACTION_TYPE_SECURITY
> >   *
> > @@ -4899,10 +4948,20 @@ struct rte_flow_port_info {
> >          */
> >         uint32_t max_nb_aging_objects;
> >         /**
> > -        * Maximum number traffic meters.
> > +        * Maximum number of traffic meters.
> >          * @see RTE_FLOW_ACTION_TYPE_METER
> >          */
> >         uint32_t max_nb_meters;
> > +       /**
> > +        * Maximum number of traffic meter profiles.
> > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > +        */
> > +       uint32_t max_nb_meter_profiles;
> > +       /**
> > +        * Maximum number of traffic meters policices.
> > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > +        */
> > +       uint32_t max_nb_meter_policies;
> >  };
> >
> >  /**
> > @@ -4972,6 +5031,16 @@ struct rte_flow_port_attr {
> >          * @see RTE_FLOW_ACTION_TYPE_METER
> >          */
> >         uint32_t nb_meters;
> > +       /**
> > +        * Number of meter profiles to configure.
> > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > +        */
> > +       uint32_t nb_meter_profiles;
> > +       /**
> > +        * Number of meter policies to configure.
> > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > +        */
> > +       uint32_t nb_meter_policies;
> >  };
> >
> >  /**
> > --
> > 2.18.2
> >

Regards,
Cristian

RE: [RFC] ethdev: datapath-focused meter actions

[Dumitrescu, Cristian]

I forgot to mention: besides the my statement at the top of my reply, there are many comments inline below :)

> -----Original Message-----
> From: Dumitrescu, Cristian
> Sent: Tuesday, April 26, 2022 2:44 PM
> To: Jerin Jacob <jerinjacobk@gmail.com>; Alexander Kozyrev
> <akozyrev@nvidia.com>
> Cc: dpdk-dev <dev@dpdk.org>; Ori Kam <orika@nvidia.com>; Thomas
> Monjalon <thomas@monjalon.net>; Ivan Malov <ivan.malov@oktetlabs.ru>;
> Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Yigit, Ferruh
> <ferruh.yigit@intel.com>; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Jerin Jacob <jerinj@marvell.com>; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: RE: [RFC] ethdev: datapath-focused meter actions
> 
> Hi Alexander,
> 
> After reviewing this RFC, I have to say that your proposal is very unclear to me.
> I don't understand what is the problem you're trying to solve and what exactly
> is that you cannot do with the current meter and flow APIs.
> 
> I suggest we get together for a community call with all the interested folks
> invited in order to get more clarity on your proposal, thank you!
> 
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Friday, April 8, 2022 9:21 AM
> > To: Alexander Kozyrev <akozyrev@nvidia.com>; Dumitrescu, Cristian
> > <cristian.dumitrescu@intel.com>
> > Cc: dpdk-dev <dev@dpdk.org>; Ori Kam <orika@nvidia.com>; Thomas
> > Monjalon <thomas@monjalon.net>; Ivan Malov <ivan.malov@oktetlabs.ru>;
> > Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Yigit, Ferruh
> > <ferruh.yigit@intel.com>; Awal, Mohammad Abdul
> > <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> > Jerin Jacob <jerinj@marvell.com>; Ajit Khaparde
> > <ajit.khaparde@broadcom.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Subject: Re: [RFC] ethdev: datapath-focused meter actions
> >
> > + @Cristian Dumitrescu meter maintainer.
> >
> >
> > On Fri, Apr 8, 2022 at 8:17 AM Alexander Kozyrev <akozyrev@nvidia.com>
> > wrote:
> > >
> > > The introduction of asynchronous flow rules operations allowed users
> > > to create/destroy flow rules as part of the datapath without blocking
> > > on Flow API and slowing the packet processing down.
> > >
> > > That applies to every possible action that has no preparation steps.
> > > Unfortunately, one notable exception is the meter action.
> > > There is a separate API to prepare a meter profile and a meter policy
> > > before any meter object can be used as a flow rule action.
> 
> I disagree. Creation of meter policies and meter objects is decoupled from the
> flow creation. Meter policies and meter objects can all be created at
> initialization or on-the-fly, and their creation does not directly require the data
> plane to be stopped.
> 
> Please explain what problem are you trying to fix here. I suggest you provide
> the sequence diagram and tell us where the problem is.
> 
> > >
> > > The application logic is the following:
> > > 1. rte_mtr_meter_profile_add() is called to create the meter profile
> > > first to define how to classify incoming packets and to assign an
> > > appropriate color to them.
> > > 2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
> > > based on its color (practically creating flow rules, matching colors).
> 
> Nope, the policy add does not create any flows. In fact, it does not create any
> meter objects either. It simply defines a configuration pattern that can be
> reused many times when meter objects are created afterwards.
> 
> > > 3. rte_mtr_create() is then needed to search (with locks) for previously
> > > created profile and policy in order to create the meter object.
> 
> The rte_mtr_create() is not created at the time the flow is created, but at a
> prior decoupled moment. I don't see any issue here.
> 
> > > 4. rte_flow_create() is now finally can be used to specify the created
> > > meter as an action.
> > >
> > > This approach doesn't fit into the asynchronous rule creation model
> > > and can be improved with the following proposal:
> 
> Again, the creation of meter policies and objects is decoupled from the flow
> creation; in fact, the meter policies and objects must be created before the
> flows using them are created.
> 
> > > 1. Creating a policy may be replaced with the creation of a group with
> > > up to 3 different rules for every color using asynchronous Flow API.
> > > That requires the introduction of a new pattern item - meter color.
> > > Then creation a flow rule with the meter means a simple jump to a group:
> > > rte_flow_async_create(group=1, pattern=color, actions=...);
> > > rte_flow_async_create(group=0, pattern=5-tuple,
> > >                       actions=meter,jump group 1);
> > > This allows to classify packets and act upon their color classifications.
> > > The Meter action assigns a color to a packet and an appropriate action
> > > is selected based on the Meter color in group 1.
> > >
> 
> The meter objects requires a relatively complex configuration procedure. This
> is one of the reasons meters have their own API, so we can keep that
> complexity away from the flow API.
> 
> You seem to indicate that your desired behavior is to create the meter objects
> when the flow is created rather than in advance. Did I get it correctly? This is
> possible with the current API as well by simply creating the meter object
> immediately before the flow gets created.
> 
> Stitching the creation of new meter object to the flow creation (if I understand
> your approach right) doe not allow for some important features, such as:
> -reusing meter objects that were previously created by reassigning them to a
> different flow
> -having multiple flows use the same shared meter.
> 
> > > 2. Preparing a meter object should be the part of flow rule creation
> 
> Why?? Please take some time to clearly explain this, your entire proposal
> seems to be predicated on this assertion being true.
> 
> > > and use the same flow queue to benefit from asynchronous operations:
> > > rte_flow_async_create(group=0, pattern=5-tuple,
> > >                       actions=meter id 1 profile rfc2697, jump group 1);
> > > Creation of the meter object takes time and flow creation must wait
> > > until it is ready before inserting the rule. Using the same queue allows
> > > ensuring that. There is no need to create a meter object outside of the
> > > Flow API, but this approach won't affect the old Meter API in any way.
> > >
> > > 3. Another point of optimization is to prepare all the resources needed
> > > in advance in rte_flow_configure().
> 
> This seems to directly contradict you previous statement that meter objects
> need to be created at the same time when the flow is created (exact quote of
> your statement from above: " Preparing a meter object should be the part of
> flow rule creation").
> 
> All the policy rules can be created
> > > during the initialization stage easily and put into several groups.
> > > These groups can be used by many meter objects by simple jump action to
> > > an appropriate group. Meter objects can be preallocated as well and
> > > configured with required profile parameters later at the flow rule
> > > creation stage. The number of pre-allocated profiles/policies is
> > > specified in the Flow engine resources settings.
> > >
> > > These optimizations alongside already existing pattern/actions templates
> > > can improve the insertion rate significantly and allow meter usage as
> > > part of the datapath. The introduction of the new API is intended to be
> > > used with the asynchronous Flow API. Deprecation of the old Meter API
> > > is not planned at this point.
> > >
> > > Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> > > ---
> > >  lib/ethdev/rte_flow.h | 71
> > ++++++++++++++++++++++++++++++++++++++++++-
> > >  1 file changed, 70 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> > > index d8827dd184..aec36a9f0a 100644
> > > --- a/lib/ethdev/rte_flow.h
> > > +++ b/lib/ethdev/rte_flow.h
> > > @@ -33,6 +33,7 @@
> > >  #include <rte_bitops.h>
> > >  #include <rte_mbuf.h>
> > >  #include <rte_mbuf_dyn.h>
> > > +#include <rte_mtr.h>
> > >  #include <rte_meter.h>
> > >  #include <rte_gtp.h>
> > >  #include <rte_l2tpv2.h>
> > > @@ -671,6 +672,13 @@ enum rte_flow_item_type {
> > >          * See struct rte_flow_item_gre_opt.
> > >          */
> > >         RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> > > +
> > > +       /**
> > > +        * Matches Meter Color.
> > > +        *
> > > +        * See struct rte_flow_item_meter_color.
> > > +        */
> > > +       RTE_FLOW_ITEM_TYPE_METER_COLOR,
> 
> As discussed in the previous community call on meters, it makes perfect sense
> to me to be able to use the meter color as one of the flow match fields.
> 
> We just need to make sure that when this is needed, it is guaranteed that the
> packet has a color, i.e. there is a meter action previously in this chain that got
> executed, or there is a default packet color if not. How do we make sure of
> this?
> 
> > >  };
> > >
> > >  /**
> > > @@ -1990,6 +1998,26 @@ static const struct rte_flow_item_ppp
> > rte_flow_item_ppp_mask = {
> > >  };
> > >  #endif
> > >
> > > +/**
> > > + * @warning
> > > + * @b EXPERIMENTAL: this structure may change without prior notice
> > > + *
> > > + * RTE_FLOW_ITEM_TYPE_METER_COLOR
> > > + *
> > > + * Matches a meter color set in the packet meta-data
> > > + * (i.e. struct rte_mbuf::sched::color).
> > > + */
> > > +struct rte_flow_item_meter_color {
> > > +       enum rte_color color; /**< Packet color. */
> > > +};
> > > +
> > > +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> > > +#ifndef __cplusplus
> > > +static const struct rte_flow_item_meter_color
> > rte_flow_item_meter_color_mask = {
> > > +       .color = 0x3,
> > > +};
> > > +#endif
> > > +
> > >  /**
> > >   * Matching pattern item definition.
> > >   *
> > > @@ -2376,6 +2404,14 @@ enum rte_flow_action_type {
> > >          */
> > >         RTE_FLOW_ACTION_TYPE_METER,
> > >
> > > +       /**
> > > +        * Extended Traffic metering and policing (MTR).
> > > +        *
> > > +        * See struct rte_flow_action_meter_ext.
> > > +        * See file rte_mtr.h for MTR object configuration.
> > > +        */
> > > +       RTE_FLOW_ACTION_TYPE_METER_EXT,
> > > +
> > >         /**
> > >          * Redirects packets to security engine of current device for security
> > >          * processing as specified by security session.
> > > @@ -3128,6 +3164,19 @@ struct rte_flow_action_meter {
> > >         uint32_t mtr_id; /**< MTR object ID created with rte_mtr_create(). */
> > >  };
> > >
> > > +/**
> > > + * RTE_FLOW_ACTION_TYPE_METER_EXT
> > > + *
> > > + * Extended Traffic metering and policing (MTR).
> > > + *
> > > + * Packets matched by items of this type can be either dropped or passed
> to
> > the
> > > + * next item with their color set by the MTR object.
> > > + */
> > > +struct rte_flow_action_meter_ext {
> > > +       uint32_t mtr_id; /**< MTR object ID. */
> > > +       struct rte_meter_profile *profile; /**< MTR profile. */
> > > +};
> > > +
> 
> How is this proposed meter extended action different from the existing meter
> action? This is not explained at all here, please explain.
> 
> The comment seems to indicate a copy & paste error, as "packets matched by
> items of this type ..." indicates a match item, and this is an action item.
> 
> > >  /**
> > >   * RTE_FLOW_ACTION_TYPE_SECURITY
> > >   *
> > > @@ -4899,10 +4948,20 @@ struct rte_flow_port_info {
> > >          */
> > >         uint32_t max_nb_aging_objects;
> > >         /**
> > > -        * Maximum number traffic meters.
> > > +        * Maximum number of traffic meters.
> > >          * @see RTE_FLOW_ACTION_TYPE_METER
> > >          */
> > >         uint32_t max_nb_meters;
> > > +       /**
> > > +        * Maximum number of traffic meter profiles.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t max_nb_meter_profiles;
> > > +       /**
> > > +        * Maximum number of traffic meters policices.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t max_nb_meter_policies;
> > >  };
> > >
> > >  /**
> > > @@ -4972,6 +5031,16 @@ struct rte_flow_port_attr {
> > >          * @see RTE_FLOW_ACTION_TYPE_METER
> > >          */
> > >         uint32_t nb_meters;
> > > +       /**
> > > +        * Number of meter profiles to configure.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t nb_meter_profiles;
> > > +       /**
> > > +        * Number of meter policies to configure.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t nb_meter_policies;
> > >  };
> > >
> > >  /**
> > > --
> > > 2.18.2
> > >
> 
> Regards,
> Cristian

RE: [RFC] ethdev: datapath-focused meter actions

[Alexander Kozyrev]

On Tuesday, April 26, 2022 9:44 Dumitrescu, Cristian <cristian.dumitrescu@intel.com>:
> After reviewing this RFC, I have to say that your proposal is very unclear to me. I
> don't understand what is the problem you're trying to solve and what exactly is
> that you cannot do with the current meter and flow APIs.
> 
> I suggest we get together for a community call with all the interested folks
> invited in order to get more clarity on your proposal, thank you!

Completely agree, I scheduled a discussion on May 10.

> > > The introduction of asynchronous flow rules operations allowed users
> > > to create/destroy flow rules as part of the datapath without blocking
> > > on Flow API and slowing the packet processing down.
> > >
> > > That applies to every possible action that has no preparation steps.
> > > Unfortunately, one notable exception is the meter action.
> > > There is a separate API to prepare a meter profile and a meter policy
> > > before any meter object can be used as a flow rule action.
> 
> I disagree. Creation of meter policies and meter objects is decoupled from the
> flow creation. Meter policies and meter objects can all be created at
> initialization or on-the-fly, and their creation does not directly require the data
> plane to be stopped.

Unfortunately we cannot create all meter objects at the initialization stage
since an application may not know the split between the meters, profiles and policies.
Any particular profile and/or policy may be created based on 5-tuple, for example.
In this case additional delays are introduced in order to find the proper profile/policy
and to create meter object if they were not created in the hardware before.

> Please explain what problem are you trying to fix here. I suggest you provide the
> sequence diagram and tell us where the problem is.

We are trying to remove any unnecessary latencies in case where flow rules are inserted
as part of the packet processing. An application parses a packet, and, based on its content,
inserts some flow rule to the hardware. This action should be as fast as possible in order
not to slow down the packet processing. Any locks to find meter objects or any resource
allocations should be optimized out if possible.

> > >
> > > The application logic is the following:
> > > 1. rte_mtr_meter_profile_add() is called to create the meter profile
> > > first to define how to classify incoming packets and to assign an
> > > appropriate color to them.
> > > 2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
> > > based on its color (practically creating flow rules, matching colors).
> 
> Nope, the policy add does not create any flows. In fact, it does not create any
> meter objects either. It simply defines a configuration pattern that can be reused
> many times when meter objects are created afterwards.

This is true for the software implementation of the metering library.
But NIC has to create some real objects at some point in case of hardware implementation.
In case of a policy there will be some flow rules offloaded to the NIC eventually.
Offloading those rules will cause additional delays during the first usage of the policy.

> > > 3. rte_mtr_create() is then needed to search (with locks) for previously
> > > created profile and policy in order to create the meter object.
> 
> The rte_mtr_create() is not created at the time the flow is created, but at a prior
> decoupled moment. I don't see any issue here.

It may be created right before the flow is inserted, based on the packet content.
An application may select different profile/policy for different 5-tuple values.

> > > 4. rte_flow_create() is now finally can be used to specify the created
> > > meter as an action.
> > >
> > > This approach doesn't fit into the asynchronous rule creation model
> > > and can be improved with the following proposal:
> 
> Again, the creation of meter policies and objects is decoupled from the flow
> creation; in fact, the meter policies and objects must be created before the
> flows using them are created.

Again, the decision on the which meter to use may be taken only when an app sees the packets.
That is the main focus of this RFC, speed up these decisions with all possible means.

> > > 1. Creating a policy may be replaced with the creation of a group with
> > > up to 3 different rules for every color using asynchronous Flow API.
> > > That requires the introduction of a new pattern item - meter color.
> > > Then creation a flow rule with the meter means a simple jump to a group:
> > > rte_flow_async_create(group=1, pattern=color, actions=...);
> > > rte_flow_async_create(group=0, pattern=5-tuple,
> > >                       actions=meter,jump group 1);
> > > This allows to classify packets and act upon their color classifications.
> > > The Meter action assigns a color to a packet and an appropriate action
> > > is selected based on the Meter color in group 1.
> > >
> 
> The meter objects requires a relatively complex configuration procedure. This is
> one of the reasons meters have their own API, so we can keep that complexity
> away from the flow API.

Agree, the logic on how to create the profile stays in the meter API.
We are merely proposing additional flexibility on how and when we can use it.
 
> You seem to indicate that your desired behavior is to create the meter objects
> when the flow is created rather than in advance. Did I get it correctly? This is
> possible with the current API as well by simply creating the meter object
> immediately before the flow gets created.

There are two stages for meter creation: meter allocation and meter configuration.
Meter allocation can be done at the application startup with rte_flow_configure().
And meter configuration that will be done during traffic and we need them to be as fast as possible.
Since new RTE flow API is multi-threaded without locks, and asynchronous while the meter one is not
this means that the application will be blocked during its datapath processing.

> Stitching the creation of new meter object to the flow creation (if I understand
> your approach right) doe not allow for some important features, such as:
> -reusing meter objects that were previously created by reassigning them to a
> different flow
> -having multiple flows use the same shared meter.

Not true, while the configuration is done per flow, we can still give the same meter id.
And (profile == null) in the meter_ext action means that the meter will be shared.

> > > 2. Preparing a meter object should be the part of flow rule creation
> 
> Why?? Please take some time to clearly explain this, your entire proposal seems
> to be predicated on this assertion being true.

Since the profile and policy are dependent on the traffic.

> > > and use the same flow queue to benefit from asynchronous operations:
> > > rte_flow_async_create(group=0, pattern=5-tuple,
> > >                       actions=meter id 1 profile rfc2697, jump group 1);
> > > Creation of the meter object takes time and flow creation must wait
> > > until it is ready before inserting the rule. Using the same queue allows
> > > ensuring that. There is no need to create a meter object outside of the
> > > Flow API, but this approach won't affect the old Meter API in any way.
> > >
> > > 3. Another point of optimization is to prepare all the resources needed
> > > in advance in rte_flow_configure().
> 
> This seems to directly contradict you previous statement that meter objects
> need to be created at the same time when the flow is created (exact quote of
> your statement from above: " Preparing a meter object should be the part of
> flow rule creation").

Again, allocation should be done at startup, while configuration is done  during rule creation.

> All the policy rules can be created
> > > during the initialization stage easily and put into several groups.
> > > These groups can be used by many meter objects by simple jump action to
> > > an appropriate group. Meter objects can be preallocated as well and
> > > configured with required profile parameters later at the flow rule
> > > creation stage. The number of pre-allocated profiles/policies is
> > > specified in the Flow engine resources settings.
> > >
> > > These optimizations alongside already existing pattern/actions templates
> > > can improve the insertion rate significantly and allow meter usage as
> > > part of the datapath. The introduction of the new API is intended to be
> > > used with the asynchronous Flow API. Deprecation of the old Meter API
> > > is not planned at this point.
> > >
> > > Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> > > ---
> > >  lib/ethdev/rte_flow.h | 71
> > ++++++++++++++++++++++++++++++++++++++++++-
> > >  1 file changed, 70 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> > > index d8827dd184..aec36a9f0a 100644
> > > --- a/lib/ethdev/rte_flow.h
> > > +++ b/lib/ethdev/rte_flow.h
> > > @@ -33,6 +33,7 @@
> > >  #include <rte_bitops.h>
> > >  #include <rte_mbuf.h>
> > >  #include <rte_mbuf_dyn.h>
> > > +#include <rte_mtr.h>
> > >  #include <rte_meter.h>
> > >  #include <rte_gtp.h>
> > >  #include <rte_l2tpv2.h>
> > > @@ -671,6 +672,13 @@ enum rte_flow_item_type {
> > >          * See struct rte_flow_item_gre_opt.
> > >          */
> > >         RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> > > +
> > > +       /**
> > > +        * Matches Meter Color.
> > > +        *
> > > +        * See struct rte_flow_item_meter_color.
> > > +        */
> > > +       RTE_FLOW_ITEM_TYPE_METER_COLOR,
> 
> As discussed in the previous community call on meters, it makes perfect sense to
> me to be able to use the meter color as one of the flow match fields.
> 
> We just need to make sure that when this is needed, it is guaranteed that the
> packet has a color, i.e. there is a meter action previously in this chain that got
> executed, or there is a default packet color if not. How do we make sure of this?

We can have modify filed (set color) to update the color item. I'll include this change in v2.

> > >  };
> > >
> > >  /**
> > > @@ -1990,6 +1998,26 @@ static const struct rte_flow_item_ppp
> > rte_flow_item_ppp_mask = {
> > >  };
> > >  #endif
> > >
> > > +/**
> > > + * @warning
> > > + * @b EXPERIMENTAL: this structure may change without prior notice
> > > + *
> > > + * RTE_FLOW_ITEM_TYPE_METER_COLOR
> > > + *
> > > + * Matches a meter color set in the packet meta-data
> > > + * (i.e. struct rte_mbuf::sched::color).
> > > + */
> > > +struct rte_flow_item_meter_color {
> > > +       enum rte_color color; /**< Packet color. */
> > > +};
> > > +
> > > +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> > > +#ifndef __cplusplus
> > > +static const struct rte_flow_item_meter_color
> > rte_flow_item_meter_color_mask = {
> > > +       .color = 0x3,
> > > +};
> > > +#endif
> > > +
> > >  /**
> > >   * Matching pattern item definition.
> > >   *
> > > @@ -2376,6 +2404,14 @@ enum rte_flow_action_type {
> > >          */
> > >         RTE_FLOW_ACTION_TYPE_METER,
> > >
> > > +       /**
> > > +        * Extended Traffic metering and policing (MTR).
> > > +        *
> > > +        * See struct rte_flow_action_meter_ext.
> > > +        * See file rte_mtr.h for MTR object configuration.
> > > +        */
> > > +       RTE_FLOW_ACTION_TYPE_METER_EXT,
> > > +
> > >         /**
> > >          * Redirects packets to security engine of current device for security
> > >          * processing as specified by security session.
> > > @@ -3128,6 +3164,19 @@ struct rte_flow_action_meter {
> > >         uint32_t mtr_id; /**< MTR object ID created with rte_mtr_create(). */
> > >  };
> > >
> > > +/**
> > > + * RTE_FLOW_ACTION_TYPE_METER_EXT
> > > + *
> > > + * Extended Traffic metering and policing (MTR).
> > > + *
> > > + * Packets matched by items of this type can be either dropped or passed to
> > the
> > > + * next item with their color set by the MTR object.
> > > + */
> > > +struct rte_flow_action_meter_ext {
> > > +       uint32_t mtr_id; /**< MTR object ID. */
> > > +       struct rte_meter_profile *profile; /**< MTR profile. */
> > > +};
> > > +
> 
> How is this proposed meter extended action different from the existing meter
> action? This is not explained at all here, please explain.
> 
> The comment seems to indicate a copy & paste error, as "packets matched by
> items of this type ..." indicates a match item, and this is an action item.

We supply the profile and this action will simply set the proper color based on profile, nothing else.
What to do with assigned packet colors will be decided later via ITEM_TYPE_METER_COLOR.

> > >  /**
> > >   * RTE_FLOW_ACTION_TYPE_SECURITY
> > >   *
> > > @@ -4899,10 +4948,20 @@ struct rte_flow_port_info {
> > >          */
> > >         uint32_t max_nb_aging_objects;
> > >         /**
> > > -        * Maximum number traffic meters.
> > > +        * Maximum number of traffic meters.
> > >          * @see RTE_FLOW_ACTION_TYPE_METER
> > >          */
> > >         uint32_t max_nb_meters;
> > > +       /**
> > > +        * Maximum number of traffic meter profiles.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t max_nb_meter_profiles;
> > > +       /**
> > > +        * Maximum number of traffic meters policices.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t max_nb_meter_policies;
> > >  };
> > >
> > >  /**
> > > @@ -4972,6 +5031,16 @@ struct rte_flow_port_attr {
> > >          * @see RTE_FLOW_ACTION_TYPE_METER
> > >          */
> > >         uint32_t nb_meters;
> > > +       /**
> > > +        * Number of meter profiles to configure.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t nb_meter_profiles;
> > > +       /**
> > > +        * Number of meter policies to configure.
> > > +        * @see RTE_FLOW_ACTION_TYPE_METER
> > > +        */
> > > +       uint32_t nb_meter_policies;
> > >  };
> > >
> > >  /**
> > > --
> > > 2.18.2
> > >

[PATCH v2 0/3] ethdev: datapath-focused meter actions

[Alexander Kozyrev]

The introduction of asynchronous flow rules operations allowed users
to create/destroy flow rules as part of the datapath without blocking
on Flow API and slowing the packet processing down.

That applies to every possible action that has no preparation steps.
Unfortunately, one notable exception is the meter action.
There is a separate API to prepare a meter profile and a meter policy
before any meter object can be used as a flow rule action.

The application logic is the following:
1. rte_mtr_meter_profile_add() is called to create the meter profile
first to define how to classify incoming packets and to assign an
appropriate color to them.
2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
based on its color (practically creating flow rules, matching colors).
3. rte_mtr_create() is then needed to search (with locks) for previously
created profile and policy in order to create the meter object.
4. rte_flow_create() is now finally can be used to specify the created
meter as an action.

This approach doesn't fit into the asynchronous rule creation model
and can be improved with the following proposal:
1. Creating a policy may be replaced with the creation of a group with
up to 3 different rules for every color using asynchronous Flow API.
That requires the introduction of a new pattern item - meter color.
Then creation a flow rule with the meter means a simple jump to a group:
rte_flow_async_create(group=1, pattern=color, actions=...);
rte_flow_async_create(group=0, pattern=5-tuple,
		      actions=meter,jump group 1);
This allows to classify packets and act upon their color classifications.
The Meter action assigns a color to a packet and an appropriate action
is selected based on the Meter color in group 1.

2. Preparing a meter object should be the part of flow rule creation
and use the same flow queue to benefit from asynchronous operations:
rte_flow_async_create(group=0, pattern=5-tuple,
		      actions=meter id 1 profile rfc2697, jump group 1);
Creation of the meter object takes time and flow creation must wait
until it is ready before inserting the rule. Using the same queue allows
ensuring that. There is no need to create a meter object outside of the
Flow API, but this approach won't affect the old Meter API in any way.

3. Another point of optimization is to prepare all the resources needed
in advance in rte_flow_configure(). All the policy rules can be created
during the initialization stage easily and put into several groups.
These groups can be used by many meter objects by simple jump action to
an appropriate group. Meter objects can be preallocated as well and
configured with required profile parameters later at the flow rule
creation stage. The number of pre-allocated profiles/policies is
specified in the Flow engine resources settings.

These optimizations alongside already existing pattern/actions templates
can improve the insertion rate significantly and allow meter usage as
part of the datapath. The introduction of the new API is intended to be
used with the asynchronous Flow API. Deprecation of the old Meter API
is not planned at this point.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Alexander Kozyrev (3):
  ethdev: add meter color matching to Flow API
  ethdev: allow meter color modification
  ethdev: add extended meter action to Flow API

 lib/ethdev/rte_flow.h | 49 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

-- 
2.18.2

[RFC v2 0/3] ethdev: datapath-focused meter actions

[Alexander Kozyrev]

The introduction of asynchronous flow rules operations allowed users
to create/destroy flow rules as part of the datapath without blocking
on Flow API and slowing the packet processing down.

That applies to every possible action that has no preparation steps.
Unfortunately, one notable exception is the meter action.
There is a separate API to prepare a meter profile and a meter policy
before any meter object can be used as a flow rule action.

The application logic is the following:
1. rte_mtr_meter_profile_add() is called to create the meter profile
first to define how to classify incoming packets and to assign an
appropriate color to them.
2. rte_mtr_meter_policy_add() is invoked to define the fate of a packet,
based on its color (practically creating flow rules, matching colors).
3. rte_mtr_create() is then needed to search (with locks) for previously
created profile and policy in order to create the meter object.
4. rte_flow_create() is now finally can be used to specify the created
meter as an action.

This approach doesn't fit into the asynchronous rule creation model
and can be improved with the following proposal:
1. Creating a policy may be replaced with the creation of a group with
up to 3 different rules for every color using asynchronous Flow API.
That requires the introduction of a new pattern item - meter color.
Then creation a flow rule with the meter means a simple jump to a group:
rte_flow_async_create(group=1, pattern=color, actions=...);
rte_flow_async_create(group=0, pattern=5-tuple,
		      actions=meter,jump group 1);
This allows to classify packets and act upon their color classifications.
The Meter action assigns a color to a packet and an appropriate action
is selected based on the Meter color in group 1.

2. Preparing a meter object should be the part of flow rule creation
and use the same flow queue to benefit from asynchronous operations:
rte_flow_async_create(group=0, pattern=5-tuple,
		      actions=meter id 1 profile rfc2697, jump group 1);
Creation of the meter object takes time and flow creation must wait
until it is ready before inserting the rule. Using the same queue allows
ensuring that. There is no need to create a meter object outside of the
Flow API, but this approach won't affect the old Meter API in any way.

3. Another point of optimization is to prepare all the resources needed
in advance in rte_flow_configure(). All the policy rules can be created
during the initialization stage easily and put into several groups.
These groups can be used by many meter objects by simple jump action to
an appropriate group. Meter objects can be preallocated as well and
configured with required profile parameters later at the flow rule
creation stage. The number of pre-allocated profiles/policies is
specified in the Flow engine resources settings.

These optimizations alongside already existing pattern/actions templates
can improve the insertion rate significantly and allow meter usage as
part of the datapath. The introduction of the new API is intended to be
used with the asynchronous Flow API. Deprecation of the old Meter API
is not planned at this point.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Alexander Kozyrev (3):
  ethdev: add meter color matching to Flow API
  ethdev: allow meter color modification
  ethdev: add extended meter action to Flow API

 lib/ethdev/rte_flow.h | 49 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

-- 
2.18.2

[PATCH v2 1/3] ethdev: add meter color matching to Flow API

[Alexander Kozyrev]

Introduce Meter Color pattern item to provide greater
flexibility for Traffic Metering handling in the RTE Flow API.
Matching on a color marker allows a user to manage
Metering policies as part of common flow rules management.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 lib/ethdev/rte_flow.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d8827dd184..67f54fba0e 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -671,6 +671,12 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+	/**
+	 * Matches Meter Color.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -1990,6 +1996,26 @@ static const struct rte_flow_item_ppp rte_flow_item_ppp_mask = {
 };
 #endif
 
+/**
+ * @warning
+ * @b EXPERIMENTAL: this structure may change without prior notice
+ *
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR
+ *
+ * Matches a meter color set in the packet meta-data
+ * (i.e. struct rte_mbuf::sched::color).
+ */
+struct rte_flow_item_meter_color {
+		enum rte_color color; /**< Packet color. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+		.color = RTE_COLORS,
+};
+#endif
+
 /**
  * Matching pattern item definition.
  *
-- 
2.18.2

[PATCH v2 2/3] ethdev: allow meter color modification

[Alexander Kozyrev]

Extend modify_field Flow API with ability to set the metering
color of a packet. This brings a possibility to set an initial
color before proceeding with Meter classification and assign a
color outside of Meter object based on other traffic patterns.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 lib/ethdev/rte_flow.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 67f54fba0e..870d1b4a06 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3716,6 +3716,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_META,		/**< Metadata value. */
 	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
 	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
+	RTE_FLOW_FIELD_METER_COLOR, /**< Metering color */
 };
 
 /**
-- 
2.18.2

[PATCH v2 3/3] ethdev: add extended meter action to Flow API

[Alexander Kozyrev]

Configure a meter profile at the time of the flow rule insertion.
Pass all the needed parameters to configure the meter profile
and insert the metering rule simultaneously.

The meter profile can be reused in multiple flow rules in order
to share the profile and optimize the flow insertion process.
Setting meter profile to NULL means keeping old profile config.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 lib/ethdev/rte_flow.h | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 870d1b4a06..964f057fa2 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -33,6 +33,7 @@
 #include <rte_bitops.h>
 #include <rte_mbuf.h>
 #include <rte_mbuf_dyn.h>
+#include <rte_mtr.h>
 #include <rte_meter.h>
 #include <rte_gtp.h>
 #include <rte_l2tpv2.h>
@@ -2908,6 +2909,13 @@ enum rte_flow_action_type {
 	 * @see struct rte_flow_action_ethdev
 	 */
 	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+	/**
+	 * Extended Traffic metering and policing (MTR).
+	 *
+	 * See struct rte_flow_action_meter_ext.
+	 * See file rte_mtr.h for MTR object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_EXT,
 };
 
 /**
@@ -3154,6 +3162,20 @@ struct rte_flow_action_meter {
 	uint32_t mtr_id; /**< MTR object ID created with rte_mtr_create(). */
 };
 
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_EXT
+ *
+ * Traffic metering and policing (MTR).
+ *
+ * Meters an IP packet stream and marks its packets either
+ * green, yellow, or red for future processing, according to a policy.
+ */
+struct rte_flow_action_meter_ext {
+	uint32_t mtr_id; /**< MTR object ID. */
+	struct rte_meter_profile *profile; /**< MTR profile. */
+	struct rte_mtr_params *params; /**< MTR parameters. */
+};
+
 /**
  * RTE_FLOW_ACTION_TYPE_SECURITY
  *
-- 
2.18.2

[PATCH 0/4] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

Extend Metering and Marking support in the Flow API:
1. Add METER_COLOR item to match Color Marker set by a Meter.
2. Add the ability to set Color Marker via modify_field Flow API.
3. Add Meter API to calculate profile configuration values.
4. Add METER_MARK action to perform Meter color marking only.
Provide greater flexibility in how Metering can be used.

RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Alexander Kozyrev (4):
  ethdev: add meter color flow matching item
  ethdev: allow meter color marker modification
  ethdev: add meter profile config calculation
  ethdev: add meter color mark flow action

 doc/guides/prog_guide/rte_flow.rst            | 33 ++++++++++
 .../traffic_metering_and_policing.rst         |  2 +
 doc/guides/rel_notes/release_22_07.rst        |  7 +++
 lib/ethdev/rte_flow.c                         |  1 +
 lib/ethdev/rte_flow.h                         | 61 +++++++++++++++++++
 lib/ethdev/rte_mtr.c                          | 12 ++++
 lib/ethdev/rte_mtr.h                          | 24 ++++++++
 lib/ethdev/rte_mtr_driver.h                   |  9 +++
 lib/ethdev/version.map                        |  3 +
 9 files changed, 152 insertions(+)

-- 
2.18.2

[PATCH 1/4] ethdev: add meter color flow matching item

[Alexander Kozyrev]

Provide an ability to use a Color Marker set by a Meter
as a matching item in Flow API. The Color Marker reflects
the metering result by setting the DS field of a
packet to a particular codepoint: green, yellow or red.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
 doc/guides/rel_notes/release_22_07.rst |  4 ++++
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
 4 files changed, 36 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 588914b231..1877d4b9b5 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1651,6 +1651,13 @@ Matches a PPP header.
 - ``proto_id``: PPP protocol identifier.
 - Default ``mask`` matches addr, ctrl, proto_id.
 
+Item: ``METER_COLOR``
+^^^^^^^^^^^^^^^^^^^^^
+
+Matches Color Marker set by a Meter in the DS field of a packet.
+
+- ``color``: Metering color code.
+
 Actions
 ~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index a60a0d5f16..a61a4d77dc 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -60,6 +60,10 @@ New Features
   Added an API which can get the number of in-flight packets in
   vhost async data path without using lock.
 
+* **Extended Metering and Marking support in the Flow API.**
+
+  * Added METER_COLOR item to match Color Marker set by a Meter.
+
 * **Updated Intel iavf driver.**
 
   * Added Tx QoS queue rate limitation support.
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 2c35a2f13e..07d7e998c0 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -161,6 +161,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
 			rte_flow_item_flex_conv),
 	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
 	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
+	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d8827dd184..e3f0e86eb6 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -671,6 +671,14 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+
+	/**
+	 * Matches Meter Color Marker.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -2201,6 +2209,22 @@ struct rte_flow_item_flex_conf {
 	uint32_t nb_outputs;
 };
 
+/**
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR.
+ *
+ * Matches Color Marker set by a Meter in the DS field of a packet.
+ */
+struct rte_flow_item_meter_color {
+	enum rte_color color; /**< Meter color marker. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+	.color = RTE_COLORS,
+};
+#endif
+
 /**
  * Action types.
  *
-- 
2.18.2

[PATCH 2/4] ethdev: allow meter color marker modification

[Alexander Kozyrev]

Extend modify_field Flow API with support of Meter Color Marker
modifications. It allows setting the DS field of a packet to any
color marker: green, yellow or red. User is able to specify
an initial packet color for Meter API or create simple Metering
and Marking flow rules based on his own coloring alghorithm.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_22_07.rst | 1 +
 lib/ethdev/rte_flow.h                  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index a61a4d77dc..ec83178d98 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -63,6 +63,7 @@ New Features
 * **Extended Metering and Marking support in the Flow API.**
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
+  * Added ability to set Color Marker via modify_field Flow API.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index e3f0e86eb6..817df34ed0 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3714,6 +3714,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_META,		/**< Metadata value. */
 	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
 	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
+	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter Color Marker. */
 };
 
 /**
-- 
2.18.2

[PATCH 3/4] ethdev: add meter profile config calculation

[Alexander Kozyrev]

Introduce a new Meter API to calculate PMD-specific profile
configuration values based on user-provided CIR, CBS and EBS.
That will allow fast Meter configuration in hardware without
the need to convert these values every time we use a profile.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 .../traffic_metering_and_policing.rst         |  2 ++
 doc/guides/rel_notes/release_22_07.rst        |  1 +
 lib/ethdev/rte_mtr.c                          | 12 ++++++++++
 lib/ethdev/rte_mtr.h                          | 24 +++++++++++++++++++
 lib/ethdev/rte_mtr_driver.h                   |  9 +++++++
 lib/ethdev/version.map                        |  3 +++
 6 files changed, 51 insertions(+)

diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst
index ceb5a96488..99c25f8937 100644
--- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
+++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
@@ -105,3 +105,5 @@ traffic meter and policing library.
    * Adding one (or multiple) actions of the type ``RTE_FLOW_ACTION_TYPE_METER``
      to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
      specified per color as show in :numref:`figure_rte_mtr_chaining`.
+#.  Traffic metering algorithm configuration may be converted to the PDM-specific
+   values using ``rte_mtr_meter_profile_calculate()`` API function for reuse.
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index ec83178d98..5f19223663 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -64,6 +64,7 @@ New Features
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
+  * Added Meter API to calculate a profile configuration values.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
index e49fcf271c..e236cac09b 100644
--- a/lib/ethdev/rte_mtr.c
+++ b/lib/ethdev/rte_mtr.c
@@ -91,6 +91,18 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 		meter_profile_id, error);
 }
 
+/** MTR meter profile calculate */
+int
+rte_mtr_meter_profile_calculate(uint16_t port_id,
+	uint32_t meter_profile_id,
+	void *meter_profile_cfg,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_FUNC(port_id, meter_profile_calculate)(dev,
+		meter_profile_id, meter_profile_cfg, error);
+}
+
 /* MTR meter policy validate */
 int
 rte_mtr_meter_policy_validate(uint16_t port_id,
diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
index 40df0888c8..d7b916b168 100644
--- a/lib/ethdev/rte_mtr.h
+++ b/lib/ethdev/rte_mtr.h
@@ -524,6 +524,30 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter profile calculate
+ *
+ * Calculate CIR, CBS and EBS values for a given meter profile ID.
+ * Convert user-provided values to PMD-specific configuration.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] meter_profile_id
+ *   Meter profile ID. Needs to be the valid.
+ * @param[out] meter_profile_cfg
+ *   Meter profile configuration filled by PMD.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   0 on success, non-zero error code otherwise.
+ */
+__rte_experimental
+int
+rte_mtr_meter_profile_calculate(uint16_t port_id,
+	uint32_t meter_profile_id,
+	void *meter_profile_cfg,
+	struct rte_mtr_error *error);
+
 /**
  * Check whether a meter policy can be created on a given port.
  *
diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
index ee8c6ef7ad..f1982ff220 100644
--- a/lib/ethdev/rte_mtr_driver.h
+++ b/lib/ethdev/rte_mtr_driver.h
@@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter profile calculate. */
+typedef int (*rte_mtr_meter_profile_calculate_t)(struct rte_eth_dev *dev,
+	uint32_t meter_profile_id,
+	void *meter_profile_cfg,
+	struct rte_mtr_error *error);
+
 /** @internal MTR meter policy validate. */
 typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
 	struct rte_mtr_meter_policy_params *policy,
@@ -156,6 +162,9 @@ struct rte_mtr_ops {
 
 	/** MTR object meter policy update */
 	rte_mtr_meter_policy_update_t meter_policy_update;
+
+	/** MTR meter profile calculate */
+	rte_mtr_meter_profile_calculate_t meter_profile_calculate;
 };
 
 /**
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index 20391ab29e..76024331cb 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -279,6 +279,9 @@ EXPERIMENTAL {
 	rte_flow_async_action_handle_create;
 	rte_flow_async_action_handle_destroy;
 	rte_flow_async_action_handle_update;
+
+	# added in 22.07
+	rte_mtr_meter_profile_calculate;
 };
 
 INTERNAL {
-- 
2.18.2

[PATCH 4/4] ethdev: add meter color mark flow action

[Alexander Kozyrev]

Create a new Flow API action: METER_MARK.
It Meters an IP packet stream and marks its packets with colors.
Unlike the METER action, it performs no policing at all.
An user has the flexibility to create any policies with the help of
the METER_COLOR item later, only meter profile is required now.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     | 26 +++++++++++++++++++
 doc/guides/rel_notes/release_22_07.rst |  1 +
 lib/ethdev/rte_flow.h                  | 36 ++++++++++++++++++++++++++
 3 files changed, 63 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 1877d4b9b5..f5a2150f86 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3411,6 +3411,32 @@ This action is meant to use the same structure as `Action: PORT_REPRESENTOR`_.
 
 See also `Item: REPRESENTED_PORT`_.
 
+Action: ``METER_MARK``
+^^^^^^^^^^^^^^^^^^^^^^
+
+Meters an IP packet stream and marks its packets with colors.
+
+Applies a stage of metering only. Unlike the ``METER`` action,
+no policing is performed and the ``METER_COLOR`` item may be used later.
+The profle object has to be created using the rte_mtr_profile_create() API.
+The ID of the profile object may be specified as an action parameter.
+Or PMD-specific profile configuration may be used to save the time to calculate
+the profile configuration via rte_mtr_profile_calculate() API.
+
+.. _table_rte_flow_action_meter_mark:
+
+.. table:: METER_MARK
+
+   +------------------+----------------------+
+   | Field            | Value                |
+   +==================+======================+
+   | ``mode``         | Meter profile mode   |
+   +------------------+----------------------+
+   | ``profile_id``   | Meter profile ID     |
+   +------------------+----------------------+
+   | ``profile_cfg``  | Meter profile config |
+   +------------------+----------------------+
+
 Negative types
 ~~~~~~~~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 5f19223663..591ac7cfba 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -65,6 +65,7 @@ New Features
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
   * Added Meter API to calculate a profile configuration values.
+  * Added METER_MARK action to perform Meter color marking only.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 817df34ed0..ac3a3b338f 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -2906,6 +2906,15 @@ enum rte_flow_action_type {
 	 * @see struct rte_flow_action_ethdev
 	 */
 	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+
+	/**
+	 * Traffic metering and marking (MTR).
+	 * the entity represented by the given ethdev.
+	 *
+	 * @see struct rte_flow_action_meter_mark
+	 * See file rte_mtr.h for MTR profile object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_MARK,
 };
 
 /**
@@ -3775,6 +3784,33 @@ struct rte_flow_action_modify_field {
 	uint32_t width; /**< Number of bits to use from a source field. */
 };
 
+/**
+ * Meter profile mode for METER_MARK action.
+ */
+enum rte_flow_meter_profile_mode {
+	RTE_FLOW_METER_PROFILE_ID = 0, /**< Use profile ID. */
+	RTE_FLOW_METER_PROFILE_CFG,    /**< Use profile config. */
+};
+
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_MARK
+ *
+ * Traffic metering and marking (MTR).
+ *
+ * Meters an IP packet stream and marks its packets either
+ * green, yellow, or red according to the specified profile.
+ */
+struct rte_flow_action_meter_mark {
+	enum rte_flow_meter_profile_mode mode; /** Meter profile mode. */
+	RTE_STD_C11
+	union {
+		/**< Profile ID create with rte_mtr_profile_create(). */
+		uint32_t profile_id;
+		/**< Profile config calculated with rte_mtr_profile_calculate(). */
+		void *profile_cfg;
+	};
+};
+
 /* Mbuf dynamic field offset for metadata. */
 extern int32_t rte_flow_dynf_metadata_offs;
 
-- 
2.18.2

RE: [PATCH 1/4] ethdev: add meter color flow matching item

[Dumitrescu, Cristian]

> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index d8827dd184..e3f0e86eb6 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -671,6 +671,14 @@ enum rte_flow_item_type {
>  	 * See struct rte_flow_item_gre_opt.
>  	 */
>  	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> +
> +	/**
> +	 * Matches Meter Color Marker.
> +	 *
> +	 * See struct rte_flow_item_meter_color.
> +	 */
> +
> +	RTE_FLOW_ITEM_TYPE_METER_COLOR,
>  };
> 
>  /**
> @@ -2201,6 +2209,22 @@ struct rte_flow_item_flex_conf {
>  	uint32_t nb_outputs;
>  };
> 
> +/**
> + * RTE_FLOW_ITEM_TYPE_METER_COLOR.
> + *
> + * Matches Color Marker set by a Meter in the DS field of a packet.
> + */
> +struct rte_flow_item_meter_color {
> +	enum rte_color color; /**< Meter color marker. */
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> +#ifndef __cplusplus
> +static const struct rte_flow_item_meter_color
> rte_flow_item_meter_color_mask = {
> +	.color = RTE_COLORS,
> +};
> +#endif

Adding color as one of the match items makes sense to me as well.

RE: [PATCH 2/4] ethdev: allow meter color marker modification

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, May 18, 2022 5:35 AM
> To: dev@dpdk.org
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> jerinjacobk@gmail.com; orika@nvidia.com; thomas@monjalon.net;
> ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
> ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [PATCH 2/4] ethdev: allow meter color marker modification
> 
> Extend modify_field Flow API with support of Meter Color Marker
> modifications. It allows setting the DS field of a packet to any
> color marker: green, yellow or red. User is able to specify
> an initial packet color for Meter API or create simple Metering
> and Marking flow rules based on his own coloring alghorithm.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/rel_notes/release_22_07.rst | 1 +
>  lib/ethdev/rte_flow.h                  | 1 +
>  2 files changed, 2 insertions(+)
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst
> b/doc/guides/rel_notes/release_22_07.rst
> index a61a4d77dc..ec83178d98 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -63,6 +63,7 @@ New Features
>  * **Extended Metering and Marking support in the Flow API.**
> 
>    * Added METER_COLOR item to match Color Marker set by a Meter.
> +  * Added ability to set Color Marker via modify_field Flow API.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index e3f0e86eb6..817df34ed0 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3714,6 +3714,7 @@ enum rte_flow_field_id {
>  	RTE_FLOW_FIELD_META,		/**< Metadata value. */
>  	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
>  	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
> +	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter Color Marker. */
>  };
> 
>  /**
> --
> 2.18.2

This is not really correct. The packet color is a sort if meta-data field that does not get sent out directly.

Yes, it can translate to a header field, but there are many possibilities: outer/inner VLAN PCP&DEI, IPv4/IPv6 DSCP, MPLS QoS, etc. We need to specify exactly which field from the packet needs to be modified.

We had the same problem for the input color mechanism, I suggest you take a look at Jerin's patch.

RE: [PATCH 3/4] ethdev: add meter profile config calculation

[Dumitrescu, Cristian]

> diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
> index 40df0888c8..d7b916b168 100644
> --- a/lib/ethdev/rte_mtr.h
> +++ b/lib/ethdev/rte_mtr.h
> @@ -524,6 +524,30 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>  	uint32_t meter_profile_id,
>  	struct rte_mtr_error *error);
> 
> +/**
> + * Meter profile calculate
> + *
> + * Calculate CIR, CBS and EBS values for a given meter profile ID.
> + * Convert user-provided values to PMD-specific configuration.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] meter_profile_id
> + *   Meter profile ID. Needs to be the valid.
> + * @param[out] meter_profile_cfg
> + *   Meter profile configuration filled by PMD.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   0 on success, non-zero error code otherwise.
> + */
> +__rte_experimental
> +int
> +rte_mtr_meter_profile_calculate(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	void *meter_profile_cfg,
> +	struct rte_mtr_error *error);
> +
>  /**
>   * Check whether a meter policy can be created on a given port.
>   *

I don't understand the purpose of this proposed API function, can you please explain?

It looks to me that you want to have an alternative way to create a meter profile, so why not use the existing API function rte_mtr_meter_profile_add() ? It does the same thing.

Also, who allocates this opaque array meter_profile_cfg? Assuming it is the user that needs to allocate it, how does the user know its size?

[PATCH v2 0/4] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

Extend Metering and Marking support in the Flow API:
1. Add METER_COLOR item to match Color Marker set by a Meter.
2. Add the ability to set Color Marker via modify_field Flow API.
3. Add Meter API to get profile/policy objects.
4. Add METER_MARK action to perform Meter color metering and marking.
Provide greater flexibility in how Metering can be used.

RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/

Traditional Meter Usage:

profile_id = rte_mtr_meter_profile_add(RFC_params);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
meter_id = rte_mtr_create(profile_id, policy_id);
rte_flow_create(pattern=5-tuple,actions=METER(meter_id));

The METER action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.
3. Jump to a policy group.
4. Match on a color.
5. Execute assigned policy actions for the color.

New Meter Usage Model:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr),JUMP);
rte_flow_create(pattern=COLOR, actions=...);

The METER_MARK action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.

A user is able to match the color later with the COLOR item.
In order to do this we add the JUMP action after the METER action.

3. Jump to a policy group.
4. Match on a color.
5. Execute actions for the color.

Here we decoupled the meter profile usage from the meter policy usage
for greater flexibility and got rid of any locks related to meter_id lookup.

Another example of the meter creation to mimic the old model entirely:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
*policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr, policy_obj_ptr));

In this case, we define the policy actions right away.
The main advantage is not having to lookup for profile_id/policy_id.

To free the meter obects we need to do the following:
rte_flow_destroy(flow_handle);
rte_mtr_meter_policy_delete(policy_id);
rte_mtr_meter_profile_delete(profile_id);.
profile_obj_ptr and policy_obj_ptr are no longer valid after that.

The meter profile configuration cannot be updated dynamically
with the current set of patches, but can be supported later on.
Now you have to destroy flows and profiles and recreate them.
But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
can have the corresponding siblings without mtr_id parameters.
In this case, we can update the config and all the flows using them.

The meter sharing is done via the indirect action Flow API:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_prof8ile_get(profile_id);
handle = rte_flow_action_handle_create(action=METER(profile_obj_ptr, NULL));
flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));

Once we are done with the flow rules we can free everything.
rte_flow_destroy(flow1);
rte_flow_destroy(flow2);
rte_flow_action_handle_destroy(handle);
rte_mtr_meter_profile_delete(profile_id);

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Alexander Kozyrev (4):
  ethdev: add meter color flow matching item
  ethdev: allow meter color marker modification
  ethdev: get meter profile/policy objects
  ethdev: add meter color mark flow action

 doc/guides/prog_guide/rte_flow.rst            | 32 ++++++++++
 .../traffic_metering_and_policing.rst         |  7 +++
 doc/guides/rel_notes/release_22_07.rst        |  7 +++
 lib/ethdev/rte_flow.c                         |  1 +
 lib/ethdev/rte_flow.h                         | 60 +++++++++++++++++++
 lib/ethdev/rte_mtr.c                          | 41 +++++++++++++
 lib/ethdev/rte_mtr.h                          | 40 +++++++++++++
 lib/ethdev/rte_mtr_driver.h                   | 19 ++++++
 lib/ethdev/version.map                        |  2 +
 9 files changed, 209 insertions(+)

-- 
2.18.2

[PATCH v2 1/4] ethdev: add meter color flow matching item

[Alexander Kozyrev]

Provide an ability to use a Color Marker set by a Meter
as a matching item in Flow API. The Color Marker reflects
the metering result by setting the metadata for a
packet to a particular codepoint: green, yellow or red.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
 doc/guides/rel_notes/release_22_07.rst |  4 ++++
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
 4 files changed, 36 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 588914b231..018def1033 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1651,6 +1651,13 @@ Matches a PPP header.
 - ``proto_id``: PPP protocol identifier.
 - Default ``mask`` matches addr, ctrl, proto_id.
 
+Item: ``METER_COLOR``
+^^^^^^^^^^^^^^^^^^^^^
+
+Matches Color Marker set by a Meter.
+
+- ``color``: Metering color marker.
+
 Actions
 ~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 0ed4f92820..c1804bc74f 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -69,6 +69,10 @@ New Features
   Added an API which can get the number of in-flight packets in
   vhost async data path without using lock.
 
+* **Extended Metering and Marking support in the Flow API.**
+
+  * Added METER_COLOR item to match Color Marker set by a Meter.
+
 * **Updated Intel iavf driver.**
 
   * Added Tx QoS queue rate limitation support.
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 2c35a2f13e..07d7e998c0 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -161,6 +161,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
 			rte_flow_item_flex_conv),
 	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
 	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
+	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d8827dd184..75ec6621ac 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -671,6 +671,14 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+
+	/**
+	 * Matches Meter Color Marker.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -2201,6 +2209,22 @@ struct rte_flow_item_flex_conf {
 	uint32_t nb_outputs;
 };
 
+/**
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR.
+ *
+ * Matches Color Marker set by a Meter.
+ */
+struct rte_flow_item_meter_color {
+	enum rte_color color; /**< Meter color marker. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+	.color = RTE_COLORS,
+};
+#endif
+
 /**
  * Action types.
  *
-- 
2.18.2

[PATCH v2 2/4] ethdev: allow meter color marker modification

[Alexander Kozyrev]

Extend modify_field Flow API with support of Meter Color Marker
modifications. It allows setting the packet's metadata to any
color marker: green, yellow or red. User is able to specify
an initial packet color for Meter API or create simple Metering
and Marking flow rules based on his own coloring algorithm.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_22_07.rst | 1 +
 lib/ethdev/rte_flow.h                  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index c1804bc74f..451ff8d460 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -72,6 +72,7 @@ New Features
 * **Extended Metering and Marking support in the Flow API.**
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
+  * Added ability to set Color Marker via modify_field Flow API.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 75ec6621ac..68af109554 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3714,6 +3714,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_META,		/**< Metadata value. */
 	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
 	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
+	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
 };
 
 /**
-- 
2.18.2

[PATCH v2 3/4] ethdev: get meter profile/policy objects

[Alexander Kozyrev]

Introduce a new Meter API to retrieve a Meter profile and policy
objects using the profile/policy ID previously created with
meter_profile_add() and meter_policy_create() functions.
That allows to save the pointer and avoid any lookups in the
corresponding lists for a quick access during a flow rule creation.
Also, it eliminates the need for CIR, CBS and EBS calculations
and conversion to a PMD-specific format when the profile is used.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 .../traffic_metering_and_policing.rst         |  7 ++++
 doc/guides/rel_notes/release_22_07.rst        |  1 +
 lib/ethdev/rte_flow.h                         |  7 ++++
 lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
 lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
 lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
 lib/ethdev/version.map                        |  2 +
 7 files changed, 117 insertions(+)

diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst
index d1958a023d..cae35bccf6 100644
--- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
+++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
@@ -107,6 +107,13 @@ traffic meter and policing library.
      to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
      specified per color as show in :numref:`figure_rte_mtr_chaining`.
 
+#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
+   API functions are available for getting the object pointers directly.
+   These pointers allow quick access to profile/policy objects and are
+   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
+   This action may omit the polciy definition to preovide a flexibility
+   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
+
 Protocol based input color selection
 ------------------------------------
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 451ff8d460..6d030bead5 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -73,6 +73,7 @@ New Features
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
+  * Added Meter API to get a pointer to profile/policy by their ID.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 68af109554..9754f6630a 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3827,6 +3827,13 @@ struct rte_flow_action {
  */
 struct rte_flow;
 
+/**
+ * Opaque type for Meter profile object returned by MTR API.
+ *
+ * This handle can be used to create Meter actions instead of profile ID.
+ */
+struct rte_flow_meter_profile;
+
 /**
  * @warning
  * @b EXPERIMENTAL: this structure may change without prior notice
diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
index 441ea1dca9..90fd277040 100644
--- a/lib/ethdev/rte_mtr.c
+++ b/lib/ethdev/rte_mtr.c
@@ -57,6 +57,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
 	ops->func;					\
 })
 
+#define RTE_MTR_HNDL_FUNC(port_id, func)		\
+({							\
+	const struct rte_mtr_ops *ops =			\
+		rte_mtr_ops_get(port_id, error);	\
+	if (ops == NULL)				\
+		return NULL;				\
+							\
+	if (ops->func == NULL) {			\
+		rte_mtr_error_set(error,		\
+			ENOSYS,				\
+			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
+			NULL,				\
+			rte_strerror(ENOSYS));		\
+		return NULL;				\
+	}						\
+							\
+	ops->func;					\
+})
+
 /* MTR capabilities get */
 int
 rte_mtr_capabilities_get(uint16_t port_id,
@@ -91,6 +110,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 		meter_profile_id, error);
 }
 
+/** MTR meter profile get */
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
+		meter_profile_id, error);
+}
+
 /* MTR meter policy validate */
 int
 rte_mtr_meter_policy_validate(uint16_t port_id,
@@ -125,6 +155,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
 		policy_id, error);
 }
 
+/** MTR meter policy get */
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
+		policy_id, error);
+}
+
 /** MTR object create */
 int
 rte_mtr_create(uint16_t port_id,
diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
index 008bc84f0d..58f0d26215 100644
--- a/lib/ethdev/rte_mtr.h
+++ b/lib/ethdev/rte_mtr.h
@@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter profile object get
+ *
+ * Get meter profile object for a given meter profile ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] meter_profile_id
+ *   Meter profile ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /**
  * Check whether a meter policy can be created on a given port.
  *
@@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
 	struct rte_mtr_meter_policy_params *policy,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter policy object get
+ *
+ * Get meter policy object for a given meter policy ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] policy_id
+ *   Meter policy ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
 /**
  * Define meter policy action list:
  * GREEN - GREEN, YELLOW - YELLOW, RED - RED
diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
index f7dca9a54c..948a629b93 100644
--- a/lib/ethdev/rte_mtr_driver.h
+++ b/lib/ethdev/rte_mtr_driver.h
@@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter profile get. */
+typedef struct rte_flow_meter_profile *
+(*rte_mtr_meter_profile_get_t)(struct rte_eth_dev *dev,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /** @internal MTR meter policy validate. */
 typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
 	struct rte_mtr_meter_policy_params *policy,
@@ -57,6 +63,13 @@ typedef int (*rte_mtr_meter_policy_delete_t)(struct rte_eth_dev *dev,
 	uint32_t policy_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter policy get. */
+typedef struct rte_flow_meter_policy *
+(*rte_mtr_meter_policy_get_t)(struct rte_eth_dev *dev,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
+
 /** @internal MTR object create. */
 typedef int (*rte_mtr_create_t)(struct rte_eth_dev *dev,
 	uint32_t mtr_id,
@@ -194,6 +207,12 @@ struct rte_mtr_ops {
 
 	/** MTR object meter policy update */
 	rte_mtr_meter_policy_update_t meter_policy_update;
+
+	/** MTR meter profile get */
+	rte_mtr_meter_profile_get_t meter_profile_get;
+
+	/** MTR meter policy get */
+	rte_mtr_meter_policy_get_t meter_policy_get;
 };
 
 /**
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index daca7851f2..2609f2e709 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -285,6 +285,8 @@ EXPERIMENTAL {
 	rte_mtr_color_in_protocol_priority_get;
 	rte_mtr_color_in_protocol_set;
 	rte_mtr_meter_vlan_table_update;
+	rte_mtr_meter_profile_get;
+	rte_mtr_meter_policy_get;
 };
 
 INTERNAL {
-- 
2.18.2

[PATCH v2 4/4] ethdev: add meter color mark flow action

[Alexander Kozyrev]

Create a new Flow API action: METER_MARK.
It Meters an IP packet stream and marks its packets with colors.
Unlike the METER action, it performs no policing at all.
A user has the flexibility to create any policies with the help of
the METER_COLOR item later, only meter profile is required here.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     | 25 +++++++++++++++++++++++
 doc/guides/rel_notes/release_22_07.rst |  1 +
 lib/ethdev/rte_flow.h                  | 28 ++++++++++++++++++++++++++
 3 files changed, 54 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 018def1033..90f0ebc9bc 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3411,6 +3411,31 @@ This action is meant to use the same structure as `Action: PORT_REPRESENTOR`_.
 
 See also `Item: REPRESENTED_PORT`_.
 
+Action: ``METER_MARK``
+^^^^^^^^^^^^^^^^^^^^^^
+
+Meters an IP packet stream and marks its packets with colors.
+
+Unlike the ``METER`` action, policing is optional and may be
+performed later with the help of the ``METER_COLOR`` item.
+The profile and/or policy objects have to be created
+using the rte_mtr_profile_add()/rte_mtr_policy_add() API.
+Pointers to these objects are used as action parameters
+and need to be retrieved using the rte_mtr_profile_get() API
+and rte_mtr_policy_get() API respectively.
+
+.. _table_rte_flow_action_meter_mark:
+
+.. table:: METER_MARK
+
+   +------------------+----------------------+
+   | Field            | Value                |
+   +==================+======================+
+   | ``profile``      | Meter profile object |
+   +------------------+----------------------+
+   | ``policy``       | Meter policy object  |
+   +------------------+----------------------+
+
 Negative types
 ~~~~~~~~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 6d030bead5..fca7d07e4b 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -74,6 +74,7 @@ New Features
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
   * Added Meter API to get a pointer to profile/policy by their ID.
+  * Added METER_MARK action for Metering with lockless profile/policy access.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 9754f6630a..bb16108f16 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -2906,6 +2906,15 @@ enum rte_flow_action_type {
 	 * @see struct rte_flow_action_ethdev
 	 */
 	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+
+	/**
+	 * Traffic metering and marking (MTR).
+	 * the entity represented by the given ethdev.
+	 *
+	 * @see struct rte_flow_action_meter_mark
+	 * See file rte_mtr.h for MTR profile object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_MARK,
 };
 
 /**
@@ -3775,6 +3784,25 @@ struct rte_flow_action_modify_field {
 	uint32_t width; /**< Number of bits to use from a source field. */
 };
 
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_MARK
+ *
+ * Traffic metering and marking (MTR).
+ *
+ * Meters an IP packet stream and marks its packets either
+ * green, yellow, or red according to the specified profile.
+ * The policy is optional and may be specified for defining
+ * subsequent actions based on a color asigned by MTR.
+ * Alternatively, the METER_COLOR item may be used for this.
+ */
+struct rte_flow_action_meter_mark {
+
+	/**< Profile config retrieved with rte_mtr_profile_get(). */
+	struct rte_flow_meter_profile *profile;
+	/**< Profile config retrieved with rte_mtr_policy_get(). */
+	struct rte_flow_meter_policy *policy;
+};
+
 /* Mbuf dynamic field offset for metadata. */
 extern int32_t rte_flow_dynf_metadata_offs;
 
-- 
2.18.2

RE: [PATCH 2/4] ethdev: allow meter color marker modification

[Alexander Kozyrev]

On Thur, May 19, 2022 10:49 Dumitrescu, Cristian <cristian.dumitrescu@intel.com>:
> This is not really correct. The packet color is a sort if meta-data field that does
> not get sent out directly.
> 
> Yes, it can translate to a header field, but there are many possibilities:
> outer/inner VLAN PCP&DEI, IPv4/IPv6 DSCP, MPLS QoS, etc. We need to specify
> exactly which field from the packet needs to be modified.
> 
> We had the same problem for the input color mechanism, I suggest you take a
> look at Jerin's patch.

You are absolutely correct, I'm not changing any way of color storing internally.
I changed the description in the v2 patch to reflect it is metadata field. Thanks.

RE: [PATCH 3/4] ethdev: add meter profile config calculation

[Alexander Kozyrev]

On  Thur, May 19, 2022 10:55 Dumitrescu, Cristian <cristian.dumitrescu@intel.com>:
> I don't understand the purpose of this proposed API function, can you please
> explain?
> 
> It looks to me that you want to have an alternative way to create a meter
> profile, so why not use the existing API function rte_mtr_meter_profile_add() ?
> It does the same thing.
> 
> Also, who allocates this opaque array meter_profile_cfg? Assuming it is the user
> that needs to allocate it, how does the user know its size?

The creation of a meter is still done thru the rte_mtr_meter_profile_add() API.
It is responsible for memory allocation and parameters calculation.
The rte_mtr_meter_profile_get() API just returns the pointer to the object
allocated earlier, based on the Meter ID, saving the lookup time.
User still have to manage the lifecycle with add/destroy functions.
PMD allocates this opaque pointer and knows its size, user doesn't need to worry.
I hope that my examples in the meeting minutes explain this concept clear enough.

RE: [PATCH v2 1/4] ethdev: add meter color flow matching item

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Sunday, May 22, 2022 1:51 PM
> Subject: [PATCH v2 1/4] ethdev: add meter color flow matching item
> 
> Provide an ability to use a Color Marker set by a Meter
> as a matching item in Flow API. The Color Marker reflects
> the metering result by setting the metadata for a
> packet to a particular codepoint: green, yellow or red.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
>  doc/guides/rel_notes/release_22_07.rst |  4 ++++
>  lib/ethdev/rte_flow.c                  |  1 +
>  lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
>  4 files changed, 36 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index 588914b231..018def1033 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1651,6 +1651,13 @@ Matches a PPP header.
>  - ``proto_id``: PPP protocol identifier.
>  - Default ``mask`` matches addr, ctrl, proto_id.
> 
> +Item: ``METER_COLOR``
> +^^^^^^^^^^^^^^^^^^^^^
> +
> +Matches Color Marker set by a Meter.
> +
> +- ``color``: Metering color marker.
> +
>  Actions
>  ~~~~~~~
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 0ed4f92820..c1804bc74f 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -69,6 +69,10 @@ New Features
>    Added an API which can get the number of in-flight packets in
>    vhost async data path without using lock.
> 
> +* **Extended Metering and Marking support in the Flow API.**
> +
> +  * Added METER_COLOR item to match Color Marker set by a Meter.
> +
>  * **Updated Intel iavf driver.**
> 
>    * Added Tx QoS queue rate limitation support.
> diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
> index 2c35a2f13e..07d7e998c0 100644
> --- a/lib/ethdev/rte_flow.c
> +++ b/lib/ethdev/rte_flow.c
> @@ -161,6 +161,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
>  			rte_flow_item_flex_conv),
>  	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
>  	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
> +	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
>  };
> 
>  /** Generate flow_action[] entry. */
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index d8827dd184..75ec6621ac 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -671,6 +671,14 @@ enum rte_flow_item_type {
>  	 * See struct rte_flow_item_gre_opt.
>  	 */
>  	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> +
> +	/**
> +	 * Matches Meter Color Marker.
> +	 *
> +	 * See struct rte_flow_item_meter_color.
> +	 */
> +
> +	RTE_FLOW_ITEM_TYPE_METER_COLOR,
>  };
> 
>  /**
> @@ -2201,6 +2209,22 @@ struct rte_flow_item_flex_conf {
>  	uint32_t nb_outputs;
>  };
> 
> +/**
> + * RTE_FLOW_ITEM_TYPE_METER_COLOR.
> + *
> + * Matches Color Marker set by a Meter.
> + */
> +struct rte_flow_item_meter_color {
> +	enum rte_color color; /**< Meter color marker. */
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> +#ifndef __cplusplus
> +static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
> +	.color = RTE_COLORS,
> +};
> +#endif
> +
>  /**
>   * Action types.
>   *
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v2 2/4] ethdev: allow meter color marker modification

[Ori Kam]

Hi Alex,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Sunday, May 22, 2022 1:51 PM
> Subject: [PATCH v2 2/4] ethdev: allow meter color marker modification
> 
> Extend modify_field Flow API with support of Meter Color Marker
> modifications. It allows setting the packet's metadata to any
> color marker: green, yellow or red. User is able to specify
> an initial packet color for Meter API or create simple Metering
> and Marking flow rules based on his own coloring algorithm.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/rel_notes/release_22_07.rst | 1 +
>  lib/ethdev/rte_flow.h                  | 1 +
>  2 files changed, 2 insertions(+)
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index c1804bc74f..451ff8d460 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -72,6 +72,7 @@ New Features
>  * **Extended Metering and Marking support in the Flow API.**
> 
>    * Added METER_COLOR item to match Color Marker set by a Meter.
> +  * Added ability to set Color Marker via modify_field Flow API.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 75ec6621ac..68af109554 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3714,6 +3714,7 @@ enum rte_flow_field_id {
>  	RTE_FLOW_FIELD_META,		/**< Metadata value. */
>  	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
>  	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
> +	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
>  };
> 
>  /**
> --
> 2.18.2

Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v2 3/4] ethdev: get meter profile/policy objects

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Sunday, May 22, 2022 1:51 PM
> Subject: [PATCH v2 3/4] ethdev: get meter profile/policy objects
> 
> Introduce a new Meter API to retrieve a Meter profile and policy
> objects using the profile/policy ID previously created with
> meter_profile_add() and meter_policy_create() functions.
> That allows to save the pointer and avoid any lookups in the
> corresponding lists for a quick access during a flow rule creation.
> Also, it eliminates the need for CIR, CBS and EBS calculations
> and conversion to a PMD-specific format when the profile is used.

I think you should say also what happens to this pointer or values
in case of modification of the policy or destroy of the policy.

> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  .../traffic_metering_and_policing.rst         |  7 ++++
>  doc/guides/rel_notes/release_22_07.rst        |  1 +
>  lib/ethdev/rte_flow.h                         |  7 ++++
>  lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
>  lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
>  lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
>  lib/ethdev/version.map                        |  2 +
>  7 files changed, 117 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst
> b/doc/guides/prog_guide/traffic_metering_and_policing.rst
> index d1958a023d..cae35bccf6 100644
> --- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
> +++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
> @@ -107,6 +107,13 @@ traffic meter and policing library.
>       to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
>       specified per color as show in :numref:`figure_rte_mtr_chaining`.
> 
> +#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
> +   API functions are available for getting the object pointers directly.
> +   These pointers allow quick access to profile/policy objects and are
> +   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
> +   This action may omit the polciy definition to preovide a flexibility
> +   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
I think it is important to explain what happens if a profile is changed to destroyed.

> +
>  Protocol based input color selection
>  ------------------------------------
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 451ff8d460..6d030bead5 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -73,6 +73,7 @@ New Features
> 
>    * Added METER_COLOR item to match Color Marker set by a Meter.
>    * Added ability to set Color Marker via modify_field Flow API.
> +  * Added Meter API to get a pointer to profile/policy by their ID.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 68af109554..9754f6630a 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3827,6 +3827,13 @@ struct rte_flow_action {
>   */
>  struct rte_flow;
> 
> +/**
> + * Opaque type for Meter profile object returned by MTR API.
> + *
> + * This handle can be used to create Meter actions instead of profile ID.
> + */
> +struct rte_flow_meter_profile;
> +
>  /**
>   * @warning
>   * @b EXPERIMENTAL: this structure may change without prior notice
> diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
> index 441ea1dca9..90fd277040 100644
> --- a/lib/ethdev/rte_mtr.c
> +++ b/lib/ethdev/rte_mtr.c
> @@ -57,6 +57,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
>  	ops->func;					\
>  })
> 
> +#define RTE_MTR_HNDL_FUNC(port_id, func)		\
> +({							\
> +	const struct rte_mtr_ops *ops =			\
> +		rte_mtr_ops_get(port_id, error);	\
> +	if (ops == NULL)				\
> +		return NULL;				\
> +							\
> +	if (ops->func == NULL) {			\
> +		rte_mtr_error_set(error,		\
> +			ENOSYS,				\
> +			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
> +			NULL,				\
> +			rte_strerror(ENOSYS));		\
> +		return NULL;				\
> +	}						\
> +							\
> +	ops->func;					\
> +})
> +
>  /* MTR capabilities get */
>  int
>  rte_mtr_capabilities_get(uint16_t port_id,
> @@ -91,6 +110,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>  		meter_profile_id, error);
>  }
> 
> +/** MTR meter profile get */
> +struct rte_flow_meter_profile *
> +rte_mtr_meter_profile_get(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error)
> +{
> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
> +		meter_profile_id, error);
> +}
> +
>  /* MTR meter policy validate */
>  int
>  rte_mtr_meter_policy_validate(uint16_t port_id,
> @@ -125,6 +155,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
>  		policy_id, error);
>  }
> 
> +/** MTR meter policy get */
> +struct rte_flow_meter_policy *
> +rte_mtr_meter_policy_get(uint16_t port_id,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error)
> +{
> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
> +		policy_id, error);
> +}
> +
>  /** MTR object create */
>  int
>  rte_mtr_create(uint16_t port_id,
> diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
> index 008bc84f0d..58f0d26215 100644
> --- a/lib/ethdev/rte_mtr.h
> +++ b/lib/ethdev/rte_mtr.h
> @@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>  	uint32_t meter_profile_id,
>  	struct rte_mtr_error *error);
> 
> +/**
> + * Meter profile object get
> + *
> + * Get meter profile object for a given meter profile ID.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] meter_profile_id
> + *   Meter profile ID. Needs to be the valid.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +__rte_experimental
> +struct rte_flow_meter_profile *
> +rte_mtr_meter_profile_get(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error);
> +
>  /**
>   * Check whether a meter policy can be created on a given port.
>   *
> @@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
>  	struct rte_mtr_meter_policy_params *policy,
>  	struct rte_mtr_error *error);
> 
> +/**
> + * Meter policy object get
> + *
> + * Get meter policy object for a given meter policy ID.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] policy_id
> + *   Meter policy ID. Needs to be the valid.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +__rte_experimental
> +struct rte_flow_meter_policy *
> +rte_mtr_meter_policy_get(uint16_t port_id,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error);
> +
>  /**
>   * Define meter policy action list:
>   * GREEN - GREEN, YELLOW - YELLOW, RED - RED
> diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
> index f7dca9a54c..948a629b93 100644
> --- a/lib/ethdev/rte_mtr_driver.h
> +++ b/lib/ethdev/rte_mtr_driver.h
> @@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
>  	uint32_t meter_profile_id,
>  	struct rte_mtr_error *error);
> 
> +/** @internal MTR meter profile get. */
> +typedef struct rte_flow_meter_profile *
> +(*rte_mtr_meter_profile_get_t)(struct rte_eth_dev *dev,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error);
> +
>  /** @internal MTR meter policy validate. */
>  typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
>  	struct rte_mtr_meter_policy_params *policy,
> @@ -57,6 +63,13 @@ typedef int (*rte_mtr_meter_policy_delete_t)(struct rte_eth_dev *dev,
>  	uint32_t policy_id,
>  	struct rte_mtr_error *error);
> 
> +/** @internal MTR meter policy get. */
> +typedef struct rte_flow_meter_policy *
> +(*rte_mtr_meter_policy_get_t)(struct rte_eth_dev *dev,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error);
> +
> +
>  /** @internal MTR object create. */
>  typedef int (*rte_mtr_create_t)(struct rte_eth_dev *dev,
>  	uint32_t mtr_id,
> @@ -194,6 +207,12 @@ struct rte_mtr_ops {
> 
>  	/** MTR object meter policy update */
>  	rte_mtr_meter_policy_update_t meter_policy_update;
> +
> +	/** MTR meter profile get */
> +	rte_mtr_meter_profile_get_t meter_profile_get;
> +
> +	/** MTR meter policy get */
> +	rte_mtr_meter_policy_get_t meter_policy_get;
>  };
> 
>  /**
> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index daca7851f2..2609f2e709 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -285,6 +285,8 @@ EXPERIMENTAL {
>  	rte_mtr_color_in_protocol_priority_get;
>  	rte_mtr_color_in_protocol_set;
>  	rte_mtr_meter_vlan_table_update;
> +	rte_mtr_meter_profile_get;
> +	rte_mtr_meter_policy_get;
>  };
> 
>  INTERNAL {
> --
> 2.18.2

Best,
Ori

RE: [PATCH v2 4/4] ethdev: add meter color mark flow action

[Ori Kam]

Hi Alexander,


> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Sunday, May 22, 2022 1:51 PM
> Subject: [PATCH v2 4/4] ethdev: add meter color mark flow action
> 
> Create a new Flow API action: METER_MARK.
> It Meters an IP packet stream and marks its packets with colors.
> Unlike the METER action, it performs no policing at all.
> A user has the flexibility to create any policies with the help of
> the METER_COLOR item later, only meter profile is required here.

I think required should be mandatory.

> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst     | 25 +++++++++++++++++++++++
>  doc/guides/rel_notes/release_22_07.rst |  1 +
>  lib/ethdev/rte_flow.h                  | 28 ++++++++++++++++++++++++++
>  3 files changed, 54 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index 018def1033..90f0ebc9bc 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -3411,6 +3411,31 @@ This action is meant to use the same structure as `Action:
> PORT_REPRESENTOR`_.
> 
>  See also `Item: REPRESENTED_PORT`_.
> 
> +Action: ``METER_MARK``
> +^^^^^^^^^^^^^^^^^^^^^^
> +
> +Meters an IP packet stream and marks its packets with colors.
> +
> +Unlike the ``METER`` action, policing is optional and may be
> +performed later with the help of the ``METER_COLOR`` item.
> +The profile and/or policy objects have to be created
> +using the rte_mtr_profile_add()/rte_mtr_policy_add() API.
> +Pointers to these objects are used as action parameters
> +and need to be retrieved using the rte_mtr_profile_get() API
> +and rte_mtr_policy_get() API respectively.
> +
> +.. _table_rte_flow_action_meter_mark:
> +
> +.. table:: METER_MARK
> +
> +   +------------------+----------------------+
> +   | Field            | Value                |
> +   +==================+======================+
> +   | ``profile``      | Meter profile object |
> +   +------------------+----------------------+
> +   | ``policy``       | Meter policy object  |
> +   +------------------+----------------------+
> +
>  Negative types
>  ~~~~~~~~~~~~~~
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 6d030bead5..fca7d07e4b 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -74,6 +74,7 @@ New Features
>    * Added METER_COLOR item to match Color Marker set by a Meter.
>    * Added ability to set Color Marker via modify_field Flow API.
>    * Added Meter API to get a pointer to profile/policy by their ID.
> +  * Added METER_MARK action for Metering with lockless profile/policy access.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 9754f6630a..bb16108f16 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -2906,6 +2906,15 @@ enum rte_flow_action_type {
>  	 * @see struct rte_flow_action_ethdev
>  	 */
>  	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
> +
> +	/**
> +	 * Traffic metering and marking (MTR).
> +	 * the entity represented by the given ethdev.
> +	 *
> +	 * @see struct rte_flow_action_meter_mark
> +	 * See file rte_mtr.h for MTR profile object configuration.
> +	 */
> +	RTE_FLOW_ACTION_TYPE_METER_MARK,
>  };
> 
>  /**
> @@ -3775,6 +3784,25 @@ struct rte_flow_action_modify_field {
>  	uint32_t width; /**< Number of bits to use from a source field. */
>  };
> 
> +/**
> + * RTE_FLOW_ACTION_TYPE_METER_MARK
> + *
> + * Traffic metering and marking (MTR).
> + *
> + * Meters an IP packet stream and marks its packets either
> + * green, yellow, or red according to the specified profile.
> + * The policy is optional and may be specified for defining
> + * subsequent actions based on a color asigned by MTR.
> + * Alternatively, the METER_COLOR item may be used for this.
> + */
> +struct rte_flow_action_meter_mark {
> +
> +	/**< Profile config retrieved with rte_mtr_profile_get(). */
> +	struct rte_flow_meter_profile *profile;
> +	/**< Profile config retrieved with rte_mtr_policy_get(). */

I think you mean Profile --> policy.

> +	struct rte_flow_meter_policy *policy;
> +};
> +
>  /* Mbuf dynamic field offset for metadata. */
>  extern int32_t rte_flow_dynf_metadata_offs;
> 
> --
> 2.18.2

Best,
Ori

RE: [PATCH v2 0/4] ethdev: separate metering and marking from policing

[Ori Kam]

Please add testpmd implenetation for this patch set.
It can be part of each patch or just one patch.

Best,
Ori

Re: [PATCH v2 0/4] ethdev: separate metering and marking from policing

[Jerin Jacob]

On Sun, May 22, 2022 at 4:21 PM Alexander Kozyrev <akozyrev@nvidia.com> wrote:
>
> Extend Metering and Marking support in the Flow API:
> 1. Add METER_COLOR item to match Color Marker set by a Meter.
> 2. Add the ability to set Color Marker via modify_field Flow API.
> 3. Add Meter API to get profile/policy objects.
> 4. Add METER_MARK action to perform Meter color metering and marking.
> Provide greater flexibility in how Metering can be used.
>
> RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/
>
> Traditional Meter Usage:
>
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> meter_id = rte_mtr_create(profile_id, policy_id);
> rte_flow_create(pattern=5-tuple,actions=METER(meter_id));
>
> The METER action effectively translates to the following:
> 1. Metering a packet stream.
> 2. Marking packets with an appropriate color.
> 3. Jump to a policy group.
> 4. Match on a color.
> 5. Execute assigned policy actions for the color.
>
> New Meter Usage Model:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> rte_flow_create(pattern=5-tuple,
>                                 actions=METER(profile_obj_ptr),JUMP);
> rte_flow_create(pattern=COLOR, actions=...);
>
> The METER_MARK action effectively translates to the following:
> 1. Metering a packet stream.
> 2. Marking packets with an appropriate color.
>
> A user is able to match the color later with the COLOR item.
> In order to do this we add the JUMP action after the METER action.
>
> 3. Jump to a policy group.
> 4. Match on a color.
> 5. Execute actions for the color.
>
> Here we decoupled the meter profile usage from the meter policy usage
> for greater flexibility and got rid of any locks related to meter_id lookup.
>
> Another example of the meter creation to mimic the old model entirely:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> *policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
> rte_flow_create(pattern=5-tuple,
>                                 actions=METER(profile_obj_ptr, policy_obj_ptr));
>
> In this case, we define the policy actions right away.
> The main advantage is not having to lookup for profile_id/policy_id.
>
> To free the meter obects we need to do the following:
> rte_flow_destroy(flow_handle);
> rte_mtr_meter_policy_delete(policy_id);
> rte_mtr_meter_profile_delete(profile_id);.
> profile_obj_ptr and policy_obj_ptr are no longer valid after that.

The overall approach looks good to me.

Could you update
doc/guides/prog_guide/traffic_metering_and_policing.rst for the new
scheme.
I think, we need to add "struct rte_mtr_capabilities" one more filed
for the capability for the application to
query to pick one way or another or both.

Re: [PATCH v2 0/4] ethdev: separate metering and marking from policing

[Jerin Jacob]

On Thu, May 26, 2022 at 6:51 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
>
> On Sun, May 22, 2022 at 4:21 PM Alexander Kozyrev <akozyrev@nvidia.com> wrote:
> >
> > Extend Metering and Marking support in the Flow API:
> > 1. Add METER_COLOR item to match Color Marker set by a Meter.
> > 2. Add the ability to set Color Marker via modify_field Flow API.
> > 3. Add Meter API to get profile/policy objects.
> > 4. Add METER_MARK action to perform Meter color metering and marking.
> > Provide greater flexibility in how Metering can be used.
> >
> > RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/
> >
> > Traditional Meter Usage:
> >
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> > meter_id = rte_mtr_create(profile_id, policy_id);
> > rte_flow_create(pattern=5-tuple,actions=METER(meter_id));
> >
> > The METER action effectively translates to the following:
> > 1. Metering a packet stream.
> > 2. Marking packets with an appropriate color.
> > 3. Jump to a policy group.
> > 4. Match on a color.
> > 5. Execute assigned policy actions for the color.
> >
> > New Meter Usage Model:
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> > rte_flow_create(pattern=5-tuple,
> >                                 actions=METER(profile_obj_ptr),JUMP);
> > rte_flow_create(pattern=COLOR, actions=...);
> >
> > The METER_MARK action effectively translates to the following:
> > 1. Metering a packet stream.
> > 2. Marking packets with an appropriate color.
> >
> > A user is able to match the color later with the COLOR item.
> > In order to do this we add the JUMP action after the METER action.
> >
> > 3. Jump to a policy group.
> > 4. Match on a color.
> > 5. Execute actions for the color.
> >
> > Here we decoupled the meter profile usage from the meter policy usage
> > for greater flexibility and got rid of any locks related to meter_id lookup.
> >
> > Another example of the meter creation to mimic the old model entirely:
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> > policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> > *policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
> > rte_flow_create(pattern=5-tuple,
> >                                 actions=METER(profile_obj_ptr, policy_obj_ptr));
> >
> > In this case, we define the policy actions right away.
> > The main advantage is not having to lookup for profile_id/policy_id.
> >
> > To free the meter obects we need to do the following:
> > rte_flow_destroy(flow_handle);
> > rte_mtr_meter_policy_delete(policy_id);
> > rte_mtr_meter_profile_delete(profile_id);.
> > profile_obj_ptr and policy_obj_ptr are no longer valid after that.
>
> The overall approach looks good to me.
>
> Could you update
> doc/guides/prog_guide/traffic_metering_and_policing.rst for the new
> scheme.
> I think, we need to add "struct rte_mtr_capabilities" one more filed
> for the capability for the application to
> query to pick one way or another or both.

Also, I think, the driver patches need to submit and merge before rc2 to
avoid the case where there is no single implementation for API.

RE: [PATCH v2 0/4] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

> On Thursday, May 26, 2022 9:23 Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > The overall approach looks good to me.
> >
> > Could you update
> > doc/guides/prog_guide/traffic_metering_and_policing.rst for the new
> > scheme.
There is a documentation update in patch #3.

> > I think, we need to add "struct rte_mtr_capabilities" one more filed
> > for the capability for the application to
> > query to pick one way or another or both.
I believe that returning NULL pointer gives a hint that new approach is not supported.

> Also, I think, the driver patches need to submit and merge before rc2 to
> avoid the case where there is no single implementation for API.
PMD patches will be sent to the mailing list shortly.

RE: [PATCH v2 3/4] ethdev: get meter profile/policy objects

[Alexander Kozyrev]

On Thursday, May 26, 2022 8:28 Ori Kam <orika@nvidia.com> wrote:
> I think you should say also what happens to this pointer or values
> in case of modification of the policy or destroy of the policy.

Will mention this in the commit message in v3.
Pointer is expected to be invalid and cannot be used once policy destruction.

> I think it is important to explain what happens if a profile is changed to
> destroyed.
I thought it is pretty straight-forward.
Pointer is not valid in case an object is gone.
If it is updated then you can continue to use it.

RE: [PATCH v2 4/4] ethdev: add meter color mark flow action

[Alexander Kozyrev]

On Thursday, May 26, 2022 8:34 Ori Kam <orika@nvidia.com> wrote:
> I think required should be mandatory.
Ok.

> > +struct rte_flow_action_meter_mark {
> > +
> > +	/**< Profile config retrieved with rte_mtr_profile_get(). */
> > +	struct rte_flow_meter_profile *profile;
> > +	/**< Profile config retrieved with rte_mtr_policy_get(). */
> 
> I think you mean Profile --> policy.
Right, thanks for catching.

[PATCH v3 0/7] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

Extend Metering and Marking support in the Flow API:
1. Add METER_COLOR item to match Color Marker set by a Meter.
2. Add the ability to set Color Marker via modify_field Flow API.
3. Add Meter API to get profile/policy objects.
4. Add METER_MARK action to perform Meter color metering and marking.
Provide greater flexibility in how Metering can be used.

RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/

Traditional Meter Usage:

profile_id = rte_mtr_meter_profile_add(RFC_params);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
meter_id = rte_mtr_create(profile_id, policy_id);
rte_flow_create(pattern=5-tuple,actions=METER(meter_id));

The METER action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.
3. Jump to a policy group.
4. Match on a color.
5. Execute assigned policy actions for the color.

New Meter Usage Model:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr),JUMP);
rte_flow_create(pattern=COLOR, actions=...);

The METER_MARK action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.

A user is able to match the color later with the COLOR item.
In order to do this we add the JUMP action after the METER action.

3. Jump to a policy group.
4. Match on a color.
5. Execute actions for the color.

Here we decoupled the meter profile usage from the meter policy usage
for greater flexibility and got rid of any locks related to meter_id lookup.

Another example of the meter creation to mimic the old model entirely:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
*policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr, policy_obj_ptr));

In this case, we define the policy actions right away.
The main advantage is not having to lookup for profile_id/policy_id.

To free the meter obects we need to do the following:
rte_flow_destroy(flow_handle);
rte_mtr_meter_policy_delete(policy_id);
rte_mtr_meter_profile_delete(profile_id);.
profile_obj_ptr and policy_obj_ptr are no longer valid after that.

The meter profile configuration cannot be updated dynamically
with the current set of patches, but can be supported later on.
Now you have to destroy flows and profiles and recreate them.
But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
can have the corresponding siblings without mtr_id parameters.
In this case, we can update the config and all the flows using them.

The meter sharing is done via the indirect action Flow API:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_prof8ile_get(profile_id);
handle = rte_flow_action_handle_create(action=METER(profile_obj_ptr, NULL));
flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));

Once we are done with the flow rules we can free everything.
rte_flow_destroy(flow1);
rte_flow_destroy(flow2);
rte_flow_action_handle_destroy(handle);
rte_mtr_meter_profile_delete(profile_id);

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Alexander Kozyrev (7):
  ethdev: add meter color flow matching item
  ethdev: allow meter color marker modification
  ethdev: get meter profile/policy objects
  ethdev: add meter color mark flow action
  app/testpmd: add meter color flow matching item
  app/testpmd: allow meter color marker modification
  app/testpmd: add meter color mark flow action

 app/test-pmd/cmdline_flow.c                   | 213 +++++++++++++++++-
 app/test-pmd/config.c                         |  26 +++
 app/test-pmd/testpmd.h                        |   4 +
 doc/guides/prog_guide/rte_flow.rst            |  32 +++
 .../traffic_metering_and_policing.rst         |   7 +
 doc/guides/rel_notes/release_22_07.rst        |   7 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   9 +
 lib/ethdev/rte_flow.c                         |   2 +
 lib/ethdev/rte_flow.h                         |  60 +++++
 lib/ethdev/rte_mtr.c                          |  41 ++++
 lib/ethdev/rte_mtr.h                          |  40 ++++
 lib/ethdev/rte_mtr_driver.h                   |  19 ++
 lib/ethdev/version.map                        |   2 +
 13 files changed, 461 insertions(+), 1 deletion(-)

-- 
2.18.2

[PATCH v3 1/7] ethdev: add meter color flow matching item

[Alexander Kozyrev]

Provide an ability to use a Color Marker set by a Meter
as a matching item in Flow API. The Color Marker reflects
the metering result by setting the metadata for a
packet to a particular codepoint: green, yellow or red.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
 doc/guides/rel_notes/release_22_07.rst |  4 ++++
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
 4 files changed, 36 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 588914b231..018def1033 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1651,6 +1651,13 @@ Matches a PPP header.
 - ``proto_id``: PPP protocol identifier.
 - Default ``mask`` matches addr, ctrl, proto_id.
 
+Item: ``METER_COLOR``
+^^^^^^^^^^^^^^^^^^^^^
+
+Matches Color Marker set by a Meter.
+
+- ``color``: Metering color marker.
+
 Actions
 ~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 0ed4f92820..c1804bc74f 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -69,6 +69,10 @@ New Features
   Added an API which can get the number of in-flight packets in
   vhost async data path without using lock.
 
+* **Extended Metering and Marking support in the Flow API.**
+
+  * Added METER_COLOR item to match Color Marker set by a Meter.
+
 * **Updated Intel iavf driver.**
 
   * Added Tx QoS queue rate limitation support.
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 2c35a2f13e..07d7e998c0 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -161,6 +161,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
 			rte_flow_item_flex_conv),
 	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
 	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
+	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d8827dd184..75ec6621ac 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -671,6 +671,14 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+
+	/**
+	 * Matches Meter Color Marker.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -2201,6 +2209,22 @@ struct rte_flow_item_flex_conf {
 	uint32_t nb_outputs;
 };
 
+/**
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR.
+ *
+ * Matches Color Marker set by a Meter.
+ */
+struct rte_flow_item_meter_color {
+	enum rte_color color; /**< Meter color marker. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+	.color = RTE_COLORS,
+};
+#endif
+
 /**
  * Action types.
  *
-- 
2.18.2

[PATCH v3 2/7] ethdev: allow meter color marker modification

[Alexander Kozyrev]

Extend modify_field Flow API with support of Meter Color Marker
modifications. It allows setting the packet's metadata to any
color marker: green, yellow or red. A user is able to specify
an initial packet color for Meter API or create simple Metering
and Marking flow rules based on his own coloring algorithm.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_22_07.rst | 1 +
 lib/ethdev/rte_flow.h                  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index c1804bc74f..451ff8d460 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -72,6 +72,7 @@ New Features
 * **Extended Metering and Marking support in the Flow API.**
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
+  * Added ability to set Color Marker via modify_field Flow API.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 75ec6621ac..68af109554 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3714,6 +3714,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_META,		/**< Metadata value. */
 	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
 	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
+	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
 };
 
 /**
-- 
2.18.2

[PATCH v3 3/7] ethdev: get meter profile/policy objects

[Alexander Kozyrev]

Introduce a new Meter API to retrieve a Meter profile and policy
objects using the profile/policy ID previously created with
meter_profile_add() and meter_policy_create() functions.
That allows to save the pointer and avoid any lookups in the
corresponding lists for quick access during a flow rule creation.
Also, it eliminates the need for CIR, CBS and EBS calculations
and conversion to a PMD-specific format when the profile is used.
Pointers are destroyed and cannot be used after thecorresponding
meter_profile_delete() or meter_policy_delete() are called.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 .../traffic_metering_and_policing.rst         |  7 ++++
 doc/guides/rel_notes/release_22_07.rst        |  1 +
 lib/ethdev/rte_flow.h                         |  7 ++++
 lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
 lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
 lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
 lib/ethdev/version.map                        |  2 +
 7 files changed, 117 insertions(+)

diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst
index d1958a023d..2ce3236ad8 100644
--- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
+++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
@@ -107,6 +107,13 @@ traffic meter and policing library.
      to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
      specified per color as show in :numref:`figure_rte_mtr_chaining`.
 
+#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
+   API functions are available for getting the object pointers directly.
+   These pointers allow quick access to profile/policy objects and are
+   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
+   This action may omit the policy definition to providei flexibility
+   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
+
 Protocol based input color selection
 ------------------------------------
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 451ff8d460..6d030bead5 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -73,6 +73,7 @@ New Features
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
+  * Added Meter API to get a pointer to profile/policy by their ID.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 68af109554..9754f6630a 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3827,6 +3827,13 @@ struct rte_flow_action {
  */
 struct rte_flow;
 
+/**
+ * Opaque type for Meter profile object returned by MTR API.
+ *
+ * This handle can be used to create Meter actions instead of profile ID.
+ */
+struct rte_flow_meter_profile;
+
 /**
  * @warning
  * @b EXPERIMENTAL: this structure may change without prior notice
diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
index 441ea1dca9..90fd277040 100644
--- a/lib/ethdev/rte_mtr.c
+++ b/lib/ethdev/rte_mtr.c
@@ -57,6 +57,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
 	ops->func;					\
 })
 
+#define RTE_MTR_HNDL_FUNC(port_id, func)		\
+({							\
+	const struct rte_mtr_ops *ops =			\
+		rte_mtr_ops_get(port_id, error);	\
+	if (ops == NULL)				\
+		return NULL;				\
+							\
+	if (ops->func == NULL) {			\
+		rte_mtr_error_set(error,		\
+			ENOSYS,				\
+			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
+			NULL,				\
+			rte_strerror(ENOSYS));		\
+		return NULL;				\
+	}						\
+							\
+	ops->func;					\
+})
+
 /* MTR capabilities get */
 int
 rte_mtr_capabilities_get(uint16_t port_id,
@@ -91,6 +110,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 		meter_profile_id, error);
 }
 
+/** MTR meter profile get */
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
+		meter_profile_id, error);
+}
+
 /* MTR meter policy validate */
 int
 rte_mtr_meter_policy_validate(uint16_t port_id,
@@ -125,6 +155,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
 		policy_id, error);
 }
 
+/** MTR meter policy get */
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
+		policy_id, error);
+}
+
 /** MTR object create */
 int
 rte_mtr_create(uint16_t port_id,
diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
index 008bc84f0d..58f0d26215 100644
--- a/lib/ethdev/rte_mtr.h
+++ b/lib/ethdev/rte_mtr.h
@@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter profile object get
+ *
+ * Get meter profile object for a given meter profile ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] meter_profile_id
+ *   Meter profile ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /**
  * Check whether a meter policy can be created on a given port.
  *
@@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
 	struct rte_mtr_meter_policy_params *policy,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter policy object get
+ *
+ * Get meter policy object for a given meter policy ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] policy_id
+ *   Meter policy ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
 /**
  * Define meter policy action list:
  * GREEN - GREEN, YELLOW - YELLOW, RED - RED
diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
index f7dca9a54c..948a629b93 100644
--- a/lib/ethdev/rte_mtr_driver.h
+++ b/lib/ethdev/rte_mtr_driver.h
@@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter profile get. */
+typedef struct rte_flow_meter_profile *
+(*rte_mtr_meter_profile_get_t)(struct rte_eth_dev *dev,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /** @internal MTR meter policy validate. */
 typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
 	struct rte_mtr_meter_policy_params *policy,
@@ -57,6 +63,13 @@ typedef int (*rte_mtr_meter_policy_delete_t)(struct rte_eth_dev *dev,
 	uint32_t policy_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter policy get. */
+typedef struct rte_flow_meter_policy *
+(*rte_mtr_meter_policy_get_t)(struct rte_eth_dev *dev,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
+
 /** @internal MTR object create. */
 typedef int (*rte_mtr_create_t)(struct rte_eth_dev *dev,
 	uint32_t mtr_id,
@@ -194,6 +207,12 @@ struct rte_mtr_ops {
 
 	/** MTR object meter policy update */
 	rte_mtr_meter_policy_update_t meter_policy_update;
+
+	/** MTR meter profile get */
+	rte_mtr_meter_profile_get_t meter_profile_get;
+
+	/** MTR meter policy get */
+	rte_mtr_meter_policy_get_t meter_policy_get;
 };
 
 /**
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index daca7851f2..2609f2e709 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -285,6 +285,8 @@ EXPERIMENTAL {
 	rte_mtr_color_in_protocol_priority_get;
 	rte_mtr_color_in_protocol_set;
 	rte_mtr_meter_vlan_table_update;
+	rte_mtr_meter_profile_get;
+	rte_mtr_meter_policy_get;
 };
 
 INTERNAL {
-- 
2.18.2

[PATCH v3 4/7] ethdev: add meter color mark flow action

[Alexander Kozyrev]

Create a new Flow API action: METER_MARK.
It Meters an IP packet stream and marks its packets with colors.
Unlike the METER action, it performs no policing at all.
A user has the flexibility to create any policies with the help of
the METER_COLOR item later, only meter profile is mandatory here.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     | 25 +++++++++++++++++++++++
 doc/guides/rel_notes/release_22_07.rst |  1 +
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 28 ++++++++++++++++++++++++++
 4 files changed, 55 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 018def1033..90f0ebc9bc 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3411,6 +3411,31 @@ This action is meant to use the same structure as `Action: PORT_REPRESENTOR`_.
 
 See also `Item: REPRESENTED_PORT`_.
 
+Action: ``METER_MARK``
+^^^^^^^^^^^^^^^^^^^^^^
+
+Meters an IP packet stream and marks its packets with colors.
+
+Unlike the ``METER`` action, policing is optional and may be
+performed later with the help of the ``METER_COLOR`` item.
+The profile and/or policy objects have to be created
+using the rte_mtr_profile_add()/rte_mtr_policy_add() API.
+Pointers to these objects are used as action parameters
+and need to be retrieved using the rte_mtr_profile_get() API
+and rte_mtr_policy_get() API respectively.
+
+.. _table_rte_flow_action_meter_mark:
+
+.. table:: METER_MARK
+
+   +------------------+----------------------+
+   | Field            | Value                |
+   +==================+======================+
+   | ``profile``      | Meter profile object |
+   +------------------+----------------------+
+   | ``policy``       | Meter policy object  |
+   +------------------+----------------------+
+
 Negative types
 ~~~~~~~~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
index 6d030bead5..fca7d07e4b 100644
--- a/doc/guides/rel_notes/release_22_07.rst
+++ b/doc/guides/rel_notes/release_22_07.rst
@@ -74,6 +74,7 @@ New Features
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
   * Added Meter API to get a pointer to profile/policy by their ID.
+  * Added METER_MARK action for Metering with lockless profile/policy access.
 
 * **Updated Intel iavf driver.**
 
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 07d7e998c0..9049d3bad3 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -262,6 +262,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = {
 	MK_FLOW_ACTION(CONNTRACK, sizeof(struct rte_flow_action_conntrack)),
 	MK_FLOW_ACTION(PORT_REPRESENTOR, sizeof(struct rte_flow_action_ethdev)),
 	MK_FLOW_ACTION(REPRESENTED_PORT, sizeof(struct rte_flow_action_ethdev)),
+	MK_FLOW_ACTION(METER_MARK, sizeof(struct rte_flow_action_meter_mark)),
 };
 
 int
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 9754f6630a..2dc80ad9ef 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -2906,6 +2906,15 @@ enum rte_flow_action_type {
 	 * @see struct rte_flow_action_ethdev
 	 */
 	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+
+	/**
+	 * Traffic metering and marking (MTR).
+	 * the entity represented by the given ethdev.
+	 *
+	 * @see struct rte_flow_action_meter_mark
+	 * See file rte_mtr.h for MTR profile object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_MARK,
 };
 
 /**
@@ -3775,6 +3784,25 @@ struct rte_flow_action_modify_field {
 	uint32_t width; /**< Number of bits to use from a source field. */
 };
 
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_MARK
+ *
+ * Traffic metering and marking (MTR).
+ *
+ * Meters an IP packet stream and marks its packets either
+ * green, yellow, or red according to the specified profile.
+ * The policy is optional and may be specified for defining
+ * subsequent actions based on a color assigned by MTR.
+ * Alternatively, the METER_COLOR item may be used for this.
+ */
+struct rte_flow_action_meter_mark {
+
+	/**< Profile config retrieved with rte_mtr_profile_get(). */
+	struct rte_flow_meter_profile *profile;
+	/**< Policy config retrieved with rte_mtr_policy_get(). */
+	struct rte_flow_meter_policy *policy;
+};
+
 /* Mbuf dynamic field offset for metadata. */
 extern int32_t rte_flow_dynf_metadata_offs;
 
-- 
2.18.2

[PATCH v3 5/7] app/testpmd: add meter color flow matching item

[Alexander Kozyrev]

Add testpmd command line to match on a meter color:
	flow create 0 ingress group 0 pattern meter color is green / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c                 | 83 +++++++++++++++++++++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  4 +
 2 files changed, 87 insertions(+)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index e3269e278d..c66da76f25 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -458,6 +458,9 @@ enum index {
 	ITEM_PPP_ADDR,
 	ITEM_PPP_CTRL,
 	ITEM_PPP_PROTO_ID,
+	ITEM_METER,
+	ITEM_METER_COLOR,
+	ITEM_METER_COLOR_NAME,
 
 	/* Validate/create actions. */
 	ACTIONS,
@@ -798,6 +801,10 @@ static const char *const modify_field_ids[] = {
 	"tag", "mark", "meta", "pointer", "value", NULL
 };
 
+static const char *const meter_colors[] = {
+	"green", "yellow", "red", "all", NULL
+};
+
 /** Maximum number of subsequent tokens and arguments on the stack. */
 #define CTX_STACK_SIZE 16
 
@@ -1331,6 +1338,7 @@ static const enum index next_item[] = {
 	ITEM_FLEX,
 	ITEM_L2TPV2,
 	ITEM_PPP,
+	ITEM_METER,
 	END_SET,
 	ZERO,
 };
@@ -1803,6 +1811,12 @@ static const enum index item_ppp[] = {
 	ZERO,
 };
 
+static const enum index item_meter[] = {
+	ITEM_METER_COLOR,
+	ITEM_NEXT,
+	ZERO,
+};
+
 static const enum index next_action[] = {
 	ACTION_END,
 	ACTION_VOID,
@@ -2371,6 +2385,9 @@ static int parse_ia_id2ptr(struct context *ctx, const struct token *token,
 static int parse_mp(struct context *, const struct token *,
 		    const char *, unsigned int,
 		    void *, unsigned int);
+static int parse_meter_color(struct context *ctx, const struct token *token,
+			     const char *str, unsigned int len, void *buf,
+			     unsigned int size);
 static int comp_none(struct context *, const struct token *,
 		     unsigned int, char *, unsigned int);
 static int comp_boolean(struct context *, const struct token *,
@@ -2401,6 +2418,8 @@ static int comp_table_id(struct context *, const struct token *,
 			 unsigned int, char *, unsigned int);
 static int comp_queue_id(struct context *, const struct token *,
 			 unsigned int, char *, unsigned int);
+static int comp_meter_color(struct context *, const struct token *,
+			    unsigned int, char *, unsigned int);
 
 /** Token definitions. */
 static const struct token token_list[] = {
@@ -5063,6 +5082,29 @@ static const struct token token_list[] = {
 		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_ppp,
 					hdr.proto_id)),
 	},
+	[ITEM_METER] = {
+		.name = "meter",
+		.help = "match meter color",
+		.priv = PRIV_ITEM(METER_COLOR,
+				  sizeof(struct rte_flow_item_meter_color)),
+		.next = NEXT(item_meter),
+		.call = parse_vc,
+	},
+	[ITEM_METER_COLOR] = {
+		.name = "color",
+		.help = "meter color",
+		.next = NEXT(item_meter,
+			     NEXT_ENTRY(ITEM_METER_COLOR_NAME),
+			     item_param),
+		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_meter_color,
+					color)),
+	},
+	[ITEM_METER_COLOR_NAME] = {
+		.name = "color_name",
+		.help = "meter color name",
+		.call = parse_meter_color,
+		.comp = comp_meter_color,
+	},
 	/* Validate/create actions. */
 	[ACTIONS] = {
 		.name = "actions",
@@ -9866,6 +9908,30 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	return ret;
 }
 
+/** Parse Meter color name */
+static int
+parse_meter_color(struct context *ctx, const struct token *token,
+		  const char *str, unsigned int len, void *buf,
+		  unsigned int size)
+{
+	struct rte_flow_item_meter_color *meter_color;
+	unsigned int i;
+
+	(void)token;
+	(void)buf;
+	(void)size;
+	for (i = 0; meter_colors[i]; ++i)
+		if (!strcmp_partial(meter_colors[i], str, len))
+			break;
+	if (!meter_colors[i])
+		return -1;
+	if (!ctx->object)
+		return len;
+	meter_color = ctx->object;
+	meter_color->color = (enum rte_color)i;
+	return len;
+}
+
 /** No completion. */
 static int
 comp_none(struct context *ctx, const struct token *token,
@@ -10157,6 +10223,20 @@ comp_queue_id(struct context *ctx, const struct token *token,
 	return i;
 }
 
+/** Complete available Meter colors. */
+static int
+comp_meter_color(struct context *ctx, const struct token *token,
+		 unsigned int ent, char *buf, unsigned int size)
+{
+	RTE_SET_USED(ctx);
+	RTE_SET_USED(token);
+	if (!buf)
+		return RTE_DIM(meter_colors);
+	if (ent < RTE_DIM(meter_colors) - 1)
+		return strlcpy(buf, meter_colors[ent], size);
+	return -1;
+}
+
 /** Internal context. */
 static struct context cmd_flow_context;
 
@@ -10772,6 +10852,9 @@ flow_item_default_mask(const struct rte_flow_item *item)
 	case RTE_FLOW_ITEM_TYPE_PPP:
 		mask = &rte_flow_item_ppp_mask;
 		break;
+	case RTE_FLOW_ITEM_TYPE_METER_COLOR:
+		mask = &rte_flow_item_meter_color_mask;
+		break;
 	default:
 		break;
 	}
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 1083c6d538..5e2f5ffb40 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -4164,6 +4164,10 @@ This section lists supported pattern items and their attributes, if any.
   - ``ctrl {unsigned}``: PPP control.
   - ``proto_id {unsigned}``: PPP protocol identifier.
 
+- ``meter``: match Meter color.
+
+  - ``color {value}``: Meter color value(green/yellow/red).
+
 Actions list
 ^^^^^^^^^^^^
 
-- 
2.18.2

[PATCH v3 6/7] app/testpmd: allow meter color marker modification

[Alexander Kozyrev]

Extend the list of available modify_field IDs to include
recently added meter color marker item in testpmd CLI.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index c66da76f25..401922fc34 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -798,7 +798,8 @@ static const char *const modify_field_ids[] = {
 	"tcp_seq_num", "tcp_ack_num", "tcp_flags",
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
-	"tag", "mark", "meta", "pointer", "value", NULL
+	"tag", "mark", "meta", "pointer", "value", "meter_color",
+	NULL
 };
 
 static const char *const meter_colors[] = {
-- 
2.18.2

[PATCH v3 7/7] app/testpmd: add meter color mark flow action

[Alexander Kozyrev]

Add testpmd command line to match for METER_MARK action:
	flow create ... actions meter_mark mtr_profile 20 / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c                 | 127 ++++++++++++++++++++
 app/test-pmd/config.c                       |  26 ++++
 app/test-pmd/testpmd.h                      |   4 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +
 4 files changed, 162 insertions(+)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 401922fc34..7261d6f52e 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -53,6 +53,7 @@ enum index {
 	COMMON_GROUP_ID,
 	COMMON_PRIORITY_LEVEL,
 	COMMON_INDIRECT_ACTION_ID,
+	COMMON_PROFILE_ID,
 	COMMON_POLICY_ID,
 	COMMON_FLEX_HANDLE,
 	COMMON_FLEX_TOKEN,
@@ -508,6 +509,11 @@ enum index {
 	ACTION_METER_COLOR_YELLOW,
 	ACTION_METER_COLOR_RED,
 	ACTION_METER_ID,
+	ACTION_METER_MARK,
+	ACTION_METER_PROFILE,
+	ACTION_METER_PROFILE_ID2PTR,
+	ACTION_METER_POLICY,
+	ACTION_METER_POLICY_ID2PTR,
 	ACTION_OF_SET_MPLS_TTL,
 	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
 	ACTION_OF_DEC_MPLS_TTL,
@@ -1835,6 +1841,7 @@ static const enum index next_action[] = {
 	ACTION_PORT_ID,
 	ACTION_METER,
 	ACTION_METER_COLOR,
+	ACTION_METER_MARK,
 	ACTION_OF_SET_MPLS_TTL,
 	ACTION_OF_DEC_MPLS_TTL,
 	ACTION_OF_SET_NW_TTL,
@@ -1951,6 +1958,13 @@ static const enum index action_meter_color[] = {
 	ZERO,
 };
 
+static const enum index action_meter_mark[] = {
+	ACTION_METER_PROFILE,
+	ACTION_METER_POLICY,
+	ACTION_NEXT,
+	ZERO,
+};
+
 static const enum index action_of_set_mpls_ttl[] = {
 	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
 	ACTION_NEXT,
@@ -2389,6 +2403,14 @@ static int parse_mp(struct context *, const struct token *,
 static int parse_meter_color(struct context *ctx, const struct token *token,
 			     const char *str, unsigned int len, void *buf,
 			     unsigned int size);
+static int parse_meter_profile_id2ptr(struct context *ctx,
+				      const struct token *token,
+				      const char *str, unsigned int len,
+				      void *buf, unsigned int size);
+static int parse_meter_policy_id2ptr(struct context *ctx,
+				     const struct token *token,
+				     const char *str, unsigned int len,
+				     void *buf, unsigned int size);
 static int comp_none(struct context *, const struct token *,
 		     unsigned int, char *, unsigned int);
 static int comp_boolean(struct context *, const struct token *,
@@ -2550,6 +2572,13 @@ static const struct token token_list[] = {
 		.call = parse_int,
 		.comp = comp_none,
 	},
+	[COMMON_PROFILE_ID] = {
+		.name = "{profile_id}",
+		.type = "PROFILE_ID",
+		.help = "profile id",
+		.call = parse_int,
+		.comp = comp_none,
+	},
 	[COMMON_POLICY_ID] = {
 		.name = "{policy_id}",
 		.type = "POLICY_ID",
@@ -5428,6 +5457,42 @@ static const struct token token_list[] = {
 		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter, mtr_id)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_METER_MARK] = {
+		.name = "meter_mark",
+		.help = "meter the directed packets using profile and policy",
+		.priv = PRIV_ACTION(METER_MARK,
+				    sizeof(struct rte_flow_action_meter_mark)),
+		.next = NEXT(action_meter_mark),
+		.call = parse_vc,
+	},
+	[ACTION_METER_PROFILE] = {
+		.name = "mtr_profile",
+		.help = "meter profile id to use",
+		.next = NEXT(NEXT_ENTRY(ACTION_METER_PROFILE_ID2PTR)),
+		.args = ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
+	},
+	[ACTION_METER_PROFILE_ID2PTR] = {
+		.name = "{mtr_profile_id}",
+		.type = "PROFILE_ID",
+		.help = "meter profile id",
+		.next = NEXT(action_meter_mark),
+		.call = parse_meter_profile_id2ptr,
+		.comp = comp_none,
+	},
+	[ACTION_METER_POLICY] = {
+		.name = "mtr_policy",
+		.help = "meter policy id to use",
+		.next = NEXT(NEXT_ENTRY(ACTION_METER_POLICY_ID2PTR)),
+		ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
+	},
+	[ACTION_METER_POLICY_ID2PTR] = {
+		.name = "{mtr_policy_id}",
+		.type = "POLICY_ID",
+		.help = "meter policy id",
+		.next = NEXT(action_meter_mark),
+		.call = parse_meter_policy_id2ptr,
+		.comp = comp_none,
+	},
 	[ACTION_OF_SET_MPLS_TTL] = {
 		.name = "of_set_mpls_ttl",
 		.help = "OpenFlow's OFPAT_SET_MPLS_TTL",
@@ -9747,6 +9812,68 @@ parse_ia_id2ptr(struct context *ctx, const struct token *token,
 	return ret;
 }
 
+static int
+parse_meter_profile_id2ptr(struct context *ctx, const struct token *token,
+			   const char *str, unsigned int len,
+			   void *buf, unsigned int size)
+{
+	struct rte_flow_action *action = ctx->object;
+	struct rte_flow_action_meter_mark *meter;
+	struct rte_flow_meter_profile *profile = NULL;
+	uint32_t id;
+	int ret;
+
+	(void)buf;
+	(void)size;
+	ctx->objdata = 0;
+	ctx->object = &id;
+	ctx->objmask = NULL;
+	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
+	ctx->object = action;
+	if (ret != (int)len)
+		return ret;
+	/* set meter profile */
+	if (action) {
+		meter = (struct rte_flow_action_meter_mark *)
+				(uintptr_t)(action->conf);
+		profile = port_meter_profile_get_by_id(ctx->port, id);
+		meter->profile = profile;
+		ret = (profile) ? ret : -1;
+	}
+	return ret;
+}
+
+static int
+parse_meter_policy_id2ptr(struct context *ctx, const struct token *token,
+			  const char *str, unsigned int len,
+			  void *buf, unsigned int size)
+{
+	struct rte_flow_action *action = ctx->object;
+	struct rte_flow_action_meter_mark *meter;
+	struct rte_flow_meter_policy *policy = NULL;
+	uint32_t id;
+	int ret;
+
+	(void)buf;
+	(void)size;
+	ctx->objdata = 0;
+	ctx->object = &id;
+	ctx->objmask = NULL;
+	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
+	ctx->object = action;
+	if (ret != (int)len)
+		return ret;
+	/* set meter policy */
+	if (action) {
+		meter = (struct rte_flow_action_meter_mark *)
+				(uintptr_t)(action->conf);
+		policy = port_meter_policy_get_by_id(ctx->port, id);
+		meter->policy = policy;
+		ret = (policy) ? ret : -1;
+	}
+	return ret;
+}
+
 /** Parse set command, initialize output buffer for subsequent tokens. */
 static int
 parse_set_raw_encap_decap(struct context *ctx, const struct token *token,
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 1b1e738f83..413dcb6e9d 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -2184,6 +2184,32 @@ port_meter_policy_add(portid_t port_id, uint32_t policy_id,
 	return ret;
 }
 
+/** Get port meter profile */
+struct rte_flow_meter_profile *
+port_meter_profile_get_by_id(portid_t port_id, uint32_t id)
+{
+	struct rte_mtr_error error;
+	struct rte_flow_meter_profile *profile;
+
+	profile = rte_mtr_meter_profile_get(port_id, id, &error);
+	if (!profile)
+		print_mtr_err_msg(&error);
+	return profile;
+}
+
+/** Get port meter policy */
+struct rte_flow_meter_policy *
+port_meter_policy_get_by_id(portid_t port_id, uint32_t id)
+{
+	struct rte_mtr_error error;
+	struct rte_flow_meter_policy *policy;
+
+	policy = rte_mtr_meter_policy_get(port_id, id, &error);
+	if (!policy)
+		print_mtr_err_msg(&error);
+	return policy;
+}
+
 /** Validate flow rule. */
 int
 port_flow_validate(portid_t port_id,
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index f04a9a11b4..5fed46ae17 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -985,6 +985,10 @@ void port_flow_tunnel_create(portid_t port_id, const struct tunnel_ops *ops);
 int port_flow_isolate(portid_t port_id, int set);
 int port_meter_policy_add(portid_t port_id, uint32_t policy_id,
 		const struct rte_flow_action *actions);
+struct rte_flow_meter_profile *port_meter_profile_get_by_id(portid_t port_id,
+							    uint32_t id);
+struct rte_flow_meter_policy *port_meter_policy_get_by_id(portid_t port_id,
+							  uint32_t id);
 
 void rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id);
 void tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id);
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 5e2f5ffb40..fd5a05dbad 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -4453,6 +4453,11 @@ This section lists supported actions and their attributes, if any.
 
   - ``ethdev_port_id {unsigned}``: ethdev port ID
 
+- ``meter_mark``:  meter the directed packets using profile and policy
+
+  - ``mtr_profile {unsigned}``: meter profile id to use
+  - ``mtr_policy {unsigned}``: meter policy id to use
+
 Destroying flow rules
 ~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.18.2

RE: [PATCH v3 1/7] ethdev: add meter color flow matching item

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> To: dev@dpdk.org
> Subject: [PATCH v3 1/7] ethdev: add meter color flow matching item
> 
> Provide an ability to use a Color Marker set by a Meter
> as a matching item in Flow API. The Color Marker reflects
> the metering result by setting the metadata for a
> packet to a particular codepoint: green, yellow or red.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
>  doc/guides/rel_notes/release_22_07.rst |  4 ++++
>  lib/ethdev/rte_flow.c                  |  1 +
>  lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
>  4 files changed, 36 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index 588914b231..018def1033 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1651,6 +1651,13 @@ Matches a PPP header.
>  - ``proto_id``: PPP protocol identifier.
>  - Default ``mask`` matches addr, ctrl, proto_id.
> 
> +Item: ``METER_COLOR``
> +^^^^^^^^^^^^^^^^^^^^^
> +
> +Matches Color Marker set by a Meter.
> +
> +- ``color``: Metering color marker.
> +
>  Actions
>  ~~~~~~~
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 0ed4f92820..c1804bc74f 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -69,6 +69,10 @@ New Features
>    Added an API which can get the number of in-flight packets in
>    vhost async data path without using lock.
> 
> +* **Extended Metering and Marking support in the Flow API.**
> +
> +  * Added METER_COLOR item to match Color Marker set by a Meter.
> +
>  * **Updated Intel iavf driver.**
> 
>    * Added Tx QoS queue rate limitation support.
> diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
> index 2c35a2f13e..07d7e998c0 100644
> --- a/lib/ethdev/rte_flow.c
> +++ b/lib/ethdev/rte_flow.c
> @@ -161,6 +161,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
>  			rte_flow_item_flex_conv),
>  	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
>  	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
> +	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
>  };
> 
>  /** Generate flow_action[] entry. */
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index d8827dd184..75ec6621ac 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -671,6 +671,14 @@ enum rte_flow_item_type {
>  	 * See struct rte_flow_item_gre_opt.
>  	 */
>  	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
> +
> +	/**
> +	 * Matches Meter Color Marker.
> +	 *
> +	 * See struct rte_flow_item_meter_color.
> +	 */
> +
> +	RTE_FLOW_ITEM_TYPE_METER_COLOR,
>  };
> 
>  /**
> @@ -2201,6 +2209,22 @@ struct rte_flow_item_flex_conf {
>  	uint32_t nb_outputs;
>  };
> 
> +/**
> + * RTE_FLOW_ITEM_TYPE_METER_COLOR.
> + *
> + * Matches Color Marker set by a Meter.
> + */
> +struct rte_flow_item_meter_color {
> +	enum rte_color color; /**< Meter color marker. */
> +};
> +
> +/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
> +#ifndef __cplusplus
> +static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
> +	.color = RTE_COLORS,
> +};
> +#endif
> +
>  /**
>   * Action types.
>   *
> --
> 2.18.2

Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 2/7] ethdev: allow meter color marker modification

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> To: dev@dpdk.org
> Subject: [PATCH v3 2/7] ethdev: allow meter color marker modification
> 
> Extend modify_field Flow API with support of Meter Color Marker
> modifications. It allows setting the packet's metadata to any
> color marker: green, yellow or red. A user is able to specify
> an initial packet color for Meter API or create simple Metering
> and Marking flow rules based on his own coloring algorithm.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/rel_notes/release_22_07.rst | 1 +
>  lib/ethdev/rte_flow.h                  | 1 +
>  2 files changed, 2 insertions(+)
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index c1804bc74f..451ff8d460 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -72,6 +72,7 @@ New Features
>  * **Extended Metering and Marking support in the Flow API.**
> 
>    * Added METER_COLOR item to match Color Marker set by a Meter.
> +  * Added ability to set Color Marker via modify_field Flow API.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 75ec6621ac..68af109554 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3714,6 +3714,7 @@ enum rte_flow_field_id {
>  	RTE_FLOW_FIELD_META,		/**< Metadata value. */
>  	RTE_FLOW_FIELD_POINTER,		/**< Memory pointer. */
>  	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
> +	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
>  };
> 
>  /**
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 3/7] ethdev: get meter profile/policy objects

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> Subject: [PATCH v3 3/7] ethdev: get meter profile/policy objects
> 
> Introduce a new Meter API to retrieve a Meter profile and policy
> objects using the profile/policy ID previously created with
> meter_profile_add() and meter_policy_create() functions.
> That allows to save the pointer and avoid any lookups in the
> corresponding lists for quick access during a flow rule creation.
> Also, it eliminates the need for CIR, CBS and EBS calculations
> and conversion to a PMD-specific format when the profile is used.
> Pointers are destroyed and cannot be used after thecorresponding
> meter_profile_delete() or meter_policy_delete() are called.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  .../traffic_metering_and_policing.rst         |  7 ++++
>  doc/guides/rel_notes/release_22_07.rst        |  1 +
>  lib/ethdev/rte_flow.h                         |  7 ++++
>  lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
>  lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
>  lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
>  lib/ethdev/version.map                        |  2 +
>  7 files changed, 117 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst
> b/doc/guides/prog_guide/traffic_metering_and_policing.rst
> index d1958a023d..2ce3236ad8 100644
> --- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
> +++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
> @@ -107,6 +107,13 @@ traffic meter and policing library.
>       to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
>       specified per color as show in :numref:`figure_rte_mtr_chaining`.
> 
> +#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
> +   API functions are available for getting the object pointers directly.
> +   These pointers allow quick access to profile/policy objects and are
> +   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
> +   This action may omit the policy definition to providei flexibility
> +   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
> +
>  Protocol based input color selection
>  ------------------------------------
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 451ff8d460..6d030bead5 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -73,6 +73,7 @@ New Features
> 
>    * Added METER_COLOR item to match Color Marker set by a Meter.
>    * Added ability to set Color Marker via modify_field Flow API.
> +  * Added Meter API to get a pointer to profile/policy by their ID.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 68af109554..9754f6630a 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3827,6 +3827,13 @@ struct rte_flow_action {
>   */
>  struct rte_flow;
> 
> +/**
> + * Opaque type for Meter profile object returned by MTR API.
> + *
> + * This handle can be used to create Meter actions instead of profile ID.
> + */
> +struct rte_flow_meter_profile;
> +
>  /**
>   * @warning
>   * @b EXPERIMENTAL: this structure may change without prior notice
> diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
> index 441ea1dca9..90fd277040 100644
> --- a/lib/ethdev/rte_mtr.c
> +++ b/lib/ethdev/rte_mtr.c
> @@ -57,6 +57,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
>  	ops->func;					\
>  })
> 
> +#define RTE_MTR_HNDL_FUNC(port_id, func)		\
> +({							\
> +	const struct rte_mtr_ops *ops =			\
> +		rte_mtr_ops_get(port_id, error);	\
> +	if (ops == NULL)				\
> +		return NULL;				\
> +							\
> +	if (ops->func == NULL) {			\
> +		rte_mtr_error_set(error,		\
> +			ENOSYS,				\
> +			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
> +			NULL,				\
> +			rte_strerror(ENOSYS));		\
> +		return NULL;				\
> +	}						\
> +							\
> +	ops->func;					\
> +})
> +
>  /* MTR capabilities get */
>  int
>  rte_mtr_capabilities_get(uint16_t port_id,
> @@ -91,6 +110,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>  		meter_profile_id, error);
>  }
> 
> +/** MTR meter profile get */
> +struct rte_flow_meter_profile *
> +rte_mtr_meter_profile_get(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error)
> +{
> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
> +		meter_profile_id, error);
> +}
> +
>  /* MTR meter policy validate */
>  int
>  rte_mtr_meter_policy_validate(uint16_t port_id,
> @@ -125,6 +155,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
>  		policy_id, error);
>  }
> 
> +/** MTR meter policy get */
> +struct rte_flow_meter_policy *
> +rte_mtr_meter_policy_get(uint16_t port_id,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error)
> +{
> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
> +		policy_id, error);
> +}
> +
>  /** MTR object create */
>  int
>  rte_mtr_create(uint16_t port_id,
> diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
> index 008bc84f0d..58f0d26215 100644
> --- a/lib/ethdev/rte_mtr.h
> +++ b/lib/ethdev/rte_mtr.h
> @@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>  	uint32_t meter_profile_id,
>  	struct rte_mtr_error *error);
> 
> +/**
> + * Meter profile object get
> + *
> + * Get meter profile object for a given meter profile ID.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] meter_profile_id
> + *   Meter profile ID. Needs to be the valid.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +__rte_experimental
> +struct rte_flow_meter_profile *
> +rte_mtr_meter_profile_get(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error);
> +
>  /**
>   * Check whether a meter policy can be created on a given port.
>   *
> @@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
>  	struct rte_mtr_meter_policy_params *policy,
>  	struct rte_mtr_error *error);
> 
> +/**
> + * Meter policy object get
> + *
> + * Get meter policy object for a given meter policy ID.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] policy_id
> + *   Meter policy ID. Needs to be the valid.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +__rte_experimental
> +struct rte_flow_meter_policy *
> +rte_mtr_meter_policy_get(uint16_t port_id,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error);
> +
>  /**
>   * Define meter policy action list:
>   * GREEN - GREEN, YELLOW - YELLOW, RED - RED
> diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
> index f7dca9a54c..948a629b93 100644
> --- a/lib/ethdev/rte_mtr_driver.h
> +++ b/lib/ethdev/rte_mtr_driver.h
> @@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
>  	uint32_t meter_profile_id,
>  	struct rte_mtr_error *error);
> 
> +/** @internal MTR meter profile get. */
> +typedef struct rte_flow_meter_profile *
> +(*rte_mtr_meter_profile_get_t)(struct rte_eth_dev *dev,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error);
> +
>  /** @internal MTR meter policy validate. */
>  typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
>  	struct rte_mtr_meter_policy_params *policy,
> @@ -57,6 +63,13 @@ typedef int (*rte_mtr_meter_policy_delete_t)(struct rte_eth_dev *dev,
>  	uint32_t policy_id,
>  	struct rte_mtr_error *error);
> 
> +/** @internal MTR meter policy get. */
> +typedef struct rte_flow_meter_policy *
> +(*rte_mtr_meter_policy_get_t)(struct rte_eth_dev *dev,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error);
> +
> +
>  /** @internal MTR object create. */
>  typedef int (*rte_mtr_create_t)(struct rte_eth_dev *dev,
>  	uint32_t mtr_id,
> @@ -194,6 +207,12 @@ struct rte_mtr_ops {
> 
>  	/** MTR object meter policy update */
>  	rte_mtr_meter_policy_update_t meter_policy_update;
> +
> +	/** MTR meter profile get */
> +	rte_mtr_meter_profile_get_t meter_profile_get;
> +
> +	/** MTR meter policy get */
> +	rte_mtr_meter_policy_get_t meter_policy_get;
>  };
> 
>  /**
> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index daca7851f2..2609f2e709 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -285,6 +285,8 @@ EXPERIMENTAL {
>  	rte_mtr_color_in_protocol_priority_get;
>  	rte_mtr_color_in_protocol_set;
>  	rte_mtr_meter_vlan_table_update;
> +	rte_mtr_meter_profile_get;
> +	rte_mtr_meter_policy_get;
>  };
> 
>  INTERNAL {
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 4/7] ethdev: add meter color mark flow action

[Ori Kam]

Hi Alexander,

Small comment, maybe worth saying that the marking is done on a metadata and not on a packet field.

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> To: dev@dpdk.org
> Cc: cristian.dumitrescu@intel.com; jerinjacobk@gmail.com; Ori Kam <orika@nvidia.com>; NBU-
> Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; ivan.malov@oktetlabs.ru;
> andrew.rybchenko@oktetlabs.ru; ferruh.yigit@xilinx.com; mohammad.abdul.awal@intel.com;
> qi.z.zhang@intel.com; jerinj@marvell.com; ajit.khaparde@broadcom.com;
> bruce.richardson@intel.com
> Subject: [PATCH v3 4/7] ethdev: add meter color mark flow action
> 
> Create a new Flow API action: METER_MARK.
> It Meters an IP packet stream and marks its packets with colors.
> Unlike the METER action, it performs no policing at all.
> A user has the flexibility to create any policies with the help of
> the METER_COLOR item later, only meter profile is mandatory here.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst     | 25 +++++++++++++++++++++++
>  doc/guides/rel_notes/release_22_07.rst |  1 +
>  lib/ethdev/rte_flow.c                  |  1 +
>  lib/ethdev/rte_flow.h                  | 28 ++++++++++++++++++++++++++
>  4 files changed, 55 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
> index 018def1033..90f0ebc9bc 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -3411,6 +3411,31 @@ This action is meant to use the same structure as `Action:
> PORT_REPRESENTOR`_.
> 
>  See also `Item: REPRESENTED_PORT`_.
> 
> +Action: ``METER_MARK``
> +^^^^^^^^^^^^^^^^^^^^^^
> +
> +Meters an IP packet stream and marks its packets with colors.
> +
> +Unlike the ``METER`` action, policing is optional and may be
> +performed later with the help of the ``METER_COLOR`` item.
> +The profile and/or policy objects have to be created
> +using the rte_mtr_profile_add()/rte_mtr_policy_add() API.
> +Pointers to these objects are used as action parameters
> +and need to be retrieved using the rte_mtr_profile_get() API
> +and rte_mtr_policy_get() API respectively.
> +
> +.. _table_rte_flow_action_meter_mark:
> +
> +.. table:: METER_MARK
> +
> +   +------------------+----------------------+
> +   | Field            | Value                |
> +   +==================+======================+
> +   | ``profile``      | Meter profile object |
> +   +------------------+----------------------+
> +   | ``policy``       | Meter policy object  |
> +   +------------------+----------------------+
> +
>  Negative types
>  ~~~~~~~~~~~~~~
> 
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 6d030bead5..fca7d07e4b 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -74,6 +74,7 @@ New Features
>    * Added METER_COLOR item to match Color Marker set by a Meter.
>    * Added ability to set Color Marker via modify_field Flow API.
>    * Added Meter API to get a pointer to profile/policy by their ID.
> +  * Added METER_MARK action for Metering with lockless profile/policy access.
> 
>  * **Updated Intel iavf driver.**
> 
> diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
> index 07d7e998c0..9049d3bad3 100644
> --- a/lib/ethdev/rte_flow.c
> +++ b/lib/ethdev/rte_flow.c
> @@ -262,6 +262,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = {
>  	MK_FLOW_ACTION(CONNTRACK, sizeof(struct rte_flow_action_conntrack)),
>  	MK_FLOW_ACTION(PORT_REPRESENTOR, sizeof(struct rte_flow_action_ethdev)),
>  	MK_FLOW_ACTION(REPRESENTED_PORT, sizeof(struct rte_flow_action_ethdev)),
> +	MK_FLOW_ACTION(METER_MARK, sizeof(struct rte_flow_action_meter_mark)),
>  };
> 
>  int
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 9754f6630a..2dc80ad9ef 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -2906,6 +2906,15 @@ enum rte_flow_action_type {
>  	 * @see struct rte_flow_action_ethdev
>  	 */
>  	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
> +
> +	/**
> +	 * Traffic metering and marking (MTR).
> +	 * the entity represented by the given ethdev.
> +	 *
> +	 * @see struct rte_flow_action_meter_mark
> +	 * See file rte_mtr.h for MTR profile object configuration.
> +	 */
> +	RTE_FLOW_ACTION_TYPE_METER_MARK,
>  };
> 
>  /**
> @@ -3775,6 +3784,25 @@ struct rte_flow_action_modify_field {
>  	uint32_t width; /**< Number of bits to use from a source field. */
>  };
> 
> +/**
> + * RTE_FLOW_ACTION_TYPE_METER_MARK
> + *
> + * Traffic metering and marking (MTR).
> + *
> + * Meters an IP packet stream and marks its packets either
> + * green, yellow, or red according to the specified profile.
> + * The policy is optional and may be specified for defining
> + * subsequent actions based on a color assigned by MTR.
> + * Alternatively, the METER_COLOR item may be used for this.
> + */
> +struct rte_flow_action_meter_mark {
> +
> +	/**< Profile config retrieved with rte_mtr_profile_get(). */
> +	struct rte_flow_meter_profile *profile;
> +	/**< Policy config retrieved with rte_mtr_policy_get(). */
> +	struct rte_flow_meter_policy *policy;
> +};
> +
>  /* Mbuf dynamic field offset for metadata. */
>  extern int32_t rte_flow_dynf_metadata_offs;
> 
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 5/7] app/testpmd: add meter color flow matching item

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> Subject: [PATCH v3 5/7] app/testpmd: add meter color flow matching item
> 
> Add testpmd command line to match on a meter color:
> 	flow create 0 ingress group 0 pattern meter color is green / end
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  app/test-pmd/cmdline_flow.c                 | 83 +++++++++++++++++++++
>  doc/guides/testpmd_app_ug/testpmd_funcs.rst |  4 +
>  2 files changed, 87 insertions(+)
> 
> diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
> index e3269e278d..c66da76f25 100644
> --- a/app/test-pmd/cmdline_flow.c
> +++ b/app/test-pmd/cmdline_flow.c
> @@ -458,6 +458,9 @@ enum index {
>  	ITEM_PPP_ADDR,
>  	ITEM_PPP_CTRL,
>  	ITEM_PPP_PROTO_ID,
> +	ITEM_METER,
> +	ITEM_METER_COLOR,
> +	ITEM_METER_COLOR_NAME,
> 
>  	/* Validate/create actions. */
>  	ACTIONS,
> @@ -798,6 +801,10 @@ static const char *const modify_field_ids[] = {
>  	"tag", "mark", "meta", "pointer", "value", NULL
>  };
> 
> +static const char *const meter_colors[] = {
> +	"green", "yellow", "red", "all", NULL
> +};
> +
>  /** Maximum number of subsequent tokens and arguments on the stack. */
>  #define CTX_STACK_SIZE 16
> 
> @@ -1331,6 +1338,7 @@ static const enum index next_item[] = {
>  	ITEM_FLEX,
>  	ITEM_L2TPV2,
>  	ITEM_PPP,
> +	ITEM_METER,
>  	END_SET,
>  	ZERO,
>  };
> @@ -1803,6 +1811,12 @@ static const enum index item_ppp[] = {
>  	ZERO,
>  };
> 
> +static const enum index item_meter[] = {
> +	ITEM_METER_COLOR,
> +	ITEM_NEXT,
> +	ZERO,
> +};
> +
>  static const enum index next_action[] = {
>  	ACTION_END,
>  	ACTION_VOID,
> @@ -2371,6 +2385,9 @@ static int parse_ia_id2ptr(struct context *ctx, const struct token *token,
>  static int parse_mp(struct context *, const struct token *,
>  		    const char *, unsigned int,
>  		    void *, unsigned int);
> +static int parse_meter_color(struct context *ctx, const struct token *token,
> +			     const char *str, unsigned int len, void *buf,
> +			     unsigned int size);
>  static int comp_none(struct context *, const struct token *,
>  		     unsigned int, char *, unsigned int);
>  static int comp_boolean(struct context *, const struct token *,
> @@ -2401,6 +2418,8 @@ static int comp_table_id(struct context *, const struct token *,
>  			 unsigned int, char *, unsigned int);
>  static int comp_queue_id(struct context *, const struct token *,
>  			 unsigned int, char *, unsigned int);
> +static int comp_meter_color(struct context *, const struct token *,
> +			    unsigned int, char *, unsigned int);
> 
>  /** Token definitions. */
>  static const struct token token_list[] = {
> @@ -5063,6 +5082,29 @@ static const struct token token_list[] = {
>  		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_ppp,
>  					hdr.proto_id)),
>  	},
> +	[ITEM_METER] = {
> +		.name = "meter",
> +		.help = "match meter color",
> +		.priv = PRIV_ITEM(METER_COLOR,
> +				  sizeof(struct rte_flow_item_meter_color)),
> +		.next = NEXT(item_meter),
> +		.call = parse_vc,
> +	},
> +	[ITEM_METER_COLOR] = {
> +		.name = "color",
> +		.help = "meter color",
> +		.next = NEXT(item_meter,
> +			     NEXT_ENTRY(ITEM_METER_COLOR_NAME),
> +			     item_param),
> +		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_meter_color,
> +					color)),
> +	},
> +	[ITEM_METER_COLOR_NAME] = {
> +		.name = "color_name",
> +		.help = "meter color name",
> +		.call = parse_meter_color,
> +		.comp = comp_meter_color,
> +	},
>  	/* Validate/create actions. */
>  	[ACTIONS] = {
>  		.name = "actions",
> @@ -9866,6 +9908,30 @@ parse_flex_handle(struct context *ctx, const struct token *token,
>  	return ret;
>  }
> 
> +/** Parse Meter color name */
> +static int
> +parse_meter_color(struct context *ctx, const struct token *token,
> +		  const char *str, unsigned int len, void *buf,
> +		  unsigned int size)
> +{
> +	struct rte_flow_item_meter_color *meter_color;
> +	unsigned int i;
> +
> +	(void)token;
> +	(void)buf;
> +	(void)size;
> +	for (i = 0; meter_colors[i]; ++i)
> +		if (!strcmp_partial(meter_colors[i], str, len))
> +			break;
> +	if (!meter_colors[i])
> +		return -1;
> +	if (!ctx->object)
> +		return len;
> +	meter_color = ctx->object;
> +	meter_color->color = (enum rte_color)i;
> +	return len;
> +}
> +
>  /** No completion. */
>  static int
>  comp_none(struct context *ctx, const struct token *token,
> @@ -10157,6 +10223,20 @@ comp_queue_id(struct context *ctx, const struct token *token,
>  	return i;
>  }
> 
> +/** Complete available Meter colors. */
> +static int
> +comp_meter_color(struct context *ctx, const struct token *token,
> +		 unsigned int ent, char *buf, unsigned int size)
> +{
> +	RTE_SET_USED(ctx);
> +	RTE_SET_USED(token);
> +	if (!buf)
> +		return RTE_DIM(meter_colors);
> +	if (ent < RTE_DIM(meter_colors) - 1)
> +		return strlcpy(buf, meter_colors[ent], size);
> +	return -1;
> +}
> +
>  /** Internal context. */
>  static struct context cmd_flow_context;
> 
> @@ -10772,6 +10852,9 @@ flow_item_default_mask(const struct rte_flow_item *item)
>  	case RTE_FLOW_ITEM_TYPE_PPP:
>  		mask = &rte_flow_item_ppp_mask;
>  		break;
> +	case RTE_FLOW_ITEM_TYPE_METER_COLOR:
> +		mask = &rte_flow_item_meter_color_mask;
> +		break;
>  	default:
>  		break;
>  	}
> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> index 1083c6d538..5e2f5ffb40 100644
> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> @@ -4164,6 +4164,10 @@ This section lists supported pattern items and their attributes, if any.
>    - ``ctrl {unsigned}``: PPP control.
>    - ``proto_id {unsigned}``: PPP protocol identifier.
> 
> +- ``meter``: match Meter color.
> +
> +  - ``color {value}``: Meter color value(green/yellow/red).
> +
>  Actions list
>  ^^^^^^^^^^^^
> 
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 6/7] app/testpmd: allow meter color marker modification

[Ori Kam]

Hi Alexander.

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> Subject: [PATCH v3 6/7] app/testpmd: allow meter color marker modification
> 
> Extend the list of available modify_field IDs to include
> recently added meter color marker item in testpmd CLI.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  app/test-pmd/cmdline_flow.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
> index c66da76f25..401922fc34 100644
> --- a/app/test-pmd/cmdline_flow.c
> +++ b/app/test-pmd/cmdline_flow.c
> @@ -798,7 +798,8 @@ static const char *const modify_field_ids[] = {
>  	"tcp_seq_num", "tcp_ack_num", "tcp_flags",
>  	"udp_port_src", "udp_port_dst",
>  	"vxlan_vni", "geneve_vni", "gtp_teid",
> -	"tag", "mark", "meta", "pointer", "value", NULL
> +	"tag", "mark", "meta", "pointer", "value", "meter_color",
> +	NULL
>  };
> 
>  static const char *const meter_colors[] = {
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 7/7] app/testpmd: add meter color mark flow action

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 6:44 AM
> Subject: [PATCH v3 7/7] app/testpmd: add meter color mark flow action
> 
> Add testpmd command line to match for METER_MARK action:
> 	flow create ... actions meter_mark mtr_profile 20 / end
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>  app/test-pmd/cmdline_flow.c                 | 127 ++++++++++++++++++++
>  app/test-pmd/config.c                       |  26 ++++
>  app/test-pmd/testpmd.h                      |   4 +
>  doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +
>  4 files changed, 162 insertions(+)
> 
> diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
> index 401922fc34..7261d6f52e 100644
> --- a/app/test-pmd/cmdline_flow.c
> +++ b/app/test-pmd/cmdline_flow.c
> @@ -53,6 +53,7 @@ enum index {
>  	COMMON_GROUP_ID,
>  	COMMON_PRIORITY_LEVEL,
>  	COMMON_INDIRECT_ACTION_ID,
> +	COMMON_PROFILE_ID,
>  	COMMON_POLICY_ID,
>  	COMMON_FLEX_HANDLE,
>  	COMMON_FLEX_TOKEN,
> @@ -508,6 +509,11 @@ enum index {
>  	ACTION_METER_COLOR_YELLOW,
>  	ACTION_METER_COLOR_RED,
>  	ACTION_METER_ID,
> +	ACTION_METER_MARK,
> +	ACTION_METER_PROFILE,
> +	ACTION_METER_PROFILE_ID2PTR,
> +	ACTION_METER_POLICY,
> +	ACTION_METER_POLICY_ID2PTR,
>  	ACTION_OF_SET_MPLS_TTL,
>  	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
>  	ACTION_OF_DEC_MPLS_TTL,
> @@ -1835,6 +1841,7 @@ static const enum index next_action[] = {
>  	ACTION_PORT_ID,
>  	ACTION_METER,
>  	ACTION_METER_COLOR,
> +	ACTION_METER_MARK,
>  	ACTION_OF_SET_MPLS_TTL,
>  	ACTION_OF_DEC_MPLS_TTL,
>  	ACTION_OF_SET_NW_TTL,
> @@ -1951,6 +1958,13 @@ static const enum index action_meter_color[] = {
>  	ZERO,
>  };
> 
> +static const enum index action_meter_mark[] = {
> +	ACTION_METER_PROFILE,
> +	ACTION_METER_POLICY,
> +	ACTION_NEXT,
> +	ZERO,
> +};
> +
>  static const enum index action_of_set_mpls_ttl[] = {
>  	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
>  	ACTION_NEXT,
> @@ -2389,6 +2403,14 @@ static int parse_mp(struct context *, const struct token *,
>  static int parse_meter_color(struct context *ctx, const struct token *token,
>  			     const char *str, unsigned int len, void *buf,
>  			     unsigned int size);
> +static int parse_meter_profile_id2ptr(struct context *ctx,
> +				      const struct token *token,
> +				      const char *str, unsigned int len,
> +				      void *buf, unsigned int size);
> +static int parse_meter_policy_id2ptr(struct context *ctx,
> +				     const struct token *token,
> +				     const char *str, unsigned int len,
> +				     void *buf, unsigned int size);
>  static int comp_none(struct context *, const struct token *,
>  		     unsigned int, char *, unsigned int);
>  static int comp_boolean(struct context *, const struct token *,
> @@ -2550,6 +2572,13 @@ static const struct token token_list[] = {
>  		.call = parse_int,
>  		.comp = comp_none,
>  	},
> +	[COMMON_PROFILE_ID] = {
> +		.name = "{profile_id}",
> +		.type = "PROFILE_ID",
> +		.help = "profile id",
> +		.call = parse_int,
> +		.comp = comp_none,
> +	},
>  	[COMMON_POLICY_ID] = {
>  		.name = "{policy_id}",
>  		.type = "POLICY_ID",
> @@ -5428,6 +5457,42 @@ static const struct token token_list[] = {
>  		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter, mtr_id)),
>  		.call = parse_vc_conf,
>  	},
> +	[ACTION_METER_MARK] = {
> +		.name = "meter_mark",
> +		.help = "meter the directed packets using profile and policy",
> +		.priv = PRIV_ACTION(METER_MARK,
> +				    sizeof(struct rte_flow_action_meter_mark)),
> +		.next = NEXT(action_meter_mark),
> +		.call = parse_vc,
> +	},
> +	[ACTION_METER_PROFILE] = {
> +		.name = "mtr_profile",
> +		.help = "meter profile id to use",
> +		.next = NEXT(NEXT_ENTRY(ACTION_METER_PROFILE_ID2PTR)),
> +		.args = ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
> +	},
> +	[ACTION_METER_PROFILE_ID2PTR] = {
> +		.name = "{mtr_profile_id}",
> +		.type = "PROFILE_ID",
> +		.help = "meter profile id",
> +		.next = NEXT(action_meter_mark),
> +		.call = parse_meter_profile_id2ptr,
> +		.comp = comp_none,
> +	},
> +	[ACTION_METER_POLICY] = {
> +		.name = "mtr_policy",
> +		.help = "meter policy id to use",
> +		.next = NEXT(NEXT_ENTRY(ACTION_METER_POLICY_ID2PTR)),
> +		ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
> +	},
> +	[ACTION_METER_POLICY_ID2PTR] = {
> +		.name = "{mtr_policy_id}",
> +		.type = "POLICY_ID",
> +		.help = "meter policy id",
> +		.next = NEXT(action_meter_mark),
> +		.call = parse_meter_policy_id2ptr,
> +		.comp = comp_none,
> +	},
>  	[ACTION_OF_SET_MPLS_TTL] = {
>  		.name = "of_set_mpls_ttl",
>  		.help = "OpenFlow's OFPAT_SET_MPLS_TTL",
> @@ -9747,6 +9812,68 @@ parse_ia_id2ptr(struct context *ctx, const struct token *token,
>  	return ret;
>  }
> 
> +static int
> +parse_meter_profile_id2ptr(struct context *ctx, const struct token *token,
> +			   const char *str, unsigned int len,
> +			   void *buf, unsigned int size)
> +{
> +	struct rte_flow_action *action = ctx->object;
> +	struct rte_flow_action_meter_mark *meter;
> +	struct rte_flow_meter_profile *profile = NULL;
> +	uint32_t id;
> +	int ret;
> +
> +	(void)buf;
> +	(void)size;
> +	ctx->objdata = 0;
> +	ctx->object = &id;
> +	ctx->objmask = NULL;
> +	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
> +	ctx->object = action;
> +	if (ret != (int)len)
> +		return ret;
> +	/* set meter profile */
> +	if (action) {
> +		meter = (struct rte_flow_action_meter_mark *)
> +				(uintptr_t)(action->conf);
> +		profile = port_meter_profile_get_by_id(ctx->port, id);
> +		meter->profile = profile;
> +		ret = (profile) ? ret : -1;
> +	}
> +	return ret;
> +}
> +
> +static int
> +parse_meter_policy_id2ptr(struct context *ctx, const struct token *token,
> +			  const char *str, unsigned int len,
> +			  void *buf, unsigned int size)
> +{
> +	struct rte_flow_action *action = ctx->object;
> +	struct rte_flow_action_meter_mark *meter;
> +	struct rte_flow_meter_policy *policy = NULL;
> +	uint32_t id;
> +	int ret;
> +
> +	(void)buf;
> +	(void)size;
> +	ctx->objdata = 0;
> +	ctx->object = &id;
> +	ctx->objmask = NULL;
> +	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
> +	ctx->object = action;
> +	if (ret != (int)len)
> +		return ret;
> +	/* set meter policy */
> +	if (action) {
> +		meter = (struct rte_flow_action_meter_mark *)
> +				(uintptr_t)(action->conf);
> +		policy = port_meter_policy_get_by_id(ctx->port, id);
> +		meter->policy = policy;
> +		ret = (policy) ? ret : -1;
> +	}
> +	return ret;
> +}
> +
>  /** Parse set command, initialize output buffer for subsequent tokens. */
>  static int
>  parse_set_raw_encap_decap(struct context *ctx, const struct token *token,
> diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> index 1b1e738f83..413dcb6e9d 100644
> --- a/app/test-pmd/config.c
> +++ b/app/test-pmd/config.c
> @@ -2184,6 +2184,32 @@ port_meter_policy_add(portid_t port_id, uint32_t policy_id,
>  	return ret;
>  }
> 
> +/** Get port meter profile */
> +struct rte_flow_meter_profile *
> +port_meter_profile_get_by_id(portid_t port_id, uint32_t id)
> +{
> +	struct rte_mtr_error error;
> +	struct rte_flow_meter_profile *profile;
> +
> +	profile = rte_mtr_meter_profile_get(port_id, id, &error);
> +	if (!profile)
> +		print_mtr_err_msg(&error);
> +	return profile;
> +}
> +
> +/** Get port meter policy */
> +struct rte_flow_meter_policy *
> +port_meter_policy_get_by_id(portid_t port_id, uint32_t id)
> +{
> +	struct rte_mtr_error error;
> +	struct rte_flow_meter_policy *policy;
> +
> +	policy = rte_mtr_meter_policy_get(port_id, id, &error);
> +	if (!policy)
> +		print_mtr_err_msg(&error);
> +	return policy;
> +}
> +
>  /** Validate flow rule. */
>  int
>  port_flow_validate(portid_t port_id,
> diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
> index f04a9a11b4..5fed46ae17 100644
> --- a/app/test-pmd/testpmd.h
> +++ b/app/test-pmd/testpmd.h
> @@ -985,6 +985,10 @@ void port_flow_tunnel_create(portid_t port_id, const struct tunnel_ops
> *ops);
>  int port_flow_isolate(portid_t port_id, int set);
>  int port_meter_policy_add(portid_t port_id, uint32_t policy_id,
>  		const struct rte_flow_action *actions);
> +struct rte_flow_meter_profile *port_meter_profile_get_by_id(portid_t port_id,
> +							    uint32_t id);
> +struct rte_flow_meter_policy *port_meter_policy_get_by_id(portid_t port_id,
> +							  uint32_t id);
> 
>  void rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id);
>  void tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id);
> diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> index 5e2f5ffb40..fd5a05dbad 100644
> --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
> @@ -4453,6 +4453,11 @@ This section lists supported actions and their attributes, if any.
> 
>    - ``ethdev_port_id {unsigned}``: ethdev port ID
> 
> +- ``meter_mark``:  meter the directed packets using profile and policy
> +
> +  - ``mtr_profile {unsigned}``: meter profile id to use
> +  - ``mtr_policy {unsigned}``: meter policy id to use
> +
>  Destroying flow rules
>  ~~~~~~~~~~~~~~~~~~~~~
> 
> --
> 2.18.2


Acked-by: Ori Kam <orika@nvidia.com>
Best,
Ori

RE: [PATCH v3 0/7] ethdev: separate metering and marking from policing

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, June 1, 2022 4:44 AM
> To: dev@dpdk.org
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> jerinjacobk@gmail.com; orika@nvidia.com; thomas@monjalon.net;
> ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
> ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [PATCH v3 0/7] ethdev: separate metering and marking from
> policing
> 
> Extend Metering and Marking support in the Flow API:
> 1. Add METER_COLOR item to match Color Marker set by a Meter.
> 2. Add the ability to set Color Marker via modify_field Flow API.
> 3. Add Meter API to get profile/policy objects.
> 4. Add METER_MARK action to perform Meter color metering and marking.
> Provide greater flexibility in how Metering can be used.
> 
> RFC:
> https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-
> 1-akozyrev@nvidia.com/
> 
> Traditional Meter Usage:
> 
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> meter_id = rte_mtr_create(profile_id, policy_id);
> rte_flow_create(pattern=5-tuple,actions=METER(meter_id));
> 
> The METER action effectively translates to the following:
> 1. Metering a packet stream.
> 2. Marking packets with an appropriate color.
> 3. Jump to a policy group.
> 4. Match on a color.
> 5. Execute assigned policy actions for the color.
> 
> New Meter Usage Model:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> rte_flow_create(pattern=5-tuple,
> 				actions=METER(profile_obj_ptr),JUMP);
> rte_flow_create(pattern=COLOR, actions=...);
> 
> The METER_MARK action effectively translates to the following:
> 1. Metering a packet stream.
> 2. Marking packets with an appropriate color.
> 
> A user is able to match the color later with the COLOR item.
> In order to do this we add the JUMP action after the METER action.
> 
> 3. Jump to a policy group.
> 4. Match on a color.
> 5. Execute actions for the color.
> 
> Here we decoupled the meter profile usage from the meter policy usage
> for greater flexibility and got rid of any locks related to meter_id lookup.
> 
> Another example of the meter creation to mimic the old model entirely:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> *policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
> rte_flow_create(pattern=5-tuple,
> 				actions=METER(profile_obj_ptr,
> policy_obj_ptr));
> 
> In this case, we define the policy actions right away.
> The main advantage is not having to lookup for profile_id/policy_id.
> 
> To free the meter obects we need to do the following:
> rte_flow_destroy(flow_handle);
> rte_mtr_meter_policy_delete(policy_id);
> rte_mtr_meter_profile_delete(profile_id);.
> profile_obj_ptr and policy_obj_ptr are no longer valid after that.
> 
> The meter profile configuration cannot be updated dynamically
> with the current set of patches, but can be supported later on.
> Now you have to destroy flows and profiles and recreate them.
> But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
> can have the corresponding siblings without mtr_id parameters.
> In this case, we can update the config and all the flows using them.
> 
> The meter sharing is done via the indirect action Flow API:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_prof8ile_get(profile_id);
> handle = rte_flow_action_handle_create(action=METER(profile_obj_ptr,
> NULL));
> flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
> flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));
> 
> Once we are done with the flow rules we can free everything.
> rte_flow_destroy(flow1);
> rte_flow_destroy(flow2);
> rte_flow_action_handle_destroy(handle);
> rte_mtr_meter_profile_delete(profile_id);
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> 
> Alexander Kozyrev (7):
>   ethdev: add meter color flow matching item
>   ethdev: allow meter color marker modification
>   ethdev: get meter profile/policy objects
>   ethdev: add meter color mark flow action
>   app/testpmd: add meter color flow matching item
>   app/testpmd: allow meter color marker modification
>   app/testpmd: add meter color mark flow action
> 
>  app/test-pmd/cmdline_flow.c                   | 213 +++++++++++++++++-
>  app/test-pmd/config.c                         |  26 +++
>  app/test-pmd/testpmd.h                        |   4 +
>  doc/guides/prog_guide/rte_flow.rst            |  32 +++
>  .../traffic_metering_and_policing.rst         |   7 +
>  doc/guides/rel_notes/release_22_07.rst        |   7 +
>  doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   9 +
>  lib/ethdev/rte_flow.c                         |   2 +
>  lib/ethdev/rte_flow.h                         |  60 +++++
>  lib/ethdev/rte_mtr.c                          |  41 ++++
>  lib/ethdev/rte_mtr.h                          |  40 ++++
>  lib/ethdev/rte_mtr_driver.h                   |  19 ++
>  lib/ethdev/version.map                        |   2 +
>  13 files changed, 461 insertions(+), 1 deletion(-)
> 
> --
> 2.18.2

Hi Alexander,

Thanks very much for your pseudo-code detailing the updated meter & flow operation!

Should we setup another call sometime next week to go through it? I still see a few fuzzy things where we need to level set.

Regards,
Cristian

Re: [PATCH v3 1/7] ethdev: add meter color flow matching item

[Andrew Rybchenko]

On 6/1/22 06:44, Alexander Kozyrev wrote:
> Provide an ability to use a Color Marker set by a Meter
> as a matching item in Flow API. The Color Marker reflects
> the metering result by setting the metadata for a
> packet to a particular codepoint: green, yellow or red.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

One nit below. Please, fix in the next version if required.

[snip]

> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 0ed4f92820..c1804bc74f 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -69,6 +69,10 @@ New Features
>     Added an API which can get the number of in-flight packets in
>     vhost async data path without using lock.
>   
> +* **Extended Metering and Marking support in the Flow API.**
> +
> +  * Added METER_COLOR item to match Color Marker set by a Meter.
> +

It should be nearby ethdev features.

>   * **Updated Intel iavf driver.**
>   
>     * Added Tx QoS queue rate limitation support.

[snip]

Re: [PATCH v3 2/7] ethdev: allow meter color marker modification

[Andrew Rybchenko]

On 6/1/22 06:44, Alexander Kozyrev wrote:
> Extend modify_field Flow API with support of Meter Color Marker
> modifications. It allows setting the packet's metadata to any
> color marker: green, yellow or red. A user is able to specify
> an initial packet color for Meter API or create simple Metering
> and Marking flow rules based on his own coloring algorithm.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>

Re: [PATCH v3 3/7] ethdev: get meter profile/policy objects

[Andrew Rybchenko]

On 6/1/22 06:44, Alexander Kozyrev wrote:
> Introduce a new Meter API to retrieve a Meter profile and policy
> objects using the profile/policy ID previously created with
> meter_profile_add() and meter_policy_create() functions.
> That allows to save the pointer and avoid any lookups in the
> corresponding lists for quick access during a flow rule creation.
> Also, it eliminates the need for CIR, CBS and EBS calculations
> and conversion to a PMD-specific format when the profile is used.
> Pointers are destroyed and cannot be used after thecorresponding

missing space after "the" above

> meter_profile_delete() or meter_policy_delete() are called.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---
>   .../traffic_metering_and_policing.rst         |  7 ++++
>   doc/guides/rel_notes/release_22_07.rst        |  1 +
>   lib/ethdev/rte_flow.h                         |  7 ++++
>   lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
>   lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
>   lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
>   lib/ethdev/version.map                        |  2 +
>   7 files changed, 117 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst
> index d1958a023d..2ce3236ad8 100644
> --- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
> +++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
> @@ -107,6 +107,13 @@ traffic meter and policing library.
>        to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
>        specified per color as show in :numref:`figure_rte_mtr_chaining`.
>   
> +#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
> +   API functions are available for getting the object pointers directly.
> +   These pointers allow quick access to profile/policy objects and are
> +   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
> +   This action may omit the policy definition to providei flexibility
> +   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
> +
>   Protocol based input color selection
>   ------------------------------------
>   
> diff --git a/doc/guides/rel_notes/release_22_07.rst b/doc/guides/rel_notes/release_22_07.rst
> index 451ff8d460..6d030bead5 100644
> --- a/doc/guides/rel_notes/release_22_07.rst
> +++ b/doc/guides/rel_notes/release_22_07.rst
> @@ -73,6 +73,7 @@ New Features
>   
>     * Added METER_COLOR item to match Color Marker set by a Meter.
>     * Added ability to set Color Marker via modify_field Flow API.
> +  * Added Meter API to get a pointer to profile/policy by their ID.
>   
>   * **Updated Intel iavf driver.**
>   
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index 68af109554..9754f6630a 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3827,6 +3827,13 @@ struct rte_flow_action {
>    */
>   struct rte_flow;
>   
> +/**
> + * Opaque type for Meter profile object returned by MTR API.
> + *
> + * This handle can be used to create Meter actions instead of profile ID.
> + */
> +struct rte_flow_meter_profile;
> +
>   /**
>    * @warning
>    * @b EXPERIMENTAL: this structure may change without prior notice
> diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
> index 441ea1dca9..90fd277040 100644
> --- a/lib/ethdev/rte_mtr.c
> +++ b/lib/ethdev/rte_mtr.c
> @@ -57,6 +57,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
>   	ops->func;					\
>   })
>   
> +#define RTE_MTR_HNDL_FUNC(port_id, func)		\
> +({							\
> +	const struct rte_mtr_ops *ops =			\
> +		rte_mtr_ops_get(port_id, error);	\
> +	if (ops == NULL)				\
> +		return NULL;				\
> +							\
> +	if (ops->func == NULL) {			\
> +		rte_mtr_error_set(error,		\
> +			ENOSYS,				\
> +			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
> +			NULL,				\
> +			rte_strerror(ENOSYS));		\
> +		return NULL;				\
> +	}						\
> +							\
> +	ops->func;					\
> +})

Personally I strongly dislike such macros. It is too tricky
to realize what it does taking into account how it is used
below. IMHO, code duplication is less evil.

> +
>   /* MTR capabilities get */
>   int
>   rte_mtr_capabilities_get(uint16_t port_id,
> @@ -91,6 +110,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>   		meter_profile_id, error);
>   }
>   
> +/** MTR meter profile get */
> +struct rte_flow_meter_profile *
> +rte_mtr_meter_profile_get(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error)
> +{
> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];

It is dangerous to get dev pointer without port_id range checking
first. I realize that it is checked later, but my point still stands.

> +	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
> +		meter_profile_id, error);
> +}
> +
>   /* MTR meter policy validate */
>   int
>   rte_mtr_meter_policy_validate(uint16_t port_id,
> @@ -125,6 +155,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
>   		policy_id, error);
>   }
>   
> +/** MTR meter policy get */
> +struct rte_flow_meter_policy *
> +rte_mtr_meter_policy_get(uint16_t port_id,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error)
> +{
> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> +	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
> +		policy_id, error);
> +}
> +
>   /** MTR object create */
>   int
>   rte_mtr_create(uint16_t port_id,
> diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
> index 008bc84f0d..58f0d26215 100644
> --- a/lib/ethdev/rte_mtr.h
> +++ b/lib/ethdev/rte_mtr.h
> @@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
>   	uint32_t meter_profile_id,
>   	struct rte_mtr_error *error);
>   
> +/**

Experimental note missing

> + * Meter profile object get
> + *
> + * Get meter profile object for a given meter profile ID.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] meter_profile_id
> + *   Meter profile ID. Needs to be the valid.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +__rte_experimental
> +struct rte_flow_meter_profile *
> +rte_mtr_meter_profile_get(uint16_t port_id,
> +	uint32_t meter_profile_id,
> +	struct rte_mtr_error *error);
> +
>   /**
>    * Check whether a meter policy can be created on a given port.
>    *
> @@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
>   	struct rte_mtr_meter_policy_params *policy,
>   	struct rte_mtr_error *error);
>   
> +/**

Experimental note missing

> + * Meter policy object get
> + *
> + * Get meter policy object for a given meter policy ID.
> + *
> + * @param[in] port_id
> + *   The port identifier of the Ethernet device.
> + * @param[in] policy_id
> + *   Meter policy ID. Needs to be the valid.
> + * @param[out] error
> + *   Error details. Filled in only on error, when not NULL.
> + * @return
> + *   A valid handle in case of success, NULL otherwise.
> + */
> +__rte_experimental
> +struct rte_flow_meter_policy *
> +rte_mtr_meter_policy_get(uint16_t port_id,
> +	uint32_t policy_id,
> +	struct rte_mtr_error *error);
> +
>   /**
>    * Define meter policy action list:
>    * GREEN - GREEN, YELLOW - YELLOW, RED - RED


> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index daca7851f2..2609f2e709 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -285,6 +285,8 @@ EXPERIMENTAL {
>   	rte_mtr_color_in_protocol_priority_get;
>   	rte_mtr_color_in_protocol_set;
>   	rte_mtr_meter_vlan_table_update;
> +	rte_mtr_meter_profile_get;
> +	rte_mtr_meter_policy_get;

if I'm not mistake it should be alphabetically sorted in the
group.

>   };
>   
>   INTERNAL {

Re: [PATCH v3 4/7] ethdev: add meter color mark flow action

[Andrew Rybchenko]

On 6/1/22 06:44, Alexander Kozyrev wrote:
> Create a new Flow API action: METER_MARK.
> It Meters an IP packet stream and marks its packets with colors.

Is it really required to be *IP* packet? IPv4 or IPv6 or both?

> Unlike the METER action, it performs no policing at all.
> A user has the flexibility to create any policies with the help of
> the METER_COLOR item later, only meter profile is mandatory here.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

[PATCH v4 0/7] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

Extend Metering and Marking support in the Flow API:
1. Add METER_COLOR item to match Color Marker set by a Meter.
2. Add the ability to set Color Marker via modify_field Flow API.
3. Add Meter API to get profile/policy objects.
4. Add METER_MARK action to perform Meter color metering and marking.
Provide greater flexibility in how Metering can be used.

RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/

Traditional Meter Usage:

profile_id = rte_mtr_meter_profile_add(RFC_params);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
meter_id = rte_mtr_create(profile_id, policy_id);
rte_flow_create(pattern=5-tuple,actions=METER(meter_id));

The METER action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.
3. Jump to a policy group.
4. Match on a color.
5. Execute assigned policy actions for the color.

New Meter Usage Model:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr),JUMP);
rte_flow_create(pattern=COLOR, actions=...);

The METER_MARK action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.

A user is able to match the color later with the COLOR item.
In order to do this we add the JUMP action after the METER action.

3. Jump to a policy group.
4. Match on a color.
5. Execute actions for the color.

Here we decoupled the meter profile usage from the meter policy usage
for greater flexibility and got rid of any locks related to meter_id lookup.

Another example of the meter creation to mimic the old model entirely:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
*policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr, policy_obj_ptr));

In this case, we define the policy actions right away.
The main advantage is not having to lookup for profile_id/policy_id.

To free the meter obects we need to do the following:
rte_flow_destroy(flow_handle);
rte_mtr_meter_policy_delete(policy_id);
rte_mtr_meter_profile_delete(profile_id);.
profile_obj_ptr and policy_obj_ptr are no longer valid after that.

The meter profile configuration cannot be updated dynamically
with the current set of patches, but can be supported later on.
Now you have to destroy flows and profiles and recreate them.
But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
can have the corresponding siblings without mtr_id parameters.
In this case, we can update the config and all the flows using them.

The meter sharing is done via the indirect action Flow API:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_prof8ile_get(profile_id);
handle = rte_flow_action_handle_create(action=METER(profile_obj_ptr, NULL));
flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));

Once we are done with the flow rules we can free everything.
rte_flow_destroy(flow1);
rte_flow_destroy(flow2);
rte_flow_action_handle_destroy(handle);
rte_mtr_meter_profile_delete(profile_id);

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Alexander Kozyrev (7):
  ethdev: add meter color flow matching item
  ethdev: allow meter color marker modification
  ethdev: get meter profile/policy objects
  ethdev: add meter color mark flow action
  app/test-pmd: add meter color flow matching item
  app/test-pmd: allow meter color marker modification
  app/testpmd: add meter color mark flow action

 app/test-pmd/cmdline_flow.c                   | 212 +++++++++++++++++-
 app/test-pmd/config.c                         |  26 +++
 app/test-pmd/testpmd.h                        |   4 +
 doc/guides/prog_guide/rte_flow.rst            |  32 +++
 .../traffic_metering_and_policing.rst         |   7 +
 doc/guides/rel_notes/release_22_11.rst        |   6 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |   9 +
 lib/ethdev/rte_flow.c                         |   2 +
 lib/ethdev/rte_flow.h                         |  60 +++++
 lib/ethdev/rte_mtr.c                          |  41 ++++
 lib/ethdev/rte_mtr.h                          |  40 ++++
 lib/ethdev/rte_mtr_driver.h                   |  19 ++
 lib/ethdev/version.map                        |   4 +
 13 files changed, 461 insertions(+), 1 deletion(-)

-- 
2.18.2

[PATCH v4 1/7] ethdev: add meter color flow matching item

[Alexander Kozyrev]

Provide an ability to use a Color Marker set by a Meter
as a matching item in Flow API. The Color Marker reflects
the metering result by setting the metadata for a
packet to a particular codepoint: green, yellow or red.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
 doc/guides/rel_notes/release_22_11.rst |  3 +++
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
 4 files changed, 35 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 588914b231..018def1033 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1651,6 +1651,13 @@ Matches a PPP header.
 - ``proto_id``: PPP protocol identifier.
 - Default ``mask`` matches addr, ctrl, proto_id.
 
+Item: ``METER_COLOR``
+^^^^^^^^^^^^^^^^^^^^^
+
+Matches Color Marker set by a Meter.
+
+- ``color``: Metering color marker.
+
 Actions
 ~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 8c021cf050..f6c02bb5e7 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -55,6 +55,9 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Extended Metering and Marking support in the Flow API.**
+
+  * Added METER_COLOR item to match Color Marker set by a Meter.
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 501be9d602..99247b599d 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -159,6 +159,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
 			rte_flow_item_flex_conv),
 	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
 	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
+	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index a79f1e7ef0..d49f5fd1b7 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -668,6 +668,14 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+
+	/**
+	 * Matches Meter Color Marker.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -2198,6 +2206,22 @@ struct rte_flow_item_flex_conf {
 	uint32_t nb_outputs;
 };
 
+/**
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR.
+ *
+ * Matches Color Marker set by a Meter.
+ */
+struct rte_flow_item_meter_color {
+	enum rte_color color; /**< Meter color marker. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+	.color = RTE_COLORS,
+};
+#endif
+
 /**
  * Action types.
  *
-- 
2.18.2

[PATCH v4 2/7] ethdev: allow meter color marker modification

[Alexander Kozyrev]

Extend modify_field Flow API with support of Meter Color Marker
modifications. It allows setting the packet's metadata to any
color marker: green, yellow or red. A user is able to specify
an initial packet color for Meter API or create simple Metering
and Marking flow rules based on his own coloring algorithm.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_22_11.rst | 1 +
 lib/ethdev/rte_flow.h                  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index f6c02bb5e7..a7651f69ba 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -58,6 +58,7 @@ New Features
 * **Extended Metering and Marking support in the Flow API.**
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
+  * Added ability to set Color Marker via modify_field Flow API.
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d49f5fd1b7..fddd47e7b5 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3713,6 +3713,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
 	RTE_FLOW_FIELD_IPV4_ECN,	/**< IPv4 ECN. */
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
+	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
 };
 
 /**
-- 
2.18.2

[PATCH v4 3/7] ethdev: get meter profile/policy objects

[Alexander Kozyrev]

Introduce a new Meter API to retrieve a Meter profile and policy
objects using the profile/policy ID previously created with
meter_profile_add() and meter_policy_create() functions.
That allows to save the pointer and avoid any lookups in the
corresponding lists for quick access during a flow rule creation.
Also, it eliminates the need for CIR, CBS and EBS calculations
and conversion to a PMD-specific format when the profile is used.
Pointers are destroyed and cannot be used after the corresponding
meter_profile_delete() or meter_policy_delete() are called.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 .../traffic_metering_and_policing.rst         |  7 ++++
 doc/guides/rel_notes/release_22_11.rst        |  1 +
 lib/ethdev/rte_flow.h                         |  7 ++++
 lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
 lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
 lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
 lib/ethdev/version.map                        |  4 ++
 7 files changed, 119 insertions(+)

diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst
index d1958a023d..2ce3236ad8 100644
--- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
+++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
@@ -107,6 +107,13 @@ traffic meter and policing library.
      to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
      specified per color as show in :numref:`figure_rte_mtr_chaining`.
 
+#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
+   API functions are available for getting the object pointers directly.
+   These pointers allow quick access to profile/policy objects and are
+   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
+   This action may omit the policy definition to providei flexibility
+   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
+
 Protocol based input color selection
 ------------------------------------
 
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index a7651f69ba..7969609788 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -59,6 +59,7 @@ New Features
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
+  * Added Meter API to get a pointer to profile/policy by their ID.
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index fddd47e7b5..edf69fc44f 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3826,6 +3826,13 @@ struct rte_flow_action {
  */
 struct rte_flow;
 
+/**
+ * Opaque type for Meter profile object returned by MTR API.
+ *
+ * This handle can be used to create Meter actions instead of profile ID.
+ */
+struct rte_flow_meter_profile;
+
 /**
  * @warning
  * @b EXPERIMENTAL: this structure may change without prior notice
diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
index c460e4f4e0..9e79b744da 100644
--- a/lib/ethdev/rte_mtr.c
+++ b/lib/ethdev/rte_mtr.c
@@ -56,6 +56,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
 	ops->func;					\
 })
 
+#define RTE_MTR_HNDL_FUNC(port_id, func)		\
+({							\
+	const struct rte_mtr_ops *ops =			\
+		rte_mtr_ops_get(port_id, error);	\
+	if (ops == NULL)				\
+		return NULL;				\
+							\
+	if (ops->func == NULL) {			\
+		rte_mtr_error_set(error,		\
+			ENOSYS,				\
+			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
+			NULL,				\
+			rte_strerror(ENOSYS));		\
+		return NULL;				\
+	}						\
+							\
+	ops->func;					\
+})
+
 /* MTR capabilities get */
 int
 rte_mtr_capabilities_get(uint16_t port_id,
@@ -90,6 +109,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 		meter_profile_id, error);
 }
 
+/** MTR meter profile get */
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
+		meter_profile_id, error);
+}
+
 /* MTR meter policy validate */
 int
 rte_mtr_meter_policy_validate(uint16_t port_id,
@@ -124,6 +154,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
 		policy_id, error);
 }
 
+/** MTR meter policy get */
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
+		policy_id, error);
+}
+
 /** MTR object create */
 int
 rte_mtr_create(uint16_t port_id,
diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
index 008bc84f0d..58f0d26215 100644
--- a/lib/ethdev/rte_mtr.h
+++ b/lib/ethdev/rte_mtr.h
@@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter profile object get
+ *
+ * Get meter profile object for a given meter profile ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] meter_profile_id
+ *   Meter profile ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /**
  * Check whether a meter policy can be created on a given port.
  *
@@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
 	struct rte_mtr_meter_policy_params *policy,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter policy object get
+ *
+ * Get meter policy object for a given meter policy ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] policy_id
+ *   Meter policy ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
 /**
  * Define meter policy action list:
  * GREEN - GREEN, YELLOW - YELLOW, RED - RED
diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
index f7dca9a54c..948a629b93 100644
--- a/lib/ethdev/rte_mtr_driver.h
+++ b/lib/ethdev/rte_mtr_driver.h
@@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter profile get. */
+typedef struct rte_flow_meter_profile *
+(*rte_mtr_meter_profile_get_t)(struct rte_eth_dev *dev,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /** @internal MTR meter policy validate. */
 typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
 	struct rte_mtr_meter_policy_params *policy,
@@ -57,6 +63,13 @@ typedef int (*rte_mtr_meter_policy_delete_t)(struct rte_eth_dev *dev,
 	uint32_t policy_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter policy get. */
+typedef struct rte_flow_meter_policy *
+(*rte_mtr_meter_policy_get_t)(struct rte_eth_dev *dev,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
+
 /** @internal MTR object create. */
 typedef int (*rte_mtr_create_t)(struct rte_eth_dev *dev,
 	uint32_t mtr_id,
@@ -194,6 +207,12 @@ struct rte_mtr_ops {
 
 	/** MTR object meter policy update */
 	rte_mtr_meter_policy_update_t meter_policy_update;
+
+	/** MTR meter profile get */
+	rte_mtr_meter_profile_get_t meter_profile_get;
+
+	/** MTR meter policy get */
+	rte_mtr_meter_policy_get_t meter_policy_get;
 };
 
 /**
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index 03f52fee91..1fec250c85 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -285,6 +285,10 @@ EXPERIMENTAL {
 	rte_mtr_color_in_protocol_priority_get;
 	rte_mtr_color_in_protocol_set;
 	rte_mtr_meter_vlan_table_update;
+
+	# added in 22.11
+	rte_mtr_meter_profile_get;
+	rte_mtr_meter_policy_get;
 };
 
 INTERNAL {
-- 
2.18.2

[PATCH v4 4/7] ethdev: add meter color mark flow action

[Alexander Kozyrev]

Create a new Flow API action: METER_MARK.
It Meters a packet stream and marks its packets with colors.
The marking is done on a metadata, not on a packet field.
Unlike the METER action, it performs no policing at all.
A user has the flexibility to create any policies with the help of
the METER_COLOR item later, only meter profile is mandatory here.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     | 25 +++++++++++++++++++++++
 doc/guides/rel_notes/release_22_11.rst |  1 +
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 28 ++++++++++++++++++++++++++
 4 files changed, 55 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 018def1033..5b87d9f61e 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3411,6 +3411,31 @@ This action is meant to use the same structure as `Action: PORT_REPRESENTOR`_.
 
 See also `Item: REPRESENTED_PORT`_.
 
+Action: ``METER_MARK``
+^^^^^^^^^^^^^^^^^^^^^^
+
+Meters a packet stream and marks its packets with colors.
+
+Unlike the ``METER`` action, policing is optional and may be
+performed later with the help of the ``METER_COLOR`` item.
+The profile and/or policy objects have to be created
+using the rte_mtr_profile_add()/rte_mtr_policy_add() API.
+Pointers to these objects are used as action parameters
+and need to be retrieved using the rte_mtr_profile_get() API
+and rte_mtr_policy_get() API respectively.
+
+.. _table_rte_flow_action_meter_mark:
+
+.. table:: METER_MARK
+
+   +------------------+----------------------+
+   | Field            | Value                |
+   +==================+======================+
+   | ``profile``      | Meter profile object |
+   +------------------+----------------------+
+   | ``policy``       | Meter policy object  |
+   +------------------+----------------------+
+
 Negative types
 ~~~~~~~~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 7969609788..401552ff84 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -60,6 +60,7 @@ New Features
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
   * Added Meter API to get a pointer to profile/policy by their ID.
+  * Added METER_MARK action for Metering with lockless profile/policy access.
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 99247b599d..7ff024f33e 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -260,6 +260,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = {
 	MK_FLOW_ACTION(CONNTRACK, sizeof(struct rte_flow_action_conntrack)),
 	MK_FLOW_ACTION(PORT_REPRESENTOR, sizeof(struct rte_flow_action_ethdev)),
 	MK_FLOW_ACTION(REPRESENTED_PORT, sizeof(struct rte_flow_action_ethdev)),
+	MK_FLOW_ACTION(METER_MARK, sizeof(struct rte_flow_action_meter_mark)),
 };
 
 int
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index edf69fc44f..74e7ddf73a 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -2903,6 +2903,15 @@ enum rte_flow_action_type {
 	 * @see struct rte_flow_action_ethdev
 	 */
 	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+
+	/**
+	 * Traffic metering and marking (MTR).
+	 * the entity represented by the given ethdev.
+	 *
+	 * @see struct rte_flow_action_meter_mark
+	 * See file rte_mtr.h for MTR profile object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_MARK,
 };
 
 /**
@@ -3774,6 +3783,25 @@ struct rte_flow_action_modify_field {
 	uint32_t width; /**< Number of bits to use from a source field. */
 };
 
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_MARK
+ *
+ * Traffic metering and marking (MTR).
+ *
+ * Meters a packet stream and marks its packets either
+ * green, yellow, or red according to the specified profile.
+ * The policy is optional and may be specified for defining
+ * subsequent actions based on a color assigned by MTR.
+ * Alternatively, the METER_COLOR item may be used for this.
+ */
+struct rte_flow_action_meter_mark {
+
+	/**< Profile config retrieved with rte_mtr_profile_get(). */
+	struct rte_flow_meter_profile *profile;
+	/**< Policy config retrieved with rte_mtr_policy_get(). */
+	struct rte_flow_meter_policy *policy;
+};
+
 /* Mbuf dynamic field offset for metadata. */
 extern int32_t rte_flow_dynf_metadata_offs;
 
-- 
2.18.2

[PATCH v4 5/7] app/test-pmd: add meter color flow matching item

[Alexander Kozyrev]

Add testpmd command line to match on a meter color:
	flow create 0 ingress group 0 pattern meter color is green / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c                 | 83 +++++++++++++++++++++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  4 +
 2 files changed, 87 insertions(+)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 7f50028eb7..b8c92efb1d 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -458,6 +458,9 @@ enum index {
 	ITEM_PPP_ADDR,
 	ITEM_PPP_CTRL,
 	ITEM_PPP_PROTO_ID,
+	ITEM_METER,
+	ITEM_METER_COLOR,
+	ITEM_METER_COLOR_NAME,
 
 	/* Validate/create actions. */
 	ACTIONS,
@@ -799,6 +802,10 @@ static const char *const modify_field_ids[] = {
 	"ipv4_ecn", "ipv6_ecn", NULL
 };
 
+static const char *const meter_colors[] = {
+	"green", "yellow", "red", "all", NULL
+};
+
 /** Maximum number of subsequent tokens and arguments on the stack. */
 #define CTX_STACK_SIZE 16
 
@@ -1332,6 +1339,7 @@ static const enum index next_item[] = {
 	ITEM_FLEX,
 	ITEM_L2TPV2,
 	ITEM_PPP,
+	ITEM_METER,
 	END_SET,
 	ZERO,
 };
@@ -1804,6 +1812,12 @@ static const enum index item_ppp[] = {
 	ZERO,
 };
 
+static const enum index item_meter[] = {
+	ITEM_METER_COLOR,
+	ITEM_NEXT,
+	ZERO,
+};
+
 static const enum index next_action[] = {
 	ACTION_END,
 	ACTION_VOID,
@@ -2372,6 +2386,9 @@ static int parse_ia_id2ptr(struct context *ctx, const struct token *token,
 static int parse_mp(struct context *, const struct token *,
 		    const char *, unsigned int,
 		    void *, unsigned int);
+static int parse_meter_color(struct context *ctx, const struct token *token,
+			     const char *str, unsigned int len, void *buf,
+			     unsigned int size);
 static int comp_none(struct context *, const struct token *,
 		     unsigned int, char *, unsigned int);
 static int comp_boolean(struct context *, const struct token *,
@@ -2402,6 +2419,8 @@ static int comp_table_id(struct context *, const struct token *,
 			 unsigned int, char *, unsigned int);
 static int comp_queue_id(struct context *, const struct token *,
 			 unsigned int, char *, unsigned int);
+static int comp_meter_color(struct context *, const struct token *,
+			    unsigned int, char *, unsigned int);
 
 /** Token definitions. */
 static const struct token token_list[] = {
@@ -5064,6 +5083,29 @@ static const struct token token_list[] = {
 		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_ppp,
 					hdr.proto_id)),
 	},
+	[ITEM_METER] = {
+		.name = "meter",
+		.help = "match meter color",
+		.priv = PRIV_ITEM(METER_COLOR,
+				  sizeof(struct rte_flow_item_meter_color)),
+		.next = NEXT(item_meter),
+		.call = parse_vc,
+	},
+	[ITEM_METER_COLOR] = {
+		.name = "color",
+		.help = "meter color",
+		.next = NEXT(item_meter,
+			     NEXT_ENTRY(ITEM_METER_COLOR_NAME),
+			     item_param),
+		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_meter_color,
+					color)),
+	},
+	[ITEM_METER_COLOR_NAME] = {
+		.name = "color_name",
+		.help = "meter color name",
+		.call = parse_meter_color,
+		.comp = comp_meter_color,
+	},
 	/* Validate/create actions. */
 	[ACTIONS] = {
 		.name = "actions",
@@ -9867,6 +9909,30 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	return ret;
 }
 
+/** Parse Meter color name */
+static int
+parse_meter_color(struct context *ctx, const struct token *token,
+		  const char *str, unsigned int len, void *buf,
+		  unsigned int size)
+{
+	struct rte_flow_item_meter_color *meter_color;
+	unsigned int i;
+
+	(void)token;
+	(void)buf;
+	(void)size;
+	for (i = 0; meter_colors[i]; ++i)
+		if (!strcmp_partial(meter_colors[i], str, len))
+			break;
+	if (!meter_colors[i])
+		return -1;
+	if (!ctx->object)
+		return len;
+	meter_color = ctx->object;
+	meter_color->color = (enum rte_color)i;
+	return len;
+}
+
 /** No completion. */
 static int
 comp_none(struct context *ctx, const struct token *token,
@@ -10158,6 +10224,20 @@ comp_queue_id(struct context *ctx, const struct token *token,
 	return i;
 }
 
+/** Complete available Meter colors. */
+static int
+comp_meter_color(struct context *ctx, const struct token *token,
+		 unsigned int ent, char *buf, unsigned int size)
+{
+	RTE_SET_USED(ctx);
+	RTE_SET_USED(token);
+	if (!buf)
+		return RTE_DIM(meter_colors);
+	if (ent < RTE_DIM(meter_colors) - 1)
+		return strlcpy(buf, meter_colors[ent], size);
+	return -1;
+}
+
 /** Internal context. */
 static struct context cmd_flow_context;
 
@@ -10773,6 +10853,9 @@ flow_item_default_mask(const struct rte_flow_item *item)
 	case RTE_FLOW_ITEM_TYPE_PPP:
 		mask = &rte_flow_item_ppp_mask;
 		break;
+	case RTE_FLOW_ITEM_TYPE_METER_COLOR:
+		mask = &rte_flow_item_meter_color_mask;
+		break;
 	default:
 		break;
 	}
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 330e34427d..150cfc0664 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -3904,6 +3904,10 @@ This section lists supported pattern items and their attributes, if any.
   - ``ctrl {unsigned}``: PPP control.
   - ``proto_id {unsigned}``: PPP protocol identifier.
 
+- ``meter``: match Meter color.
+
+  - ``color {value}``: Meter color value(green/yellow/red).
+
 Actions list
 ^^^^^^^^^^^^
 
-- 
2.18.2

[PATCH v4 6/7] app/test-pmd: allow meter color marker modification

[Alexander Kozyrev]

Extend the list of available modify_field IDs to include
recently added meter color marker item in testpmd CLI.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index b8c92efb1d..e6f4bdf3b2 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -799,7 +799,7 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", NULL
+	"ipv4_ecn", "ipv6_ecn", "meter_color", NULL
 };
 
 static const char *const meter_colors[] = {
-- 
2.18.2

[PATCH v4 7/7] app/testpmd: add meter color mark flow action

[Alexander Kozyrev]

Add testpmd command line to match for METER_MARK action:
	flow create ... actions meter_mark mtr_profile 20 / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c                 | 127 ++++++++++++++++++++
 app/test-pmd/config.c                       |  26 ++++
 app/test-pmd/testpmd.h                      |   4 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   5 +
 4 files changed, 162 insertions(+)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index e6f4bdf3b2..dd837e27f7 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -53,6 +53,7 @@ enum index {
 	COMMON_GROUP_ID,
 	COMMON_PRIORITY_LEVEL,
 	COMMON_INDIRECT_ACTION_ID,
+	COMMON_PROFILE_ID,
 	COMMON_POLICY_ID,
 	COMMON_FLEX_HANDLE,
 	COMMON_FLEX_TOKEN,
@@ -508,6 +509,11 @@ enum index {
 	ACTION_METER_COLOR_YELLOW,
 	ACTION_METER_COLOR_RED,
 	ACTION_METER_ID,
+	ACTION_METER_MARK,
+	ACTION_METER_PROFILE,
+	ACTION_METER_PROFILE_ID2PTR,
+	ACTION_METER_POLICY,
+	ACTION_METER_POLICY_ID2PTR,
 	ACTION_OF_SET_MPLS_TTL,
 	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
 	ACTION_OF_DEC_MPLS_TTL,
@@ -1835,6 +1841,7 @@ static const enum index next_action[] = {
 	ACTION_PORT_ID,
 	ACTION_METER,
 	ACTION_METER_COLOR,
+	ACTION_METER_MARK,
 	ACTION_OF_SET_MPLS_TTL,
 	ACTION_OF_DEC_MPLS_TTL,
 	ACTION_OF_SET_NW_TTL,
@@ -1951,6 +1958,13 @@ static const enum index action_meter_color[] = {
 	ZERO,
 };
 
+static const enum index action_meter_mark[] = {
+	ACTION_METER_PROFILE,
+	ACTION_METER_POLICY,
+	ACTION_NEXT,
+	ZERO,
+};
+
 static const enum index action_of_set_mpls_ttl[] = {
 	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
 	ACTION_NEXT,
@@ -2389,6 +2403,14 @@ static int parse_mp(struct context *, const struct token *,
 static int parse_meter_color(struct context *ctx, const struct token *token,
 			     const char *str, unsigned int len, void *buf,
 			     unsigned int size);
+static int parse_meter_profile_id2ptr(struct context *ctx,
+				      const struct token *token,
+				      const char *str, unsigned int len,
+				      void *buf, unsigned int size);
+static int parse_meter_policy_id2ptr(struct context *ctx,
+				     const struct token *token,
+				     const char *str, unsigned int len,
+				     void *buf, unsigned int size);
 static int comp_none(struct context *, const struct token *,
 		     unsigned int, char *, unsigned int);
 static int comp_boolean(struct context *, const struct token *,
@@ -2550,6 +2572,13 @@ static const struct token token_list[] = {
 		.call = parse_int,
 		.comp = comp_none,
 	},
+	[COMMON_PROFILE_ID] = {
+		.name = "{profile_id}",
+		.type = "PROFILE_ID",
+		.help = "profile id",
+		.call = parse_int,
+		.comp = comp_none,
+	},
 	[COMMON_POLICY_ID] = {
 		.name = "{policy_id}",
 		.type = "POLICY_ID",
@@ -5428,6 +5457,42 @@ static const struct token token_list[] = {
 		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter, mtr_id)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_METER_MARK] = {
+		.name = "meter_mark",
+		.help = "meter the directed packets using profile and policy",
+		.priv = PRIV_ACTION(METER_MARK,
+				    sizeof(struct rte_flow_action_meter_mark)),
+		.next = NEXT(action_meter_mark),
+		.call = parse_vc,
+	},
+	[ACTION_METER_PROFILE] = {
+		.name = "mtr_profile",
+		.help = "meter profile id to use",
+		.next = NEXT(NEXT_ENTRY(ACTION_METER_PROFILE_ID2PTR)),
+		.args = ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
+	},
+	[ACTION_METER_PROFILE_ID2PTR] = {
+		.name = "{mtr_profile_id}",
+		.type = "PROFILE_ID",
+		.help = "meter profile id",
+		.next = NEXT(action_meter_mark),
+		.call = parse_meter_profile_id2ptr,
+		.comp = comp_none,
+	},
+	[ACTION_METER_POLICY] = {
+		.name = "mtr_policy",
+		.help = "meter policy id to use",
+		.next = NEXT(NEXT_ENTRY(ACTION_METER_POLICY_ID2PTR)),
+		ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
+	},
+	[ACTION_METER_POLICY_ID2PTR] = {
+		.name = "{mtr_policy_id}",
+		.type = "POLICY_ID",
+		.help = "meter policy id",
+		.next = NEXT(action_meter_mark),
+		.call = parse_meter_policy_id2ptr,
+		.comp = comp_none,
+	},
 	[ACTION_OF_SET_MPLS_TTL] = {
 		.name = "of_set_mpls_ttl",
 		.help = "OpenFlow's OFPAT_SET_MPLS_TTL",
@@ -9747,6 +9812,68 @@ parse_ia_id2ptr(struct context *ctx, const struct token *token,
 	return ret;
 }
 
+static int
+parse_meter_profile_id2ptr(struct context *ctx, const struct token *token,
+			   const char *str, unsigned int len,
+			   void *buf, unsigned int size)
+{
+	struct rte_flow_action *action = ctx->object;
+	struct rte_flow_action_meter_mark *meter;
+	struct rte_flow_meter_profile *profile = NULL;
+	uint32_t id;
+	int ret;
+
+	(void)buf;
+	(void)size;
+	ctx->objdata = 0;
+	ctx->object = &id;
+	ctx->objmask = NULL;
+	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
+	ctx->object = action;
+	if (ret != (int)len)
+		return ret;
+	/* set meter profile */
+	if (action) {
+		meter = (struct rte_flow_action_meter_mark *)
+				(uintptr_t)(action->conf);
+		profile = port_meter_profile_get_by_id(ctx->port, id);
+		meter->profile = profile;
+		ret = (profile) ? ret : -1;
+	}
+	return ret;
+}
+
+static int
+parse_meter_policy_id2ptr(struct context *ctx, const struct token *token,
+			  const char *str, unsigned int len,
+			  void *buf, unsigned int size)
+{
+	struct rte_flow_action *action = ctx->object;
+	struct rte_flow_action_meter_mark *meter;
+	struct rte_flow_meter_policy *policy = NULL;
+	uint32_t id;
+	int ret;
+
+	(void)buf;
+	(void)size;
+	ctx->objdata = 0;
+	ctx->object = &id;
+	ctx->objmask = NULL;
+	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
+	ctx->object = action;
+	if (ret != (int)len)
+		return ret;
+	/* set meter policy */
+	if (action) {
+		meter = (struct rte_flow_action_meter_mark *)
+				(uintptr_t)(action->conf);
+		policy = port_meter_policy_get_by_id(ctx->port, id);
+		meter->policy = policy;
+		ret = (policy) ? ret : -1;
+	}
+	return ret;
+}
+
 /** Parse set command, initialize output buffer for subsequent tokens. */
 static int
 parse_set_raw_encap_decap(struct context *ctx, const struct token *token,
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index a2939867c4..6f82bd783c 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -2316,6 +2316,32 @@ port_meter_policy_add(portid_t port_id, uint32_t policy_id,
 	return ret;
 }
 
+/** Get port meter profile */
+struct rte_flow_meter_profile *
+port_meter_profile_get_by_id(portid_t port_id, uint32_t id)
+{
+	struct rte_mtr_error error;
+	struct rte_flow_meter_profile *profile;
+
+	profile = rte_mtr_meter_profile_get(port_id, id, &error);
+	if (!profile)
+		print_mtr_err_msg(&error);
+	return profile;
+}
+
+/** Get port meter policy */
+struct rte_flow_meter_policy *
+port_meter_policy_get_by_id(portid_t port_id, uint32_t id)
+{
+	struct rte_mtr_error error;
+	struct rte_flow_meter_policy *policy;
+
+	policy = rte_mtr_meter_policy_get(port_id, id, &error);
+	if (!policy)
+		print_mtr_err_msg(&error);
+	return policy;
+}
+
 /** Validate flow rule. */
 int
 port_flow_validate(portid_t port_id,
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index fb2f5195d3..ac1e1b7d22 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -1013,6 +1013,10 @@ void port_flow_tunnel_create(portid_t port_id, const struct tunnel_ops *ops);
 int port_flow_isolate(portid_t port_id, int set);
 int port_meter_policy_add(portid_t port_id, uint32_t policy_id,
 		const struct rte_flow_action *actions);
+struct rte_flow_meter_profile *port_meter_profile_get_by_id(portid_t port_id,
+							    uint32_t id);
+struct rte_flow_meter_policy *port_meter_policy_get_by_id(portid_t port_id,
+							  uint32_t id);
 
 void rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id);
 void tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id);
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 150cfc0664..cf09dc5aef 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -4193,6 +4193,11 @@ This section lists supported actions and their attributes, if any.
 
   - ``ethdev_port_id {unsigned}``: ethdev port ID
 
+- ``meter_mark``:  meter the directed packets using profile and policy
+
+  - ``mtr_profile {unsigned}``: meter profile id to use
+  - ``mtr_policy {unsigned}``: meter policy id to use
+
 Destroying flow rules
 ~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.18.2

RE: [PATCH v4 1/7] ethdev: add meter color flow matching item

[Ori Kam]

Hi Alexander,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Wednesday, 21 September 2022 5:11
> Subject: [PATCH v4 1/7] ethdev: add meter color flow matching item
> 
> Provide an ability to use a Color Marker set by a Meter
> as a matching item in Flow API. The Color Marker reflects
> the metering result by setting the metadata for a
> packet to a particular codepoint: green, yellow or red.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---

Series-acked-by:  Ori Kam <orika@nvidia.com>
Thanks,
Ori

[PATCH v5 0/7] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

Extend Metering and Marking support in the Flow API:
1. Add METER_COLOR item to match Color Marker set by a Meter.
2. Add the ability to set Color Marker via modify_field Flow API.
3. Add Meter API to get profile/policy objects.
4. Add METER_MARK action to perform Meter color metering and marking.
Provide greater flexibility in how Metering can be used.

RFC: https://patchwork.dpdk.org/project/dpdk/cover/20220502200439.4100965-1-akozyrev@nvidia.com/

Traditional Meter Usage:

profile_id = rte_mtr_meter_profile_add(RFC_params);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
meter_id = rte_mtr_create(profile_id, policy_id);
rte_flow_create(pattern=5-tuple,actions=METER(meter_id));

The METER action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.
3. Jump to a policy group.
4. Match on a color.
5. Execute assigned policy actions for the color.

New Meter Usage Model:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr),JUMP);
rte_flow_create(pattern=COLOR, actions=...);

The METER_MARK action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.

A user is able to match the color later with the COLOR item.
In order to do this we add the JUMP action after the METER action.

3. Jump to a policy group.
4. Match on a color.
5. Execute actions for the color.

Here we decoupled the meter profile usage from the meter policy usage
for greater flexibility and got rid of any locks related to meter_id lookup.

Another example of the meter creation to mimic the old model entirely:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
*policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
rte_flow_create(pattern=5-tuple,
				actions=METER(profile_obj_ptr, policy_obj_ptr));

In this case, we define the policy actions right away.
The main advantage is not having to lookup for profile_id/policy_id.

To free the meter obects we need to do the following:
rte_flow_destroy(flow_handle);
rte_mtr_meter_policy_delete(policy_id);
rte_mtr_meter_profile_delete(profile_id);.
profile_obj_ptr and policy_obj_ptr are no longer valid after that.

The meter profile configuration cannot be updated dynamically
with the current set of patches, but can be supported later on.
Now you have to destroy flows and profiles and recreate them.
But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
can have the corresponding siblings without mtr_id parameters.
In this case, we can update the config and all the flows using them.

The meter sharing is done via the indirect action Flow API:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_prof8ile_get(profile_id);
handle = rte_flow_action_handle_create(action=METER(profile_obj_ptr, NULL));
flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));

Once we are done with the flow rules we can free everything.
rte_flow_destroy(flow1);
rte_flow_destroy(flow2);
rte_flow_action_handle_destroy(handle);
rte_mtr_meter_profile_delete(profile_id);

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
v5: added color-aware mode to METER_MARK and ability to update the action

Alexander Kozyrev (7):
  ethdev: add meter color flow matching item
  ethdev: allow meter color marker modification
  ethdev: get meter profile/policy objects
  ethdev: add meter color mark flow action
  app/testpmd: add meter color flow matching item
  app/testpmd: allow meter color marker modification
  app/testpmd: add meter color mark flow action

 app/test-pmd/cmdline_flow.c                   | 238 +++++++++++++++++-
 app/test-pmd/config.c                         |  45 +++-
 app/test-pmd/testpmd.h                        |   4 +
 doc/guides/prog_guide/rte_flow.rst            |  32 +++
 .../traffic_metering_and_policing.rst         |   7 +
 doc/guides/rel_notes/release_22_11.rst        |   8 +-
 doc/guides/testpmd_app_ug/testpmd_funcs.rst   |  12 +
 lib/ethdev/rte_flow.c                         |   2 +
 lib/ethdev/rte_flow.h                         |  95 +++++++
 lib/ethdev/rte_mtr.c                          |  41 +++
 lib/ethdev/rte_mtr.h                          |  40 +++
 lib/ethdev/rte_mtr_driver.h                   |  19 ++
 lib/ethdev/version.map                        |   4 +
 13 files changed, 544 insertions(+), 3 deletions(-)

-- 
2.18.2

[PATCH v5 1/7] ethdev: add meter color flow matching item

[Alexander Kozyrev]

Provide an ability to use a Color Marker set by a Meter
as a matching item in Flow API. The Color Marker reflects
the metering result by setting the metadata for a
packet to a particular codepoint: green, yellow or red.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     |  7 +++++++
 doc/guides/rel_notes/release_22_11.rst |  5 ++++-
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 24 ++++++++++++++++++++++++
 4 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 588914b231..018def1033 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -1651,6 +1651,13 @@ Matches a PPP header.
 - ``proto_id``: PPP protocol identifier.
 - Default ``mask`` matches addr, ctrl, proto_id.
 
+Item: ``METER_COLOR``
+^^^^^^^^^^^^^^^^^^^^^
+
+Matches Color Marker set by a Meter.
+
+- ``color``: Metering color marker.
+
 Actions
 ~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 235ac9bf94..eeed420ae3 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -55,11 +55,14 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Extended Metering and Marking support in the Flow API.**
+
+  * Added METER_COLOR item to match Color Marker set by a Meter.
+
 * **Updated Wangxun ngbe driver.**
 
   * Added support to set device link down/up.
 
-
 Removed Items
 -------------
 
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 501be9d602..99247b599d 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -159,6 +159,7 @@ static const struct rte_flow_desc_data rte_flow_desc_item[] = {
 			rte_flow_item_flex_conv),
 	MK_FLOW_ITEM(L2TPV2, sizeof(struct rte_flow_item_l2tpv2)),
 	MK_FLOW_ITEM(PPP, sizeof(struct rte_flow_item_ppp)),
+	MK_FLOW_ITEM(METER_COLOR, sizeof(struct rte_flow_item_meter_color)),
 };
 
 /** Generate flow_action[] entry. */
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 017f690798..c0409a5ada 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -668,6 +668,14 @@ enum rte_flow_item_type {
 	 * See struct rte_flow_item_gre_opt.
 	 */
 	RTE_FLOW_ITEM_TYPE_GRE_OPTION,
+
+	/**
+	 * Matches Meter Color Marker.
+	 *
+	 * See struct rte_flow_item_meter_color.
+	 */
+
+	RTE_FLOW_ITEM_TYPE_METER_COLOR,
 };
 
 /**
@@ -2198,6 +2206,22 @@ struct rte_flow_item_flex_conf {
 	uint32_t nb_outputs;
 };
 
+/**
+ * RTE_FLOW_ITEM_TYPE_METER_COLOR.
+ *
+ * Matches Color Marker set by a Meter.
+ */
+struct rte_flow_item_meter_color {
+	enum rte_color color; /**< Meter color marker. */
+};
+
+/** Default mask for RTE_FLOW_ITEM_TYPE_METER_COLOR. */
+#ifndef __cplusplus
+static const struct rte_flow_item_meter_color rte_flow_item_meter_color_mask = {
+	.color = RTE_COLORS,
+};
+#endif
+
 /**
  * Action types.
  *
-- 
2.18.2

[PATCH v5 2/7] ethdev: allow meter color marker modification

[Alexander Kozyrev]

Extend modify_field Flow API with support of Meter Color Marker
modifications. It allows setting the packet's metadata to any
color marker: green, yellow or red. A user is able to specify
an initial packet color for Meter API or create simple Metering
and Marking flow rules based on his own coloring algorithm.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_22_11.rst | 1 +
 lib/ethdev/rte_flow.h                  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index eeed420ae3..0fcf2093ed 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -58,6 +58,7 @@ New Features
 * **Extended Metering and Marking support in the Flow API.**
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
+  * Added ability to set Color Marker via modify_field Flow API.
 
 * **Updated Wangxun ngbe driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index c0409a5ada..d8e6608bcf 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3713,6 +3713,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_VALUE,		/**< Immediate value. */
 	RTE_FLOW_FIELD_IPV4_ECN,	/**< IPv4 ECN. */
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
+	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
 };
 
 /**
-- 
2.18.2

[PATCH v5 3/7] ethdev: get meter profile/policy objects

[Alexander Kozyrev]

Introduce a new Meter API to retrieve a Meter profile and policy
objects using the profile/policy ID previously created with
meter_profile_add() and meter_policy_create() functions.
That allows to save the pointer and avoid any lookups in the
corresponding lists for quick access during a flow rule creation.
Also, it eliminates the need for CIR, CBS and EBS calculations
and conversion to a PMD-specific format when the profile is used.
Pointers are destroyed and cannot be used after the corresponding
meter_profile_delete() or meter_policy_delete() are called.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 .../traffic_metering_and_policing.rst         |  7 ++++
 doc/guides/rel_notes/release_22_11.rst        |  1 +
 lib/ethdev/rte_flow.h                         | 14 +++++++
 lib/ethdev/rte_mtr.c                          | 41 +++++++++++++++++++
 lib/ethdev/rte_mtr.h                          | 40 ++++++++++++++++++
 lib/ethdev/rte_mtr_driver.h                   | 19 +++++++++
 lib/ethdev/version.map                        |  4 ++
 7 files changed, 126 insertions(+)

diff --git a/doc/guides/prog_guide/traffic_metering_and_policing.rst b/doc/guides/prog_guide/traffic_metering_and_policing.rst
index d1958a023d..2ce3236ad8 100644
--- a/doc/guides/prog_guide/traffic_metering_and_policing.rst
+++ b/doc/guides/prog_guide/traffic_metering_and_policing.rst
@@ -107,6 +107,13 @@ traffic meter and policing library.
      to the list of meter actions (``struct rte_mtr_meter_policy_params::actions``)
      specified per color as show in :numref:`figure_rte_mtr_chaining`.
 
+#. The ``rte_mtr_meter_profile_get()`` and ``rte_mtr_meter_policy_get()``
+   API functions are available for getting the object pointers directly.
+   These pointers allow quick access to profile/policy objects and are
+   required by the ``RTE_FLOW_ACTION_TYPE_METER_MARK`` action.
+   This action may omit the policy definition to providei flexibility
+   to match a color later with the ``RTE_FLOW_ITEM_TYPE_METER_COLOR`` item.
+
 Protocol based input color selection
 ------------------------------------
 
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 0fcf2093ed..47925ef005 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -59,6 +59,7 @@ New Features
 
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
+  * Added Meter API to get a pointer to profile/policy by their ID.
 
 * **Updated Wangxun ngbe driver.**
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index d8e6608bcf..906c28807a 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3822,6 +3822,20 @@ struct rte_flow_action {
  */
 struct rte_flow;
 
+/**
+ * Opaque type for Meter profile object returned by MTR API.
+ *
+ * This handle can be used to create Meter actions instead of profile ID.
+ */
+struct rte_flow_meter_profile;
+
+/**
+ * Opaque type for Meter policy object returned by MTR API.
+ *
+ * This handle can be used to create Meter actions instead of policy ID.
+ */
+struct rte_flow_meter_policy;
+
 /**
  * @warning
  * @b EXPERIMENTAL: this structure may change without prior notice
diff --git a/lib/ethdev/rte_mtr.c b/lib/ethdev/rte_mtr.c
index c460e4f4e0..9e79b744da 100644
--- a/lib/ethdev/rte_mtr.c
+++ b/lib/ethdev/rte_mtr.c
@@ -56,6 +56,25 @@ rte_mtr_ops_get(uint16_t port_id, struct rte_mtr_error *error)
 	ops->func;					\
 })
 
+#define RTE_MTR_HNDL_FUNC(port_id, func)		\
+({							\
+	const struct rte_mtr_ops *ops =			\
+		rte_mtr_ops_get(port_id, error);	\
+	if (ops == NULL)				\
+		return NULL;				\
+							\
+	if (ops->func == NULL) {			\
+		rte_mtr_error_set(error,		\
+			ENOSYS,				\
+			RTE_MTR_ERROR_TYPE_UNSPECIFIED,	\
+			NULL,				\
+			rte_strerror(ENOSYS));		\
+		return NULL;				\
+	}						\
+							\
+	ops->func;					\
+})
+
 /* MTR capabilities get */
 int
 rte_mtr_capabilities_get(uint16_t port_id,
@@ -90,6 +109,17 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 		meter_profile_id, error);
 }
 
+/** MTR meter profile get */
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_profile_get)(dev,
+		meter_profile_id, error);
+}
+
 /* MTR meter policy validate */
 int
 rte_mtr_meter_policy_validate(uint16_t port_id,
@@ -124,6 +154,17 @@ rte_mtr_meter_policy_delete(uint16_t port_id,
 		policy_id, error);
 }
 
+/** MTR meter policy get */
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error)
+{
+	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+	return RTE_MTR_HNDL_FUNC(port_id, meter_policy_get)(dev,
+		policy_id, error);
+}
+
 /** MTR object create */
 int
 rte_mtr_create(uint16_t port_id,
diff --git a/lib/ethdev/rte_mtr.h b/lib/ethdev/rte_mtr.h
index 008bc84f0d..58f0d26215 100644
--- a/lib/ethdev/rte_mtr.h
+++ b/lib/ethdev/rte_mtr.h
@@ -623,6 +623,26 @@ rte_mtr_meter_profile_delete(uint16_t port_id,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter profile object get
+ *
+ * Get meter profile object for a given meter profile ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] meter_profile_id
+ *   Meter profile ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_profile *
+rte_mtr_meter_profile_get(uint16_t port_id,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /**
  * Check whether a meter policy can be created on a given port.
  *
@@ -679,6 +699,26 @@ rte_mtr_meter_policy_add(uint16_t port_id,
 	struct rte_mtr_meter_policy_params *policy,
 	struct rte_mtr_error *error);
 
+/**
+ * Meter policy object get
+ *
+ * Get meter policy object for a given meter policy ID.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] policy_id
+ *   Meter policy ID. Needs to be the valid.
+ * @param[out] error
+ *   Error details. Filled in only on error, when not NULL.
+ * @return
+ *   A valid handle in case of success, NULL otherwise.
+ */
+__rte_experimental
+struct rte_flow_meter_policy *
+rte_mtr_meter_policy_get(uint16_t port_id,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
 /**
  * Define meter policy action list:
  * GREEN - GREEN, YELLOW - YELLOW, RED - RED
diff --git a/lib/ethdev/rte_mtr_driver.h b/lib/ethdev/rte_mtr_driver.h
index f7dca9a54c..948a629b93 100644
--- a/lib/ethdev/rte_mtr_driver.h
+++ b/lib/ethdev/rte_mtr_driver.h
@@ -41,6 +41,12 @@ typedef int (*rte_mtr_meter_profile_delete_t)(struct rte_eth_dev *dev,
 	uint32_t meter_profile_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter profile get. */
+typedef struct rte_flow_meter_profile *
+(*rte_mtr_meter_profile_get_t)(struct rte_eth_dev *dev,
+	uint32_t meter_profile_id,
+	struct rte_mtr_error *error);
+
 /** @internal MTR meter policy validate. */
 typedef int (*rte_mtr_meter_policy_validate_t)(struct rte_eth_dev *dev,
 	struct rte_mtr_meter_policy_params *policy,
@@ -57,6 +63,13 @@ typedef int (*rte_mtr_meter_policy_delete_t)(struct rte_eth_dev *dev,
 	uint32_t policy_id,
 	struct rte_mtr_error *error);
 
+/** @internal MTR meter policy get. */
+typedef struct rte_flow_meter_policy *
+(*rte_mtr_meter_policy_get_t)(struct rte_eth_dev *dev,
+	uint32_t policy_id,
+	struct rte_mtr_error *error);
+
+
 /** @internal MTR object create. */
 typedef int (*rte_mtr_create_t)(struct rte_eth_dev *dev,
 	uint32_t mtr_id,
@@ -194,6 +207,12 @@ struct rte_mtr_ops {
 
 	/** MTR object meter policy update */
 	rte_mtr_meter_policy_update_t meter_policy_update;
+
+	/** MTR meter profile get */
+	rte_mtr_meter_profile_get_t meter_profile_get;
+
+	/** MTR meter policy get */
+	rte_mtr_meter_policy_get_t meter_policy_get;
 };
 
 /**
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index 03f52fee91..1fec250c85 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -285,6 +285,10 @@ EXPERIMENTAL {
 	rte_mtr_color_in_protocol_priority_get;
 	rte_mtr_color_in_protocol_set;
 	rte_mtr_meter_vlan_table_update;
+
+	# added in 22.11
+	rte_mtr_meter_profile_get;
+	rte_mtr_meter_policy_get;
 };
 
 INTERNAL {
-- 
2.18.2

[PATCH v5 4/7] ethdev: add meter color mark flow action

[Alexander Kozyrev]

Create a new Flow API action: METER_MARK.
It Meters a packet stream and marks its packets with colors.
The marking is done on a metadata, not on a packet field.
Unlike the METER action, it performs no policing at all.
A user has the flexibility to create any policies with the help of
the METER_COLOR item later, only meter profile is mandatory here.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     | 25 ++++++++++++
 doc/guides/rel_notes/release_22_11.rst |  1 +
 lib/ethdev/rte_flow.c                  |  1 +
 lib/ethdev/rte_flow.h                  | 56 ++++++++++++++++++++++++++
 4 files changed, 83 insertions(+)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 018def1033..5b87d9f61e 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -3411,6 +3411,31 @@ This action is meant to use the same structure as `Action: PORT_REPRESENTOR`_.
 
 See also `Item: REPRESENTED_PORT`_.
 
+Action: ``METER_MARK``
+^^^^^^^^^^^^^^^^^^^^^^
+
+Meters a packet stream and marks its packets with colors.
+
+Unlike the ``METER`` action, policing is optional and may be
+performed later with the help of the ``METER_COLOR`` item.
+The profile and/or policy objects have to be created
+using the rte_mtr_profile_add()/rte_mtr_policy_add() API.
+Pointers to these objects are used as action parameters
+and need to be retrieved using the rte_mtr_profile_get() API
+and rte_mtr_policy_get() API respectively.
+
+.. _table_rte_flow_action_meter_mark:
+
+.. table:: METER_MARK
+
+   +------------------+----------------------+
+   | Field            | Value                |
+   +==================+======================+
+   | ``profile``      | Meter profile object |
+   +------------------+----------------------+
+   | ``policy``       | Meter policy object  |
+   +------------------+----------------------+
+
 Negative types
 ~~~~~~~~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
index 47925ef005..375af892ac 100644
--- a/doc/guides/rel_notes/release_22_11.rst
+++ b/doc/guides/rel_notes/release_22_11.rst
@@ -60,6 +60,7 @@ New Features
   * Added METER_COLOR item to match Color Marker set by a Meter.
   * Added ability to set Color Marker via modify_field Flow API.
   * Added Meter API to get a pointer to profile/policy by their ID.
+  * Added METER_MARK action for Metering with lockless profile/policy access.
 
 * **Updated Wangxun ngbe driver.**
 
diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
index 99247b599d..7ff024f33e 100644
--- a/lib/ethdev/rte_flow.c
+++ b/lib/ethdev/rte_flow.c
@@ -260,6 +260,7 @@ static const struct rte_flow_desc_data rte_flow_desc_action[] = {
 	MK_FLOW_ACTION(CONNTRACK, sizeof(struct rte_flow_action_conntrack)),
 	MK_FLOW_ACTION(PORT_REPRESENTOR, sizeof(struct rte_flow_action_ethdev)),
 	MK_FLOW_ACTION(REPRESENTED_PORT, sizeof(struct rte_flow_action_ethdev)),
+	MK_FLOW_ACTION(METER_MARK, sizeof(struct rte_flow_action_meter_mark)),
 };
 
 int
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 906c28807a..4531d2f7c5 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -2903,6 +2903,14 @@ enum rte_flow_action_type {
 	 * @see struct rte_flow_action_ethdev
 	 */
 	RTE_FLOW_ACTION_TYPE_REPRESENTED_PORT,
+
+	/**
+	 * Traffic metering and marking (MTR).
+	 *
+	 * @see struct rte_flow_action_meter_mark
+	 * See file rte_mtr.h for MTR profile object configuration.
+	 */
+	RTE_FLOW_ACTION_TYPE_METER_MARK,
 };
 
 /**
@@ -3774,6 +3782,54 @@ struct rte_flow_action_modify_field {
 	uint32_t width; /**< Number of bits to use from a source field. */
 };
 
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_MARK
+ *
+ * Traffic metering and marking (MTR).
+ *
+ * Meters a packet stream and marks its packets either
+ * green, yellow, or red according to the specified profile.
+ * The policy is optional and may be specified for defining
+ * subsequent actions based on a color assigned by MTR.
+ * Alternatively, the METER_COLOR item may be used for this.
+ */
+struct rte_flow_action_meter_mark {
+
+	/**< Profile config retrieved with rte_mtr_profile_get(). */
+	struct rte_flow_meter_profile *profile;
+	/**< Policy config retrieved with rte_mtr_policy_get(). */
+	struct rte_flow_meter_policy *policy;
+	/** Metering mode: 0 - Color-Blind, 1 - Color-Aware. */
+	int color_mode;
+	/** Initial Color applied to packets in Color-Aware mode. */
+	enum rte_color init_color;
+	/** Metering state: 0 - Disabled, 1 - Enabled. */
+	int state;
+};
+
+/**
+ * RTE_FLOW_ACTION_TYPE_METER_MARK
+ *
+ * Wrapper structure for the context update interface.
+ *
+ */
+struct rte_flow_update_meter_mark {
+	/** New meter_mark parameters to be updated. */
+	struct rte_flow_action_meter_mark meter_mark;
+	/** The profile will be updated. */
+	uint32_t profile_valid:1;
+	/** The policy will be updated. */
+	uint32_t policy_valid:1;
+	/** The color mode will be updated. */
+	uint32_t color_mode_valid:1;
+	/** The initial color will be updated. */
+	uint32_t init_color_valid:1;
+	/** The meter state will be updated. */
+	uint32_t state_valid:1;
+	/** Reserved bits for the future usage. */
+	uint32_t reserved:27;
+};
+
 /* Mbuf dynamic field offset for metadata. */
 extern int32_t rte_flow_dynf_metadata_offs;
 
-- 
2.18.2

[PATCH v5 5/7] app/testpmd: add meter color flow matching item

[Alexander Kozyrev]

Add testpmd command line to match on a meter color:
	flow create 0 ingress group 0 pattern meter color is green / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c                 | 83 +++++++++++++++++++++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  4 +
 2 files changed, 87 insertions(+)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index d97be6fe98..57fc30d13b 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -459,6 +459,9 @@ enum index {
 	ITEM_PPP_ADDR,
 	ITEM_PPP_CTRL,
 	ITEM_PPP_PROTO_ID,
+	ITEM_METER,
+	ITEM_METER_COLOR,
+	ITEM_METER_COLOR_NAME,
 
 	/* Validate/create actions. */
 	ACTIONS,
@@ -800,6 +803,10 @@ static const char *const modify_field_ids[] = {
 	"ipv4_ecn", "ipv6_ecn", NULL
 };
 
+static const char *const meter_colors[] = {
+	"green", "yellow", "red", "all", NULL
+};
+
 /** Maximum number of subsequent tokens and arguments on the stack. */
 #define CTX_STACK_SIZE 16
 
@@ -1333,6 +1340,7 @@ static const enum index next_item[] = {
 	ITEM_FLEX,
 	ITEM_L2TPV2,
 	ITEM_PPP,
+	ITEM_METER,
 	END_SET,
 	ZERO,
 };
@@ -1805,6 +1813,12 @@ static const enum index item_ppp[] = {
 	ZERO,
 };
 
+static const enum index item_meter[] = {
+	ITEM_METER_COLOR,
+	ITEM_NEXT,
+	ZERO,
+};
+
 static const enum index next_action[] = {
 	ACTION_END,
 	ACTION_VOID,
@@ -2373,6 +2387,9 @@ static int parse_ia_id2ptr(struct context *ctx, const struct token *token,
 static int parse_mp(struct context *, const struct token *,
 		    const char *, unsigned int,
 		    void *, unsigned int);
+static int parse_meter_color(struct context *ctx, const struct token *token,
+			     const char *str, unsigned int len, void *buf,
+			     unsigned int size);
 static int comp_none(struct context *, const struct token *,
 		     unsigned int, char *, unsigned int);
 static int comp_boolean(struct context *, const struct token *,
@@ -2403,6 +2420,8 @@ static int comp_table_id(struct context *, const struct token *,
 			 unsigned int, char *, unsigned int);
 static int comp_queue_id(struct context *, const struct token *,
 			 unsigned int, char *, unsigned int);
+static int comp_meter_color(struct context *, const struct token *,
+			    unsigned int, char *, unsigned int);
 
 /** Token definitions. */
 static const struct token token_list[] = {
@@ -5065,6 +5084,29 @@ static const struct token token_list[] = {
 		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_ppp,
 					hdr.proto_id)),
 	},
+	[ITEM_METER] = {
+		.name = "meter",
+		.help = "match meter color",
+		.priv = PRIV_ITEM(METER_COLOR,
+				  sizeof(struct rte_flow_item_meter_color)),
+		.next = NEXT(item_meter),
+		.call = parse_vc,
+	},
+	[ITEM_METER_COLOR] = {
+		.name = "color",
+		.help = "meter color",
+		.next = NEXT(item_meter,
+			     NEXT_ENTRY(ITEM_METER_COLOR_NAME),
+			     item_param),
+		.args = ARGS(ARGS_ENTRY(struct rte_flow_item_meter_color,
+					color)),
+	},
+	[ITEM_METER_COLOR_NAME] = {
+		.name = "color_name",
+		.help = "meter color name",
+		.call = parse_meter_color,
+		.comp = comp_meter_color,
+	},
 	/* Validate/create actions. */
 	[ACTIONS] = {
 		.name = "actions",
@@ -9868,6 +9910,30 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	return ret;
 }
 
+/** Parse Meter color name */
+static int
+parse_meter_color(struct context *ctx, const struct token *token,
+		  const char *str, unsigned int len, void *buf,
+		  unsigned int size)
+{
+	struct rte_flow_item_meter_color *meter_color;
+	unsigned int i;
+
+	(void)token;
+	(void)buf;
+	(void)size;
+	for (i = 0; meter_colors[i]; ++i)
+		if (!strcmp_partial(meter_colors[i], str, len))
+			break;
+	if (!meter_colors[i])
+		return -1;
+	if (!ctx->object)
+		return len;
+	meter_color = ctx->object;
+	meter_color->color = (enum rte_color)i;
+	return len;
+}
+
 /** No completion. */
 static int
 comp_none(struct context *ctx, const struct token *token,
@@ -10159,6 +10225,20 @@ comp_queue_id(struct context *ctx, const struct token *token,
 	return i;
 }
 
+/** Complete available Meter colors. */
+static int
+comp_meter_color(struct context *ctx, const struct token *token,
+		 unsigned int ent, char *buf, unsigned int size)
+{
+	RTE_SET_USED(ctx);
+	RTE_SET_USED(token);
+	if (!buf)
+		return RTE_DIM(meter_colors);
+	if (ent < RTE_DIM(meter_colors) - 1)
+		return strlcpy(buf, meter_colors[ent], size);
+	return -1;
+}
+
 /** Internal context. */
 static struct context cmd_flow_context;
 
@@ -10774,6 +10854,9 @@ flow_item_default_mask(const struct rte_flow_item *item)
 	case RTE_FLOW_ITEM_TYPE_PPP:
 		mask = &rte_flow_item_ppp_mask;
 		break;
+	case RTE_FLOW_ITEM_TYPE_METER_COLOR:
+		mask = &rte_flow_item_meter_color_mask;
+		break;
 	default:
 		break;
 	}
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index 5fbec06c35..b586e92649 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -3697,6 +3697,10 @@ This section lists supported pattern items and their attributes, if any.
   - ``ctrl {unsigned}``: PPP control.
   - ``proto_id {unsigned}``: PPP protocol identifier.
 
+- ``meter``: match Meter color.
+
+  - ``color {value}``: Meter color value(green/yellow/red).
+
 Actions list
 ^^^^^^^^^^^^
 
-- 
2.18.2

[PATCH v5 6/7] app/testpmd: allow meter color marker modification

[Alexander Kozyrev]

Extend the list of available modify_field IDs to include
recently added meter color marker item in testpmd CLI.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 57fc30d13b..601df52f38 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -800,7 +800,7 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", NULL
+	"ipv4_ecn", "ipv6_ecn", "meter_color", NULL
 };
 
 static const char *const meter_colors[] = {
-- 
2.18.2

[PATCH v5 7/7] app/testpmd: add meter color mark flow action

[Alexander Kozyrev]

Add testpmd command line to match for METER_MARK action:
	flow create ... actions meter_mark mtr_profile 20 / end

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 app/test-pmd/cmdline_flow.c                 | 153 ++++++++++++++++++++
 app/test-pmd/config.c                       |  45 +++++-
 app/test-pmd/testpmd.h                      |   4 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |   8 +
 4 files changed, 209 insertions(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 601df52f38..417655eca8 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -54,6 +54,7 @@ enum index {
 	COMMON_GROUP_ID,
 	COMMON_PRIORITY_LEVEL,
 	COMMON_INDIRECT_ACTION_ID,
+	COMMON_PROFILE_ID,
 	COMMON_POLICY_ID,
 	COMMON_FLEX_HANDLE,
 	COMMON_FLEX_TOKEN,
@@ -509,6 +510,14 @@ enum index {
 	ACTION_METER_COLOR_YELLOW,
 	ACTION_METER_COLOR_RED,
 	ACTION_METER_ID,
+	ACTION_METER_MARK,
+	ACTION_METER_PROFILE,
+	ACTION_METER_PROFILE_ID2PTR,
+	ACTION_METER_POLICY,
+	ACTION_METER_POLICY_ID2PTR,
+	ACTION_METER_COLOR_MODE,
+	ACTION_METER_INIT_COLOR,
+	ACTION_METER_STATE,
 	ACTION_OF_SET_MPLS_TTL,
 	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
 	ACTION_OF_DEC_MPLS_TTL,
@@ -1836,6 +1845,7 @@ static const enum index next_action[] = {
 	ACTION_PORT_ID,
 	ACTION_METER,
 	ACTION_METER_COLOR,
+	ACTION_METER_MARK,
 	ACTION_OF_SET_MPLS_TTL,
 	ACTION_OF_DEC_MPLS_TTL,
 	ACTION_OF_SET_NW_TTL,
@@ -1952,6 +1962,16 @@ static const enum index action_meter_color[] = {
 	ZERO,
 };
 
+static const enum index action_meter_mark[] = {
+	ACTION_METER_PROFILE,
+	ACTION_METER_POLICY,
+	ACTION_METER_COLOR_MODE,
+	ACTION_METER_INIT_COLOR,
+	ACTION_METER_STATE,
+	ACTION_NEXT,
+	ZERO,
+};
+
 static const enum index action_of_set_mpls_ttl[] = {
 	ACTION_OF_SET_MPLS_TTL_MPLS_TTL,
 	ACTION_NEXT,
@@ -2387,6 +2407,14 @@ static int parse_ia_id2ptr(struct context *ctx, const struct token *token,
 static int parse_mp(struct context *, const struct token *,
 		    const char *, unsigned int,
 		    void *, unsigned int);
+static int parse_meter_profile_id2ptr(struct context *ctx,
+				      const struct token *token,
+				      const char *str, unsigned int len,
+				      void *buf, unsigned int size);
+static int parse_meter_policy_id2ptr(struct context *ctx,
+				     const struct token *token,
+				     const char *str, unsigned int len,
+				     void *buf, unsigned int size);
 static int parse_meter_color(struct context *ctx, const struct token *token,
 			     const char *str, unsigned int len, void *buf,
 			     unsigned int size);
@@ -2551,6 +2579,13 @@ static const struct token token_list[] = {
 		.call = parse_int,
 		.comp = comp_none,
 	},
+	[COMMON_PROFILE_ID] = {
+		.name = "{profile_id}",
+		.type = "PROFILE_ID",
+		.help = "profile id",
+		.call = parse_int,
+		.comp = comp_none,
+	},
 	[COMMON_POLICY_ID] = {
 		.name = "{policy_id}",
 		.type = "POLICY_ID",
@@ -5429,6 +5464,62 @@ static const struct token token_list[] = {
 		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter, mtr_id)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_METER_MARK] = {
+		.name = "meter_mark",
+		.help = "meter the directed packets using profile and policy",
+		.priv = PRIV_ACTION(METER_MARK,
+				    sizeof(struct rte_flow_action_meter_mark)),
+		.next = NEXT(action_meter_mark),
+		.call = parse_vc,
+	},
+	[ACTION_METER_PROFILE] = {
+		.name = "mtr_profile",
+		.help = "meter profile id to use",
+		.next = NEXT(NEXT_ENTRY(ACTION_METER_PROFILE_ID2PTR)),
+		.args = ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
+	},
+	[ACTION_METER_PROFILE_ID2PTR] = {
+		.name = "{mtr_profile_id}",
+		.type = "PROFILE_ID",
+		.help = "meter profile id",
+		.next = NEXT(action_meter_mark),
+		.call = parse_meter_profile_id2ptr,
+		.comp = comp_none,
+	},
+	[ACTION_METER_POLICY] = {
+		.name = "mtr_policy",
+		.help = "meter policy id to use",
+		.next = NEXT(NEXT_ENTRY(ACTION_METER_POLICY_ID2PTR)),
+		ARGS(ARGS_ENTRY_ARB(0, sizeof(uint32_t))),
+	},
+	[ACTION_METER_POLICY_ID2PTR] = {
+		.name = "{mtr_policy_id}",
+		.type = "POLICY_ID",
+		.help = "meter policy id",
+		.next = NEXT(action_meter_mark),
+		.call = parse_meter_policy_id2ptr,
+		.comp = comp_none,
+	},
+	[ACTION_METER_COLOR_MODE] = {
+		.name = "mtr_color_mode",
+		.help = "meter color awareness mode",
+		.next = NEXT(action_meter_mark, NEXT_ENTRY(COMMON_UNSIGNED)),
+		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter_mark, color_mode)),
+		.call = parse_vc_conf,
+	},
+	[ACTION_METER_INIT_COLOR] = {
+		.name = "mtr_init_color",
+		.help = "meter initial color",
+		.next = NEXT(action_meter_mark, NEXT_ENTRY(ITEM_METER_COLOR_NAME)),
+		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter_mark, init_color)),
+	},
+	[ACTION_METER_STATE] = {
+		.name = "mtr_state",
+		.help = "meter state",
+		.next = NEXT(action_meter_mark, NEXT_ENTRY(COMMON_UNSIGNED)),
+		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_meter_mark, state)),
+		.call = parse_vc_conf,
+	},
 	[ACTION_OF_SET_MPLS_TTL] = {
 		.name = "of_set_mpls_ttl",
 		.help = "OpenFlow's OFPAT_SET_MPLS_TTL",
@@ -9748,6 +9839,68 @@ parse_ia_id2ptr(struct context *ctx, const struct token *token,
 	return ret;
 }
 
+static int
+parse_meter_profile_id2ptr(struct context *ctx, const struct token *token,
+		const char *str, unsigned int len,
+		void *buf, unsigned int size)
+{
+	struct rte_flow_action *action = ctx->object;
+	struct rte_flow_action_meter_mark *meter;
+	struct rte_flow_meter_profile *profile = NULL;
+	uint32_t id = 0;
+	int ret;
+
+	(void)buf;
+	(void)size;
+	ctx->objdata = 0;
+	ctx->object = &id;
+	ctx->objmask = NULL;
+	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
+	ctx->object = action;
+	if (ret != (int)len)
+		return ret;
+	/* set meter profile */
+	if (action) {
+		meter = (struct rte_flow_action_meter_mark *)
+			(uintptr_t)(action->conf);
+		profile = port_meter_profile_get_by_id(ctx->port, id);
+		meter->profile = profile;
+		ret = (profile) ? ret : -1;
+	}
+	return ret;
+}
+
+static int
+parse_meter_policy_id2ptr(struct context *ctx, const struct token *token,
+		const char *str, unsigned int len,
+		void *buf, unsigned int size)
+{
+	struct rte_flow_action *action = ctx->object;
+	struct rte_flow_action_meter_mark *meter;
+	struct rte_flow_meter_policy *policy = NULL;
+	uint32_t id = 0;
+	int ret;
+
+	(void)buf;
+	(void)size;
+	ctx->objdata = 0;
+	ctx->object = &id;
+	ctx->objmask = NULL;
+	ret = parse_int(ctx, token, str, len, ctx->object, sizeof(id));
+	ctx->object = action;
+	if (ret != (int)len)
+		return ret;
+	/* set meter policy */
+	if (action) {
+		meter = (struct rte_flow_action_meter_mark *)
+			(uintptr_t)(action->conf);
+		policy = port_meter_policy_get_by_id(ctx->port, id);
+		meter->policy = policy;
+		ret = (policy) ? ret : -1;
+	}
+	return ret;
+}
+
 /** Parse set command, initialize output buffer for subsequent tokens. */
 static int
 parse_set_raw_encap_decap(struct context *ctx, const struct token *token,
diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index 568b0881d4..06d6dc3546 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -2122,6 +2122,29 @@ port_meter_policy_add(portid_t port_id, uint32_t policy_id,
 	return ret;
 }
 
+struct rte_flow_meter_profile *
+port_meter_profile_get_by_id(portid_t port_id, uint32_t id)
+{
+	struct rte_mtr_error error;
+	struct rte_flow_meter_profile *profile;
+
+	profile = rte_mtr_meter_profile_get(port_id, id, &error);
+	if (!profile)
+		print_mtr_err_msg(&error);
+	return profile;
+}
+struct rte_flow_meter_policy *
+port_meter_policy_get_by_id(portid_t port_id, uint32_t id)
+{
+	struct rte_mtr_error error;
+	struct rte_flow_meter_policy *policy;
+
+	policy = rte_mtr_meter_policy_get(port_id, id, &error);
+	if (!policy)
+		print_mtr_err_msg(&error);
+	return policy;
+}
+
 /** Validate flow rule. */
 int
 port_flow_validate(portid_t port_id,
@@ -2709,6 +2732,9 @@ port_queue_action_handle_update(portid_t port_id,
 	struct rte_port *port;
 	struct rte_flow_error error;
 	struct rte_flow_action_handle *action_handle;
+	struct port_indirect_action *pia;
+	struct rte_flow_update_meter_mark mtr_update;
+	const void *update;
 
 	action_handle = port_action_handle_get_by_id(port_id, id);
 	if (!action_handle)
@@ -2720,8 +2746,25 @@ port_queue_action_handle_update(portid_t port_id,
 		return -EINVAL;
 	}
 
+	pia = action_get_by_id(port_id, id);
+	if (!pia)
+		return -EINVAL;
+
+	if (pia->type == RTE_FLOW_ACTION_TYPE_METER_MARK) {
+		rte_memcpy(&mtr_update.meter_mark, action->conf,
+			sizeof(struct rte_flow_action_meter_mark));
+		mtr_update.profile_valid = 1;
+		mtr_update.policy_valid  = 1;
+		mtr_update.color_mode_valid  = 1;
+		mtr_update.init_color_valid  = 1;
+		mtr_update.state_valid  = 1;
+		update = &mtr_update;
+	} else {
+		update = action;
+	}
+
 	if (rte_flow_async_action_handle_update(port_id, queue_id, &attr,
-				    action_handle, action, NULL, &error)) {
+					action_handle, update, NULL, &error)) {
 		return port_flow_complain(&error);
 	}
 	printf("Indirect action #%u update queued\n", id);
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 21c5632aec..e96f563674 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -938,6 +938,10 @@ void port_flow_tunnel_create(portid_t port_id, const struct tunnel_ops *ops);
 int port_flow_isolate(portid_t port_id, int set);
 int port_meter_policy_add(portid_t port_id, uint32_t policy_id,
 		const struct rte_flow_action *actions);
+struct rte_flow_meter_profile *port_meter_profile_get_by_id(portid_t port_id,
+							    uint32_t id);
+struct rte_flow_meter_policy *port_meter_policy_get_by_id(portid_t port_id,
+							  uint32_t id);
 
 void rx_ring_desc_display(portid_t port_id, queueid_t rxq_id, uint16_t rxd_id);
 void tx_ring_desc_display(portid_t port_id, queueid_t txq_id, uint16_t txd_id);
diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
index b586e92649..d1f536b6fe 100644
--- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst
+++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst
@@ -3986,6 +3986,14 @@ This section lists supported actions and their attributes, if any.
 
   - ``ethdev_port_id {unsigned}``: ethdev port ID
 
+- ``meter_mark``:  meter the directed packets using profile and policy
+
+  - ``mtr_profile {unsigned}``: meter profile id to use
+  - ``mtr_policy {unsigned}``: meter policy id to use
+  - ``mtr_color_mode {unsigned}``: meter color-awareness mode (blind/aware)
+  - ``mtr_init_color {value}``: initial color value(green/yellow/red)
+  - ``mtr_state {unsigned}``: meter state (disabled/enabled)
+
 Destroying flow rules
 ~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.18.2

RE: [PATCH v5 0/7] ethdev: separate metering and marking from policing

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Monday, September 26, 2022 3:58 PM
> To: dev@dpdk.org
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> jerinjacobk@gmail.com; orika@nvidia.com; thomas@monjalon.net;
> ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
> ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [PATCH v5 0/7] ethdev: separate metering and marking from
> policing
> 
Hi Alexander,

Thank you for your work!

Series-acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

Regards,
Cristian

RE: [PATCH v5 0/7] ethdev: separate metering and marking from policing

[Ori Kam]

> -----Original Message-----
> From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Sent: Tuesday, 27 September 2022 14:56
> 
> 
> 
> > -----Original Message-----
> > From: Alexander Kozyrev <akozyrev@nvidia.com>
> > Sent: Monday, September 26, 2022 3:58 PM
> > To: dev@dpdk.org
> > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> > jerinjacobk@gmail.com; orika@nvidia.com; thomas@monjalon.net;
> > ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
> > ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
> > <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> > jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Subject: [PATCH v5 0/7] ethdev: separate metering and marking from
> > policing
> >
> Hi Alexander,
> 
> Thank you for your work!
> 
> Series-acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> 
> Regards,
> Cristian

Series-acked-by:  Ori Kam <orika@nvidia.com>
Thanks,
Ori 

Re: [PATCH v5 0/7] ethdev: separate metering and marking from policing

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 1239 bytes --]

On Tue, Sep 27, 2022 at 11:45 PM Ori Kam <orika@nvidia.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> > Sent: Tuesday, 27 September 2022 14:56
> >
> >
> >
> > > -----Original Message-----
> > > From: Alexander Kozyrev <akozyrev@nvidia.com>
> > > Sent: Monday, September 26, 2022 3:58 PM
> > > To: dev@dpdk.org
> > > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> > > jerinjacobk@gmail.com; orika@nvidia.com; thomas@monjalon.net;
> > > ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
> > > ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
> > > <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> > > jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
> > > <bruce.richardson@intel.com>
> > > Subject: [PATCH v5 0/7] ethdev: separate metering and marking from
> > > policing
> > >
> > Hi Alexander,
> >
> > Thank you for your work!
> >
> > Series-acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
> >
> > Regards,
> > Cristian
>
> Series-acked-by:  Ori Kam <orika@nvidia.com>
Took a while for me to get back to this. This is good.

Series-acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

> Thanks,
> Ori

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4218 bytes --]

Re: [PATCH v5 0/7] ethdev: separate metering and marking from policing

[Andrew Rybchenko]

On 9/28/22 19:38, Ajit Khaparde wrote:
> On Tue, Sep 27, 2022 at 11:45 PM Ori Kam <orika@nvidia.com> wrote:
>>
>>
>>
>>> -----Original Message-----
>>> From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
>>> Sent: Tuesday, 27 September 2022 14:56
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Alexander Kozyrev <akozyrev@nvidia.com>
>>>> Sent: Monday, September 26, 2022 3:58 PM
>>>> To: dev@dpdk.org
>>>> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
>>>> jerinjacobk@gmail.com; orika@nvidia.com; thomas@monjalon.net;
>>>> ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
>>>> ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
>>>> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
>>>> jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
>>>> <bruce.richardson@intel.com>
>>>> Subject: [PATCH v5 0/7] ethdev: separate metering and marking from
>>>> policing
>>>>
>>> Hi Alexander,
>>>
>>> Thank you for your work!
>>>
>>> Series-acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
>>>
>>> Regards,
>>> Cristian
>>
>> Series-acked-by:  Ori Kam <orika@nvidia.com>
> Took a while for me to get back to this. This is good.
> 
> Series-acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>

Series-acked-by: Andrew Rybchenko <andrew.rybchenko@oktetalbs.ru>

Applied to dpdk-next-net/main, thanks.

RE: [PATCH v5 0/7] ethdev: separate metering and marking from policing

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Monday, September 26, 2022 7:48 PM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; dev@dpdk.org
> Cc: jerinjacobk@gmail.com; Ori Kam <orika@nvidia.com>; NBU-Contact-
> Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> ivan.malov@oktetlabs.ru; andrew.rybchenko@oktetlabs.ru;
> ferruh.yigit@xilinx.com; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> jerinj@marvell.com; ajit.khaparde@broadcom.com; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: RE: [PATCH v5 0/7] ethdev: separate metering and marking from
> policing
> 
> > Hi Alexander,
> >
> > Thanks very much for your pseudo-code detailing the updated meter &
> flow
> > operation!
> >
> > Should we setup another call sometime next week to go through it? I still
> see
> > a few fuzzy things where we need to level set.
> >
> > Regards,
> > Cristian
> 
> Hi Christian, do you still have questions/concerns about the proposed
> changes?
> Do we need another call to sort them out or you got the idea of the overall
> design?

Thanks, Alexander, no questions at this time, I just acked your V5 series.

Regards,
Cristian

RE: [PATCH v5 0/7] ethdev: separate metering and marking from policing

[Alexander Kozyrev]

> Hi Alexander,
> 
> Thanks very much for your pseudo-code detailing the updated meter & flow
> operation!
> 
> Should we setup another call sometime next week to go through it? I still see
> a few fuzzy things where we need to level set.
> 
> Regards,
> Cristian

Hi Christian, do you still have questions/concerns about the proposed changes?
Do we need another call to sort them out or you got the idea of the overall design?