RE: [RFC] ethdev: datapath-focused meter actions, continue

RE: [RFC] ethdev: datapath-focused meter actions, continue

[Dumitrescu, Cristian]

Hi folks,

Here are a few takeaways of mine, with a few actions for Alex and Ori:

1. Problem with pre-definition of meter profiles and policies: In the case of the data plane adding flow rules to itself  (as opposed to the control plane adding flow rules to the data plane), the API requirement of pre-defining the profile and the policy results in a performance problem. This is because the driver needs to acquire the lock to the list of profiles/policies and to search for the profile/policy ID within the profile/policy list. Therefore, to avoid this issue, the API needs to allow passing the profile parameters (i.e. the full profile) and the policy parameters (i.e. the full policy) to the meter object creation/configuration.

2. Add the packet color to the list of possible match items: This would make the meter policy specification redundant, as the packet color that is set by a meter object could be used as match field into a subsequent table (group) to apply the actions that currently have to be specified as part of the meter policy. Therefore, in this case, passing the policy to the meter object creation/configuration is redundant, and therefore the API should accept passing a NULL policy as well as a valid policy.

3. The flow METER action needs to continue to imply the implicit action of setting the packet color (internal meta-data).

4. Translation of meter profile (i.e. srTCM/trTCM rates and bucket sizes) to the implementation-dependent low-level values: This is still required, as it is taking significant cycles, which is a problem when the data plane is adding flow rules to itself. What should be avoided is the registration of the output of the translation as a pre-defined profile ID.

5. Proposal to create/configure a meter object as part of an rte_flow action. Question to be addressed by Alexander Kozyrev and Ori Kam: How do we get an ID for the meter object for the purpose of (1) sharing the same meter object with other flows; (2) Freeing up the meter object after the flow is removed (in fact, after all the flows sharing the same meter object are removed).

6. Alexander Kozyrev to provide pseudo-code for the meter operation with the new proposal:
	(1) meter creation;
	(2) meter sharing;
	(3) meter reconfiguration: do we need to remove the flow/flows using the meter and re-add them with a new meter object that has updated configuration, or can we update the meter object itself (what API?);
	(4) meter free.

Regards,
Cristian

> -----Original Appointment-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Friday, May 13, 2022 10:37 PM
> To: Alexander Kozyrev; Ori Kam; Jerin Jacob; Dumitrescu, Cristian; NBU-
> Contact-Thomas Monjalon (EXTERNAL); Andrew Rybchenko;
> Vipin.Varghese@amd.com; Ajit Khaparde; Ferruh Yigit
> Cc: Ray Kinsella; Sunil Kumar Kori; Ivan Malov; Awal, Mohammad Abdul; Zhang,
> Qi Z; Richardson, Bruce; Konstantin Ananyev; Singh, Jasvinder; dev@dpdk.org
> Subject: [RFC] ethdev: datapath-focused meter actions, continue
> When: 19 May 2022 15:00-16:00 (UTC) Coordinated Universal Time.
> Where: https://meet.jit.si/DPDK
> 
> Agenda: continue discussion about proposed improvements to Flow API in
> regards to Meter handling (slides attached).

RE: [RFC] ethdev: datapath-focused meter actions, continue

[Alexander Kozyrev]

On Thursday, May 19, 2022 13:35 Dumitrescu, Cristian <cristian.dumitrescu@intel.com>:
> Here are a few takeaways of mine, with a few actions for Alex and Ori:

Thank you, Cristian, for compiling these takeaways. Great summary of the discussion.
 
> 6. Alexander Kozyrev to provide pseudo-code for the meter operation with
> the new proposal:
> 	(1) meter creation;
> 	(2) meter sharing;
> 	(3) meter reconfiguration: do we need to remove the flow/flows
> using the meter and re-add them with a new meter object that has updated
> configuration, or can we update the meter object itself (what API?);
> 	(4) meter free.

Traditional Meter Usage:

profile_id = rte_mtr_meter_profile_add(RFC_params);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
meter_id = rte_mtr_create(profile_id, policy_id);
rte_flow_create(pattern=5-tuple,actions=METER(meter_id));

The METER action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.
3. Jump to a policy group.
4. Match on a color.
5. Execute assigned policy actions for the color.

New Meter Usage Model:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
rte_flow_create(pattern=5-tuple,
		actions=METER_MARK(profile_obj_ptr),JUMP);
rte_flow_create(pattern=COLOR, actions=...);

The METER_MARK action effectively translates to the following:
1. Metering a packet stream.
2. Marking packets with an appropriate color.

A user is able to match the color later with the COLOR item.
In order to do this we add the JUMP action after the METER action.

3. Jump to a policy group.
4. Match on a color.
5. Execute actions for the color.

Here we decoupled the meter profile usage from the meter policy usage
for greater flexibility and got rid of any locks related to meter_id lookup.

Another example of the meter creation to mimic the old model entirely:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
*policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
rte_flow_create(pattern=5-tuple,
		actions= METER_MARK(profile_obj_ptr, policy_obj_ptr));

In this case, we define the policy actions right away.
The main advantage is not having to lookup for profile_id/policy_id.

To free the meter obects we need to do the following:
rte_flow_destroy(flow_handle);
rte_mtr_meter_policy_delete(policy_id);
rte_mtr_meter_profile_delete(profile_id);.
profile_obj_ptr and policy_obj_ptr are no longer valid after that.

The meter profile configuration cannot be updated dynamically
with the current set of patches, but can be supported later on.
Now you have to destroy flows and profiles and recreate them.
But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
can have the corresponding siblings without mtr_id parameters.
In this case, we can update the config and all the flows using them.

The meter sharing is done via the indirect action Flow API:
profile_id = rte_mtr_meter_profile_add(RFC_params);
*profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
handle = rte_flow_action_handle_create(action= METER_MARK(profile_obj_ptr, NULL));
flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));

Once we are done with the flow rules we can free everything.
rte_flow_destroy(flow1);
rte_flow_destroy(flow2);
rte_flow_action_handle_destroy(handle);
rte_mtr_meter_profile_delete(profile_id);

Re: [RFC] ethdev: datapath-focused meter actions, continue

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 4170 bytes --]

On Sat, May 21, 2022 at 7:49 PM Alexander Kozyrev <akozyrev@nvidia.com> wrote:
>
> On Thursday, May 19, 2022 13:35 Dumitrescu, Cristian <cristian.dumitrescu@intel.com>:
> > Here are a few takeaways of mine, with a few actions for Alex and Ori:
>
> Thank you, Cristian, for compiling these takeaways. Great summary of the discussion.
Agree.

>
> > 6. Alexander Kozyrev to provide pseudo-code for the meter operation with
> > the new proposal:

I was wondering how the PMD and application can negotiate whether
to use the old model or the new model.
Maybe the application can determine if the PMD supports the traditional model or
the new model by checking rte_mtr_meter_profile_get or rte_mtr_meter_policy_get?
Or use something else?


> >       (1) meter creation;
> >       (2) meter sharing;
> >       (3) meter reconfiguration: do we need to remove the flow/flows
> > using the meter and re-add them with a new meter object that has updated
> > configuration, or can we update the meter object itself (what API?);
> >       (4) meter free.
>
> Traditional Meter Usage:
>
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> meter_id = rte_mtr_create(profile_id, policy_id);
> rte_flow_create(pattern=5-tuple,actions=METER(meter_id));
>
> The METER action effectively translates to the following:
> 1. Metering a packet stream.
> 2. Marking packets with an appropriate color.
> 3. Jump to a policy group.
> 4. Match on a color.
> 5. Execute assigned policy actions for the color.
>
> New Meter Usage Model:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> rte_flow_create(pattern=5-tuple,
>                 actions=METER_MARK(profile_obj_ptr),JUMP);
> rte_flow_create(pattern=COLOR, actions=...);
>
> The METER_MARK action effectively translates to the following:
> 1. Metering a packet stream.
> 2. Marking packets with an appropriate color.
>
> A user is able to match the color later with the COLOR item.
> In order to do this we add the JUMP action after the METER action.
>
> 3. Jump to a policy group.
> 4. Match on a color.
> 5. Execute actions for the color.
>
> Here we decoupled the meter profile usage from the meter policy usage
> for greater flexibility and got rid of any locks related to meter_id lookup.
>
> Another example of the meter creation to mimic the old model entirely:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> *policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
> rte_flow_create(pattern=5-tuple,
>                 actions= METER_MARK(profile_obj_ptr, policy_obj_ptr));
>
>
> In this case, we define the policy actions right away.
> The main advantage is not having to lookup for profile_id/policy_id.
>
> To free the meter obects we need to do the following:
> rte_flow_destroy(flow_handle);
> rte_mtr_meter_policy_delete(policy_id);
> rte_mtr_meter_profile_delete(profile_id);.
> profile_obj_ptr and policy_obj_ptr are no longer valid after that.
>
> The meter profile configuration cannot be updated dynamically
> with the current set of patches, but can be supported later on.
> Now you have to destroy flows and profiles and recreate them.
> But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
> can have the corresponding siblings without mtr_id parameters.
> In this case, we can update the config and all the flows using them.
>
> The meter sharing is done via the indirect action Flow API:
> profile_id = rte_mtr_meter_profile_add(RFC_params);
> *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> handle = rte_flow_action_handle_create(action= METER_MARK(profile_obj_ptr, NULL));
> flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
> flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));
>
> Once we are done with the flow rules we can free everything.
> rte_flow_destroy(flow1);
> rte_flow_destroy(flow2);
> rte_flow_action_handle_destroy(handle);
> rte_mtr_meter_profile_delete(profile_id);

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4218 bytes --]

RE: [RFC] ethdev: datapath-focused meter actions, continue

[Ori Kam]

> -----Original Message-----
> From: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Sent: Sunday, May 22, 2022 8:01 AM
> Subject: Re: [RFC] ethdev: datapath-focused meter actions, continue
> 
> On Sat, May 21, 2022 at 7:49 PM Alexander Kozyrev <akozyrev@nvidia.com> wrote:
> >
> > On Thursday, May 19, 2022 13:35 Dumitrescu, Cristian <cristian.dumitrescu@intel.com>:
> > > Here are a few takeaways of mine, with a few actions for Alex and Ori:
> >
> > Thank you, Cristian, for compiling these takeaways. Great summary of the discussion.
> Agree.
> 
+1

> >
> > > 6. Alexander Kozyrev to provide pseudo-code for the meter operation with
> > > the new proposal:
> 
> I was wondering how the PMD and application can negotiate whether
> to use the old model or the new model.
> Maybe the application can determine if the PMD supports the traditional model or
> the new model by checking rte_mtr_meter_profile_get or rte_mtr_meter_policy_get?
> Or use something else?
> 
Good idea, and it can know if the new API is not supported, like any other rte_flow API validate it.

> 
> > >       (1) meter creation;
> > >       (2) meter sharing;
> > >       (3) meter reconfiguration: do we need to remove the flow/flows
> > > using the meter and re-add them with a new meter object that has updated
> > > configuration, or can we update the meter object itself (what API?);
> > >       (4) meter free.
> >
> > Traditional Meter Usage:
> >
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> > meter_id = rte_mtr_create(profile_id, policy_id);
> > rte_flow_create(pattern=5-tuple,actions=METER(meter_id));
> >
> > The METER action effectively translates to the following:
> > 1. Metering a packet stream.
> > 2. Marking packets with an appropriate color.
> > 3. Jump to a policy group.
> > 4. Match on a color.
> > 5. Execute assigned policy actions for the color.
> >
> > New Meter Usage Model:
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> > rte_flow_create(pattern=5-tuple,
> >                 actions=METER_MARK(profile_obj_ptr),JUMP);
> > rte_flow_create(pattern=COLOR, actions=...);
> >
> > The METER_MARK action effectively translates to the following:
> > 1. Metering a packet stream.
> > 2. Marking packets with an appropriate color.
> >
> > A user is able to match the color later with the COLOR item.
> > In order to do this we add the JUMP action after the METER action.
> >
> > 3. Jump to a policy group.
> > 4. Match on a color.
> > 5. Execute actions for the color.
> >
> > Here we decoupled the meter profile usage from the meter policy usage
> > for greater flexibility and got rid of any locks related to meter_id lookup.
> >
> > Another example of the meter creation to mimic the old model entirely:
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> > policy_id = rte_mtr_meter_policy_add(actions[RED/YELLOW/GREEN]);
> > *policy_obj_ptr = rte_mtr_meter_policy_get(policy_id);
> > rte_flow_create(pattern=5-tuple,
> >                 actions= METER_MARK(profile_obj_ptr, policy_obj_ptr));
> >
> >
> > In this case, we define the policy actions right away.
> > The main advantage is not having to lookup for profile_id/policy_id.
> >
> > To free the meter obects we need to do the following:
> > rte_flow_destroy(flow_handle);
> > rte_mtr_meter_policy_delete(policy_id);
> > rte_mtr_meter_profile_delete(profile_id);.
> > profile_obj_ptr and policy_obj_ptr are no longer valid after that.
> >
> > The meter profile configuration cannot be updated dynamically
> > with the current set of patches, but can be supported later on.
> > Now you have to destroy flows and profiles and recreate them.
> > But rte_mtr_meter_profile_update()/rte_mtr_meter_policy_update()
> > can have the corresponding siblings without mtr_id parameters.
> > In this case, we can update the config and all the flows using them.
> >
> > The meter sharing is done via the indirect action Flow API:
> > profile_id = rte_mtr_meter_profile_add(RFC_params);
> > *profile_obj_ptr = rte_mtr_meter_profile_get(profile_id);
> > handle = rte_flow_action_handle_create(action= METER_MARK(profile_obj_ptr, NULL));
> > flow1 = rte_flow_create(pattern=5-tuple-1, actions=INDIRECT(handle));
> > flow2 = rte_flow_create(pattern=5-tuple-2, actions=INDIRECT(handle));
> >
> > Once we are done with the flow rules we can free everything.
> > rte_flow_destroy(flow1);
> > rte_flow_destroy(flow2);
> > rte_flow_action_handle_destroy(handle);
> > rte_mtr_meter_profile_delete(profile_id);

[RFC] ethdev: datapath-focused meter actions, continue

[Alexander Kozyrev]

[-- Attachment #1: Type: text/plain, Size: 117 bytes --]

Agenda: continue discussion about proposed improvements to Flow API in regards to Meter handling (slides attached).

[-- Attachment #2: Type: text/calendar, Size: 3543 bytes --]

BEGIN:VCALENDAR
METHOD:REQUEST
PRODID:Microsoft Exchange Server 2010
VERSION:2.0
BEGIN:VTIMEZONE
TZID:UTC
BEGIN:STANDARD
DTSTART:16010101T000000
TZOFFSETFROM:+0000
TZOFFSETTO:+0000
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T000000
TZOFFSETFROM:+0000
TZOFFSETTO:+0000
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ORGANIZER;CN=Alexander Kozyrev:mailto:akozyrev@nvidia.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Ori Kam:ma
 ilto:orika@nvidia.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Jerin Jaco
 b:mailto:jerinj@marvell.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Cristian D
 umitrescu:mailto:cristian.dumitrescu@intel.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=NBU-Contac
 t-Thomas Monjalon (EXTERNAL):mailto:thomas@monjalon.net
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Andrew Ryb
 chenko:mailto:andrew.rybchenko@oktetlabs.ru
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Vipin.Varg
 hese@amd.com:mailto:Vipin.Varghese@amd.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Ajit Khapa
 rde:mailto:ajit.khaparde@broadcom.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Ferruh Yig
 it:mailto:ferruh.yigit@xilinx.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Ray Kinsel
 la:mailto:mdr@ashroe.eu
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Sunil Kuma
 r Kori:mailto:skori@marvell.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Ivan Malov
 :mailto:ivan.malov@oktetlabs.ru
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="Awal, Moha
 mmad Abdul":mailto:mohammad.abdul.awal@intel.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="Zhang, Qi 
 Z":mailto:qi.z.zhang@intel.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="Richardson
 , Bruce":mailto:bruce.richardson@intel.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Konstantin
  Ananyev:mailto:konstantin.ananyev@intel.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="Singh, Jas
 vinder":mailto:jasvinder.singh@intel.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=dev@dpdk.o
 rg:mailto:dev@dpdk.org
DESCRIPTION;LANGUAGE=en-US:Agenda: continue discussion about proposed impro
 vements to Flow API in regards to Meter handling (slides attached).\n
UID:040000008200E00074C5B7101A82E00800000000A0F44E41EF66D801000000000000000
 01000000036E44E661B98544FB074243B872E93EB
SUMMARY;LANGUAGE=en-US:[RFC] ethdev: datapath-focused meter actions\, conti
 nue
DTSTART;TZID=UTC:20220519T150000
DTEND;TZID=UTC:20220519T160000
CLASS:PUBLIC
PRIORITY:5
DTSTAMP:20220516T181455Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:1
LOCATION;LANGUAGE=en-US:https://meet.jit.si/DPDK
X-MICROSOFT-CDO-APPT-SEQUENCE:1
X-MICROSOFT-CDO-OWNERAPPTID:-1072867354
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
X-MICROSOFT-DONOTFORWARDMEETING:FALSE
X-MICROSOFT-DISALLOW-COUNTER:FALSE
X-MICROSOFT-LOCATIONS:[ { "DisplayName" : "https://meet.jit.si/DPDK"\, "Loc
 ationAnnotation" : ""\, "LocationSource" : 0\, "Unresolved" : true\, "Loca
 tionUri" : "" } ]
BEGIN:VALARM
DESCRIPTION:REMINDER
TRIGGER;RELATED=START:-PT15M
ACTION:DISPLAY
END:VALARM
END:VEVENT
END:VCALENDAR

Re: [RFC] ethdev: datapath-focused meter actions, continue

[Andrew Rybchenko]

Hi all,

I'm sorry, I'm not sure that I can take part in tomorrow meeting, so, 
I'd like to drop my thoughts on the topic via E-mail.

Existing "meter" object which pulls profile and policy together allows 
do apply metering in one flow-based lookup for different flows.
I.e. we can route absolutely different flows to one meter object to 
share metering counters. When we know meter ID for a flow, everything 
becomes simple - just get corresponding metering counters, apply it and 
do actions based on color. Yes, it is not flexible, but very simple. As 
I understand the configuration model enforces to define actions for all 
colors.

A new model, if I'm not mistaken, will require three flow-based lookups:
  1. To assign a TAG based on flow fields (to handle different flows in 
one meter)
  2. To do metering for packets with a TAG
  3. To find actions based on color
Of course, (2) and (3) are done in existing model with meter ID, but 
here it is a generic flow-based lookups with extra matching criteria. 
Yes, it is true that it gives extra flexibility, but everything has its 
price.

Theoretically old model could be expressed using new one (and, 
therefore, supported on old HW), but it is a bit tricky and raises many 
questions on how to handle it correctly in all cases. E.g. if a TAG is 
the only pattern in non-zero table and used for meter+jump actions only, 
it could be associated with meter ID.
Above jump table specified after meter action could be associated with a 
policy ID. If action for a color is not specified in a table, it should 
be drop by default.

Indirect actions or action templates could help to do meter profile job 
- define profile in single place.

To sum up, since some HW could support the flexibility provided by 
suggested flow API items/actions. I see no reason to block it. Solution 
looks good from flow API design point of view.

May be I'm missing something since I'm not expert in QoS and have no 
hands-on experience with meters in DPDK.

Andrew.

RE: [RFC] ethdev: datapath-focused meter actions, continue

[Alexander Kozyrev]

On Wed, May 18, 2022 12:51 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>:
> I'm sorry, I'm not sure that I can take part in tomorrow meeting, so,
> I'd like to drop my thoughts on the topic via E-mail.

Thank you for taking some time and reviewing this RFC.

> Existing "meter" object which pulls profile and policy together allows
> do apply metering in one flow-based lookup for different flows.
> I.e. we can route absolutely different flows to one meter object to
> share metering counters. When we know meter ID for a flow, everything
> becomes simple - just get corresponding metering counters, apply it and
> do actions based on color. Yes, it is not flexible, but very simple. As
> I understand the configuration model enforces to define actions for all
> colors.

Yes, and what I propose is the flexible version of Meters where both
profiles and policies can be used separately. But flexibility comes with
a price of taking care of both of them separately as well, of course.

> A new model, if I'm not mistaken, will require three flow-based lookups:
>   1. To assign a TAG based on flow fields (to handle different flows in
> one meter)
>   2. To do metering for packets with a TAG
>   3. To find actions based on color
> Of course, (2) and (3) are done in existing model with meter ID, but
> here it is a generic flow-based lookups with extra matching criteria.
> Yes, it is true that it gives extra flexibility, but everything has its
> price.

We don't need to assign a TAG. I used the TAG as example on how we can
combine color matching with any other item matching.  Model stays the same.
You still do color marking with a meter and find actions based on a color.

> Theoretically old model could be expressed using new one (and,
> therefore, supported on old HW), but it is a bit tricky and raises many
> questions on how to handle it correctly in all cases. E.g. if a TAG is
> the only pattern in non-zero table and used for meter+jump actions only,
> it could be associated with meter ID.
> Above jump table specified after meter action could be associated with a
> policy ID. If action for a color is not specified in a table, it should
> be drop by default.

Yes, old model can be expressed via new API and new model can be
simulated with the old API. Efficiency and performance is the key.

> Indirect actions or action templates could help to do meter profile job
> - define profile in single place.

True, meter color marking may be used for meter sharing, for example.

> To sum up, since some HW could support the flexibility provided by
> suggested flow API items/actions. I see no reason to block it. Solution
> looks good from flow API design point of view.

Thank you.

> May be I'm missing something since I'm not expert in QoS and have no
> hands-on experience with meters in DPDK.
> 
> Andrew.

Re: [RFC] ethdev: datapath-focused meter actions, continue

[Andrew Rybchenko]

On 5/19/22 05:17, Alexander Kozyrev wrote:
> On Wed, May 18, 2022 12:51 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>:
>> I'm sorry, I'm not sure that I can take part in tomorrow meeting, so,
>> I'd like to drop my thoughts on the topic via E-mail.
> 
> Thank you for taking some time and reviewing this RFC.
> 
>> Existing "meter" object which pulls profile and policy together allows
>> do apply metering in one flow-based lookup for different flows.
>> I.e. we can route absolutely different flows to one meter object to
>> share metering counters. When we know meter ID for a flow, everything
>> becomes simple - just get corresponding metering counters, apply it and
>> do actions based on color. Yes, it is not flexible, but very simple. As
>> I understand the configuration model enforces to define actions for all
>> colors.
> 
> Yes, and what I propose is the flexible version of Meters where both
> profiles and policies can be used separately. But flexibility comes with
> a price of taking care of both of them separately as well, of course.
> 
>> A new model, if I'm not mistaken, will require three flow-based lookups:
>>    1. To assign a TAG based on flow fields (to handle different flows in
>> one meter)
>>    2. To do metering for packets with a TAG
>>    3. To find actions based on color
>> Of course, (2) and (3) are done in existing model with meter ID, but
>> here it is a generic flow-based lookups with extra matching criteria.
>> Yes, it is true that it gives extra flexibility, but everything has its
>> price.
> 
> We don't need to assign a TAG. I used the TAG as example on how we can
> combine color matching with any other item matching.  Model stays the same.
> You still do color marking with a meter and find actions based on a color.

I general yes, but I've tried to highlight the case when the
same meter should be applied to different flows. If so, you
need a way to combine. Old way allows to do it in a single step - just 
specify meter ID. New way needs extra rule, for example,
using TAG or MARK.

> 
>> Theoretically old model could be expressed using new one (and,
>> therefore, supported on old HW), but it is a bit tricky and raises many
>> questions on how to handle it correctly in all cases. E.g. if a TAG is
>> the only pattern in non-zero table and used for meter+jump actions only,
>> it could be associated with meter ID.
>> Above jump table specified after meter action could be associated with a
>> policy ID. If action for a color is not specified in a table, it should
>> be drop by default.
> 
> Yes, old model can be expressed via new API and new model can be
> simulated with the old API. Efficiency and performance is the key.
> 
>> Indirect actions or action templates could help to do meter profile job
>> - define profile in single place.
> 
> True, meter color marking may be used for meter sharing, for example.
> 
>> To sum up, since some HW could support the flexibility provided by
>> suggested flow API items/actions. I see no reason to block it. Solution
>> looks good from flow API design point of view.
> 
> Thank you.
> 
>> May be I'm missing something since I'm not expert in QoS and have no
>> hands-on experience with meters in DPDK.
>>
>> Andrew.
> 

RE: [RFC] ethdev: datapath-focused meter actions, continue

[Ori Kam]

Hi Andrew,

> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Thursday, May 19, 2022 9:45 AM
> Subject: Re: [RFC] ethdev: datapath-focused meter actions, continue
> 
> On 5/19/22 05:17, Alexander Kozyrev wrote:
> > On Wed, May 18, 2022 12:51 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>:
> >> I'm sorry, I'm not sure that I can take part in tomorrow meeting, so,
> >> I'd like to drop my thoughts on the topic via E-mail.
> >
> > Thank you for taking some time and reviewing this RFC.
> >
> >> Existing "meter" object which pulls profile and policy together allows
> >> do apply metering in one flow-based lookup for different flows.
> >> I.e. we can route absolutely different flows to one meter object to
> >> share metering counters. When we know meter ID for a flow, everything
> >> becomes simple - just get corresponding metering counters, apply it and
> >> do actions based on color. Yes, it is not flexible, but very simple. As
> >> I understand the configuration model enforces to define actions for all
> >> colors.
> >
> > Yes, and what I propose is the flexible version of Meters where both
> > profiles and policies can be used separately. But flexibility comes with
> > a price of taking care of both of them separately as well, of course.
> >
> >> A new model, if I'm not mistaken, will require three flow-based lookups:
> >>    1. To assign a TAG based on flow fields (to handle different flows in
> >> one meter)
> >>    2. To do metering for packets with a TAG
> >>    3. To find actions based on color
> >> Of course, (2) and (3) are done in existing model with meter ID, but
> >> here it is a generic flow-based lookups with extra matching criteria.
> >> Yes, it is true that it gives extra flexibility, but everything has its
> >> price.
> >
> > We don't need to assign a TAG. I used the TAG as example on how we can
> > combine color matching with any other item matching.  Model stays the same.
> > You still do color marking with a meter and find actions based on a color.
> 
> I general yes, but I've tried to highlight the case when the
> same meter should be applied to different flows. If so, you
> need a way to combine. Old way allows to do it in a single step - just
> specify meter ID. New way needs extra rule, for example,
> using TAG or MARK.
> 
If you want the same meter to be used by different rules, you can just
use the action handle (indirect action).
The tag is used in order to demux the rules after the color matching.

Example for such use case:
You have a share meter for a user, The user has number of connections,
(Youtube, web, game)
So the first thing the application does it checks the user meter
Group 0:
Match (user youtube 5 tuple) actions shared_meter set_tag(youtube) jump to group 1
Match (user web 5 tuple) actions shared_meter set_tag(web) jump to group 1
Match (user game 5 tuple) actions shared_meter set_tag(game) jump to group 1

Group 1:  (this is what we currently call profile,)
Match color == green jump group 2
Match color == red drop

Group 2:
Match tag == youtube actions youtube related actions
Match tag == web actions web related actions
Match tag == game actions game related actions

Using the new API you can combine Group 1 and 2 (group 2 will no be needed anymore)
Group 1':
Match tag == youtube and color == green  actions youtube related actions
Match tag == weband color == green actions web related actions
Match tag == game and color == green actions game related actions
Match color == red drop

As you can see, to implement the logic in current API we must use 3 groups while,
using the suggested APi we can just use 2 groups.


> >
> >> Theoretically old model could be expressed using new one (and,
> >> therefore, supported on old HW), but it is a bit tricky and raises many
> >> questions on how to handle it correctly in all cases. E.g. if a TAG is
> >> the only pattern in non-zero table and used for meter+jump actions only,
> >> it could be associated with meter ID.
> >> Above jump table specified after meter action could be associated with a
> >> policy ID. If action for a color is not specified in a table, it should
> >> be drop by default.
> >
> > Yes, old model can be expressed via new API and new model can be
> > simulated with the old API. Efficiency and performance is the key.
> >
> >> Indirect actions or action templates could help to do meter profile job
> >> - define profile in single place.
> >
> > True, meter color marking may be used for meter sharing, for example.
> >
> >> To sum up, since some HW could support the flexibility provided by
> >> suggested flow API items/actions. I see no reason to block it. Solution
> >> looks good from flow API design point of view.
> >
> > Thank you.
> >
> >> May be I'm missing something since I'm not expert in QoS and have no
> >> hands-on experience with meters in DPDK.
> >>
> >> Andrew.
> >

Re: [RFC] ethdev: datapath-focused meter actions, continue

[Andrew Rybchenko]

Hi Ori,

On 5/19/22 11:41, Ori Kam wrote:
> Hi Andrew,
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Sent: Thursday, May 19, 2022 9:45 AM
>> Subject: Re: [RFC] ethdev: datapath-focused meter actions, continue
>>
>> On 5/19/22 05:17, Alexander Kozyrev wrote:
>>> On Wed, May 18, 2022 12:51 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>:
>>>> I'm sorry, I'm not sure that I can take part in tomorrow meeting, so,
>>>> I'd like to drop my thoughts on the topic via E-mail.
>>>
>>> Thank you for taking some time and reviewing this RFC.
>>>
>>>> Existing "meter" object which pulls profile and policy together allows
>>>> do apply metering in one flow-based lookup for different flows.
>>>> I.e. we can route absolutely different flows to one meter object to
>>>> share metering counters. When we know meter ID for a flow, everything
>>>> becomes simple - just get corresponding metering counters, apply it and
>>>> do actions based on color. Yes, it is not flexible, but very simple. As
>>>> I understand the configuration model enforces to define actions for all
>>>> colors.
>>>
>>> Yes, and what I propose is the flexible version of Meters where both
>>> profiles and policies can be used separately. But flexibility comes with
>>> a price of taking care of both of them separately as well, of course.
>>>
>>>> A new model, if I'm not mistaken, will require three flow-based lookups:
>>>>     1. To assign a TAG based on flow fields (to handle different flows in
>>>> one meter)
>>>>     2. To do metering for packets with a TAG
>>>>     3. To find actions based on color
>>>> Of course, (2) and (3) are done in existing model with meter ID, but
>>>> here it is a generic flow-based lookups with extra matching criteria.
>>>> Yes, it is true that it gives extra flexibility, but everything has its
>>>> price.
>>>
>>> We don't need to assign a TAG. I used the TAG as example on how we can
>>> combine color matching with any other item matching.  Model stays the same.
>>> You still do color marking with a meter and find actions based on a color.
>>
>> I general yes, but I've tried to highlight the case when the
>> same meter should be applied to different flows. If so, you
>> need a way to combine. Old way allows to do it in a single step - just
>> specify meter ID. New way needs extra rule, for example,
>> using TAG or MARK.
>>
> If you want the same meter to be used by different rules, you can just
> use the action handle (indirect action).

Good point. I've missed such usage of an indirect action.

> The tag is used in order to demux the rules after the color matching.
> 
> Example for such use case:
> You have a share meter for a user, The user has number of connections,
> (Youtube, web, game)
> So the first thing the application does it checks the user meter
> Group 0:
> Match (user youtube 5 tuple) actions shared_meter set_tag(youtube) jump to group 1
> Match (user web 5 tuple) actions shared_meter set_tag(web) jump to group 1
> Match (user game 5 tuple) actions shared_meter set_tag(game) jump to group 1
> 
> Group 1:  (this is what we currently call profile,)
> Match color == green jump group 2
> Match color == red drop
> 
> Group 2:
> Match tag == youtube actions youtube related actions
> Match tag == web actions web related actions
> Match tag == game actions game related actions
> 
> Using the new API you can combine Group 1 and 2 (group 2 will no be needed anymore)
> Group 1':
> Match tag == youtube and color == green  actions youtube related actions
> Match tag == weband color == green actions web related actions
> Match tag == game and color == green actions game related actions
> Match color == red drop
> 
> As you can see, to implement the logic in current API we must use 3 groups while,
> using the suggested APi we can just use 2 groups.
> 
> 
>>>
>>>> Theoretically old model could be expressed using new one (and,
>>>> therefore, supported on old HW), but it is a bit tricky and raises many
>>>> questions on how to handle it correctly in all cases. E.g. if a TAG is
>>>> the only pattern in non-zero table and used for meter+jump actions only,
>>>> it could be associated with meter ID.
>>>> Above jump table specified after meter action could be associated with a
>>>> policy ID. If action for a color is not specified in a table, it should
>>>> be drop by default.
>>>
>>> Yes, old model can be expressed via new API and new model can be
>>> simulated with the old API. Efficiency and performance is the key.
>>>
>>>> Indirect actions or action templates could help to do meter profile job
>>>> - define profile in single place.
>>>
>>> True, meter color marking may be used for meter sharing, for example.
>>>
>>>> To sum up, since some HW could support the flexibility provided by
>>>> suggested flow API items/actions. I see no reason to block it. Solution
>>>> looks good from flow API design point of view.
>>>
>>> Thank you.
>>>
>>>> May be I'm missing something since I'm not expert in QoS and have no
>>>> hands-on experience with meters in DPDK.
>>>>
>>>> Andrew.
>>>
> 

Re: [RFC] ethdev: datapath-focused meter actions, continue

[Jerin Jacob]

On Mon, May 16, 2022 at 11:45 PM Alexander Kozyrev <akozyrev@nvidia.com> wrote:
>
> Agenda: continue discussion about proposed improvements to Flow API in regards to Meter handling (slides attached).



I think, primary difference between the old and new methods are where
the meter HW objects are available.
In the old method, it is more in NIC HW and in the new method it is
more in flow processor HW.

Also, I think, new method is more aligned with p4
(https://p4.org/p4-spec/docs/PSA.html#sec-meter) where things are done
using a flow processor kind of HW.

Emulating each approach is costly. So I don't see any harm in keeping
a new method(without removing the old method) for a specific set of
HWs that has such features.

But looking at the API specification it is not easy to understand how
to enable this example use case as it is complicated.
Could you share pseudo code from application perspective for the
following use case.

1) Meter0 has profile of srtcm_rfc2697 of RFC 2697 with packet based metering
2) Meter1 has profile of trtcm_rfc2698 of RFC 2698 with byte based metering
3) Meter2 has profile of trtcm_rfc4115 of  RFC 4115 with packet based metering
4)If VLAN ID is X then do Meter1 ,
- if output color is RED then drop the packet
- if output color is YELLOW then do Meter0
--If the out color for Meter 0 is not RED then forward to Queue 0 else
drop the packet.
5)If VLAN ID is Y then do Meter2 and define the input color from VLAN
PCP field(000-means Green, Remaining means Yellow)
--If the output color is Green and packet is IPV4 and forward the packet Queue 0
--If the output color is Green and packet is IPV6 and forward the packet Queue 1
- If the output color is not Green then drop the packet.

Listed above use case to just test API specification to be sure that
we have not missed anything so that new approach we can address all
metering requirements.

Thanks,
Jerin.

RE: [RFC] ethdev: datapath-focused meter actions, continue

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, May 19, 2022 2:56 PM
> To: Alexander Kozyrev <akozyrev@nvidia.com>
> Cc: Ori Kam <orika@nvidia.com>; Jerin Jacob <jerinj@marvell.com>;
> Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; NBU-Contact-Thomas
> Monjalon (EXTERNAL) <thomas@monjalon.net>; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>; Vipin.Varghese@amd.com; Ajit Khaparde
> <ajit.khaparde@broadcom.com>; Ferruh Yigit <ferruh.yigit@xilinx.com>; Ray
> Kinsella <mdr@ashroe.eu>; Sunil Kumar Kori <skori@marvell.com>; Ivan Malov
> <ivan.malov@oktetlabs.ru>; Awal, Mohammad Abdul
> <mohammad.abdul.awal@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Konstantin Ananyev
> <konstantin.ananyev@intel.com>; Singh, Jasvinder
> <jasvinder.singh@intel.com>; dev@dpdk.org
> Subject: Re: [RFC] ethdev: datapath-focused meter actions, continue
> 
> On Mon, May 16, 2022 at 11:45 PM Alexander Kozyrev <akozyrev@nvidia.com>
> wrote:
> >
> > Agenda: continue discussion about proposed improvements to Flow API in
> regards to Meter handling (slides attached).
> 
> 
> 
> I think, primary difference between the old and new methods are where
> the meter HW objects are available.
> In the old method, it is more in NIC HW and in the new method it is
> more in flow processor HW.
> 
> Also, I think, new method is more aligned with p4
> (https://p4.org/p4-spec/docs/PSA.html#sec-meter) where things are done
> using a flow processor kind of HW.
> 

Not really. In P4 PSA, the array of meters is defined upfront (initialization). Yes, the meters are configured at run-time by the control plane, with a default output color of green specified by the spec for an unconfigured meter.


> Emulating each approach is costly. So I don't see any harm in keeping
> a new method(without removing the old method) for a specific set of
> HWs that has such features.
> 

Yes, but we should try to avoid duplicating APIs for the same functionality, if possible.

> But looking at the API specification it is not easy to understand how
> to enable this example use case as it is complicated.
> Could you share pseudo code from application perspective for the
> following use case.
> 
> 1) Meter0 has profile of srtcm_rfc2697 of RFC 2697 with packet based
> metering
> 2) Meter1 has profile of trtcm_rfc2698 of RFC 2698 with byte based metering
> 3) Meter2 has profile of trtcm_rfc4115 of  RFC 4115 with packet based
> metering
> 4)If VLAN ID is X then do Meter1 ,
> - if output color is RED then drop the packet
> - if output color is YELLOW then do Meter0
> --If the out color for Meter 0 is not RED then forward to Queue 0 else
> drop the packet.
> 5)If VLAN ID is Y then do Meter2 and define the input color from VLAN
> PCP field(000-means Green, Remaining means Yellow)
> --If the output color is Green and packet is IPV4 and forward the packet Queue
> 0
> --If the output color is Green and packet is IPV6 and forward the packet Queue
> 1
> - If the output color is not Green then drop the packet.
> 

I have the same problem, it is difficult for me to visualize what happens when by whom, a simple pseudocode example would help a lot.

> Listed above use case to just test API specification to be sure that
> we have not missed anything so that new approach we can address all
> metering requirements.
> 
> Thanks,
> Jerin.