Traffic Management API Questions

Traffic Management API Questions

[Venky Venkatesh]

[-- Attachment #1: Type: text/plain, Size: 1733 bytes --]

Hi,
I was looking at the DPDK Traffic Management API. I wanted to clarify some
things that I understand from the code (for software based TM
implementation (at 20.11)) vs the documentation.

   - The documentation says "Traffic shaping: single/*dual rate,* private (*per
   node*) and shared (by *multiple nodes*) shapers" are supported. However
   it appears that the code supports only *single *rate shapers. Is my
   understanding correct?
      - If not, pls point me to where dual rate shaping is supported in the
      software based TM implementation code.
      - However, if my understanding is correct, can the authors clarify
      the nature of issues they ran into in supporting dual rate (which thus
      prevented them from implementing it)?
   - The documentation comment above sounds like every node can have
   shapers. However it appears that the code does not support shaping at the
   port level. Again the same questions as above(regarding the accuracy of my
   understanding and if it is accurate, the reasons from the author for not
   supporting it)
   - At the level of the TM API (*and* the associated software TM
   implementation) are there any restrictions on the number of levels of QoS
   hierarchy we can construct?
   - Lastly, does the QoS framework API (which I suppose is built on lower
   level building blocks including the TM API) expose the entire capabilities
   of the TM API (e.g. dual rate shapers, shapers at port level, > 4 levels of
   shaping etc.)? From the reading of the documentation it appears that there
   may be restrictions imposed by the QoS framework API on top of what TM API
   imposes. Can someone pls confirm this (and if so, the reason for doing so)?


Thanks
-Venky

[-- Attachment #2: Type: text/html, Size: 2161 bytes --]

[2nd Try]:Re: Traffic Management API Questions

[Venky Venkatesh]

[-- Attachment #1: Type: text/plain, Size: 1946 bytes --]

Hi,
Can someone pls get back on these
Thanks
-Venky

On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <
vvenkatesh@paloaltonetworks.com> wrote:

> Hi,
> I was looking at the DPDK Traffic Management API. I wanted to clarify some
> things that I understand from the code (for software based TM
> implementation (at 20.11)) vs the documentation.
>
>    - The documentation says "Traffic shaping: single/*dual rate,* private
>    (*per node*) and shared (by *multiple nodes*) shapers" are supported.
>    However it appears that the code supports only *single *rate shapers.
>    Is my understanding correct?
>       - If not, pls point me to where dual rate shaping is supported in
>       the software based TM implementation code.
>       - However, if my understanding is correct, can the authors clarify
>       the nature of issues they ran into in supporting dual rate (which thus
>       prevented them from implementing it)?
>    - The documentation comment above sounds like every node can have
>    shapers. However it appears that the code does not support shaping at the
>    port level. Again the same questions as above(regarding the accuracy of my
>    understanding and if it is accurate, the reasons from the author for not
>    supporting it)
>    - At the level of the TM API (*and* the associated software TM
>    implementation) are there any restrictions on the number of levels of QoS
>    hierarchy we can construct?
>    - Lastly, does the QoS framework API (which I suppose is built on
>    lower level building blocks including the TM API) expose the entire
>    capabilities of the TM API (e.g. dual rate shapers, shapers at port level,
>    > 4 levels of shaping etc.)? From the reading of the documentation it
>    appears that there may be restrictions imposed by the QoS framework API on
>    top of what TM API imposes. Can someone pls confirm this (and if so, the
>    reason for doing so)?
>
>
> Thanks
> -Venky
>
>

[-- Attachment #2: Type: text/html, Size: 2649 bytes --]

RE: [2nd Try]:Re: Traffic Management API Questions

[Singh, Jasvinder]

[-- Attachment #1: Type: text/plain, Size: 3497 bytes --]

Hi Venky,

Please see inline.

Thanks,
Jasvinder


From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
Sent: Tuesday, January 10, 2023 8:52 AM
To: dev@dpdk.org
Subject: [2nd Try]:Re: Traffic Management API Questions

Hi,
Can someone pls get back on these
Thanks
-Venky

On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <vvenkatesh@paloaltonetworks.com<mailto:vvenkatesh@paloaltonetworks.com>> wrote:
Hi,
I was looking at the DPDK Traffic Management API. I wanted to clarify some things that I understand from the code (for software based TM implementation (at 20.11)) vs the documentation.
·         The documentation says "Traffic shaping: single/dual rate, private (per node) and shared (by multiple nodes) shapers" are supported. However it appears that the code supports only single rate shapers. Is my understanding correct?
[JS] – Yes, TM API supports single and dual rate shapers, privately per node as well as shared across multiple nodes. However, DPDK QoS scheduler library implements single rate shaper at nodes.
o    If not, pls point me to where dual rate shaping is supported in the software based TM implementation code.
o    However, if my understanding is correct, can the authors clarify the nature of issues they ran into in supporting dual rate (which thus prevented them from implementing it)?
[JS] – There isn’t any issue except more complexity. Author can rework the library to implement the dual rate shapers for the desired nodes depending upon the requirement.
·         The documentation comment above sounds like every node can have shapers. However it appears that the code does not support shaping at the port level. Again the same questions as above(regarding the accuracy of my understanding and if it is accurate, the reasons from the author for not supporting it)
[JS] – Implementation supports shapers at subport (group of pipes) and pipe level. The bandwidth available at the port level is distributed among the subports with the condition that aggregate bandwidth of subports should not exceed the port bandwidth. Each subport buffers and shape the traffic from the pipes depending upon the port bandwidth allocated to it. Implementation doesn’t support distribution of unused bandwidth of one subport to another subport. However, one can modify this behaviour if needed.
·         At the level of the TM API (and the associated software TM implementation) are there any restrictions on the number of levels of QoS hierarchy we can construct?
[JS] – TM API doesn’t restrict the number of QoS scheduler levels and generic enough to work with hierarchical schedulers with any number of levels. The current dpdk sched library implementation supports fixed 5 level scheduler hierarchy.
·         Lastly, does the QoS framework API (which I suppose is built on lower level building blocks including the TM API) expose the entire capabilities of the TM API (e.g. dual rate shapers, shapers at port level, > 4 levels of shaping etc.)? From the reading of the documentation it appears that there may be restrictions imposed by the QoS framework API on top of what TM API imposes. Can someone pls confirm this (and if so, the reason for doing so)?
[JS] – No, QoS framework API (DPDK sched library) presents only one flavour of hierarchical scheduler and doesn’t implements all the features exposed through TM API.  However, more features can be added to library and configured through TM API.

Thanks
-Venky


[-- Attachment #2: Type: text/html, Size: 12233 bytes --]

Re: [2nd Try]:Re: Traffic Management API Questions

[Venky Venkatesh]

[-- Attachment #1: Type: text/plain, Size: 5287 bytes --]

Hi Jasvinder,
Thanks for the detailed answers. Our need is to have shaping at the port
level as well. I am trying to see what would be the way to accomplish this
given the current limitations of the sched library implementation in this
regard. I see 2 options:

   - The top level (i.e. port level) documentation says the following:
   "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled in
   round robin order with all ports having equal priority". Questions:
      - Do all the ports have to be of the same speed OR can it be a
      heterogeneous set of port speeds?
      - If it can be a heterogeneous set of ports, is the scheduling across
      those ports *weighted* round robin as opposed to round robin?
      - Are Speeds other than  1/10/40 GbE not supported?
      - I suppose this heterogeneous mix of port speeds is implemented by
      playing with the weights across ports, correct?
      - If so, what problem do you foresee if we provide arbitrary
      bandwidth ports by regulating the above weights?
   - The other alternative would be to add another layer (which has a
   shaper)  to the hierarchy by mimicking one of the existing layers: how
   amenable is the current implementation to that?

Do either of the above look like workable ideas? Are there any other
approaches where we could accomplish our requirement with minimal changes
to the code logic?
Thanks
-Venky

On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder <jasvinder.singh@intel.com>
wrote:

> Hi Venky,
>
>
>
> Please see inline.
>
>
>
> Thanks,
>
> Jasvinder
>
>
>
>
>
> *From:* Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
> *Sent:* Tuesday, January 10, 2023 8:52 AM
> *To:* dev@dpdk.org
> *Subject:* [2nd Try]:Re: Traffic Management API Questions
>
>
>
> Hi,
>
> Can someone pls get back on these
>
> Thanks
>
> -Venky
>
>
>
> On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <
> vvenkatesh@paloaltonetworks.com> wrote:
>
> Hi,
>
> I was looking at the DPDK Traffic Management API. I wanted to clarify some
> things that I understand from the code (for software based TM
> implementation (at 20.11)) vs the documentation.
>
> ·         The documentation says "Traffic shaping: single/*dual rate**,* private
> (*per node*) and shared (by *multiple nodes*) shapers" are supported.
> However it appears that the code supports only *single *rate shapers. Is
> my understanding correct?
>
> [JS] – Yes, TM API supports single and dual rate shapers, privately per
> node as well as shared across multiple nodes. However, DPDK QoS scheduler
> library implements single rate shaper at nodes.
>
> o    If not, pls point me to where dual rate shaping is supported in the
> software based TM implementation code.
>
> o    However, if my understanding is correct, can the authors clarify the
> nature of issues they ran into in supporting dual rate (which thus
> prevented them from implementing it)?
>
> [JS] – There isn’t any issue except more complexity. Author can rework the
> library to implement the dual rate shapers for the desired nodes depending
> upon the requirement.
>
> ·         The documentation comment above sounds like every node can have
> shapers. However it appears that the code does not support shaping at the
> port level. Again the same questions as above(regarding the accuracy of my
> understanding and if it is accurate, the reasons from the author for not
> supporting it)
>
> [JS] – Implementation supports shapers at subport (group of pipes) and
> pipe level. The bandwidth available at the port level is distributed among
> the subports with the condition that aggregate bandwidth of subports should
> not exceed the port bandwidth. Each subport buffers and shape the traffic
> from the pipes depending upon the port bandwidth allocated to it.
> Implementation doesn’t support distribution of unused bandwidth of one
> subport to another subport. However, one can modify this behaviour if
> needed.
>
> ·         At the level of the TM API (*and* the associated software TM
> implementation) are there any restrictions on the number of levels of QoS
> hierarchy we can construct?
>
> [JS] – TM API doesn’t restrict the number of QoS scheduler levels and
> generic enough to work with hierarchical schedulers with any number of
> levels. The current dpdk sched library implementation supports fixed 5
> level scheduler hierarchy.
>
> ·         Lastly, does the QoS framework API (which I suppose is built on
> lower level building blocks including the TM API) expose the entire
> capabilities of the TM API (e.g. dual rate shapers, shapers at port level,
> > 4 levels of shaping etc.)? From the reading of the documentation it
> appears that there may be restrictions imposed by the QoS framework API on
> top of what TM API imposes. Can someone pls confirm this (and if so, the
> reason for doing so)?
>
> [JS] – No, QoS framework API (DPDK sched library) presents only one
> flavour of hierarchical scheduler and doesn’t implements all the features
> exposed through TM API.  However, more features can be added to library and
> configured through TM API.
>
>
>
> Thanks
>
> -Venky
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 9343 bytes --]

RE: [2nd Try]:Re: Traffic Management API Questions

[Singh, Jasvinder]

[-- Attachment #1: Type: text/plain, Size: 6344 bytes --]

Hi Venky,

Please see inline;

Jasvinder

From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
Sent: Wednesday, January 11, 2023 11:56 AM
To: Singh, Jasvinder <jasvinder.singh@intel.com>
Cc: dev@dpdk.org
Subject: Re: [2nd Try]:Re: Traffic Management API Questions

Hi Jasvinder,
Thanks for the detailed answers. Our need is to have shaping at the port level as well. I am trying to see what would be the way to accomplish this given the current limitations of the sched library implementation in this regard. I see 2 options:

  *   The top level (i.e. port level) documentation says the following: "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled in round robin order with all ports having equal priority". Questions:

     *   Do all the ports have to be of the same speed OR can it be a heterogeneous set of port speeds?
[JS] – the library supports single port (root node) of the hierarchy. Each port can have multiple subports configured using different shaping rates. If you desire to have multiple ports, each port would have separate hierarchical tree underneath. Different ports could have different speed.
o If it can be a heterogeneous set of ports, is the scheduling across those ports weighted round robin as opposed to round robin?
[JS] – Scheduling across multiple ports is not supported in current sched library. At the application level, one can think of invoking enqueue/dequeue sched API in round robin or weighted round robin manner.

     *   Are Speeds other than  1/10/40 GbE not supported?
[JS] – Speeds other than above is supported, for eg. 25G, 50G etc.

     *   I suppose this heterogeneous mix of port speeds is implemented by playing with the weights across ports, correct?
[JS] -please see above answers

     *   If so, what problem do you foresee if we provide arbitrary bandwidth ports by regulating the above weights?
[JS] – I don’t see any issue.

  *   The other alternative would be to add another layer (which has a shaper)  to the hierarchy by mimicking one of the existing layers: how amenable is the current implementation to that?
Do either of the above look like workable ideas? Are there any other approaches where we could accomplish our requirement with minimal changes to the code logic?
[JS] – adding another layer will require considerable work in library. How about using multiple subports with different shaping bandwidth where each subport maintain #subcribers and send traffic out through single physical port (root node). It will need less effort and library supports multiple subports under single port (root node).


Thanks
-Venky

On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder <jasvinder.singh@intel.com<mailto:jasvinder.singh@intel.com>> wrote:
Hi Venky,

Please see inline.

Thanks,
Jasvinder


From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com<mailto:vvenkatesh@paloaltonetworks.com>>
Sent: Tuesday, January 10, 2023 8:52 AM
To: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: [2nd Try]:Re: Traffic Management API Questions

Hi,
Can someone pls get back on these
Thanks
-Venky

On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <vvenkatesh@paloaltonetworks.com<mailto:vvenkatesh@paloaltonetworks.com>> wrote:
Hi,
I was looking at the DPDK Traffic Management API. I wanted to clarify some things that I understand from the code (for software based TM implementation (at 20.11)) vs the documentation.
•         The documentation says "Traffic shaping: single/dual rate, private (per node) and shared (by multiple nodes) shapers" are supported. However it appears that the code supports only single rate shapers. Is my understanding correct?
[JS] – Yes, TM API supports single and dual rate shapers, privately per node as well as shared across multiple nodes. However, DPDK QoS scheduler library implements single rate shaper at nodes.
o    If not, pls point me to where dual rate shaping is supported in the software based TM implementation code.
o    However, if my understanding is correct, can the authors clarify the nature of issues they ran into in supporting dual rate (which thus prevented them from implementing it)?
[JS] – There isn’t any issue except more complexity. Author can rework the library to implement the dual rate shapers for the desired nodes depending upon the requirement.
•         The documentation comment above sounds like every node can have shapers. However it appears that the code does not support shaping at the port level. Again the same questions as above(regarding the accuracy of my understanding and if it is accurate, the reasons from the author for not supporting it)
[JS] – Implementation supports shapers at subport (group of pipes) and pipe level. The bandwidth available at the port level is distributed among the subports with the condition that aggregate bandwidth of subports should not exceed the port bandwidth. Each subport buffers and shape the traffic from the pipes depending upon the port bandwidth allocated to it. Implementation doesn’t support distribution of unused bandwidth of one subport to another subport. However, one can modify this behaviour if needed.
•         At the level of the TM API (and the associated software TM implementation) are there any restrictions on the number of levels of QoS hierarchy we can construct?
[JS] – TM API doesn’t restrict the number of QoS scheduler levels and generic enough to work with hierarchical schedulers with any number of levels. The current dpdk sched library implementation supports fixed 5 level scheduler hierarchy.
•         Lastly, does the QoS framework API (which I suppose is built on lower level building blocks including the TM API) expose the entire capabilities of the TM API (e.g. dual rate shapers, shapers at port level, > 4 levels of shaping etc.)? From the reading of the documentation it appears that there may be restrictions imposed by the QoS framework API on top of what TM API imposes. Can someone pls confirm this (and if so, the reason for doing so)?
[JS] – No, QoS framework API (DPDK sched library) presents only one flavour of hierarchical scheduler and doesn’t implements all the features exposed through TM API.  However, more features can be added to library and configured through TM API.

Thanks
-Venky


[-- Attachment #2: Type: text/html, Size: 31499 bytes --]

Re: [2nd Try]:Re: Traffic Management API Questions

[Venky Venkatesh]

[-- Attachment #1: Type: text/plain, Size: 8254 bytes --]

Hi Jasvinder,
Thanks for the insights on the complexity of adding a layer.
As for the workaround that you suggested using multiple subports, if I
understand it correctly (*pls correct if I misunderstood*) it would not
meet our needs:

   - We require multiple heterogeneous ports (i.e. ports with different
   bandwidths -- with no excess sharing since these are port limits). That
   would probably need some shaper attached there too since WRR (at the
   application) would share the instantaneous excess among the siblings.
   - As for the 2nd second suggestion (increase the number of subports):
   our need (*in addition to* multiple ports of different bandwidths at the
   top level) is to have  4 more TM layers for a total of 5. I am *not*
   looking at the assignment of the terms port/subport/user/pipe etc in the
   DPDK documentation -- instead am looking at it as abstract scheduling
   (and/or shaping) layers with differing abilities in some layers. So in
   order to compensate for the missing shaper at the port level I was planning
   to add 1 additional layer (so that what in DPDK documentation is referred
   to as subport is actually a port -- since the subport layer has the
   property of *not sharing* excess between siblings. With that principle,
   I am not clear how adding width to the subport layer (as I understand your
   suggestion) would help.


Thanks
-Venky


On Wed, Jan 11, 2023 at 9:24 AM Singh, Jasvinder <jasvinder.singh@intel.com>
wrote:

> Hi Venky,
>
>
>
> Please see inline;
>
>
>
> Jasvinder
>
>
>
> *From:* Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
> *Sent:* Wednesday, January 11, 2023 11:56 AM
> *To:* Singh, Jasvinder <jasvinder.singh@intel.com>
> *Cc:* dev@dpdk.org
> *Subject:* Re: [2nd Try]:Re: Traffic Management API Questions
>
>
>
> Hi Jasvinder,
>
> Thanks for the detailed answers. Our need is to have shaping at the port
> level as well. I am trying to see what would be the way to accomplish this
> given the current limitations of the sched library implementation in this
> regard. I see 2 options:
>
>    - The top level (i.e. port level) documentation says the following:
>    "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled
>    in round robin order with all ports having equal priority". Questions:
>
>
>    - Do all the ports have to be of the same speed OR can it be a
>       heterogeneous set of port speeds?
>
> [JS] – the library supports single port (root node) of the hierarchy. Each
> port can have multiple subports configured using different shaping rates.
> If you desire to have multiple ports, each port would have separate
> hierarchical tree underneath. Different ports could have different speed.
>
> o If it can be a heterogeneous set of ports, is the scheduling across
> those ports *weighted* round robin as opposed to round robin?
>
> [JS] – Scheduling across multiple ports is not supported in current sched
> library. At the application level, one can think of invoking
> enqueue/dequeue sched API in round robin or weighted round robin manner.
>
>    - Are Speeds other than  1/10/40 GbE not supported?
>
> [JS] – Speeds other than above is supported, for eg. 25G, 50G etc.
>
>    - I suppose this heterogeneous mix of port speeds is implemented by
>       playing with the weights across ports, correct?
>
> [JS] -please see above answers
>
>    - If so, what problem do you foresee if we provide arbitrary bandwidth
>       ports by regulating the above weights?
>
> [JS] – I don’t see any issue.
>
>    - The other alternative would be to add another layer (which has a
>    shaper)  to the hierarchy by mimicking one of the existing layers: how
>    amenable is the current implementation to that?
>
> Do either of the above look like workable ideas? Are there any other
> approaches where we could accomplish our requirement with minimal changes
> to the code logic?
>
> [JS] – adding another layer will require considerable work in library. How
> about using multiple subports with different shaping bandwidth where each
> subport maintain #subcribers and send traffic out through single physical
> port (root node). It will need less effort and library supports multiple
> subports under single port (root node).
>
>
>
>
>
> Thanks
>
> -Venky
>
>
>
> On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder <
> jasvinder.singh@intel.com> wrote:
>
> Hi Venky,
>
>
>
> Please see inline.
>
>
>
> Thanks,
>
> Jasvinder
>
>
>
>
>
> *From:* Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
> *Sent:* Tuesday, January 10, 2023 8:52 AM
> *To:* dev@dpdk.org
> *Subject:* [2nd Try]:Re: Traffic Management API Questions
>
>
>
> Hi,
>
> Can someone pls get back on these
>
> Thanks
>
> -Venky
>
>
>
> On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <
> vvenkatesh@paloaltonetworks.com> wrote:
>
> Hi,
>
> I was looking at the DPDK Traffic Management API. I wanted to clarify some
> things that I understand from the code (for software based TM
> implementation (at 20.11)) vs the documentation.
>
> ·         The documentation says "Traffic shaping: single/*dual rate**,* private
> (*per node*) and shared (by *multiple nodes*) shapers" are supported.
> However it appears that the code supports only *single *rate shapers. Is
> my understanding correct?
>
> [JS] – Yes, TM API supports single and dual rate shapers, privately per
> node as well as shared across multiple nodes. However, DPDK QoS scheduler
> library implements single rate shaper at nodes.
>
> o    If not, pls point me to where dual rate shaping is supported in the
> software based TM implementation code.
>
> o    However, if my understanding is correct, can the authors clarify the
> nature of issues they ran into in supporting dual rate (which thus
> prevented them from implementing it)?
>
> [JS] – There isn’t any issue except more complexity. Author can rework the
> library to implement the dual rate shapers for the desired nodes depending
> upon the requirement.
>
> ·         The documentation comment above sounds like every node can have
> shapers. However it appears that the code does not support shaping at the
> port level. Again the same questions as above(regarding the accuracy of my
> understanding and if it is accurate, the reasons from the author for not
> supporting it)
>
> [JS] – Implementation supports shapers at subport (group of pipes) and
> pipe level. The bandwidth available at the port level is distributed among
> the subports with the condition that aggregate bandwidth of subports should
> not exceed the port bandwidth. Each subport buffers and shape the traffic
> from the pipes depending upon the port bandwidth allocated to it.
> Implementation doesn’t support distribution of unused bandwidth of one
> subport to another subport. However, one can modify this behaviour if
> needed.
>
> ·         At the level of the TM API (*and* the associated software TM
> implementation) are there any restrictions on the number of levels of QoS
> hierarchy we can construct?
>
> [JS] – TM API doesn’t restrict the number of QoS scheduler levels and
> generic enough to work with hierarchical schedulers with any number of
> levels. The current dpdk sched library implementation supports fixed 5
> level scheduler hierarchy.
>
> ·         Lastly, does the QoS framework API (which I suppose is built on
> lower level building blocks including the TM API) expose the entire
> capabilities of the TM API (e.g. dual rate shapers, shapers at port level,
> > 4 levels of shaping etc.)? From the reading of the documentation it
> appears that there may be restrictions imposed by the QoS framework API on
> top of what TM API imposes. Can someone pls confirm this (and if so, the
> reason for doing so)?
>
> [JS] – No, QoS framework API (DPDK sched library) presents only one
> flavour of hierarchical scheduler and doesn’t implements all the features
> exposed through TM API.  However, more features can be added to library and
> configured through TM API.
>
>
>
> Thanks
>
> -Venky
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 14657 bytes --]

RE: [2nd Try]:Re: Traffic Management API Questions

[Singh, Jasvinder]

[-- Attachment #1: Type: text/plain, Size: 8722 bytes --]

Hi Venky,

Please see inline;

Thanks,
Jasvinder


From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
Sent: Monday, January 16, 2023 8:06 AM
To: Singh, Jasvinder <jasvinder.singh@intel.com>
Cc: dev@dpdk.org
Subject: Re: [2nd Try]:Re: Traffic Management API Questions

Hi Jasvinder,
Thanks for the insights on the complexity of adding a layer.
As for the workaround that you suggested using multiple subports, if I understand it correctly (pls correct if I misunderstood) it would not meet our needs:

  *   We require multiple heterogeneous ports (i.e. ports with different bandwidths -- with no excess sharing since these are port limits). That would probably need some shaper attached there too since WRR (at the application) would share the instantaneous excess among the siblings.
·  As for the 2nd second suggestion (increase the number of subports): our need (in addition to multiple ports of different bandwidths at the top level) is to have  4 more TM layers for a total of 5. I am not looking at the assignment of the terms port/subport/user/pipe etc in the DPDK documentation -- instead am looking at it as abstract scheduling (and/or shaping) layers with differing abilities in some layers. So in order to compensate for the missing shaper at the port level I was planning to add 1 additional layer (so that what in DPDK documentation is referred to as subport is actually a port -- since the subport layer has the property of not sharing excess between siblings. With that principle, I am not clear how adding width to the subport layer (as I understand your suggestion) would help.
[JS] – I was suggesting to assume subports as ports in the existing implementation and assign fixed bandwidth to each of the subports. By doing so, you would have multiple subports (re-named as ports) with shaper attached. Only limitation in such solution is that hierarchy would have single root node with the bandwidth equal to sum of subports bandwidth and all the subports would be served individually in round robin manner. If it doesn’t suit your  requirement, you need to make changes as you suggested above.

Thanks
-Venky


On Wed, Jan 11, 2023 at 9:24 AM Singh, Jasvinder <jasvinder.singh@intel.com<mailto:jasvinder.singh@intel.com>> wrote:
Hi Venky,

Please see inline;

Jasvinder

From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com<mailto:vvenkatesh@paloaltonetworks.com>>
Sent: Wednesday, January 11, 2023 11:56 AM
To: Singh, Jasvinder <jasvinder.singh@intel.com<mailto:jasvinder.singh@intel.com>>
Cc: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [2nd Try]:Re: Traffic Management API Questions

Hi Jasvinder,
Thanks for the detailed answers. Our need is to have shaping at the port level as well. I am trying to see what would be the way to accomplish this given the current limitations of the sched library implementation in this regard. I see 2 options:

  *   The top level (i.e. port level) documentation says the following: "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled in round robin order with all ports having equal priority". Questions:

     *   Do all the ports have to be of the same speed OR can it be a heterogeneous set of port speeds?
[JS] – the library supports single port (root node) of the hierarchy. Each port can have multiple subports configured using different shaping rates. If you desire to have multiple ports, each port would have separate hierarchical tree underneath. Different ports could have different speed.
o If it can be a heterogeneous set of ports, is the scheduling across those ports weighted round robin as opposed to round robin?
[JS] – Scheduling across multiple ports is not supported in current sched library. At the application level, one can think of invoking enqueue/dequeue sched API in round robin or weighted round robin manner.

     *   Are Speeds other than  1/10/40 GbE not supported?
[JS] – Speeds other than above is supported, for eg. 25G, 50G etc.

     *   I suppose this heterogeneous mix of port speeds is implemented by playing with the weights across ports, correct?
[JS] -please see above answers

     *   If so, what problem do you foresee if we provide arbitrary bandwidth ports by regulating the above weights?
[JS] – I don’t see any issue.

  *   The other alternative would be to add another layer (which has a shaper)  to the hierarchy by mimicking one of the existing layers: how amenable is the current implementation to that?
Do either of the above look like workable ideas? Are there any other approaches where we could accomplish our requirement with minimal changes to the code logic?
[JS] – adding another layer will require considerable work in library. How about using multiple subports with different shaping bandwidth where each subport maintain #subcribers and send traffic out through single physical port (root node). It will need less effort and library supports multiple subports under single port (root node).


Thanks
-Venky

On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder <jasvinder.singh@intel.com<mailto:jasvinder.singh@intel.com>> wrote:
Hi Venky,

Please see inline.

Thanks,
Jasvinder


From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com<mailto:vvenkatesh@paloaltonetworks.com>>
Sent: Tuesday, January 10, 2023 8:52 AM
To: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: [2nd Try]:Re: Traffic Management API Questions

Hi,
Can someone pls get back on these
Thanks
-Venky

On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <vvenkatesh@paloaltonetworks.com<mailto:vvenkatesh@paloaltonetworks.com>> wrote:
Hi,
I was looking at the DPDK Traffic Management API. I wanted to clarify some things that I understand from the code (for software based TM implementation (at 20.11)) vs the documentation.
•         The documentation says "Traffic shaping: single/dual rate, private (per node) and shared (by multiple nodes) shapers" are supported. However it appears that the code supports only single rate shapers. Is my understanding correct?
[JS] – Yes, TM API supports single and dual rate shapers, privately per node as well as shared across multiple nodes. However, DPDK QoS scheduler library implements single rate shaper at nodes.
o    If not, pls point me to where dual rate shaping is supported in the software based TM implementation code.
o    However, if my understanding is correct, can the authors clarify the nature of issues they ran into in supporting dual rate (which thus prevented them from implementing it)?
[JS] – There isn’t any issue except more complexity. Author can rework the library to implement the dual rate shapers for the desired nodes depending upon the requirement.
•         The documentation comment above sounds like every node can have shapers. However it appears that the code does not support shaping at the port level. Again the same questions as above(regarding the accuracy of my understanding and if it is accurate, the reasons from the author for not supporting it)
[JS] – Implementation supports shapers at subport (group of pipes) and pipe level. The bandwidth available at the port level is distributed among the subports with the condition that aggregate bandwidth of subports should not exceed the port bandwidth. Each subport buffers and shape the traffic from the pipes depending upon the port bandwidth allocated to it. Implementation doesn’t support distribution of unused bandwidth of one subport to another subport. However, one can modify this behaviour if needed.
•         At the level of the TM API (and the associated software TM implementation) are there any restrictions on the number of levels of QoS hierarchy we can construct?
[JS] – TM API doesn’t restrict the number of QoS scheduler levels and generic enough to work with hierarchical schedulers with any number of levels. The current dpdk sched library implementation supports fixed 5 level scheduler hierarchy.
•         Lastly, does the QoS framework API (which I suppose is built on lower level building blocks including the TM API) expose the entire capabilities of the TM API (e.g. dual rate shapers, shapers at port level, > 4 levels of shaping etc.)? From the reading of the documentation it appears that there may be restrictions imposed by the QoS framework API on top of what TM API imposes. Can someone pls confirm this (and if so, the reason for doing so)?
[JS] – No, QoS framework API (DPDK sched library) presents only one flavour of hierarchical scheduler and doesn’t implements all the features exposed through TM API.  However, more features can be added to library and configured through TM API.

Thanks
-Venky


[-- Attachment #2: Type: text/html, Size: 37221 bytes --]

Re: [2nd Try]:Re: Traffic Management API Questions

[Venky Venkatesh]

[-- Attachment #1: Type: text/plain, Size: 9735 bytes --]

Thanks Jasvinder. I guess we are on the same page. With the design that you
mention we run short by 1 level of hierarchy -- which is why I was
originally asking for the difficulty of adding a layer. I think I
understand your assessment in that regard i.e. it is easier to add a shaped
dequeue at the roots in the application as opposed to add an additional
layer

Pls correct me if I am wrong. Otherwise Thanks for all your inputs
-Venky

On Mon, Jan 16, 2023 at 3:39 AM Singh, Jasvinder <jasvinder.singh@intel.com>
wrote:

> Hi Venky,
>
>
>
> Please see inline;
>
>
>
> Thanks,
>
> Jasvinder
>
>
>
>
>
> *From:* Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
> *Sent:* Monday, January 16, 2023 8:06 AM
> *To:* Singh, Jasvinder <jasvinder.singh@intel.com>
> *Cc:* dev@dpdk.org
> *Subject:* Re: [2nd Try]:Re: Traffic Management API Questions
>
>
>
> Hi Jasvinder,
>
> Thanks for the insights on the complexity of adding a layer.
>
> As for the workaround that you suggested using multiple subports, if I
> understand it correctly (*pls correct if I misunderstood*) it would not
> meet our needs:
>
>    - We require multiple heterogeneous ports (i.e. ports with different
>    bandwidths -- with no excess sharing since these are port limits). That
>    would probably need some shaper attached there too since WRR (at the
>    application) would share the instantaneous excess among the siblings.
>
> ·  As for the 2nd second suggestion (increase the number of subports):
> our need (*in addition to* multiple ports of different bandwidths at the
> top level) is to have  4 more TM layers for a total of 5. I am *not*
> looking at the assignment of the terms port/subport/user/pipe etc in the
> DPDK documentation -- instead am looking at it as abstract scheduling
> (and/or shaping) layers with differing abilities in some layers. So in
> order to compensate for the missing shaper at the port level I was planning
> to add 1 additional layer (so that what in DPDK documentation is referred
> to as subport is actually a port -- since the subport layer has the
> property of *not sharing* excess between siblings. With that principle, I
> am not clear how adding width to the subport layer (as I understand your
> suggestion) would help.
>
> [JS] – I was suggesting to assume subports as ports in the existing
> implementation and assign fixed bandwidth to each of the subports. By doing
> so, you would have multiple subports (re-named as ports) with shaper
> attached. Only limitation in such solution is that hierarchy would have
> single root node with the bandwidth equal to sum of subports bandwidth and
> all the subports would be served individually in round robin manner. If it
> doesn’t suit your  requirement, you need to make changes as you suggested
> above.
>
>
>
> Thanks
>
> -Venky
>
>
>
>
>
> On Wed, Jan 11, 2023 at 9:24 AM Singh, Jasvinder <
> jasvinder.singh@intel.com> wrote:
>
> Hi Venky,
>
>
>
> Please see inline;
>
>
>
> Jasvinder
>
>
>
> *From:* Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
> *Sent:* Wednesday, January 11, 2023 11:56 AM
> *To:* Singh, Jasvinder <jasvinder.singh@intel.com>
> *Cc:* dev@dpdk.org
> *Subject:* Re: [2nd Try]:Re: Traffic Management API Questions
>
>
>
> Hi Jasvinder,
>
> Thanks for the detailed answers. Our need is to have shaping at the port
> level as well. I am trying to see what would be the way to accomplish this
> given the current limitations of the sched library implementation in this
> regard. I see 2 options:
>
>    - The top level (i.e. port level) documentation says the following:
>    "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled
>    in round robin order with all ports having equal priority". Questions:
>
>
>    - Do all the ports have to be of the same speed OR can it be a
>       heterogeneous set of port speeds?
>
> [JS] – the library supports single port (root node) of the hierarchy. Each
> port can have multiple subports configured using different shaping rates.
> If you desire to have multiple ports, each port would have separate
> hierarchical tree underneath. Different ports could have different speed.
>
> o If it can be a heterogeneous set of ports, is the scheduling across
> those ports *weighted* round robin as opposed to round robin?
>
> [JS] – Scheduling across multiple ports is not supported in current sched
> library. At the application level, one can think of invoking
> enqueue/dequeue sched API in round robin or weighted round robin manner.
>
>    - Are Speeds other than  1/10/40 GbE not supported?
>
> [JS] – Speeds other than above is supported, for eg. 25G, 50G etc.
>
>    - I suppose this heterogeneous mix of port speeds is implemented by
>       playing with the weights across ports, correct?
>
> [JS] -please see above answers
>
>    - If so, what problem do you foresee if we provide arbitrary bandwidth
>       ports by regulating the above weights?
>
> [JS] – I don’t see any issue.
>
>    - The other alternative would be to add another layer (which has a
>    shaper)  to the hierarchy by mimicking one of the existing layers: how
>    amenable is the current implementation to that?
>
> Do either of the above look like workable ideas? Are there any other
> approaches where we could accomplish our requirement with minimal changes
> to the code logic?
>
> [JS] – adding another layer will require considerable work in library. How
> about using multiple subports with different shaping bandwidth where each
> subport maintain #subcribers and send traffic out through single physical
> port (root node). It will need less effort and library supports multiple
> subports under single port (root node).
>
>
>
>
>
> Thanks
>
> -Venky
>
>
>
> On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder <
> jasvinder.singh@intel.com> wrote:
>
> Hi Venky,
>
>
>
> Please see inline.
>
>
>
> Thanks,
>
> Jasvinder
>
>
>
>
>
> *From:* Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
> *Sent:* Tuesday, January 10, 2023 8:52 AM
> *To:* dev@dpdk.org
> *Subject:* [2nd Try]:Re: Traffic Management API Questions
>
>
>
> Hi,
>
> Can someone pls get back on these
>
> Thanks
>
> -Venky
>
>
>
> On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <
> vvenkatesh@paloaltonetworks.com> wrote:
>
> Hi,
>
> I was looking at the DPDK Traffic Management API. I wanted to clarify some
> things that I understand from the code (for software based TM
> implementation (at 20.11)) vs the documentation.
>
> ·         The documentation says "Traffic shaping: single/*dual rate**,* private
> (*per node*) and shared (by *multiple nodes*) shapers" are supported.
> However it appears that the code supports only *single *rate shapers. Is
> my understanding correct?
>
> [JS] – Yes, TM API supports single and dual rate shapers, privately per
> node as well as shared across multiple nodes. However, DPDK QoS scheduler
> library implements single rate shaper at nodes.
>
> o    If not, pls point me to where dual rate shaping is supported in the
> software based TM implementation code.
>
> o    However, if my understanding is correct, can the authors clarify the
> nature of issues they ran into in supporting dual rate (which thus
> prevented them from implementing it)?
>
> [JS] – There isn’t any issue except more complexity. Author can rework the
> library to implement the dual rate shapers for the desired nodes depending
> upon the requirement.
>
> ·         The documentation comment above sounds like every node can have
> shapers. However it appears that the code does not support shaping at the
> port level. Again the same questions as above(regarding the accuracy of my
> understanding and if it is accurate, the reasons from the author for not
> supporting it)
>
> [JS] – Implementation supports shapers at subport (group of pipes) and
> pipe level. The bandwidth available at the port level is distributed among
> the subports with the condition that aggregate bandwidth of subports should
> not exceed the port bandwidth. Each subport buffers and shape the traffic
> from the pipes depending upon the port bandwidth allocated to it.
> Implementation doesn’t support distribution of unused bandwidth of one
> subport to another subport. However, one can modify this behaviour if
> needed.
>
> ·         At the level of the TM API (*and* the associated software TM
> implementation) are there any restrictions on the number of levels of QoS
> hierarchy we can construct?
>
> [JS] – TM API doesn’t restrict the number of QoS scheduler levels and
> generic enough to work with hierarchical schedulers with any number of
> levels. The current dpdk sched library implementation supports fixed 5
> level scheduler hierarchy.
>
> ·         Lastly, does the QoS framework API (which I suppose is built on
> lower level building blocks including the TM API) expose the entire
> capabilities of the TM API (e.g. dual rate shapers, shapers at port level,
> > 4 levels of shaping etc.)? From the reading of the documentation it
> appears that there may be restrictions imposed by the QoS framework API on
> top of what TM API imposes. Can someone pls confirm this (and if so, the
> reason for doing so)?
>
> [JS] – No, QoS framework API (DPDK sched library) presents only one
> flavour of hierarchical scheduler and doesn’t implements all the features
> exposed through TM API.  However, more features can be added to library and
> configured through TM API.
>
>
>
> Thanks
>
> -Venky
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 18309 bytes --]