Thanks Jasvinder. I guess we are on the same page. With the design that you mention we run short by 1 level of hierarchy -- which is why I was originally asking for the difficulty of adding a layer. I think I understand your assessment in that regard i.e. it is easier to add a shaped dequeue at the roots in the application as opposed to add an additional layer Pls correct me if I am wrong. Otherwise Thanks for all your inputs -Venky On Mon, Jan 16, 2023 at 3:39 AM Singh, Jasvinder wrote: > Hi Venky, > > > > Please see inline; > > > > Thanks, > > Jasvinder > > > > > > *From:* Venky Venkatesh > *Sent:* Monday, January 16, 2023 8:06 AM > *To:* Singh, Jasvinder > *Cc:* dev@dpdk.org > *Subject:* Re: [2nd Try]:Re: Traffic Management API Questions > > > > Hi Jasvinder, > > Thanks for the insights on the complexity of adding a layer. > > As for the workaround that you suggested using multiple subports, if I > understand it correctly (*pls correct if I misunderstood*) it would not > meet our needs: > > - We require multiple heterogeneous ports (i.e. ports with different > bandwidths -- with no excess sharing since these are port limits). That > would probably need some shaper attached there too since WRR (at the > application) would share the instantaneous excess among the siblings. > > · As for the 2nd second suggestion (increase the number of subports): > our need (*in addition to* multiple ports of different bandwidths at the > top level) is to have 4 more TM layers for a total of 5. I am *not* > looking at the assignment of the terms port/subport/user/pipe etc in the > DPDK documentation -- instead am looking at it as abstract scheduling > (and/or shaping) layers with differing abilities in some layers. So in > order to compensate for the missing shaper at the port level I was planning > to add 1 additional layer (so that what in DPDK documentation is referred > to as subport is actually a port -- since the subport layer has the > property of *not sharing* excess between siblings. With that principle, I > am not clear how adding width to the subport layer (as I understand your > suggestion) would help. > > [JS] – I was suggesting to assume subports as ports in the existing > implementation and assign fixed bandwidth to each of the subports. By doing > so, you would have multiple subports (re-named as ports) with shaper > attached. Only limitation in such solution is that hierarchy would have > single root node with the bandwidth equal to sum of subports bandwidth and > all the subports would be served individually in round robin manner. If it > doesn’t suit your requirement, you need to make changes as you suggested > above. > > > > Thanks > > -Venky > > > > > > On Wed, Jan 11, 2023 at 9:24 AM Singh, Jasvinder < > jasvinder.singh@intel.com> wrote: > > Hi Venky, > > > > Please see inline; > > > > Jasvinder > > > > *From:* Venky Venkatesh > *Sent:* Wednesday, January 11, 2023 11:56 AM > *To:* Singh, Jasvinder > *Cc:* dev@dpdk.org > *Subject:* Re: [2nd Try]:Re: Traffic Management API Questions > > > > Hi Jasvinder, > > Thanks for the detailed answers. Our need is to have shaping at the port > level as well. I am trying to see what would be the way to accomplish this > given the current limitations of the sched library implementation in this > regard. I see 2 options: > > - The top level (i.e. port level) documentation says the following: > "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled > in round robin order with all ports having equal priority". Questions: > > > - Do all the ports have to be of the same speed OR can it be a > heterogeneous set of port speeds? > > [JS] – the library supports single port (root node) of the hierarchy. Each > port can have multiple subports configured using different shaping rates. > If you desire to have multiple ports, each port would have separate > hierarchical tree underneath. Different ports could have different speed. > > o If it can be a heterogeneous set of ports, is the scheduling across > those ports *weighted* round robin as opposed to round robin? > > [JS] – Scheduling across multiple ports is not supported in current sched > library. At the application level, one can think of invoking > enqueue/dequeue sched API in round robin or weighted round robin manner. > > - Are Speeds other than 1/10/40 GbE not supported? > > [JS] – Speeds other than above is supported, for eg. 25G, 50G etc. > > - I suppose this heterogeneous mix of port speeds is implemented by > playing with the weights across ports, correct? > > [JS] -please see above answers > > - If so, what problem do you foresee if we provide arbitrary bandwidth > ports by regulating the above weights? > > [JS] – I don’t see any issue. > > - The other alternative would be to add another layer (which has a > shaper) to the hierarchy by mimicking one of the existing layers: how > amenable is the current implementation to that? > > Do either of the above look like workable ideas? Are there any other > approaches where we could accomplish our requirement with minimal changes > to the code logic? > > [JS] – adding another layer will require considerable work in library. How > about using multiple subports with different shaping bandwidth where each > subport maintain #subcribers and send traffic out through single physical > port (root node). It will need less effort and library supports multiple > subports under single port (root node). > > > > > > Thanks > > -Venky > > > > On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder < > jasvinder.singh@intel.com> wrote: > > Hi Venky, > > > > Please see inline. > > > > Thanks, > > Jasvinder > > > > > > *From:* Venky Venkatesh > *Sent:* Tuesday, January 10, 2023 8:52 AM > *To:* dev@dpdk.org > *Subject:* [2nd Try]:Re: Traffic Management API Questions > > > > Hi, > > Can someone pls get back on these > > Thanks > > -Venky > > > > On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh < > vvenkatesh@paloaltonetworks.com> wrote: > > Hi, > > I was looking at the DPDK Traffic Management API. I wanted to clarify some > things that I understand from the code (for software based TM > implementation (at 20.11)) vs the documentation. > > · The documentation says "Traffic shaping: single/*dual rate**,* private > (*per node*) and shared (by *multiple nodes*) shapers" are supported. > However it appears that the code supports only *single *rate shapers. Is > my understanding correct? > > [JS] – Yes, TM API supports single and dual rate shapers, privately per > node as well as shared across multiple nodes. However, DPDK QoS scheduler > library implements single rate shaper at nodes. > > o If not, pls point me to where dual rate shaping is supported in the > software based TM implementation code. > > o However, if my understanding is correct, can the authors clarify the > nature of issues they ran into in supporting dual rate (which thus > prevented them from implementing it)? > > [JS] – There isn’t any issue except more complexity. Author can rework the > library to implement the dual rate shapers for the desired nodes depending > upon the requirement. > > · The documentation comment above sounds like every node can have > shapers. However it appears that the code does not support shaping at the > port level. Again the same questions as above(regarding the accuracy of my > understanding and if it is accurate, the reasons from the author for not > supporting it) > > [JS] – Implementation supports shapers at subport (group of pipes) and > pipe level. The bandwidth available at the port level is distributed among > the subports with the condition that aggregate bandwidth of subports should > not exceed the port bandwidth. Each subport buffers and shape the traffic > from the pipes depending upon the port bandwidth allocated to it. > Implementation doesn’t support distribution of unused bandwidth of one > subport to another subport. However, one can modify this behaviour if > needed. > > · At the level of the TM API (*and* the associated software TM > implementation) are there any restrictions on the number of levels of QoS > hierarchy we can construct? > > [JS] – TM API doesn’t restrict the number of QoS scheduler levels and > generic enough to work with hierarchical schedulers with any number of > levels. The current dpdk sched library implementation supports fixed 5 > level scheduler hierarchy. > > · Lastly, does the QoS framework API (which I suppose is built on > lower level building blocks including the TM API) expose the entire > capabilities of the TM API (e.g. dual rate shapers, shapers at port level, > > 4 levels of shaping etc.)? From the reading of the documentation it > appears that there may be restrictions imposed by the QoS framework API on > top of what TM API imposes. Can someone pls confirm this (and if so, the > reason for doing so)? > > [JS] – No, QoS framework API (DPDK sched library) presents only one > flavour of hierarchical scheduler and doesn’t implements all the features > exposed through TM API. However, more features can be added to library and > configured through TM API. > > > > Thanks > > -Venky > > > >