From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9FF67423ED; Mon, 16 Jan 2023 09:05:50 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 90CA940A7D; Mon, 16 Jan 2023 09:05:50 +0100 (CET) Received: from mx0b-00169c01.pphosted.com (mx0b-00169c01.pphosted.com [67.231.156.123]) by mails.dpdk.org (Postfix) with ESMTP id F339F40042 for ; Mon, 16 Jan 2023 09:05:48 +0100 (CET) Received: from pps.filterd (m0048189.ppops.net [127.0.0.1]) by mx0b-00169c01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 30G3NJaq003105 for ; Mon, 16 Jan 2023 00:05:48 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paloaltonetworks.com; h=mime-version : references : in-reply-to : from : date : message-id : subject : to : cc : content-type; s=PPS12012017; bh=7GERzlH3iLDHZ0pLawOi/WwGGaZHEeMSV2iEfNv3GaM=; b=bJkcJEWHJadTlfNM74oBNivbvMo+bR3Y83eT7MMsct68FGBJfwASKeGjwIsqeysmNL9A zpgaG51oPARojDOEiPUdDdgQsfQFphrUkvQsyidO/q0FACWf4h4teE7IUUC5Pbkw+bqJ pH0hCP3UpevpxcKPg2JE0+otxHv/WC9A1ns90ELGRzW/03tWge3RzBuXsovCNigqagsu xycTNB5GnGJWCb8eVHrs5xyuxUrmf0cyCOBCm7PM4ybaN4LALvU3sc8JscfX9dgDCX4R 7DlCj/0BvAT3C01gefHAr+JRir/AQrOdDJpYo00W651vsT1zENtFKKeWPXS4CB7x8apE eg== Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by mx0b-00169c01.pphosted.com (PPS) with ESMTPS id 3n4xmn0h28-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 16 Jan 2023 00:05:48 -0800 Received: by mail-pj1-f72.google.com with SMTP id lk5-20020a17090b33c500b00228cb369d7aso11953586pjb.5 for ; Mon, 16 Jan 2023 00:05:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paloaltonetworks.com; s=google.paloaltonetworks.com; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=7GERzlH3iLDHZ0pLawOi/WwGGaZHEeMSV2iEfNv3GaM=; b=ZjD009yKY81poqlFWFepHGiiWmWZe8IJmYfExD5b9/zZgrpX/Yc18fZKaCsEJvMzc2 RwUhcdjVvwUhBfypQF9XpdB0nncKtTyjGZ40EuSDB3dQG992JmmlK0OQCHJknK4JcVjd 2BwSmMw2qwDUJ9AKn4H++ix5+Ysxbh9T37tIFNVmSvMCUo56qeHC0JYfvMtNGG6pZ9Fq /HHbfLEu+c1yudrysD0xgTQTe/0atoKMhTfFWfICSSZXV+ufXcBxPg7bimf66i1q0TmC Yygzf5FjpzEldhsSUM91a8SIRB6qaqSzz3HHgO5NR6ovg7u731irbfiUop0LwQmfFJrg CUHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7GERzlH3iLDHZ0pLawOi/WwGGaZHEeMSV2iEfNv3GaM=; b=3M/WbEWgKC9p+lZvOZepeNDdcQ5o3ryIgIVy7NlWKdMaY1zzDSs7VcgWTSilnfGgYC XqnBSy2gU2/rqv5JU+rl9Awr78BJvVpNKGioqiyiowAhp4mbUGnP0hbSXdBntEL6erw0 6ZHAWPO0wFlA9x0spWeo+YoZwYk89J9iBQW1csG/AUCIKJZZzt07/RuPRj9lL2N1wTPy n3oMSoukrYNLzkeMAORgj+e1LJiYCFXdwahlhNxmzzdyUar0CmwrsPtbtkB1JJ6qpahP Sbjz/E8YB1s5YOGqxYR3lP0a3K79Qq7ksh6Tb59M7W8142ZQbdKmcFl0yHKVMiq8Bx29 8d6Q== X-Gm-Message-State: AFqh2kpF3fLOUrt0jA4T31/kN6+Gmck6Va94GXXDk0bQAevkJvj5KMFH PfuiA87Aa1hPK4ev4/MtZahG7enkHjf77AlL0gdk2c425oA2ax0R0Ft1Qaa4rGLQfo2qpSY7Qfe mE6Gq7IRH74EwLY1bJkA= X-Received: by 2002:a17:90a:9c8:b0:229:3cec:913 with SMTP id 66-20020a17090a09c800b002293cec0913mr632647pjo.48.1673856346453; Mon, 16 Jan 2023 00:05:46 -0800 (PST) X-Google-Smtp-Source: AMrXdXsQWe/7wp8Ka0REkNrNjoy4YEO4e4dxkYveqy9Be4vtOaRlp6tTeqbKObalamarq9El/UsJCnr2mjvsDkwAQwg= X-Received: by 2002:a17:90a:9c8:b0:229:3cec:913 with SMTP id 66-20020a17090a09c800b002293cec0913mr632642pjo.48.1673856346051; Mon, 16 Jan 2023 00:05:46 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Venky Venkatesh Date: Mon, 16 Jan 2023 00:05:34 -0800 Message-ID: Subject: Re: [2nd Try]:Re: Traffic Management API Questions To: "Singh, Jasvinder" Cc: "dev@dpdk.org" Content-Type: multipart/alternative; boundary="000000000000e9f01405f25d0cc5" X-Proofpoint-GUID: QyoRkzgtdFvXz6_7tDjpB5GrHNpIyKxK X-Proofpoint-ORIG-GUID: QyoRkzgtdFvXz6_7tDjpB5GrHNpIyKxK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.923,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-01-16_06,2023-01-13_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 mlxscore=0 suspectscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 spamscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301160059 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000e9f01405f25d0cc5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Jasvinder, Thanks for the insights on the complexity of adding a layer. As for the workaround that you suggested using multiple subports, if I understand it correctly (*pls correct if I misunderstood*) it would not meet our needs: - We require multiple heterogeneous ports (i.e. ports with different bandwidths -- with no excess sharing since these are port limits). That would probably need some shaper attached there too since WRR (at the application) would share the instantaneous excess among the siblings. - As for the 2nd second suggestion (increase the number of subports): our need (*in addition to* multiple ports of different bandwidths at the top level) is to have 4 more TM layers for a total of 5. I am *not* looking at the assignment of the terms port/subport/user/pipe etc in the DPDK documentation -- instead am looking at it as abstract scheduling (and/or shaping) layers with differing abilities in some layers. So in order to compensate for the missing shaper at the port level I was plann= ing to add 1 additional layer (so that what in DPDK documentation is referre= d to as subport is actually a port -- since the subport layer has the property of *not sharing* excess between siblings. With that principle, I am not clear how adding width to the subport layer (as I understand yo= ur suggestion) would help. Thanks -Venky On Wed, Jan 11, 2023 at 9:24 AM Singh, Jasvinder wrote: > Hi Venky, > > > > Please see inline; > > > > Jasvinder > > > > *From:* Venky Venkatesh > *Sent:* Wednesday, January 11, 2023 11:56 AM > *To:* Singh, Jasvinder > *Cc:* dev@dpdk.org > *Subject:* Re: [2nd Try]:Re: Traffic Management API Questions > > > > Hi Jasvinder, > > Thanks for the detailed answers. Our need is to have shaping at the port > level as well. I am trying to see what would be the way to accomplish thi= s > given the current limitations of the sched library implementation in this > regard. I see 2 options: > > - The top level (i.e. port level) documentation says the following: > "Output Ethernet port 1/10/40 GbE" and "Multiple ports are scheduled > in round robin order with all ports having equal priority". Questions: > > > - Do all the ports have to be of the same speed OR can it be a > heterogeneous set of port speeds? > > [JS] =E2=80=93 the library supports single port (root node) of the hierar= chy. Each > port can have multiple subports configured using different shaping rates. > If you desire to have multiple ports, each port would have separate > hierarchical tree underneath. Different ports could have different speed. > > o If it can be a heterogeneous set of ports, is the scheduling across > those ports *weighted* round robin as opposed to round robin? > > [JS] =E2=80=93 Scheduling across multiple ports is not supported in curre= nt sched > library. At the application level, one can think of invoking > enqueue/dequeue sched API in round robin or weighted round robin manner. > > - Are Speeds other than 1/10/40 GbE not supported? > > [JS] =E2=80=93 Speeds other than above is supported, for eg. 25G, 50G etc= . > > - I suppose this heterogeneous mix of port speeds is implemented by > playing with the weights across ports, correct? > > [JS] -please see above answers > > - If so, what problem do you foresee if we provide arbitrary bandwidth > ports by regulating the above weights? > > [JS] =E2=80=93 I don=E2=80=99t see any issue. > > - The other alternative would be to add another layer (which has a > shaper) to the hierarchy by mimicking one of the existing layers: how > amenable is the current implementation to that? > > Do either of the above look like workable ideas? Are there any other > approaches where we could accomplish our requirement with minimal changes > to the code logic? > > [JS] =E2=80=93 adding another layer will require considerable work in lib= rary. How > about using multiple subports with different shaping bandwidth where each > subport maintain #subcribers and send traffic out through single physical > port (root node). It will need less effort and library supports multiple > subports under single port (root node). > > > > > > Thanks > > -Venky > > > > On Tue, Jan 10, 2023 at 2:54 AM Singh, Jasvinder < > jasvinder.singh@intel.com> wrote: > > Hi Venky, > > > > Please see inline. > > > > Thanks, > > Jasvinder > > > > > > *From:* Venky Venkatesh > *Sent:* Tuesday, January 10, 2023 8:52 AM > *To:* dev@dpdk.org > *Subject:* [2nd Try]:Re: Traffic Management API Questions > > > > Hi, > > Can someone pls get back on these > > Thanks > > -Venky > > > > On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh < > vvenkatesh@paloaltonetworks.com> wrote: > > Hi, > > I was looking at the DPDK Traffic Management API. I wanted to clarify som= e > things that I understand from the code (for software based TM > implementation (at 20.11)) vs the documentation. > > =C2=B7 The documentation says "Traffic shaping: single/*dual rate= **,* private > (*per node*) and shared (by *multiple nodes*) shapers" are supported. > However it appears that the code supports only *single *rate shapers. Is > my understanding correct? > > [JS] =E2=80=93 Yes, TM API supports single and dual rate shapers, private= ly per > node as well as shared across multiple nodes. However, DPDK QoS scheduler > library implements single rate shaper at nodes. > > o If not, pls point me to where dual rate shaping is supported in the > software based TM implementation code. > > o However, if my understanding is correct, can the authors clarify the > nature of issues they ran into in supporting dual rate (which thus > prevented them from implementing it)? > > [JS] =E2=80=93 There isn=E2=80=99t any issue except more complexity. Auth= or can rework the > library to implement the dual rate shapers for the desired nodes dependin= g > upon the requirement. > > =C2=B7 The documentation comment above sounds like every node can= have > shapers. However it appears that the code does not support shaping at the > port level. Again the same questions as above(regarding the accuracy of m= y > understanding and if it is accurate, the reasons from the author for not > supporting it) > > [JS] =E2=80=93 Implementation supports shapers at subport (group of pipes= ) and > pipe level. The bandwidth available at the port level is distributed amon= g > the subports with the condition that aggregate bandwidth of subports shou= ld > not exceed the port bandwidth. Each subport buffers and shape the traffic > from the pipes depending upon the port bandwidth allocated to it. > Implementation doesn=E2=80=99t support distribution of unused bandwidth o= f one > subport to another subport. However, one can modify this behaviour if > needed. > > =C2=B7 At the level of the TM API (*and* the associated software = TM > implementation) are there any restrictions on the number of levels of QoS > hierarchy we can construct? > > [JS] =E2=80=93 TM API doesn=E2=80=99t restrict the number of QoS schedule= r levels and > generic enough to work with hierarchical schedulers with any number of > levels. The current dpdk sched library implementation supports fixed 5 > level scheduler hierarchy. > > =C2=B7 Lastly, does the QoS framework API (which I suppose is bui= lt on > lower level building blocks including the TM API) expose the entire > capabilities of the TM API (e.g. dual rate shapers, shapers at port level= , > > 4 levels of shaping etc.)? From the reading of the documentation it > appears that there may be restrictions imposed by the QoS framework API o= n > top of what TM API imposes. Can someone pls confirm this (and if so, the > reason for doing so)? > > [JS] =E2=80=93 No, QoS framework API (DPDK sched library) presents only o= ne > flavour of hierarchical scheduler and doesn=E2=80=99t implements all the = features > exposed through TM API. However, more features can be added to library a= nd > configured through TM API. > > > > Thanks > > -Venky > > > > --000000000000e9f01405f25d0cc5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Jasvinder,
Thanks for the insights on the complexit= y of adding a layer.
As for the workaround that you suggested usi= ng multiple subports, if I understand it correctly (pls correct if I mis= understood) it would not meet our needs:
  • We require m= ultiple heterogeneous ports (i.e. ports with different bandwidths -- with n= o excess sharing since these are port limits). That would probably need som= e shaper attached there too since WRR (at the application) would share the = instantaneous excess among the siblings.
  • As for the 2nd second sugg= estion (increase the number of subports): our need (in addition to m= ultiple ports of different bandwidths at the top level) is to have=C2=A0 4 = more TM layers for a total of 5. I am not looking at the assignment = of the terms port/subport/user/pipe etc in the DPDK documentation -- instea= d am looking at it as abstract scheduling (and/or shaping) layers with diff= ering abilities in some layers. So in order to compensate for the missing s= haper at the port level I was planning to add 1 additional layer (so that w= hat in DPDK documentation is referred to as subport is actually a port -- s= ince the subport layer has the property of not sharing excess betwee= n siblings. With that principle, I am not clear how adding width to the sub= port layer (as I understand your suggestion) would help.

Thanks
-Venky


On Wed, Jan 11= , 2023 at 9:24 AM Singh, Jasvinder <jasvinder.singh@intel.com> wrote:

Hi Venky,

=C2=A0

Please see inline;

=C2=A0

Jasvinder

=C2=A0

From: Venky Venkatesh <vvenkatesh@paloaltonetworks.com>
Sent: Wednesday, January 11, 2023 11:56 AM
To: Singh, Jasvinder <jasvinder.singh@intel.com>
Cc: dev@dpdk.org
Subject: Re: [2nd Try]:Re: Traffic Management API Questions

=C2=A0

Hi Jasvinder,

Thanks for the detailed answers. Our need is to have= shaping at the port level as well. I am trying to see what would be the wa= y to accomplish this given the current limitations of the sched library imp= lementation in this regard. I see 2 options:

  • The top level (i.e. port level) documentation says the following: "Out= put Ethernet port 1/10/40 GbE" and "Multiple por= ts are scheduled in round robin order with all ports having equal priority". Questions:
    • Do all the ports have to be of the same speed OR can it be a heterogeneous = set of port speeds?

[JS] =E2=80=93 the library supports single port (roo= t node) of the hierarchy. Each port can have multiple subports configured u= sing different shaping rates. If you desire to have multiple ports, each port would have separate hierarchical tree underneath. Differe= nt ports could have different speed.

<= span>o If it can be a heterogeneous set of ports, is t= he scheduling across those ports=C2=A0weighted round robin as= opposed to round robin?

[JS] =E2=80=93 Scheduling across multiple ports is n= ot supported in current sched library. At the application level, one can th= ink of invoking enqueue/dequeue sched API in round robin or weighted round robin manner.

    • Are Speeds other than=C2=A0=C2=A01/10/40 GbE not supported?=C2=A0=

[JS] =E2=80=93 Speeds other than above is supported,= for eg. 25G, 50G etc.

    • I suppose this heterogeneous=C2=A0mix of port speeds is implemented by play= ing with the weights across ports, correct?

[JS] -please see above answers

    • If so, what problem do you foresee if we provide arbitrary bandwidth ports = by regulating the above weights?

[JS] =E2=80=93 I don=E2=80=99t see any issue.

  • The other alternative would be to add another layer (which has a shaper)=C2= =A0=C2=A0to the hierarchy by mimicking one of the existing layers: how amen= able is the current implementation to that?

Do either of the above look like workable ideas? Are= there any other approaches where we could accomplish our requirement with = minimal changes to the code logic?

[JS] =E2=80=93 adding another layer will require con= siderable work in library. How about using multiple subports with different= shaping bandwidth where each subport maintain #subcribers and send traffic out through single physical port (root node). It will nee= d less effort and library supports multiple subports under single port (roo= t node).

=C2=A0

=C2=A0

Thanks

-Venky

=C2=A0

Hi Venky,

=C2=A0

Please see inline.

=C2=A0

Thanks,

Jasvinder

=C2=A0

=C2=A0

From: Venky Venkatesh <vvenkatesh@paloaltone= tworks.com>
Sent: Tuesday, January 10, 2023 8:52 AM
To: dev@dpdk.org
Subject: [2nd Try]:Re: Traffic Management API Questions

=C2=A0

Hi,

Can someone pls get back on these

Thanks

-Venky

=C2=A0

On Thu, Jan 5, 2023 at 4:07 AM Venky Venkatesh <<= a href=3D"mailto:vvenkatesh@paloaltonetworks.com" target=3D"_blank">vvenkat= esh@paloaltonetworks.com> wrote:

Hi,

I was looking at the DPDK Traffic Management API. I = wanted to clarify some things that I understand from the code (for software= based TM implementation (at 20.11)) vs the documentation.

=C2=B7=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 The documentation says "Traffic shapin= g: single/dual rate,=C2=A0private (per node) an= d shared (by=C2=A0multiple nodes) shapers" are supported= . However it appears that the code supports only=C2=A0single=C2=A0rate shaper= s. Is my understanding correct?=C2=A0

[JS] =E2=80=93 Yes, TM API supports single and dual = rate shapers, privately per node as well as shared across multiple nodes. H= owever, DPDK QoS scheduler library implements single rate shaper at nodes.

o= =C2=A0=C2=A0=C2=A0 If not, pls point me to where dual rate shaping is supported in the = software based TM implementation code.=C2=A0

o= =C2=A0=C2=A0=C2=A0 However, if my understanding is correct, can the authors clarify the= nature of issues they ran into in supporting dual rate (which thus prevent= ed them from implementing it)?

[JS] =E2=80=93 There isn=E2=80=99t any issue except = more complexity. Author can rework the library to implement the dual rate s= hapers for the desired nodes depending upon the requirement. =C2=A0<= u>

=C2=B7=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 The documentation comment above sounds like every node can have shap= ers. However it appears that the code does not support shaping at the port = level. Again the same questions as above(regarding the accuracy of my under= standing and if it is accurate, the reasons from the author for not supporting it)

[JS] =E2=80=93 Implementation supports shapers at su= bport (group of pipes) and pipe level. The bandwidth available at the port = level is distributed among the subports with the condition that aggregate bandwidth of subports should not exceed the port bandwidth.= Each subport buffers and shape the traffic from the pipes depending upon t= he port bandwidth allocated to it. Implementation doesn=E2=80=99t support d= istribution of unused bandwidth of one subport to another subport. However, one can modify this behaviour if needed. =C2= =A0=C2=A0

=C2=B7=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 At the level of the TM API (and=C2=A0the associated software = TM implementation) are there any restrictions on the number of levels of Qo= S hierarchy we can construct?

[JS] =E2=80=93 TM API doesn=E2=80=99t restrict the n= umber of QoS scheduler levels and generic enough to work with hierarchical = schedulers with any number of levels. The current dpdk sched library implementation supports fixed 5 level scheduler hierarchy. =C2=A0=C2=A0=C2= =A0

=C2=B7=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Lastly, does the QoS framework API (which I suppose is built on lowe= r level building blocks including the TM API) expose the entire capabilitie= s of the TM API (e.g. dual rate shapers, shapers at port level, > 4 leve= ls of shaping etc.)? From the reading of the documentation it appears that there may be restrictions imposed by = the QoS framework API on top of what TM API imposes. Can someone pls confir= m this (and if so, the reason for=C2=A0doing so)?

[JS] =E2=80=93 No, QoS framework API (DPDK sched lib= rary) presents only one flavour of hierarchical scheduler and doesn=E2=80= =99t implements all the features exposed through TM API.=C2=A0 However, mor= e features can be added to library and configured through TM API. =

=C2=A0

Thanks

-Venky=

=C2=A0=

--000000000000e9f01405f25d0cc5--