[dpdk-dev] DPDK-QoS - link sharing across classes

[dpdk-dev] DPDK-QoS - link sharing across classes

[sreenaath vasudevan]

Hi
I currently have QoS implemented in hardware and I am thinking of using
DPDK's QoS feature to move it to software.
Currently in the hardware,Based on the 4 class per pipe and 4 queues per
class limitation, I was thinking of using 4 classes in DPDK-QoS and spread
out the 8 h/w queues across the 4 classes.
Let me explain with an example. Currently, this is how the h/w queue is
represented
Q0 - 10% b/w
Q1- 10%  b/w
Q2- 15% b/w
Q3 - 15% b/w
Q4 - 15% b/w
Q5 - 15% b/w
Q6 - 10% b/w
Q7 - 10% b/w

Translating the above config to DPDK-QoS, based on my application need, Q0
and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2, Q3,
Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
Q7 can be logically grouped under class 3 with super b/w = 20%.

However, in the h/w, link sharing is available across all the 8 queues.
DPDK materials say link sharing "typically" is enabled for last class, in
this case class2. However, I also want class 1 or class 0 to use the
remaining bandwidth when class2 does not have any traffic and so on. Can
this be done in DPDK ? Do we have a concept of min and max b/w guarantee at
the class level in DPDK-QoS ?

Thanks !

-- 
regards
sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of sreenaath
> vasudevan
> Sent: Tuesday, February 2, 2016 9:09 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
> 
> Hi
> I currently have QoS implemented in hardware and I am thinking of using
> DPDK's QoS feature to move it to software.
> Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> class limitation, I was thinking of using 4 classes in DPDK-QoS and spread
> out the 8 h/w queues across the 4 classes.
> Let me explain with an example. Currently, this is how the h/w queue is
> represented
> Q0 - 10% b/w
> Q1- 10%  b/w
> Q2- 15% b/w
> Q3 - 15% b/w
> Q4 - 15% b/w
> Q5 - 15% b/w
> Q6 - 10% b/w
> Q7 - 10% b/w
> 
> Translating the above config to DPDK-QoS, based on my application need, Q0
> and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2, Q3,
> Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> Q7 can be logically grouped under class 3 with super b/w = 20%.
> 
> However, in the h/w, link sharing is available across all the 8 queues.
> DPDK materials say link sharing "typically" is enabled for last class, in
> this case class2. However, I also want class 1 or class 0 to use the
> remaining bandwidth when class2 does not have any traffic and so on. Can
> this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> at
> the class level in DPDK-QoS ?

Your requirements seem to be:
- minimal rate for each class (with rates fully provisioned, i.e. sum of rate not exceeding 100%)
- avoid b/w waste by redistributing unused b/w to the traffic classes that currently have traffic according to their relative weights

In our implementation, traffic classes are scheduled in strict priority (with upper limit defined, to avoid starvation), so TC0 traffic is always prioritized for the current pipe over TC1 .. TC3 traffic (as long as current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore, the traffic class hierarchy level is not going to help you here.

On the other hand, if you map all your queues/traffic classes to the queues of one of our pipe traffic classes (it does not matter which one of TC0 .. TC3 you pick, as long as you pick just one), your requirements become possible, as we have 4 queues per each pipe traffic class, and they are scheduled using weighted fair queuing (byte-level weighted round robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%), your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which should work exactly what you need. Makes sense?

Regards,
Cristian

> 
> Thanks !
> 
> --
> regards
> sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[sreenaath vasudevan]

Hi Cristian
Thanks for detailed response.
Your solution works so long as I have four queues in my current
implementation.

Following are the two issues I have now
1. I have 8 queues in the current implementation. This means I need to map
the existing 8 queues to two sets of 4 queues across two different classes
(C0 and C1) in DPDK-QOs right? The problem with that approach is that queue
weights are not relative across classes. Is there a way to work around this?
2. B/w redistribution -
     a) The moment I map the current implementation's 8 queues across two
different classes (say C0 and C1), would remaining b/w be distributed
across the two classes C0 and C1? Is it true that in the current DPDK-QoS
implementation, unused b/w gets distributed only to the last class C3?
Would not un-used b/w from C0 come to C1?
     b) In current DPDK QoS implemention, if C1 has un-used b/w would that
get used by C0? Or is it only "lower priority class (C3, more specifically)
uses the un-used b/w from higher priority classes (C0,C1,C2) ?




On Tue, Feb 16, 2016 at 2:46 PM, Dumitrescu, Cristian <
cristian.dumitrescu@intel.com> wrote:

>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of sreenaath
> > vasudevan
> > Sent: Tuesday, February 2, 2016 9:09 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
> >
> > Hi
> > I currently have QoS implemented in hardware and I am thinking of using
> > DPDK's QoS feature to move it to software.
> > Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> > class limitation, I was thinking of using 4 classes in DPDK-QoS and
> spread
> > out the 8 h/w queues across the 4 classes.
> > Let me explain with an example. Currently, this is how the h/w queue is
> > represented
> > Q0 - 10% b/w
> > Q1- 10%  b/w
> > Q2- 15% b/w
> > Q3 - 15% b/w
> > Q4 - 15% b/w
> > Q5 - 15% b/w
> > Q6 - 10% b/w
> > Q7 - 10% b/w
> >
> > Translating the above config to DPDK-QoS, based on my application need,
> Q0
> > and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2,
> Q3,
> > Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> > Q7 can be logically grouped under class 3 with super b/w = 20%.
> >
> > However, in the h/w, link sharing is available across all the 8 queues.
> > DPDK materials say link sharing "typically" is enabled for last class, in
> > this case class2. However, I also want class 1 or class 0 to use the
> > remaining bandwidth when class2 does not have any traffic and so on. Can
> > this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> > at
> > the class level in DPDK-QoS ?
>
> Your requirements seem to be:
> - minimal rate for each class (with rates fully provisioned, i.e. sum of
> rate not exceeding 100%)
> - avoid b/w waste by redistributing unused b/w to the traffic classes that
> currently have traffic according to their relative weights
>
> In our implementation, traffic classes are scheduled in strict priority
> (with upper limit defined, to avoid starvation), so TC0 traffic is always
> prioritized for the current pipe over TC1 .. TC3 traffic (as long as
> current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore,
> the traffic class hierarchy level is not going to help you here.
>
> On the other hand, if you map all your queues/traffic classes to the
> queues of one of our pipe traffic classes (it does not matter which one of
> TC0 .. TC3 you pick, as long as you pick just one), your requirements
> become possible, as we have 4 queues per each pipe traffic class, and they
> are scheduled using weighted fair queuing (byte-level weighted round
> robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%),
> your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our
> pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which
> should work exactly what you need. Makes sense?
>
> Regards,
> Cristian
>
> >
> > Thanks !
> >
> > --
> > regards
> > sreenaath
>



-- 
regards
sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[Dumitrescu, Cristian]

From: sreenaath vasudevan [mailto:sreenaathkv@gmail.com]
Sent: Thursday, February 18, 2016 8:01 PM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK-QoS - link sharing across classes

Hi Cristian
Thanks for detailed response.
Your solution works so long as I have four queues in my current implementation.
[Cristian] Yes, I was relying on you saying you can actually group Q0 and Q1 together, as well as Q2 and Q3, Q4 and Q5, Q6 and Q7.

Following are the two issues I have now
1. I have 8 queues in the current implementation. This means I need to map the existing 8 queues to two sets of 4 queues across two different classes  (C0 and C1) in DPDK-QOs right? The problem with that approach is that queue weights are not relative across classes. Is there a way to work around this?
[Cristian] You can, of course, change to code to implement 8 queues per pipe traffic class, but this is not going to be a trivial exercise. With the current implementation unmodified, you could simply map 4 queues to e.g. TC0 and 4 queues to e.g. TC1, provided that it makes sense to prioritize the 4 queues of TC0 as higher priority than those 4 queues mapped to TC1, so basically you are OK with having strict priority (up to an upper limit) between the 2 sets of 4 queues.


2. B/w redistribution -
     a) The moment I map the current implementation's 8 queues across two different classes (say C0 and C1), would remaining b/w be distributed across the two classes C0 and C1? Is it true that in the current DPDK-QoS implementation, unused b/w gets distributed only to the last class C3? Would not un-used b/w from C0 come to C1?
[Cristian] With 4 queues mapped to TC0 and 4 queues mapped to TC1, you can set TC0 rate to X% of pipe rate and TC1 rate to 100% of pipe rate. This is the idea behind strict priority: the sum of TC rates is usually bigger than 100% of pipe rate, but this is fine, as due to strict priority the lowest priority TC used (which in this case is TC1, not TC3, as you are not using TC2 and TC3 at all) only gets 100% of the pipe rate when no traffic from higher priority TCs exists. In this case, the extremes are: (1) TC0 demand is high, so TC0 uses everything up to X%, so TC1 can use a max of (100 – X)% and (2) TC0 demand is zero, in which case TC1 is free to use up to 100% of pipe rate. So the answers to your 3 questions are: yes/no/yes.

     b) In current DPDK QoS implemention, if C1 has un-used b/w would that get used by C0? Or is it only "lower priority class (C3, more specifically) uses the un-used b/w from higher priority classes (C0,C1,C2) ?
[Cristian] Due to strict priority, it does not make sense to think about higher priority classes using whatever is not used by low priority classes: If high priority classes have traffic to send, they will always be picked in the detriment of lower ones; when the high priority classes hit their upper limit rate, then they are not allowed to send more, otherwise the upper limit makes no sense. So the bandwidth reuse concept makes sense only for low prio TCs to reuse whatever is not used from high prio TCs. Usually, this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100% of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 .. TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as well when only TC0 and TC1 are actually used. So the answers to your 2 questions are: no/no.


On Tue, Feb 16, 2016 at 2:46 PM, Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>> wrote:


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of sreenaath
> vasudevan
> Sent: Tuesday, February 2, 2016 9:09 PM
> To: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
>
> Hi
> I currently have QoS implemented in hardware and I am thinking of using
> DPDK's QoS feature to move it to software.
> Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> class limitation, I was thinking of using 4 classes in DPDK-QoS and spread
> out the 8 h/w queues across the 4 classes.
> Let me explain with an example. Currently, this is how the h/w queue is
> represented
> Q0 - 10% b/w
> Q1- 10%  b/w
> Q2- 15% b/w
> Q3 - 15% b/w
> Q4 - 15% b/w
> Q5 - 15% b/w
> Q6 - 10% b/w
> Q7 - 10% b/w
>
> Translating the above config to DPDK-QoS, based on my application need, Q0
> and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2, Q3,
> Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> Q7 can be logically grouped under class 3 with super b/w = 20%.
>
> However, in the h/w, link sharing is available across all the 8 queues.
> DPDK materials say link sharing "typically" is enabled for last class, in
> this case class2. However, I also want class 1 or class 0 to use the
> remaining bandwidth when class2 does not have any traffic and so on. Can
> this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> at
> the class level in DPDK-QoS ?
Your requirements seem to be:
- minimal rate for each class (with rates fully provisioned, i.e. sum of rate not exceeding 100%)
- avoid b/w waste by redistributing unused b/w to the traffic classes that currently have traffic according to their relative weights

In our implementation, traffic classes are scheduled in strict priority (with upper limit defined, to avoid starvation), so TC0 traffic is always prioritized for the current pipe over TC1 .. TC3 traffic (as long as current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore, the traffic class hierarchy level is not going to help you here.

On the other hand, if you map all your queues/traffic classes to the queues of one of our pipe traffic classes (it does not matter which one of TC0 .. TC3 you pick, as long as you pick just one), your requirements become possible, as we have 4 queues per each pipe traffic class, and they are scheduled using weighted fair queuing (byte-level weighted round robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%), your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which should work exactly what you need. Makes sense?

Regards,
Cristian

>
> Thanks !
>
> --
> regards
> sreenaath



--
regards
sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[sreenaath vasudevan]

b) In current DPDK QoS implemention, if C1 has un-used b/w would that get
used by C0? Or is it only "lower priority class (C3, more specifically)
uses the un-used b/w from higher priority classes (C0,C1,C2) ?

[Cristian] Due to strict priority, it does not make sense to think about
higher priority classes using whatever is not used by low priority classes:
If high priority classes have traffic to send, they will always be picked
in the detriment of lower ones; when the high priority classes hit their
upper limit rate, then they are not allowed to send more, otherwise the
upper limit makes no sense. So the bandwidth reuse concept makes sense only
for low prio TCs to reuse whatever is not used from high prio TCs. Usually,
this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100%
of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 ..
TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and
100% of pipe rate. As explained above, this can be applicable to e.g TC1 as
well when only TC0 and TC1 are actually used. So the answers to your 2
questions are: no/no.



>> Sreenaath - Let's say I have two classes TC0 and TC1 each having a
single queue. Let's say I give TC0 60% and TC1 100%. So according to DPDK's
implementation, TC1 can use 100% of the bandwidth if TC0 is inactive, while
TC0 at any time cannot exceed 60%, even if TC1 is inactive. Is that correct
? And since you mentioned that this is strict priority, according to this
implementation, high priority traffic cannot use 100% of the bw even if
there is no low priority traffic, but low priority traffic can use 100% of
the bw if there is no high priority traffic. Is my understanding correct ?
This doesn't make sense.

On Fri, Feb 19, 2016 at 7:02 AM, Dumitrescu, Cristian <
cristian.dumitrescu@intel.com> wrote:

>
>
>
>
> *From:* sreenaath vasudevan [mailto:sreenaathkv@gmail.com]
> *Sent:* Thursday, February 18, 2016 8:01 PM
> *To:* Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> *Cc:* dev@dpdk.org
> *Subject:* Re: [dpdk-dev] DPDK-QoS - link sharing across classes
>
>
>
> Hi Cristian
>
> Thanks for detailed response.
>
> Your solution works so long as I have four queues in my current
> implementation.
>
> [Cristian] Yes, I was relying on you saying you can actually group Q0 and
> Q1 together, as well as Q2 and Q3, Q4 and Q5, Q6 and Q7.
>
>
>
> Following are the two issues I have now
>
> 1. I have 8 queues in the current implementation. This means I need to map
> the existing 8 queues to two sets of 4 queues across two different classes
> (C0 and C1) in DPDK-QOs right? The problem with that approach is that queue
> weights are not relative across classes. Is there a way to work around this?
>
> [Cristian] You can, of course, change to code to implement 8 queues per
> pipe traffic class, but this is not going to be a trivial exercise. With
> the current implementation unmodified, you could simply map 4 queues to
> e.g. TC0 and 4 queues to e.g. TC1, provided that it makes sense to
> prioritize the 4 queues of TC0 as higher priority than those 4 queues
> mapped to TC1, so basically you are OK with having strict priority (up to
> an upper limit) between the 2 sets of 4 queues.
>
>
>
>
>
> 2. B/w redistribution -
>      a) The moment I map the current implementation's 8 queues across two
> different classes (say C0 and C1), would remaining b/w be distributed
> across the two classes C0 and C1? Is it true that in the current DPDK-QoS
> implementation, unused b/w gets distributed only to the last class C3?
> Would not un-used b/w from C0 come to C1?
>
> [Cristian] With 4 queues mapped to TC0 and 4 queues mapped to TC1, you can
> set TC0 rate to X% of pipe rate and TC1 rate to 100% of pipe rate. This is
> the idea behind strict priority: the sum of TC rates is usually bigger than
> 100% of pipe rate, but this is fine, as due to strict priority the lowest
> priority TC used (which in this case is TC1, not TC3, as you are not using
> TC2 and TC3 at all) only gets 100% of the pipe rate when no traffic from
> higher priority TCs exists. In this case, the extremes are: (1) TC0 demand
> is high, so TC0 uses everything up to X%, so TC1 can use a max of (100 –
> X)% and (2) TC0 demand is zero, in which case TC1 is free to use up to 100%
> of pipe rate. So the answers to your 3 questions are: yes/no/yes.
>
>
>
>      b) In current DPDK QoS implemention, if C1 has un-used b/w would that
> get used by C0? Or is it only "lower priority class (C3, more specifically)
> uses the un-used b/w from higher priority classes (C0,C1,C2) ?
>
> [Cristian] Due to strict priority, it does not make sense to think about
> higher priority classes using whatever is not used by low priority classes:
> If high priority classes have traffic to send, they will always be picked
> in the detriment of lower ones; when the high priority classes hit their
> upper limit rate, then they are not allowed to send more, otherwise the
> upper limit makes no sense. So the bandwidth reuse concept makes sense only
> for low prio TCs to reuse whatever is not used from high prio TCs. Usually,
> this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100%
> of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 ..
> TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and
> 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as
> well when only TC0 and TC1 are actually used. So the answers to your 2
> questions are: no/no.
>
>
>
>
>
> On Tue, Feb 16, 2016 at 2:46 PM, Dumitrescu, Cristian <
> cristian.dumitrescu@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of sreenaath
> > vasudevan
> > Sent: Tuesday, February 2, 2016 9:09 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
> >
> > Hi
> > I currently have QoS implemented in hardware and I am thinking of using
> > DPDK's QoS feature to move it to software.
> > Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> > class limitation, I was thinking of using 4 classes in DPDK-QoS and
> spread
> > out the 8 h/w queues across the 4 classes.
> > Let me explain with an example. Currently, this is how the h/w queue is
> > represented
> > Q0 - 10% b/w
> > Q1- 10%  b/w
> > Q2- 15% b/w
> > Q3 - 15% b/w
> > Q4 - 15% b/w
> > Q5 - 15% b/w
> > Q6 - 10% b/w
> > Q7 - 10% b/w
> >
> > Translating the above config to DPDK-QoS, based on my application need,
> Q0
> > and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2,
> Q3,
> > Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> > Q7 can be logically grouped under class 3 with super b/w = 20%.
> >
> > However, in the h/w, link sharing is available across all the 8 queues.
> > DPDK materials say link sharing "typically" is enabled for last class, in
> > this case class2. However, I also want class 1 or class 0 to use the
> > remaining bandwidth when class2 does not have any traffic and so on. Can
> > this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> > at
> > the class level in DPDK-QoS ?
>
> Your requirements seem to be:
> - minimal rate for each class (with rates fully provisioned, i.e. sum of
> rate not exceeding 100%)
> - avoid b/w waste by redistributing unused b/w to the traffic classes that
> currently have traffic according to their relative weights
>
> In our implementation, traffic classes are scheduled in strict priority
> (with upper limit defined, to avoid starvation), so TC0 traffic is always
> prioritized for the current pipe over TC1 .. TC3 traffic (as long as
> current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore,
> the traffic class hierarchy level is not going to help you here.
>
> On the other hand, if you map all your queues/traffic classes to the
> queues of one of our pipe traffic classes (it does not matter which one of
> TC0 .. TC3 you pick, as long as you pick just one), your requirements
> become possible, as we have 4 queues per each pipe traffic class, and they
> are scheduled using weighted fair queuing (byte-level weighted round
> robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%),
> your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our
> pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which
> should work exactly what you need. Makes sense?
>
> Regards,
> Cristian
>
>
> >
> > Thanks !
> >
> > --
> > regards
> > sreenaath
>
>
>
>
> --
>
> regards
> sreenaath
>



-- 
regards
sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[Dumitrescu, Cristian]

From: sreenaath vasudevan [mailto:sreenaathkv@gmail.com]
Sent: Wednesday, February 24, 2016 12:55 AM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK-QoS - link sharing across classes


b) In current DPDK QoS implemention, if C1 has un-used b/w would that get used by C0? Or is it only "lower priority class (C3, more specifically) uses the un-used b/w from higher priority classes (C0,C1,C2) ?

[Cristian] Due to strict priority, it does not make sense to think about higher priority classes using whatever is not used by low priority classes: If high priority classes have traffic to send, they will always be picked in the detriment of lower ones; when the high priority classes hit their upper limit rate, then they are not allowed to send more, otherwise the upper limit makes no sense. So the bandwidth reuse concept makes sense only for low prio TCs to reuse whatever is not used from high prio TCs. Usually, this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100% of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 .. TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as well when only TC0 and TC1 are actually used. So the answers to your 2 questions are: no/no.



>> Sreenaath - Let's say I have two classes TC0 and TC1 each having a single queue. Let's say I give TC0 60% and TC1 100%. So according to DPDK's implementation, TC1 can use 100% of the bandwidth if TC0 is inactive, while TC0 at any time cannot exceed 60%, even if TC1 is inactive. Is that correct ? And since you mentioned that this is strict priority, according to this implementation, high priority traffic cannot use 100% of the bw even if there is no low priority traffic, but low priority traffic can use 100% of the bw if there is no high priority traffic. Is my understanding correct ? This doesn't make sense.
[Cristian] It does to me, as high priority traffic is usually low bandwidth and typically fully provisioned (how many voice connections, RTP video conferences, 911 calls, signalling traffic, etc can you have active for a given user at a given time?), while most of the Internet traffic is best effort (low priority) and it is typically overprovisioned 20:1 or more.
You can set TC0 rate to 100% of pipe rate, but this is totally not recommended, because when TC0 demand is exactly 100% you are going to starve the low priority TC1 .. TC2 completely. You can make Weighted Fair Queuing (WFQ) behave like strict priority by using big weight ratios like 4:1 or 16:1, but you cannot make strict priority behave like WFQ (which is what you seem to be looking for).


On Fri, Feb 19, 2016 at 7:02 AM, Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>> wrote:


From: sreenaath vasudevan [mailto:sreenaathkv@gmail.com<mailto:sreenaathkv@gmail.com>]
Sent: Thursday, February 18, 2016 8:01 PM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>>
Cc: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [dpdk-dev] DPDK-QoS - link sharing across classes

Hi Cristian
Thanks for detailed response.
Your solution works so long as I have four queues in my current implementation.
[Cristian] Yes, I was relying on you saying you can actually group Q0 and Q1 together, as well as Q2 and Q3, Q4 and Q5, Q6 and Q7.

Following are the two issues I have now
1. I have 8 queues in the current implementation. This means I need to map the existing 8 queues to two sets of 4 queues across two different classes  (C0 and C1) in DPDK-QOs right? The problem with that approach is that queue weights are not relative across classes. Is there a way to work around this?
[Cristian] You can, of course, change to code to implement 8 queues per pipe traffic class, but this is not going to be a trivial exercise. With the current implementation unmodified, you could simply map 4 queues to e.g. TC0 and 4 queues to e.g. TC1, provided that it makes sense to prioritize the 4 queues of TC0 as higher priority than those 4 queues mapped to TC1, so basically you are OK with having strict priority (up to an upper limit) between the 2 sets of 4 queues.


2. B/w redistribution -
     a) The moment I map the current implementation's 8 queues across two different classes (say C0 and C1), would remaining b/w be distributed across the two classes C0 and C1? Is it true that in the current DPDK-QoS implementation, unused b/w gets distributed only to the last class C3? Would not un-used b/w from C0 come to C1?
[Cristian] With 4 queues mapped to TC0 and 4 queues mapped to TC1, you can set TC0 rate to X% of pipe rate and TC1 rate to 100% of pipe rate. This is the idea behind strict priority: the sum of TC rates is usually bigger than 100% of pipe rate, but this is fine, as due to strict priority the lowest priority TC used (which in this case is TC1, not TC3, as you are not using TC2 and TC3 at all) only gets 100% of the pipe rate when no traffic from higher priority TCs exists. In this case, the extremes are: (1) TC0 demand is high, so TC0 uses everything up to X%, so TC1 can use a max of (100 – X)% and (2) TC0 demand is zero, in which case TC1 is free to use up to 100% of pipe rate. So the answers to your 3 questions are: yes/no/yes.

     b) In current DPDK QoS implemention, if C1 has un-used b/w would that get used by C0? Or is it only "lower priority class (C3, more specifically) uses the un-used b/w from higher priority classes (C0,C1,C2) ?
[Cristian] Due to strict priority, it does not make sense to think about higher priority classes using whatever is not used by low priority classes: If high priority classes have traffic to send, they will always be picked in the detriment of lower ones; when the high priority classes hit their upper limit rate, then they are not allowed to send more, otherwise the upper limit makes no sense. So the bandwidth reuse concept makes sense only for low prio TCs to reuse whatever is not used from high prio TCs. Usually, this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100% of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 .. TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as well when only TC0 and TC1 are actually used. So the answers to your 2 questions are: no/no.


On Tue, Feb 16, 2016 at 2:46 PM, Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>> wrote:


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of sreenaath
> vasudevan
> Sent: Tuesday, February 2, 2016 9:09 PM
> To: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
>
> Hi
> I currently have QoS implemented in hardware and I am thinking of using
> DPDK's QoS feature to move it to software.
> Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> class limitation, I was thinking of using 4 classes in DPDK-QoS and spread
> out the 8 h/w queues across the 4 classes.
> Let me explain with an example. Currently, this is how the h/w queue is
> represented
> Q0 - 10% b/w
> Q1- 10%  b/w
> Q2- 15% b/w
> Q3 - 15% b/w
> Q4 - 15% b/w
> Q5 - 15% b/w
> Q6 - 10% b/w
> Q7 - 10% b/w
>
> Translating the above config to DPDK-QoS, based on my application need, Q0
> and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2, Q3,
> Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> Q7 can be logically grouped under class 3 with super b/w = 20%.
>
> However, in the h/w, link sharing is available across all the 8 queues.
> DPDK materials say link sharing "typically" is enabled for last class, in
> this case class2. However, I also want class 1 or class 0 to use the
> remaining bandwidth when class2 does not have any traffic and so on. Can
> this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> at
> the class level in DPDK-QoS ?
Your requirements seem to be:
- minimal rate for each class (with rates fully provisioned, i.e. sum of rate not exceeding 100%)
- avoid b/w waste by redistributing unused b/w to the traffic classes that currently have traffic according to their relative weights

In our implementation, traffic classes are scheduled in strict priority (with upper limit defined, to avoid starvation), so TC0 traffic is always prioritized for the current pipe over TC1 .. TC3 traffic (as long as current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore, the traffic class hierarchy level is not going to help you here.

On the other hand, if you map all your queues/traffic classes to the queues of one of our pipe traffic classes (it does not matter which one of TC0 .. TC3 you pick, as long as you pick just one), your requirements become possible, as we have 4 queues per each pipe traffic class, and they are scheduled using weighted fair queuing (byte-level weighted round robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%), your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which should work exactly what you need. Makes sense?

Regards,
Cristian

>
> Thanks !
>
> --
> regards
> sreenaath



--
regards
sreenaath



--
regards
sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[sreenaath vasudevan]

[Cristian] It does to me, as high priority traffic is usually low bandwidth
and typically fully provisioned (how many voice connections, RTP video
conferences, 911 calls, signalling traffic, etc can you have active for a
given user at a given time?), while most of the Internet traffic is best
effort (low priority) and it is typically overprovisioned 20:1 or more.
You can set TC0 rate to 100% of pipe rate, but this is totally not
recommended, because when TC0 demand is exactly 100% you are going to
starve the low priority TC1 .. TC2 completely. You can make Weighted Fair
Queuing (WFQ) behave like strict priority by using big weight ratios like
4:1 or 16:1, but you cannot make strict priority behave like WFQ (which is
what you seem to be looking for).

>> [Sreenaath] You are right that high priority traffic is usually
low-bandwidth. But to me it is no harm to provision un-used link bandwidth
to higher priority class. But this is what DPDK provides today and that's
fine. But if its possible, I would like to submit to you future feature
request for high-priority TC to use un-used bw.



I have another question regarding using only two classes within a pipe.
When I try to provison only two classes TC0 and TC1, then I hit the
following code in "rte_sched_port_check_params()  function, which forces me
to give some bw to TC2 and TC3 even if I do not want to use them.



/* TC rate: non-zero, less than pipe rate */

        for (j = 0; j < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; j ++) {

            if ((p->tc_rate[j] == 0) || (p->tc_rate[j] > p->tb_rate)) {

                return -12;

            }

        }



Is there a way around this? How can I provision bw for only two classes TC0
and TC1 and do not run in to the above issue ?


On Wed, Feb 24, 2016 at 3:02 AM, Dumitrescu, Cristian <
cristian.dumitrescu@intel.com> wrote:

>
>
>
>
> *From:* sreenaath vasudevan [mailto:sreenaathkv@gmail.com]
> *Sent:* Wednesday, February 24, 2016 12:55 AM
> *To:* Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> *Cc:* dev@dpdk.org
> *Subject:* Re: [dpdk-dev] DPDK-QoS - link sharing across classes
>
>
>
> b) In current DPDK QoS implemention, if C1 has un-used b/w would that get
> used by C0? Or is it only "lower priority class (C3, more specifically)
> uses the un-used b/w from higher priority classes (C0,C1,C2) ?
>
> [Cristian] Due to strict priority, it does not make sense to think about
> higher priority classes using whatever is not used by low priority classes:
> If high priority classes have traffic to send, they will always be picked
> in the detriment of lower ones; when the high priority classes hit their
> upper limit rate, then they are not allowed to send more, otherwise the
> upper limit makes no sense. So the bandwidth reuse concept makes sense only
> for low prio TCs to reuse whatever is not used from high prio TCs. Usually,
> this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100%
> of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 ..
> TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and
> 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as
> well when only TC0 and TC1 are actually used. So the answers to your 2
> questions are: no/no.
>
>
>
> >> Sreenaath - Let's say I have two classes TC0 and TC1 each having a
> single queue. Let's say I give TC0 60% and TC1 100%. So according to DPDK's
> implementation, TC1 can use 100% of the bandwidth if TC0 is inactive, while
> TC0 at any time cannot exceed 60%, even if TC1 is inactive. Is that correct
> ? And since you mentioned that this is strict priority, according to this
> implementation, high priority traffic cannot use 100% of the bw even if
> there is no low priority traffic, but low priority traffic can use 100% of
> the bw if there is no high priority traffic. Is my understanding correct ?
> This doesn't make sense.
>
> [Cristian] It does to me, as high priority traffic is usually low
> bandwidth and typically fully provisioned (how many voice connections, RTP
> video conferences, 911 calls, signalling traffic, etc can you have active
> for a given user at a given time?), while most of the Internet traffic is
> best effort (low priority) and it is typically overprovisioned 20:1 or more.
>
> You can set TC0 rate to 100% of pipe rate, but this is totally not
> recommended, because when TC0 demand is exactly 100% you are going to
> starve the low priority TC1 .. TC2 completely. You can make Weighted Fair
> Queuing (WFQ) behave like strict priority by using big weight ratios like
> 4:1 or 16:1, but you cannot make strict priority behave like WFQ (which is
> what you seem to be looking for).
>
>
>
>
>
> On Fri, Feb 19, 2016 at 7:02 AM, Dumitrescu, Cristian <
> cristian.dumitrescu@intel.com> wrote:
>
>
>
>
>
> *From:* sreenaath vasudevan [mailto:sreenaathkv@gmail.com]
> *Sent:* Thursday, February 18, 2016 8:01 PM
> *To:* Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> *Cc:* dev@dpdk.org
> *Subject:* Re: [dpdk-dev] DPDK-QoS - link sharing across classes
>
>
>
> Hi Cristian
>
> Thanks for detailed response.
>
> Your solution works so long as I have four queues in my current
> implementation.
>
> [Cristian] Yes, I was relying on you saying you can actually group Q0 and
> Q1 together, as well as Q2 and Q3, Q4 and Q5, Q6 and Q7.
>
>
>
> Following are the two issues I have now
>
> 1. I have 8 queues in the current implementation. This means I need to map
> the existing 8 queues to two sets of 4 queues across two different classes
> (C0 and C1) in DPDK-QOs right? The problem with that approach is that queue
> weights are not relative across classes. Is there a way to work around this?
>
> [Cristian] You can, of course, change to code to implement 8 queues per
> pipe traffic class, but this is not going to be a trivial exercise. With
> the current implementation unmodified, you could simply map 4 queues to
> e.g. TC0 and 4 queues to e.g. TC1, provided that it makes sense to
> prioritize the 4 queues of TC0 as higher priority than those 4 queues
> mapped to TC1, so basically you are OK with having strict priority (up to
> an upper limit) between the 2 sets of 4 queues.
>
>
>
>
>
> 2. B/w redistribution -
>      a) The moment I map the current implementation's 8 queues across two
> different classes (say C0 and C1), would remaining b/w be distributed
> across the two classes C0 and C1? Is it true that in the current DPDK-QoS
> implementation, unused b/w gets distributed only to the last class C3?
> Would not un-used b/w from C0 come to C1?
>
> [Cristian] With 4 queues mapped to TC0 and 4 queues mapped to TC1, you can
> set TC0 rate to X% of pipe rate and TC1 rate to 100% of pipe rate. This is
> the idea behind strict priority: the sum of TC rates is usually bigger than
> 100% of pipe rate, but this is fine, as due to strict priority the lowest
> priority TC used (which in this case is TC1, not TC3, as you are not using
> TC2 and TC3 at all) only gets 100% of the pipe rate when no traffic from
> higher priority TCs exists. In this case, the extremes are: (1) TC0 demand
> is high, so TC0 uses everything up to X%, so TC1 can use a max of (100 –
> X)% and (2) TC0 demand is zero, in which case TC1 is free to use up to 100%
> of pipe rate. So the answers to your 3 questions are: yes/no/yes.
>
>
>
>      b) In current DPDK QoS implemention, if C1 has un-used b/w would that
> get used by C0? Or is it only "lower priority class (C3, more specifically)
> uses the un-used b/w from higher priority classes (C0,C1,C2) ?
>
> [Cristian] Due to strict priority, it does not make sense to think about
> higher priority classes using whatever is not used by low priority classes:
> If high priority classes have traffic to send, they will always be picked
> in the detriment of lower ones; when the high priority classes hit their
> upper limit rate, then they are not allowed to send more, otherwise the
> upper limit makes no sense. So the bandwidth reuse concept makes sense only
> for low prio TCs to reuse whatever is not used from high prio TCs. Usually,
> this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100%
> of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 ..
> TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and
> 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as
> well when only TC0 and TC1 are actually used. So the answers to your 2
> questions are: no/no.
>
>
>
>
>
> On Tue, Feb 16, 2016 at 2:46 PM, Dumitrescu, Cristian <
> cristian.dumitrescu@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of sreenaath
> > vasudevan
> > Sent: Tuesday, February 2, 2016 9:09 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
> >
> > Hi
> > I currently have QoS implemented in hardware and I am thinking of using
> > DPDK's QoS feature to move it to software.
> > Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> > class limitation, I was thinking of using 4 classes in DPDK-QoS and
> spread
> > out the 8 h/w queues across the 4 classes.
> > Let me explain with an example. Currently, this is how the h/w queue is
> > represented
> > Q0 - 10% b/w
> > Q1- 10%  b/w
> > Q2- 15% b/w
> > Q3 - 15% b/w
> > Q4 - 15% b/w
> > Q5 - 15% b/w
> > Q6 - 10% b/w
> > Q7 - 10% b/w
> >
> > Translating the above config to DPDK-QoS, based on my application need,
> Q0
> > and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2,
> Q3,
> > Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> > Q7 can be logically grouped under class 3 with super b/w = 20%.
> >
> > However, in the h/w, link sharing is available across all the 8 queues.
> > DPDK materials say link sharing "typically" is enabled for last class, in
> > this case class2. However, I also want class 1 or class 0 to use the
> > remaining bandwidth when class2 does not have any traffic and so on. Can
> > this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> > at
> > the class level in DPDK-QoS ?
>
> Your requirements seem to be:
> - minimal rate for each class (with rates fully provisioned, i.e. sum of
> rate not exceeding 100%)
> - avoid b/w waste by redistributing unused b/w to the traffic classes that
> currently have traffic according to their relative weights
>
> In our implementation, traffic classes are scheduled in strict priority
> (with upper limit defined, to avoid starvation), so TC0 traffic is always
> prioritized for the current pipe over TC1 .. TC3 traffic (as long as
> current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore,
> the traffic class hierarchy level is not going to help you here.
>
> On the other hand, if you map all your queues/traffic classes to the
> queues of one of our pipe traffic classes (it does not matter which one of
> TC0 .. TC3 you pick, as long as you pick just one), your requirements
> become possible, as we have 4 queues per each pipe traffic class, and they
> are scheduled using weighted fair queuing (byte-level weighted round
> robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%),
> your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our
> pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which
> should work exactly what you need. Makes sense?
>
> Regards,
> Cristian
>
>
> >
> > Thanks !
> >
> > --
> > regards
> > sreenaath
>
>
>
>
> --
>
> regards
> sreenaath
>
>
>
>
> --
>
> regards
> sreenaath
>



-- 
regards
sreenaath

Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[Dumitrescu, Cristian]

From: sreenaath vasudevan [mailto:sreenaathkv@gmail.com]
Sent: Wednesday, February 24, 2016 11:23 PM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK-QoS - link sharing across classes

[Cristian] It does to me, as high priority traffic is usually low bandwidth and typically fully provisioned (how many voice connections, RTP video conferences, 911 calls, signalling traffic, etc can you have active for a given user at a given time?), while most of the Internet traffic is best effort (low priority) and it is typically overprovisioned 20:1 or more.
You can set TC0 rate to 100% of pipe rate, but this is totally not recommended, because when TC0 demand is exactly 100% you are going to starve the low priority TC1 .. TC2 completely. You can make Weighted Fair Queuing (WFQ) behave like strict priority by using big weight ratios like 4:1 or 16:1, but you cannot make strict priority behave like WFQ (which is what you seem to be looking for).


>> [Sreenaath] You are right that high priority traffic is usually low-bandwidth. But to me it is no harm to provision un-used link bandwidth to higher priority class. But this is what DPDK provides today and that's fine. But if its possible, I would like to submit to you future feature request for high-priority TC to use un-used bw.



I have another question regarding using only two classes within a pipe. When I try to provison only two classes TC0 and TC1, then I hit the following code in "rte_sched_port_check_params()  function, which forces me to give some bw to TC2 and TC3 even if I do not want to use them.



/* TC rate: non-zero, less than pipe rate */

        for (j = 0; j < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; j ++) {

            if ((p->tc_rate[j] == 0) || (p->tc_rate[j] > p->tb_rate)) {

                return -12;

            }

        }



Is there a way around this? How can I provision bw for only two classes TC0 and TC1 and do not run in to the above issue ?

[Cristian] As you are not using TC2 and TC3, you can probably set any value 0 .. 100% for their rate.


On Wed, Feb 24, 2016 at 3:02 AM, Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>> wrote:


From: sreenaath vasudevan [mailto:sreenaathkv@gmail.com<mailto:sreenaathkv@gmail.com>]
Sent: Wednesday, February 24, 2016 12:55 AM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>>
Cc: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [dpdk-dev] DPDK-QoS - link sharing across classes


b) In current DPDK QoS implemention, if C1 has un-used b/w would that get used by C0? Or is it only "lower priority class (C3, more specifically) uses the un-used b/w from higher priority classes (C0,C1,C2) ?

[Cristian] Due to strict priority, it does not make sense to think about higher priority classes using whatever is not used by low priority classes: If high priority classes have traffic to send, they will always be picked in the detriment of lower ones; when the high priority classes hit their upper limit rate, then they are not allowed to send more, otherwise the upper limit makes no sense. So the bandwidth reuse concept makes sense only for low prio TCs to reuse whatever is not used from high prio TCs. Usually, this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100% of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 .. TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as well when only TC0 and TC1 are actually used. So the answers to your 2 questions are: no/no.



>> Sreenaath - Let's say I have two classes TC0 and TC1 each having a single queue. Let's say I give TC0 60% and TC1 100%. So according to DPDK's implementation, TC1 can use 100% of the bandwidth if TC0 is inactive, while TC0 at any time cannot exceed 60%, even if TC1 is inactive. Is that correct ? And since you mentioned that this is strict priority, according to this implementation, high priority traffic cannot use 100% of the bw even if there is no low priority traffic, but low priority traffic can use 100% of the bw if there is no high priority traffic. Is my understanding correct ? This doesn't make sense.
[Cristian] It does to me, as high priority traffic is usually low bandwidth and typically fully provisioned (how many voice connections, RTP video conferences, 911 calls, signalling traffic, etc can you have active for a given user at a given time?), while most of the Internet traffic is best effort (low priority) and it is typically overprovisioned 20:1 or more.
You can set TC0 rate to 100% of pipe rate, but this is totally not recommended, because when TC0 demand is exactly 100% you are going to starve the low priority TC1 .. TC2 completely. You can make Weighted Fair Queuing (WFQ) behave like strict priority by using big weight ratios like 4:1 or 16:1, but you cannot make strict priority behave like WFQ (which is what you seem to be looking for).


On Fri, Feb 19, 2016 at 7:02 AM, Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>> wrote:


From: sreenaath vasudevan [mailto:sreenaathkv@gmail.com<mailto:sreenaathkv@gmail.com>]
Sent: Thursday, February 18, 2016 8:01 PM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>>
Cc: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: Re: [dpdk-dev] DPDK-QoS - link sharing across classes

Hi Cristian
Thanks for detailed response.
Your solution works so long as I have four queues in my current implementation.
[Cristian] Yes, I was relying on you saying you can actually group Q0 and Q1 together, as well as Q2 and Q3, Q4 and Q5, Q6 and Q7.

Following are the two issues I have now
1. I have 8 queues in the current implementation. This means I need to map the existing 8 queues to two sets of 4 queues across two different classes  (C0 and C1) in DPDK-QOs right? The problem with that approach is that queue weights are not relative across classes. Is there a way to work around this?
[Cristian] You can, of course, change to code to implement 8 queues per pipe traffic class, but this is not going to be a trivial exercise. With the current implementation unmodified, you could simply map 4 queues to e.g. TC0 and 4 queues to e.g. TC1, provided that it makes sense to prioritize the 4 queues of TC0 as higher priority than those 4 queues mapped to TC1, so basically you are OK with having strict priority (up to an upper limit) between the 2 sets of 4 queues.


2. B/w redistribution -
     a) The moment I map the current implementation's 8 queues across two different classes (say C0 and C1), would remaining b/w be distributed across the two classes C0 and C1? Is it true that in the current DPDK-QoS implementation, unused b/w gets distributed only to the last class C3? Would not un-used b/w from C0 come to C1?
[Cristian] With 4 queues mapped to TC0 and 4 queues mapped to TC1, you can set TC0 rate to X% of pipe rate and TC1 rate to 100% of pipe rate. This is the idea behind strict priority: the sum of TC rates is usually bigger than 100% of pipe rate, but this is fine, as due to strict priority the lowest priority TC used (which in this case is TC1, not TC3, as you are not using TC2 and TC3 at all) only gets 100% of the pipe rate when no traffic from higher priority TCs exists. In this case, the extremes are: (1) TC0 demand is high, so TC0 uses everything up to X%, so TC1 can use a max of (100 – X)% and (2) TC0 demand is zero, in which case TC1 is free to use up to 100% of pipe rate. So the answers to your 3 questions are: yes/no/yes.

     b) In current DPDK QoS implemention, if C1 has un-used b/w would that get used by C0? Or is it only "lower priority class (C3, more specifically) uses the un-used b/w from higher priority classes (C0,C1,C2) ?
[Cristian] Due to strict priority, it does not make sense to think about higher priority classes using whatever is not used by low priority classes: If high priority classes have traffic to send, they will always be picked in the detriment of lower ones; when the high priority classes hit their upper limit rate, then they are not allowed to send more, otherwise the upper limit makes no sense. So the bandwidth reuse concept makes sense only for low prio TCs to reuse whatever is not used from high prio TCs. Usually, this is configured by fully provisioning TC0 .. TC2 and setting TC3 to 100% of pipe rate, so TC3 can use its rate plus whatever gets unused by TC0 .. TC2, so TC3 rate is between: 100% - (rate TC0 + rate TC1 + rate TC2) and 100% of pipe rate. As explained above, this can be applicable to e.g TC1 as well when only TC0 and TC1 are actually used. So the answers to your 2 questions are: no/no.


On Tue, Feb 16, 2016 at 2:46 PM, Dumitrescu, Cristian <cristian.dumitrescu@intel.com<mailto:cristian.dumitrescu@intel.com>> wrote:


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of sreenaath
> vasudevan
> Sent: Tuesday, February 2, 2016 9:09 PM
> To: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: [dpdk-dev] DPDK-QoS - link sharing across classes
>
> Hi
> I currently have QoS implemented in hardware and I am thinking of using
> DPDK's QoS feature to move it to software.
> Currently in the hardware,Based on the 4 class per pipe and 4 queues per
> class limitation, I was thinking of using 4 classes in DPDK-QoS and spread
> out the 8 h/w queues across the 4 classes.
> Let me explain with an example. Currently, this is how the h/w queue is
> represented
> Q0 - 10% b/w
> Q1- 10%  b/w
> Q2- 15% b/w
> Q3 - 15% b/w
> Q4 - 15% b/w
> Q5 - 15% b/w
> Q6 - 10% b/w
> Q7 - 10% b/w
>
> Translating the above config to DPDK-QoS, based on my application need, Q0
> and Q1 can be logically grouped under class0 with upper b/w = 20%; Q2, Q3,
> Q4, Q5 can be logically grouped under class2 with upper b/w = 60%; Q6 and
> Q7 can be logically grouped under class 3 with super b/w = 20%.
>
> However, in the h/w, link sharing is available across all the 8 queues.
> DPDK materials say link sharing "typically" is enabled for last class, in
> this case class2. However, I also want class 1 or class 0 to use the
> remaining bandwidth when class2 does not have any traffic and so on. Can
> this be done in DPDK ? Do we have a concept of min and max b/w guarantee
> at
> the class level in DPDK-QoS ?
Your requirements seem to be:
- minimal rate for each class (with rates fully provisioned, i.e. sum of rate not exceeding 100%)
- avoid b/w waste by redistributing unused b/w to the traffic classes that currently have traffic according to their relative weights

In our implementation, traffic classes are scheduled in strict priority (with upper limit defined, to avoid starvation), so TC0 traffic is always prioritized for the current pipe over TC1 .. TC3 traffic (as long as current pipe has TC0 traffic and pipe TC0 rate is not reached). Therefore, the traffic class hierarchy level is not going to help you here.

On the other hand, if you map all your queues/traffic classes to the queues of one of our pipe traffic classes (it does not matter which one of TC0 .. TC3 you pick, as long as you pick just one), your requirements become possible, as we have 4 queues per each pipe traffic class, and they are scheduled using weighted fair queuing (byte-level weighted round robin). Simply map your class0  to our pipe TC0 queue 0 (weight of 20%), your class2 to our pipe TC0 queue 1 (weight of 60%) and your class3 to our pipe TC0 queue 2 (weight of 20%), with our pipe TC0 queue 3 unused, which should work exactly what you need. Makes sense?

Regards,
Cristian

>
> Thanks !
>
> --
> regards
> sreenaath



--
regards
sreenaath



--
regards
sreenaath



--
regards
sreenaath