[dpdk-dev] Fwd: RE: Please any one who can help me with librte_sched

[Ariel Rodriguez <arodriguez@callistech.com>]

---------- Forwarded message ----------
From: "Ariel Rodriguez" <arodriguez@callistech.com>
Date: May 29, 2014 8:59 PM
Subject: RE: [dpdk-dev] Please any one who can help me with librte_sched
To: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>
Cc:

Hi christian. I am aware of that... maybe i wold have to explain better my
approach.

1) The client applies a new configuration, its a requirement for our
customer that they could change port, subport,pipe rate and also pipe
profiles in a dynamic way.
So, i think the solution on this way:
First when a customer applies the config , the managment core parse the
conf and then creates a new rte_sched_port from scratch with the changes in
pipes profiles port subport and pipe rate etc. When the managment core is
done with that , it send a command update via a rte_ring and the data in
the ring is actually a structure wrapping the rte_sched_port recently
created.

2) The tx core which is also the same core where its enqueue and dequeue
packets from the "old" rte_sched_port, reads the rte_ring and extract the
rte_sched_port that the customer applies. So i will just let the two of it
lives together and the tx core enqueue in the new port only, and  dequeue
in a round robin manner from  the two rte_sched_port until the "old one"
become empty. Then this "live together" state ends, and the tx core is
switch to the old state in which just enqueue and dequeue from one
rte_sched_port only.

Just a note we have a rx core in which the packets are read from the net
port and send to severals worker ring . And we have several "workers" cores
that just read from the ring and pass the packets to the ,i believe you
know ,Qosmos ixEngine dpi.
Based on the classification that qosmos delivers , we put that packet in a
specific queue from specific pipe and send to the tx ring . We are doing
qos scheduling based on the dinamic information that qosmos delivers. Then
the tx core reads from that ring and behave like i just told.

Thats our architecture in a few words ... i wide open to suggestions je je
je.

Im planning when i have a little time to read the qos scheduler code in a
conscious way because im interesting in the aproach of scheduling packets
in that way... because my other option is a hardware solution and if so i
would have to change the perspective radically.

On May 29, 2014 8:07 PM, "Dumitrescu, Cristian" <
cristian.dumitrescu@intel.com> wrote:

>  Hi Ariel,
>
>
>
> So how are you making sure you are not getting more packets in while you
> are waiting for dequeue() to return zero for some time?
>
>
>
> As stated, I don’t like this approach, firstly because it forces the
> on-the-fly BW configuration changes to result in dropping packets for a
> considerable amount of time in an uncontrolled way, and it should not be
> this way. Secondly, because making it work with no buffer leakage and no
> race conditions is probably not that simple ;)
>
>
>
> Regards,
>
> Cristian
>
>
>
>
>
> *From:* Ariel Rodriguez [mailto:arodriguez@callistech.com]
> *Sent:* Wednesday, May 28, 2014 11:50 AM
> *To:* Dumitrescu, Cristian
> *Cc:* dev@dpdk.org; Stephen Hemminger
> *Subject:* RE: [dpdk-dev] Please any one who can help me with librte_sched
>
>
>
> Ok i can do that... but still is there a way to ask to the rte_sched_port
> something like is_empty
> ... Or simply if the dequeue function return 0 packets retrieved from the
> old port structure running in other core,
> Can i  assume that port is empty with that?
>
> Regards
>
> Ariel.
>
> On May 28, 2014 7:10 AM, "Dumitrescu, Cristian" <
> cristian.dumitrescu@intel.com> wrote:
>
>  Hi Ariel,
>
>
>
> I think you put your finger precisely on the problem associated with your
> approach: you have to iterate through all the queues and free up the
> packets, which takes a lot of time. Obviously this is not done by the
> rte_sched API.
>
>
>
> Maybe a limited workaround for this approach would be to create and
> service the parallel rte_sched using a different CPU core, while the
> previous CPU core takes its time to free up all the packets and data
> structures correctly.
>
>
>
> Regards,
>
> Cristian
>
>
>
> *From:* Ariel Rodriguez [mailto:arodriguez@callistech.com]
> *Sent:* Wednesday, May 28, 2014 1:46 AM
> *To:* Dumitrescu, Cristian
> *Cc:* Stephen Hemminger; dev@dpdk.org
> *Subject:* Re: [dpdk-dev] Please any one who can help me with librte_sched
>
>
>
> Thank you perfect explanation, i think im going to creating a new parallel
> rte_sched_port and change the reference with managment core updating the
> tx/sched core. So, what happens with the packets on the old reference if i
> just do rte_port_free on it, are them leaked? Is there a why to flush the
> rte_sched_port or maybe gets the packet total size somewhere?.
>
> Anyway the rcu algoritm fits ok in this aproach ... but maybe there is a
> way to flush the old reference port, and work from there with the recently
>  created rte_sched_port ....
>
>
>
> Regars,
>
> Ariel.
>
>
>
> On Tue, May 27, 2014 at 3:31 PM, Dumitrescu, Cristian <
> cristian.dumitrescu@intel.com> wrote:
>
> Hi Ariel,
>
> What's wrong with calling rte_sched_subport_config() and
> rte_sched_pipe_config() during run-time?
>
> This assumes that:
>
> 1. Port initialization is done, which includes the following:
> a) the number of subports, pipes per subport are fixed
> b) the queues are all created and their size is fixed
> c) the pipe profiles are defined
> d) Basically the maximal data structures get created (maximum number of
> supports, pipes and queues) with no run-time changes allowed, apart for the
> bandwidth related parameters. Queues that do not receive packets are not
> used now, they will be used as soon as they get packets. The
> packets-to-queues mapping logic can change over time, as well as the level
> of activity for different users/queues.
>
> 2. The CPU core calling the subport/pipe config functions is the same as
> the core doing enque/dequeue for this port (for thread safety reasons).
> a) As you say, the management core can send update requests to the core
> running the scheduler, with the latter sampling the request queue regularly
> and performing the updates.
>
> Regards,
> Cristian
>
>
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger
> Sent: Tuesday, May 27, 2014 5:35 PM
> To: Ariel Rodriguez
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Please any one who can help me with librte_sched
>
> On Tue, 27 May 2014 10:33:02 -0300
> Ariel Rodriguez <arodriguez@callistech.com> wrote:
>
> > Hello , this is my third mail , the previous mails have not been answered
> > yet.
> >
> > I justo need someone explains to me  how the librte_sched framework
> behaves
> > in a specific situation.
> >
> > I have a managment application , this connects with a ring with the tx
> > core, when a user applies some configuration of the bandwith mangement ,
> > the tx core read the message in the ring parse the configuration in a
> > rte_port_params struct , subport_params and pipe_params, then creates a
> new
> > rte_sched from scratch , and then changes the pointer of the current
> > rte_sched_port currently doing scheduling and then the code execurte
> > rte_sched_port_free() for the unreference (reference by temporal pointer)
> > rte_sched_port . This is the only way i found for applying dinamic
> > configuration or changes to the qos framework.
> > So, with this, what happens with the packets attached to the old
> > rte_sched_port while is deleted? are those lost packets inside the
> > rte_sched_port generates memory leaks?  how can i recover this packets _
> > just dequeing from the port scheduler? Where the port scheduler
>  indicates
> > empty packets in the queu state?
> >
> > Is there a better way to achieve this kind of behaviour? i just need to
> > update  the rte_sched_port configuration dinamically, and i want to
> change
> > the current pipe configuration and sub port configuration also.
> >
> > Regards .
>
> If you need to do dynamic changes, I would recommend using an RCU type
> algorithm where you exchange in new parameters and then cleanup/free
> after a grace period.  See http://lttng.org/urcu
>
> --------------------------------------------------------------
> Intel Shannon Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> Business address: Dromore House, East Park, Shannon, Co. Clare
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>
>
> --------------------------------------------------------------
> Intel Shannon Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> Business address: Dromore House, East Park, Shannon, Co. Clare
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>  --------------------------------------------------------------
> Intel Shannon Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> Business address: Dromore House, East Park, Shannon, Co. Clare
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>