DPDK usage discussions
 help / color / mirror / Atom feed
* [dpdk-users] running multiple pthreads in the same lcore id
       [not found] <855335141.522018.1521368574772.JavaMail.zimbra@viettel.com.vn>
@ 2018-03-18 10:25 ` longtb5
  0 siblings, 0 replies; 4+ messages in thread
From: longtb5 @ 2018-03-18 10:25 UTC (permalink / raw)
  To: vhuertas, users

Hi Victor, 

Have you look at the Packet Framework in DPDK? The sample application ip_pipeline seems to do exactly what you've described. 

-BL 

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-users] running multiple pthreads in the same lcore id
  2018-03-16 17:11 ` Van Haaren, Harry
@ 2018-03-16 18:20   ` Victor Huertas
  0 siblings, 0 replies; 4+ messages in thread
From: Victor Huertas @ 2018-03-16 18:20 UTC (permalink / raw)
  To: Van Haaren, Harry; +Cc: users

Thanks Harry for your quick response,

Mmmm. your recommendation on lighweight threads makes me think on
suitability of using DPDK in my case. It seems to me that "lightweight
threads" is specially recommended when you have to execute in parallel many
instances of the same working function (having the same per-packet
processing load). Am I right on this assumption?

The idea that comes to mind is to break the packet processing chaining
application into several processes making then simpler (in terms of
multi-threading) and independent packet processing feature. For example:

   - one process to only handle with packet-capture and packet tx through
   NICs,: here if I want to support RSS several threads shall be run.
   - one process to classify and conform the traffic: as many threads as
   the one run in the previous process to support pipelining.
   - one process to perform scheduling in order to distribute packet into
   queues: here DPDK scheduling qos library is not an option for me because I
   have already implemented my own WFQ and Strict priority Queue Server by my
   own being able to have hundreds and thousands of independent queues. Here
   it would be a good candidate for lightweight threading (one for each queue)
   for example.

The following diagram depicts why I mean:

[Process for Tx/Rx packets NICs] --> sw ring --> [Process for
classif/conform] -->sw ring-->[Process for QoS scheculing]
                 |

           |
                 |<------------------------------------------- sw ring
<---------------------------------------------------------------|


Each process would be assigned it own lcores id thanks to the DPDK EAL
initialization options.

Despite this partition, I will have to launch a good number of phreads, so
I am afraid that I will have to use non-EAL pthread functions. However, I
use several customized packet metadata that must be preserved between
processes (I would use the userdata pointer in the rte_mbuf metadata header
in order not to loose the association: packet-->metadata along all the
application).
The key point here is to combine two things: non-EAL pthreading launching
function and the DPDK software rings and memory pools that must be shared
by ALL the proceses mentioned before.

Do you agree?

Regards,

2018-03-16 18:11 GMT+01:00 Van Haaren, Harry <harry.van.haaren@intel.com>:

> > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Victor Huertas
> > Sent: Friday, March 16, 2018 4:24 PM
> > To: users@dpdk.org
> > Subject: [dpdk-users] running multiple pthreads in the same lcore id
> >
> > Hi all,
>
> Hi Victor,
>
> > I have developed a simple application which implements a network bridge.
> It
> > captures packets in one port (port0), sends them into a software ring
> > towards a second thread which receives them and sends them out via port1.
> >
> > Same implementation is performed in the direction port1->port0. So there
> > are 4 threads in total.
> >
> > [port0]-->rx thread0 -->sw ring--> tx thread1-->[port 1]
> > [port0]<--rx thread2 <--sw ring<-- tx thread3<--[port 1]
> >
> > The thing is that this application is a first step of a functionality
> > expansion by inserting additional threads (all connected with software
> > rings) in the middle of the path so that it performs some packet
> > manipulation before delivering them to the destination port:
> >
> > [port0]-->rx thread0 -->sw ring--> add_thread 1-->sw ring--> add_thread
> > N-->sw ring-> tx thread1-->[port 1]
> >
> > The total number of threads can reach hundreds (much more than number of
> > lcores) and I would like to assign every thread to a particular lcore, no
> > mattering if more than one thread is being executed by the same lcore.
> This
> > assignment would be done through an XML config file, which would be
> loaded
> > at the very beginning of the application execution.
>
> Understood.
>
>
> > the rte_eal_remote_launch function at rte_launch.h can only launch one
> > thread and attach it to one and unique lcore. So imagine that I want to
> > launch another thread on the same lcore. DPDK doesn't seem to allow me to
> > do that.
>
> Correct, DPDK uses core pinning as this enables higher performance by
> reducing
> overhead of context-switching between threads.
>
>
> > I have read that using cgroups could make this possible but I haven't
> found
> > an example application where DPDK and cgroups are used toghether.
>
> I would advise to consider some form of "lightweight threads", which don't
> actually
> perform a thread-switch, but instead run a different work function on the
> same
> DPDK lcore.
>
> DPDK provides infrastructure for doing this - which you might like to
> investigate.
> The "service-cores" library enables running various functions on the same
> DPDK lcore.
> Refer to the examples/service_cores code, and you can see different work()
> functions being
> switched between DPDK lcores dynamically (aka, runtime movement of the
> work between threads).
>
>
> > what is your recommendation on this as expert users? Is using cgroups an
> > acceptable option?
> >
> > Is DPDK contemplating in a future release to support natively multi
> pthread
> > on one lcore on the rte_launch.h lib?
>
> I'm not aware of any such ideas.
>
> Hope the above helps! -Harry
>
>
> > Thanks for your attention,
> >
> > --
> > Victor
>



-- 
Victor

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-users] running multiple pthreads in the same lcore id
  2018-03-16 16:24 Victor Huertas
@ 2018-03-16 17:11 ` Van Haaren, Harry
  2018-03-16 18:20   ` Victor Huertas
  0 siblings, 1 reply; 4+ messages in thread
From: Van Haaren, Harry @ 2018-03-16 17:11 UTC (permalink / raw)
  To: Victor Huertas, users

> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Victor Huertas
> Sent: Friday, March 16, 2018 4:24 PM
> To: users@dpdk.org
> Subject: [dpdk-users] running multiple pthreads in the same lcore id
> 
> Hi all,

Hi Victor,

> I have developed a simple application which implements a network bridge. It
> captures packets in one port (port0), sends them into a software ring
> towards a second thread which receives them and sends them out via port1.
> 
> Same implementation is performed in the direction port1->port0. So there
> are 4 threads in total.
> 
> [port0]-->rx thread0 -->sw ring--> tx thread1-->[port 1]
> [port0]<--rx thread2 <--sw ring<-- tx thread3<--[port 1]
> 
> The thing is that this application is a first step of a functionality
> expansion by inserting additional threads (all connected with software
> rings) in the middle of the path so that it performs some packet
> manipulation before delivering them to the destination port:
> 
> [port0]-->rx thread0 -->sw ring--> add_thread 1-->sw ring--> add_thread
> N-->sw ring-> tx thread1-->[port 1]
> 
> The total number of threads can reach hundreds (much more than number of
> lcores) and I would like to assign every thread to a particular lcore, no
> mattering if more than one thread is being executed by the same lcore. This
> assignment would be done through an XML config file, which would be loaded
> at the very beginning of the application execution.

Understood.


> the rte_eal_remote_launch function at rte_launch.h can only launch one
> thread and attach it to one and unique lcore. So imagine that I want to
> launch another thread on the same lcore. DPDK doesn't seem to allow me to
> do that.

Correct, DPDK uses core pinning as this enables higher performance by reducing
overhead of context-switching between threads.


> I have read that using cgroups could make this possible but I haven't found
> an example application where DPDK and cgroups are used toghether.

I would advise to consider some form of "lightweight threads", which don't actually
perform a thread-switch, but instead run a different work function on the same
DPDK lcore.

DPDK provides infrastructure for doing this - which you might like to investigate.
The "service-cores" library enables running various functions on the same DPDK lcore.
Refer to the examples/service_cores code, and you can see different work() functions being
switched between DPDK lcores dynamically (aka, runtime movement of the work between threads).


> what is your recommendation on this as expert users? Is using cgroups an
> acceptable option?
> 
> Is DPDK contemplating in a future release to support natively multi pthread
> on one lcore on the rte_launch.h lib?

I'm not aware of any such ideas.

Hope the above helps! -Harry


> Thanks for your attention,
> 
> --
> Victor

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dpdk-users] running multiple pthreads in the same lcore id
@ 2018-03-16 16:24 Victor Huertas
  2018-03-16 17:11 ` Van Haaren, Harry
  0 siblings, 1 reply; 4+ messages in thread
From: Victor Huertas @ 2018-03-16 16:24 UTC (permalink / raw)
  To: users

Hi all,

I have developed a simple application which implements a network bridge. It
captures packets in one port (port0), sends them into a software ring
towards a second thread which receives them and sends them out via port1.

Same implementation is performed in the direction port1->port0. So there
are 4 threads in total.

[port0]-->rx thread0 -->sw ring--> tx thread1-->[port 1]
[port0]<--rx thread2 <--sw ring<-- tx thread3<--[port 1]

The thing is that this application is a first step of a functionality
expansion by inserting additional threads (all connected with software
rings) in the middle of the path so that it performs some packet
manipulation before delivering them to the destination port:

[port0]-->rx thread0 -->sw ring--> add_thread 1-->sw ring--> add_thread
N-->sw ring-> tx thread1-->[port 1]

The total number of threads can reach hundreds (much more than number of
lcores) and I would like to assign every thread to a particular lcore, no
mattering if more than one thread is being executed by the same lcore. This
assignment would be done through an XML config file, which would be loaded
at the very beginning of the application execution.

the rte_eal_remote_launch function at rte_launch.h can only launch one
thread and attach it to one and unique lcore. So imagine that I want to
launch another thread on the same lcore. DPDK doesn't seem to allow me to
do that.

I have read that using cgroups could make this possible but I haven't found
an example application where DPDK and cgroups are used toghether.

what is your recommendation on this as expert users? Is using cgroups an
acceptable option?

Is DPDK contemplating in a future release to support natively multi pthread
on one lcore on the rte_launch.h lib?

Thanks for your attention,

-- 
Victor

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2018-03-18 10:25 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <855335141.522018.1521368574772.JavaMail.zimbra@viettel.com.vn>
2018-03-18 10:25 ` [dpdk-users] running multiple pthreads in the same lcore id longtb5
2018-03-16 16:24 Victor Huertas
2018-03-16 17:11 ` Van Haaren, Harry
2018-03-16 18:20   ` Victor Huertas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).