DPDK usage discussions
 help / color / mirror / Atom feed
From: "Van Haaren, Harry" <harry.van.haaren@intel.com>
To: Antoine POLLENUS <a.pollenus@deltacast.tv>,
	"users@dpdk.org" <users@dpdk.org>
Subject: Re: [dpdk-users] What is the best threading technology when using DPDK ?
Date: Mon, 1 Jul 2019 15:39:09 +0000	[thread overview]
Message-ID: <E923DB57A917B54B9182A2E928D00FA67588A75F@IRSMSX102.ger.corp.intel.com> (raw)
In-Reply-To: <1a2e3c0440bd4c0f849a5ff871a4c289@deltacast.tv>

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Antoine POLLENUS
> Sent: Monday, July 1, 2019 2:20 PM
> To: users@dpdk.org
> Subject: [dpdk-users] What is the best threading technology when using DPDK
> ?
> 
> Hello,

Hi Antoine,


> I'm developing a time critical application using DPDK that require to be
> multithreaded. I'm wondering what threading technology I should use ?
> 
> -          Can I use the standard pthread library and if yes, is there a
> trade of in term of performance ?
> 
> -          I see on this page that a lthread library also exist but is kind
> of limited in term of functionality:
> https://doc.dpdk.org/guides/sample_app_ug/performance_thread.html
> 
> -          I see also that we can launch a function on another lcore using
> the rte_eal_remote_launch(...)
> 
> Is there a recommendation when using DPDK to use a technology threading
> technology or another ?

Good questions to ask, I'll bullet a few thoughts in reply;

- DPDK provides its own threading APIs, that depending on the platform calls the OS native implementation. For Linux this means pthreads. So by using DPDK's thread APIs, you're really using pthreads, but with a wrapper layer. This wrapper layer means that you can recompile against other targets (windows support is WIP for DPDK) and you won't have to change your threading code...

- Lthreads is a method of scheduling large numbers of work items on a lower numbers of "real" threads. Think of it as a scheduler implementation (like any OS has, to multiplex tasks to HW CPU cores). If you are running in a time-critical domain, general practice is to avoid multiplexing, and to dedicate resources just to the time critical work. In short; I suggest you run a DPDK lcore dedicated to the time-critical task, and do not use lthreads.

- The DPDK threading APIs use rte_eal_remote_launch() to "spawn" a worker thread to a given hardware thread on a CPU. (With hyper-threading, or running 2 "logical" threads on one "physical" core, this enumeration becomes a little more complex, but is still valid). DPDK uses this feature to do core-pinning, which means that a worker pthread is affinitized with a specific hardware-thread on the CPU. This stops the linux scheduler from moving the software-thread to a different CPU core/thread, which is desirable as you want to minimize jitter for time sensitive workloads. (And switching to a different CPU core/thread requires work, and hence takes time).

- For time sensitive processing, my recommendation would be to spawn a worker thread into a busy-loop for the time critical task. If possible it is best to dedicate that CPU for the task and not put any other work on that thread - this will minimize the jitter/latency.

- Investigate the "isolcpus" kernel boot parameter, and IRQ affinities if you have not already done so, to reduce jitter due to Linux scheduler and IRQ subsystem interference with the DPDK thread.


> Regards
> 
> Antoine Pollenus


Hope the above helps! Regards, -Harry

  reply	other threads:[~2019-07-01 15:39 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-01 13:20 Antoine POLLENUS
2019-07-01 15:39 ` Van Haaren, Harry [this message]
2019-07-02  5:41   ` Antoine POLLENUS

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=E923DB57A917B54B9182A2E928D00FA67588A75F@IRSMSX102.ger.corp.intel.com \
    --to=harry.van.haaren@intel.com \
    --cc=a.pollenus@deltacast.tv \
    --cc=users@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).