Re: [dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 11 Sep 2014 23:47:52 +0000
Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com> wrote:

> Hi Bruce,
> 
> > Subject: [dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0
> > 
> > This patchset sets things up so that we can affinitize the interrupt,
> > vfio management, and hpet timer management threads to lcore 0, so that
> > they never interfere with data plane threads.
> 
> I don't think it works well always.
> The management threads can be floating all cpus on demand, because those
> threads are created before the master thread affinity is set. The kernel
> scheduler will take care of it. And we should isolate cpus which data plane
> threads are pinned to, so the management threads cannot run on those isolated
> cpus data plane thread run.
> In some cases, the user may run data plane thread on lcore 0, but with
> this patchset the data plane pinned to lcore 0 always run with the
> management threads. That doesn't seem good.
> 
> I think this functionality should be conditional.
> How about to add a parameter to specify the mask for the management threads
> instead of statically assignment to lcore 0?
> 
> thanks,
> Hiroshi
> 
> > 
> > Bruce Richardson (3):
> >   eal: add core id param to  eal_thread_set_affinity
> >   eal: increase scope of eal_thread_set_affinity
> >   eal: affinitize low-priority threads to lcore 0
> > 
> >  lib/librte_eal/bsdapp/eal/eal_thread.c             | 12 ++++++------
> >  lib/librte_eal/common/include/eal_private.h        | 10 ++++++++++
> >  lib/librte_eal/linuxapp/eal/eal_interrupts.c       |  5 +++++
> >  lib/librte_eal/linuxapp/eal/eal_pci_vfio_mp_sync.c |  6 ++++++
> >  lib/librte_eal/linuxapp/eal/eal_thread.c           | 12 ++++++------
> >  lib/librte_eal/linuxapp/eal/eal_timer.c            |  5 +++++
> >  6 files changed, 38 insertions(+), 12 deletions(-)
> > 
> > --
> > 1.9.3
> 

We just set the non-master DPDK worker threads to be real time by
using pthread_set_scheduler. This works very well and makes it clear to
Linux scheduler what the intention is. This also improves wakeup behavior
from sleeps.