Re: [dpdk-dev] [PATCH v3 2/2] eal: restrict ctrl threads to startup cpu affinity

[Olivier Matz <olivier.matz@6wind.com>]

On Wed, Feb 20, 2019 at 04:01:33PM +0000, Burakov, Anatoly wrote:
> On 19-Feb-19 8:41 PM, David Marchand wrote:
> > Spawning the ctrl threads on anything that is not part of the eal
> > coremask is not that polite to the rest of the system, especially
> > when you took good care to pin your processes on cpu resources with
> > tools like taskset (linux) / cpuset (freebsd).
> > 
> > Rather than introduce yet another eal options to control on which cpu
> > those ctrl threads are created, let's take the startup cpu affinity
> > as a reference and remove the eal coremask from it.
> > If no cpu is left, then we default to the master core.
> > 
> > The cpuset is computed once at init before the original cpu affinity
> > is lost.
> > 
> > Introduced a RTE_CPU_AND macro to abstract the differences between linux
> > and freebsd respective macros.
> > 
> > Examples in a 4 cores FreeBSD vm:
> > 
> > $ ./build/app/testpmd -l 2,3 --no-huge --no-pci -m 512 \
> >   -- -i --total-num-mbufs=2048
> > 
> > $ procstat -S 1057
> >    PID    TID COMM                TDNAME              CPU CSID CPU MASK
> >   1057 100131 testpmd             -                     2    1 2
> >   1057 100140 testpmd             eal-intr-thread       1    1 0-1
> >   1057 100141 testpmd             rte_mp_handle         1    1 0-1
> >   1057 100142 testpmd             lcore-slave-3         3    1 3
> > 
> > $ cpuset -l 1,2,3 ./build/app/testpmd -l 2,3 --no-huge --no-pci -m 512 \
> >   -- -i --total-num-mbufs=2048
> > 
> > $ procstat -S 1061
> >    PID    TID COMM                TDNAME              CPU CSID CPU MASK
> >   1061 100131 testpmd             -                     2    2 2
> >   1061 100144 testpmd             eal-intr-thread       1    2 1
> >   1061 100145 testpmd             rte_mp_handle         1    2 1
> >   1061 100147 testpmd             lcore-slave-3         3    2 3
> > 
> > $ cpuset -l 2,3 ./build/app/testpmd -l 2,3 --no-huge --no-pci -m 512 \
> >   -- -i --total-num-mbufs=2048
> > 
> > $ procstat -S 1065
> >    PID    TID COMM                TDNAME              CPU CSID CPU MASK
> >   1065 100131 testpmd             -                     2    2 2
> >   1065 100148 testpmd             eal-intr-thread       2    2 2
> >   1065 100149 testpmd             rte_mp_handle         2    2 2
> >   1065 100150 testpmd             lcore-slave-3         3    2 3
> > 
> > Fixes: d651ee4919cd ("eal: set affinity for control threads")
> > Cc: stable@dpdk.org
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> 
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>

Reviewed-by: Olivier Matz <olivier.matz@6wind.com>