[dpdk-users] Which coremask and "-n" (number of mem channels) should be used for a system with only "node1" (no "node0") and 2 sockets ?

[dpdk-users] Which coremask and "-n" (number of mem channels) should be used for a system with only "node1" (no "node0") and 2 sockets ?

[Kevin Wilson]

Hello,
I have a machine on which there is a bit non common setup: I have only
"node1" (no node0) under /sys/devices/system/node:
ls /sys/devices/system/node/
...
node1
...

I have 2 sockets on this machines, and 16 cores:

./tools/cpu_layout.py
============================================================
Core and Socket Information (as reported by '/proc/cpuinfo')
============================================================

cores =  [0, 1, 2, 3]
sockets =  [0, 1]

       Socket 0        Socket 1
       --------        --------
Core 0 [0, 8]          [4, 12]
Core 1 [1, 9]          [5, 13]
Core 2 [2, 10]         [6, 14]
Core 3 [3, 11]         [7, 15]

What should be the "-c" and "-n"  EAL paramaters that I should use ? I
 made several trials but
none of them succeeded, applications cannot start.

Please let me know if anything else is needed.

Regards,
Kevin

Re: [dpdk-users] Which coremask and "-n" (number of mem channels) should be used for a system with only "node1" (no "node0") and 2 sockets ?

[Matthew Hall]

On Fri, Jul 15, 2016 at 09:51:42AM +0300, Kevin Wilson wrote:
> Hello,
> I have a machine on which there is a bit non common setup: I have only
> "node1" (no node0) under /sys/devices/system/node:
> ls /sys/devices/system/node/
> ...
> node1
> ...

If I had to guess then either the BIOS is busted or the CPU is in the wrong 
socket or the RAM is in the wrong socket.

Either way all bets are off how this is going to pan out. Probably badly when 
it comes to getting the top performance.

Some exploration of the dmidecode output on the machine could be valuable.

Matthew.

Re: [dpdk-users] Which coremask and "-n" (number of mem channels) should be used for a system with only "node1" (no "node0") and 2 sockets ?

[Kevin Wilson]

Thanks Metthew.
Indeed,  something seems broken in that machine,

Running
dmidecode
gives:
.....
Invalid entry length (0). DMI table is broken! Stop

Kevin


On Mon, Jul 18, 2016 at 10:13 PM, Matthew Hall <mhall@mhcomputing.net> wrote:
> On Fri, Jul 15, 2016 at 09:51:42AM +0300, Kevin Wilson wrote:
>> Hello,
>> I have a machine on which there is a bit non common setup: I have only
>> "node1" (no node0) under /sys/devices/system/node:
>> ls /sys/devices/system/node/
>> ...
>> node1
>> ...
>
> If I had to guess then either the BIOS is busted or the CPU is in the wrong
> socket or the RAM is in the wrong socket.
>
> Either way all bets are off how this is going to pan out. Probably badly when
> it comes to getting the top performance.
>
> Some exploration of the dmidecode output on the machine could be valuable.
>
> Matthew.

Re: [dpdk-users] Which coremask and "-n" (number of mem channels) should be used for a system with only "node1" (no "node0") and 2 sockets ?

[Stephen Hemminger]

On Tue, 16 Aug 2016 20:41:42 +0300
Kevin Wilson <wkevils@gmail.com> wrote:

> Thanks Metthew.
> Indeed,  something seems broken in that machine,
> 
> Running
> dmidecode
> gives:
> .....
> Invalid entry length (0). DMI table is broken! Stop
> 
> Kevin
> 
> 
> On Mon, Jul 18, 2016 at 10:13 PM, Matthew Hall <mhall@mhcomputing.net> wrote:
> > On Fri, Jul 15, 2016 at 09:51:42AM +0300, Kevin Wilson wrote:  
> >> Hello,
> >> I have a machine on which there is a bit non common setup: I have only
> >> "node1" (no node0) under /sys/devices/system/node:
> >> ls /sys/devices/system/node/
> >> ...
> >> node1
> >> ...  
> >
> > If I had to guess then either the BIOS is busted or the CPU is in the wrong
> > socket or the RAM is in the wrong socket.
> >
> > Either way all bets are off how this is going to pan out. Probably badly when
> > it comes to getting the top performance.
> >
> > Some exploration of the dmidecode output on the machine could be valuable.
> >
> > Matthew.  

What if you have a two socket machine, and node 0 is not populated?

Also does the kernel have NUMA support enabled?

Long ago there was a bug in DPDK where it got NUMA socket info from /proc/cpuinfo.
On some machines, the 'physical id' starts at 1 because value comes from BIOS.

Re: [dpdk-users] Which coremask and "-n" (number of mem channels) should be used for a system with only "node1" (no "node0") and 2 sockets ?

[Matthew Hall]

On Tue, Aug 16, 2016 at 11:05:26AM -0700, Stephen Hemminger wrote:
> What if you have a two socket machine, and node 0 is not populated?

I did include that possibility. But most NUMA systems I have set up myself 
will not boot if Socket 1 is full and Socket 0 is empty. The early boot code 
generally assumes Socket 0 is valid when first initting itself.

> Also does the kernel have NUMA support enabled?

Good question, the vast majority of standard SMP kernels do have NUMA at this 
point I believe. It would maybe be good to see if the issue went away if the 
machine was intentionally run with a UP kernel.

> Long ago there was a bug in DPDK where it got NUMA socket info from 
> /proc/cpuinfo. On some machines, the 'physical id' starts at 1 because value 
> comes from BIOS.

Ouch... that's very evil of the BIOS. But also sadly not surprising.

Matthew.