patches for DPDK stable branches
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Nick Connolly <nick.connolly@mayadata.io>
Cc: dev@dpdk.org, nicolas.dichtel@6wind.com, stable@dpdk.org
Subject: Re: [dpdk-stable] [PATCH] mem: fix allocation failure on non-NUMA kernel
Date: Thu, 17 Sep 2020 15:18:37 +0100
Message-ID: <419f4946-958a-b166-07d0-a4df22cd2018@intel.com> (raw)
In-Reply-To: <dd8b5c35-89b1-50a0-2ed3-1a5381381e08@mayadata.io>

On 17-Sep-20 3:08 PM, Nick Connolly wrote:
> Excellent - thanks - I'll amend the patch.
> 
> On 17/09/2020 15:07, Burakov, Anatoly wrote:
>> On 17-Sep-20 2:05 PM, Nick Connolly wrote:
>>> Hi Anatoly,
>>>
>>> Thanks.  My recollection is that all of the NUMA configuration flags 
>>> were set to 'n'.
>>>
>>> Regards,
>>> Nick
>>>
>>> On 17/09/2020 13:57, Burakov, Anatoly wrote:
>>>> On 17-Sep-20 1:29 PM, Nick Connolly wrote:
>>>>> Hi Anatoly,
>>>>>
>>>>> Thanks for the response.  You are asking a good question - here's 
>>>>> what I know:
>>>>>
>>>>> The issue arose on a single socket system, running WSL2 (full Linux 
>>>>> kernel running as a lightweight VM under Windows).
>>>>> The default kernel in this environment is built with CONFIG_NUMA=n 
>>>>> which means get_mempolicy() returns an error.
>>>>> This causes the check to ensure that the allocated memory is 
>>>>> associated with the correct socket to fail.
>>>>>
>>>>> The change is to skip the allocation check if check_numa() 
>>>>> indicates that NUMA-aware memory is not supported.
>>>>>
>>>>> Researching the meaning of CONFIG_NUMA, I found 
>>>>> https://cateee.net/lkddb/web-lkddb/NUMA.html which says:
>>>>>> Enable NUMA (Non-Uniform Memory Access) support.
>>>>>> The kernel will try to allocate memory used by a CPU on the local 
>>>>>> memory controller of the CPU and add some more NUMA awareness to 
>>>>>> the kernel.
>>>>>
>>>>> Clearly CONFIG_NUMA enables memory awareness, but there's no 
>>>>> indication in the description whether information about the NUMA 
>>>>> physical architecture is 'hidden', or whether it is still exposed 
>>>>> through /sys/devices/system/node* (which is used by the rte 
>>>>> initialisation code to determine how many sockets there are). 
>>>>> Unfortunately, I don't have ready access to a multi-socket Linux 
>>>>> system that I can test this out on, so I took the conservative 
>>>>> approach that it may be possible to have CONFIG_NUMA disabled, but 
>>>>> the kernel still report more than one node, and coded the change to 
>>>>> generate a debug message if this occurs.
>>>>>
>>>>> Do you know whether CONFIG_NUMA turns off all knowledge about the 
>>>>> hardware architecture?  If it does, then I agree that the test for 
>>>>> rte_socket_count() serves no purpose and should be removed.
>>>>>
>>>>
>>>> I have a system with a custom compiled kernel, i can recompile it 
>>>> without this flag and test this. I'll report back with results :)
>>>>
>>>
>>
>> With CONFIG_NUMA set to 'n':
>>
>> [root@xxx ~]# find /sys -name "node*"
>> /sys/kernel/software_nodes/node0
>> [root@xxx ~]#
>>
>> This is confirmed by running DPDK on that machine - i can see all 
>> cores from all sockets, but they're all appearing on socket 0. So, 
>> yes, that check isn't necessary :)
>>
> 

I would also add a comment explaining why we're checking for NUMA 
support when NUMA support is defined at compiled time.

-- 
Thanks,
Anatoly

  reply	other threads:[~2020-09-17 14:18 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-05 12:26 Nick Connolly
2020-08-05 13:42 ` Nicolas Dichtel
2020-08-05 14:20   ` Nick Connolly
2020-08-05 14:36     ` Nicolas Dichtel
2020-08-05 14:53       ` Nick Connolly
2020-08-05 15:13         ` Nicolas Dichtel
2020-08-05 15:21           ` Nick Connolly
2020-09-17 11:28             ` Burakov, Anatoly
2020-09-17 11:31 ` Burakov, Anatoly
2020-09-17 12:29   ` Nick Connolly
2020-09-17 12:57     ` Burakov, Anatoly
2020-09-17 13:05       ` Nick Connolly
2020-09-17 14:07         ` Burakov, Anatoly
2020-09-17 14:08           ` Nick Connolly
2020-09-17 14:18             ` Burakov, Anatoly [this message]
2020-09-17 14:19               ` Nick Connolly
2020-10-12 19:28 ` [dpdk-stable] [PATCH v2] " Nick Connolly
2020-10-13  7:59   ` Nicolas Dichtel
2020-10-13 12:01     ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=419f4946-958a-b166-07d0-a4df22cd2018@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    --cc=nick.connolly@mayadata.io \
    --cc=nicolas.dichtel@6wind.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

patches for DPDK stable branches

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/stable/0 stable/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 stable stable/ https://inbox.dpdk.org/stable \
		stable@dpdk.org
	public-inbox-index stable

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.stable


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git