From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F035A04BB for ; Thu, 17 Sep 2020 16:18:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2B9C01D67B; Thu, 17 Sep 2020 16:18:47 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 3C7091D664; Thu, 17 Sep 2020 16:18:43 +0200 (CEST) IronPort-SDR: lOoJI9tlM2cNUYYqFGpPqiEERtItVd13PkyobC6X+7QVC+jjhrTW3nTf/k1WeipktaoYDcawaa fitvsSDwmq4g== X-IronPort-AV: E=McAfee;i="6000,8403,9746"; a="157088971" X-IronPort-AV: E=Sophos;i="5.76,437,1592895600"; d="scan'208";a="157088971" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2020 07:18:40 -0700 IronPort-SDR: DLGfbd52v/K242+BQKLEzf3sBiqgYQFR4CvpucPoiWE1w1/YIYAgo/Y2Wl9kh19lojJmjPgkWh pMRgOBlG+phw== X-IronPort-AV: E=Sophos;i="5.76,437,1592895600"; d="scan'208";a="452320299" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.206.43]) ([10.213.206.43]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Sep 2020 07:18:38 -0700 To: Nick Connolly Cc: dev@dpdk.org, nicolas.dichtel@6wind.com, stable@dpdk.org References: <20200805122640.13884-1-nick.connolly@mayadata.io> <8ebfbd1f-9a2b-3421-1d35-a3cf070ce8df@mayadata.io> <2ea114be-d606-229c-4fc0-bac06c0ea2fd@intel.com> From: "Burakov, Anatoly" Message-ID: <419f4946-958a-b166-07d0-a4df22cd2018@intel.com> Date: Thu, 17 Sep 2020 15:18:37 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-stable] [PATCH] mem: fix allocation failure on non-NUMA kernel X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" On 17-Sep-20 3:08 PM, Nick Connolly wrote: > Excellent - thanks - I'll amend the patch. > > On 17/09/2020 15:07, Burakov, Anatoly wrote: >> On 17-Sep-20 2:05 PM, Nick Connolly wrote: >>> Hi Anatoly, >>> >>> Thanks.  My recollection is that all of the NUMA configuration flags >>> were set to 'n'. >>> >>> Regards, >>> Nick >>> >>> On 17/09/2020 13:57, Burakov, Anatoly wrote: >>>> On 17-Sep-20 1:29 PM, Nick Connolly wrote: >>>>> Hi Anatoly, >>>>> >>>>> Thanks for the response.  You are asking a good question - here's >>>>> what I know: >>>>> >>>>> The issue arose on a single socket system, running WSL2 (full Linux >>>>> kernel running as a lightweight VM under Windows). >>>>> The default kernel in this environment is built with CONFIG_NUMA=n >>>>> which means get_mempolicy() returns an error. >>>>> This causes the check to ensure that the allocated memory is >>>>> associated with the correct socket to fail. >>>>> >>>>> The change is to skip the allocation check if check_numa() >>>>> indicates that NUMA-aware memory is not supported. >>>>> >>>>> Researching the meaning of CONFIG_NUMA, I found >>>>> https://cateee.net/lkddb/web-lkddb/NUMA.html which says: >>>>>> Enable NUMA (Non-Uniform Memory Access) support. >>>>>> The kernel will try to allocate memory used by a CPU on the local >>>>>> memory controller of the CPU and add some more NUMA awareness to >>>>>> the kernel. >>>>> >>>>> Clearly CONFIG_NUMA enables memory awareness, but there's no >>>>> indication in the description whether information about the NUMA >>>>> physical architecture is 'hidden', or whether it is still exposed >>>>> through /sys/devices/system/node* (which is used by the rte >>>>> initialisation code to determine how many sockets there are). >>>>> Unfortunately, I don't have ready access to a multi-socket Linux >>>>> system that I can test this out on, so I took the conservative >>>>> approach that it may be possible to have CONFIG_NUMA disabled, but >>>>> the kernel still report more than one node, and coded the change to >>>>> generate a debug message if this occurs. >>>>> >>>>> Do you know whether CONFIG_NUMA turns off all knowledge about the >>>>> hardware architecture?  If it does, then I agree that the test for >>>>> rte_socket_count() serves no purpose and should be removed. >>>>> >>>> >>>> I have a system with a custom compiled kernel, i can recompile it >>>> without this flag and test this. I'll report back with results :) >>>> >>> >> >> With CONFIG_NUMA set to 'n': >> >> [root@xxx ~]# find /sys -name "node*" >> /sys/kernel/software_nodes/node0 >> [root@xxx ~]# >> >> This is confirmed by running DPDK on that machine - i can see all >> cores from all sockets, but they're all appearing on socket 0. So, >> yes, that check isn't necessary :) >> > I would also add a comment explaining why we're checking for NUMA support when NUMA support is defined at compiled time. -- Thanks, Anatoly