From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 651A4A04BC for ; Thu, 17 Sep 2020 14:29:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DFAF71D641; Thu, 17 Sep 2020 14:29:04 +0200 (CEST) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by dpdk.org (Postfix) with ESMTP id 0EE881D62D for ; Thu, 17 Sep 2020 14:29:03 +0200 (CEST) Received: by mail-wr1-f66.google.com with SMTP id s12so1849176wrw.11 for ; Thu, 17 Sep 2020 05:29:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mayadata-io.20150623.gappssmtp.com; s=20150623; h=from:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=PsZy/4zhh40QuTUcuQgO32uxLJtl526BtvYZuto0kYw=; b=K4RdPfHeKg8/OJc/1NcVI6TBx/WEOIb/Y+F5Sd1B1QnwECE9qsYCLTkjTOM3YSQ8Wl Dm+NXc8KoybV2lF1qT9Db+x46pGgs9O8xz5iwwiWGFs+JfsUeGEpJtyCzGhXq3fMTz9m o6ZBSoEK8fnEUiRJ6eYShng/wIb798yP/+mF7wj6QBsoDIuHq+D3V8kb71YrN6SIAZzo zeq6MNOP/t+lr0EIh3OK4t4BwUD1FJXDt3d0psvYXScZ2Vd8wR17YqLFpefomGK6eUQe AViSAcnXvg7qc9DSg4ZJfYLqXbeVMFsnnippDcOeHGU7weWJ/eBH0dVtfm1eeIcCpal/ VyCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=PsZy/4zhh40QuTUcuQgO32uxLJtl526BtvYZuto0kYw=; b=YXfAGnhggSxNcIApSWZ2S6xO8TNRHIWKX7CUoOY2U4SKY/DkFAAoQhr89vlTGNCFBu 2hC9BDfW1gxoSXmVdn5rQKVMO9h3FPB3J07/mnIwOEIBwxdT5N4H0EQSFkmn7+dYRLor k0iuoQHEe6s+ihfzG7BkGpQPHAA+SENKLs0n8lCQajO4DXVdvSl3Kci2ABcPJMi3Z5mZ XFc+AcqOvl1Rhqu2VFkJ1sjZRHtfMJrIt7VzS0GY5KKyGIXK/FS9lv8IkY/+1FhanpAC inDLEx239/0uMFV1hYZVao3o3DAADKM+3PBnjCsxKjVEDeUaagzRYIoSdUki83AeyoYX Mb/A== X-Gm-Message-State: AOAM530d+3UfnvAx8w7QxBO3JAxRv8T4cSPBRsbge8Jzl4O3XJ9Pyirw NBp0J2K8WKQJCn2qhgXCuFN9CpTtEeWVhQ== X-Google-Smtp-Source: ABdhPJwBW1OS3AaYAqS+HSnfA87aPMvlDhQKeAkVVB8LxYo6uG2Bdj6WIK2gg7Zuf3m7ZIfw7nPw7w== X-Received: by 2002:adf:e690:: with SMTP id r16mr31636928wrm.15.1600345742203; Thu, 17 Sep 2020 05:29:02 -0700 (PDT) Received: from [192.168.0.33] (cpc98320-croy25-2-0-cust77.19-2.cable.virginm.net. [80.235.134.78]) by smtp.gmail.com with ESMTPSA id d5sm39844419wrb.28.2020.09.17.05.29.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Sep 2020 05:29:01 -0700 (PDT) From: Nick Connolly X-Google-Original-From: Nick Connolly To: "Burakov, Anatoly" Cc: dev@dpdk.org, nicolas.dichtel@6wind.com, stable@dpdk.org References: <20200805122640.13884-1-nick.connolly@mayadata.io> Message-ID: <8ebfbd1f-9a2b-3421-1d35-a3cf070ce8df@mayadata.io> Date: Thu, 17 Sep 2020 13:29:00 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB Subject: Re: [dpdk-stable] [PATCH] mem: fix allocation failure on non-NUMA kernel X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi Anatoly, Thanks for the response.  You are asking a good question - here's what I know: The issue arose on a single socket system, running WSL2 (full Linux kernel running as a lightweight VM under Windows). The default kernel in this environment is built with CONFIG_NUMA=n which means get_mempolicy() returns an error. This causes the check to ensure that the allocated memory is associated with the correct socket to fail. The change is to skip the allocation check if check_numa() indicates that NUMA-aware memory is not supported. Researching the meaning of CONFIG_NUMA, I found https://cateee.net/lkddb/web-lkddb/NUMA.html which says: > Enable NUMA (Non-Uniform Memory Access) support. > The kernel will try to allocate memory used by a CPU on the local > memory controller of the CPU and add some more NUMA awareness to the > kernel. Clearly CONFIG_NUMA enables memory awareness, but there's no indication in the description whether information about the NUMA physical architecture is 'hidden', or whether it is still exposed through /sys/devices/system/node* (which is used by the rte initialisation code to determine how many sockets there are). Unfortunately, I don't have ready access to a multi-socket Linux system that I can test this out on, so I took the conservative approach that it may be possible to have CONFIG_NUMA disabled, but the kernel still report more than one node, and coded the change to generate a debug message if this occurs. Do you know whether CONFIG_NUMA turns off all knowledge about the hardware architecture?  If it does, then I agree that the test for rte_socket_count() serves no purpose and should be removed. Many thanks, Nick On 17/09/2020 12:31, Burakov, Anatoly wrote: > On 05-Aug-20 1:26 PM, Nick Connolly wrote: >> Running dpdk-helloworld on Linux with lib numa present, >> but no kernel support for NUMA (CONFIG_NUMA=n) causes >> ret_service_init() to fail with EAL: error allocating >> rte services array. >> >> alloc_seg() calls get_mempolicy to verify that the allocation >> has happened on the correct socket, but receives ENOSYS from >> the kernel and fails the allocation. >> >> The allocated socket should only be verified if check_numa() is true. >> >> Fixes: 2a96c88be83e ("mem: ease init in a docker container") >> Cc: nicolas.dichtel@6wind.com >> Cc: stable@dpdk.org >> Signed-off-by: Nick Connolly >> --- >>   lib/librte_eal/linux/eal_memalloc.c | 28 +++++++++++++++++----------- >>   1 file changed, 17 insertions(+), 11 deletions(-) >> >> diff --git a/lib/librte_eal/linux/eal_memalloc.c >> b/lib/librte_eal/linux/eal_memalloc.c >> index db60e7997..179757809 100644 >> --- a/lib/librte_eal/linux/eal_memalloc.c >> +++ b/lib/librte_eal/linux/eal_memalloc.c >> @@ -610,17 +610,23 @@ alloc_seg(struct rte_memseg *ms, void *addr, >> int socket_id, >>       } >>     #ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES >> -    ret = get_mempolicy(&cur_socket_id, NULL, 0, addr, >> -                MPOL_F_NODE | MPOL_F_ADDR); >> -    if (ret < 0) { >> -        RTE_LOG(DEBUG, EAL, "%s(): get_mempolicy: %s\n", >> -            __func__, strerror(errno)); >> -        goto mapped; >> -    } else if (cur_socket_id != socket_id) { >> -        RTE_LOG(DEBUG, EAL, >> -                "%s(): allocation happened on wrong socket (wanted >> %d, got %d)\n", >> -            __func__, socket_id, cur_socket_id); >> -        goto mapped; >> +    if (check_numa()) { >> +        ret = get_mempolicy(&cur_socket_id, NULL, 0, addr, >> +                    MPOL_F_NODE | MPOL_F_ADDR); >> +        if (ret < 0) { >> +            RTE_LOG(DEBUG, EAL, "%s(): get_mempolicy: %s\n", >> +                __func__, strerror(errno)); >> +            goto mapped; >> +        } else if (cur_socket_id != socket_id) { >> +            RTE_LOG(DEBUG, EAL, >> +                    "%s(): allocation happened on wrong socket >> (wanted %d, got %d)\n", >> +                __func__, socket_id, cur_socket_id); >> +            goto mapped; >> +        } >> +    } else { >> +        if (rte_socket_count() > 1) >> +            RTE_LOG(DEBUG, EAL, "%s(): not checking socket for >> allocation (wanted %d)\n", >> +                    __func__, socket_id); >>       } > > If there is no kernel support for NUMA, how would we end up with >1 > socket count? > >>   #else >>       if (rte_socket_count() > 1) >> > >