From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E84E6A053D; Wed, 5 Aug 2020 16:53:16 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 791EA37B4; Wed, 5 Aug 2020 16:53:16 +0200 (CEST) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by dpdk.org (Postfix) with ESMTP id 489E52C23 for ; Wed, 5 Aug 2020 16:53:15 +0200 (CEST) Received: by mail-wr1-f66.google.com with SMTP id f7so41025443wrw.1 for ; Wed, 05 Aug 2020 07:53:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mayadata-io.20150623.gappssmtp.com; s=20150623; h=from:subject:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=8Cjj9/PYG4xk6m82POv8tAVyVuTiLrucemyQZ89ey/Y=; b=JMJ6nmEueCnuFAxLcARUp0bVEzN+KFk4y3JgC3+D209pb7g8EDcPe4nHRis/r0yuI8 CYORee2FbcBXOH8VK1vrAIV8HXgWDR/8PFgQSUGAxE0/PyuOkH7PnBv49K1+EbykLwdn i4xwc3iPTZ32saSz183JHkuJ2erxwM+JTOFuHWJEffoEHsprL/ZKdd28ujQFDVhLweXZ M1U3C4Y7di8r5NcnVE2evl8OGmERTr5O21pzgblmZG5SkeodAJme7JSovSGwPLNhVwxO gletiXPrr5+wuO+N7xJTKymDFpuGGGYN0znQq1U2t6ApRIBKfbIMgK3+9Xb0FkN/3G/y KdTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=8Cjj9/PYG4xk6m82POv8tAVyVuTiLrucemyQZ89ey/Y=; b=jZ2u6O3PhFiGtLLxNT658GovtmEmOgo5GOAmCLrLlRKVocUkGyN5maec1ECBQvmlOn JqAcPKzv67mxgxO2oJQ8VRLZ9rVy9GI1dRdkeEd3mJTCybgbFBV4hzC7WdsdOOWp1u1c TwZggUvp2Fpwb3SmDRa1rLLGU31c3v1dOhENeFM/v6gMx4b6TDCSdc+iSETg0ErydkYL G5j37OmuWrmiLlnmkq5HdVmmgQJz5mPvr0yXqbtb9wbibwkz+e0PZcF3oS1iZ8epTd0R WcnqzxcNBEDh5vNCBDtmkZqQEhXlsVM9VrMEzvkPIJlVqyAcbbVG+bQaLp16q6eTNmG+ PuIQ== X-Gm-Message-State: AOAM530DgIX1fHMxjysqaqVNyAsaiBkIQF9EMf0ucVTxAeX1xWhFbpIF lW/X3wm1baxBf4ltyxQ0diHNXA== X-Google-Smtp-Source: ABdhPJynTjKyGzaySKsrwVRHsVostCBU6Uorpp+bcuGKF3L8/IpmLOVJyi00nsqat/ZS3KRLb33ekQ== X-Received: by 2002:adf:dc83:: with SMTP id r3mr3414624wrj.172.1596639194671; Wed, 05 Aug 2020 07:53:14 -0700 (PDT) Received: from [192.168.0.33] (cpc98324-croy25-2-0-cust73.19-2.cable.virginm.net. [82.38.60.74]) by smtp.gmail.com with ESMTPSA id p6sm3192367wmg.0.2020.08.05.07.53.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 05 Aug 2020 07:53:14 -0700 (PDT) From: Nick Connolly X-Google-Original-From: Nick Connolly To: nicolas.dichtel@6wind.com, Anatoly Burakov Cc: dev@dpdk.org, stable@dpdk.org References: <20200805122640.13884-1-nick.connolly@mayadata.io> <4a1d578d-30e3-88de-be02-34ee0dd41ad1@mayadata.io> <906edd92-cda3-9e79-ebdd-29a944082b61@6wind.com> Message-ID: Date: Wed, 5 Aug 2020 15:53:12 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <906edd92-cda3-9e79-ebdd-29a944082b61@6wind.com> Content-Language: en-GB Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] mem: fix allocation failure on non-NUMA kernel X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 05/08/2020 15:36, Nicolas Dichtel wrote: > Le 05/08/2020 à 16:20, Nick Connolly a écrit : > [snip] >>>> Fixes: 2a96c88be83e ("mem: ease init in a docker container") >>> I'm wondering if the bug existed before this commit. >>> >>> Before this commit, it was: >>>         move_pages(getpid(), 1, &addr, NULL, &cur_socket_id, 0); >>>         if (cur_socket_id != socket_id) { >>>                 /* error */ >>> >>> Isn't it possible to hit this error case if CONFIG_NUMA is unset in the kernel? >> I've just run the previous code to test this out and you are right that >> move_pages does indeed return -1 with errno set to ENOSYS, but nothing checks >> this so execution carries on and compares cur_socket_id (which will be unchanged >> from the zero initialization) with socket_id (which is presumably also zero), >> thus allowing the allocation to succeed! > I came to this conclusion, but I didn't check if socket_id could be != from 0. > >>> [snip] >>>> +    if (check_numa()) { >>>> +        ret = get_mempolicy(&cur_socket_id, NULL, 0, addr, >>>> +                    MPOL_F_NODE | MPOL_F_ADDR); >>>> +        if (ret < 0) { >>>> +            RTE_LOG(DEBUG, EAL, "%s(): get_mempolicy: %s\n", >>>> +                __func__, strerror(errno)); >>>> +            goto mapped; >>>> +        } else if (cur_socket_id != socket_id) { >>>> +            RTE_LOG(DEBUG, EAL, >>>> +                    "%s(): allocation happened on wrong socket (wanted %d, >>>> got %d)\n", >>>> +                __func__, socket_id, cur_socket_id); >>>> +            goto mapped; >>>> +        } >>>> +    } else { >>>> +        if (rte_socket_count() > 1) >>>> +            RTE_LOG(DEBUG, EAL, "%s(): not checking socket for allocation >>>> (wanted %d)\n", >>>> +                    __func__, socket_id); >>> nit: maybe an higher log level like WARNING? >> Open to guidance here - my concern was that this is going to be generated for >> every call to alloc_seg() and I'm not sure what the frequency will be - I'm >> cautious about flooding the log with warnings under 'normal running'.  Are the >> implications of running on a multi socket system with NUMA support disabled in >> the kernel purely performance related for the DPDK or is there a functional >> correctness issue as well? > Is it really a 'normal running' to have CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES in > dpdk and not CONFIG_NUMA in the kernel? I'm not an expert of DPDK, but I think it needs to be treated as 'normal running', for the following reasons: 1. The existing code in eal_memalloc_alloc_seg_bulk() is designed to work even if check_numa() indicates that NUMA support is not enabled: #ifdef RTE_EAL_NUMA_AWARE_HUGEPAGES if (check_numa()) {         oldmask = numa_allocate_nodemask();         prepare_numa(&oldpolicy, oldmask, socket);         have_numa = true;     } #endif 2. The DPDK application could be built with CONFIG_RTE_EAL_NUMA_AWARE_HUGE_PAGES and then the binary run on different systems with and without NUMA support. Regards, Nick