From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A645A0C47; Wed, 3 Nov 2021 21:26:19 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 70D6E410E5; Wed, 3 Nov 2021 21:26:18 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 9A3F940E0F for ; Wed, 3 Nov 2021 21:26:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635971177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=O+zJkQkOzZT+VPdlPBcycn7BmorRn+l2bUuKfppR0HQ=; b=Ry3nkf02xPriHCPoWa4v6/tCBqAgccUY+C97N+NKeAH0HcRmqnneGPG0gZykGU2Ex3LORx O8duTdmhZ6L+2VvublzqXUImr8kxZLFp+qanqRAq+LWa5G560nNEKCLowj06ubIe8Zypk9 txiA8O5lrLoW0UWTeKbQ8vqAuA/CYMo= Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-273-d4C0CG52PkWAH9NORdAvtg-1; Wed, 03 Nov 2021 16:26:16 -0400 X-MC-Unique: d4C0CG52PkWAH9NORdAvtg-1 Received: by mail-lf1-f70.google.com with SMTP id r10-20020ac25c0a000000b004017ded5edeso751897lfp.17 for ; Wed, 03 Nov 2021 13:26:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=O+zJkQkOzZT+VPdlPBcycn7BmorRn+l2bUuKfppR0HQ=; b=BLla5kBTCLih5a3TJ58d+SmLCShrcMOZRG8n9nRTv+JmAqd6GWoe9+0BUmRUvkSIdG TqSBEGmCZdcKwrjuwM3AJ50ZKhGgJYPG3z7ig7MH7B7gUHKi1sf4Vo+VZmXUGx+qQFMh Qx37ffGIXYF2e4YBj4CSs05cXFcEeJ3sJL+SNrxEXRjxgRip0IQDelS58YCSfS0g8CYR W4dX2W6KciYui1g4kgP6Oqi4JdAi8baJD0ZNHwFvLDCympHKoXsm9m1U2k2Vzm3jqeEI husiDWJ1F/VS6n6R43AE79SPEgCVYS6RiIxI7yVi0zSRNwchvL5p2uUM/mr5NSABPwoP KYnA== X-Gm-Message-State: AOAM533f9E1XXBXAn13fCy70fvjwfaIrmmVolOyC9HaXPWAtJzlpAK/l sUFGuqJe1eoH5NHbSYD6Fb3erF9pgfbd37dcpLIXgnlFsPe3gtfupl+Cg1/sttbvfFjaljo8djz 3vSYCIRAHf/HzEuJqYkc= X-Received: by 2002:a2e:7f1b:: with SMTP id a27mr47706996ljd.55.1635971174020; Wed, 03 Nov 2021 13:26:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxGZyS2PqDdCFIYseSb9VG1/2/+Zucf0DqeaNhoCKPfz9aASd2HZ5B6Av3gxkrvzLrNWyIC0gVF+MNohx04ij8= X-Received: by 2002:a2e:7f1b:: with SMTP id a27mr47706952ljd.55.1635971173614; Wed, 03 Nov 2021 13:26:13 -0700 (PDT) MIME-Version: 1.0 References: <20211029094929.29864-1-olivier.matz@6wind.com> In-Reply-To: <20211029094929.29864-1-olivier.matz@6wind.com> From: David Marchand Date: Wed, 3 Nov 2021 21:26:02 +0100 Message-ID: To: Olivier Matz , Anatoly Burakov Cc: dev , Ilyes Ben Hamouda , dpdk stable Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH] eal: fix mem alloc from control thread if socket 0 is unused X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Oct 29, 2021 at 11:49 AM Olivier Matz wrote: > > From: Ilyes Ben Hamouda > > When using rte_malloc() from a control thread, the used heap is the one > from numa socket 0, which may not have available memory. > > Fix this by selecting the first socket which has available memory. > > Note: malloc_get_numa_socket() is only used from one .c file, so move > it there, and remove the inline keyword. > > Fixes: b94580d6887e ("malloc: avoid unknown socket id") > Cc: stable@dpdk.org > > Signed-off-by: Ilyes Ben Hamouda > Signed-off-by: Olivier Matz > --- > lib/eal/common/malloc_heap.c | 20 ++++++++++++++++++++ > lib/eal/common/malloc_heap.h | 11 ----------- > 2 files changed, 20 insertions(+), 11 deletions(-) > > diff --git a/lib/eal/common/malloc_heap.c b/lib/eal/common/malloc_heap.c > index ee400f38ec..6eff9a2284 100644 > --- a/lib/eal/common/malloc_heap.c > +++ b/lib/eal/common/malloc_heap.c > @@ -694,6 +694,26 @@ malloc_heap_alloc_on_heap_id(const char *type, size_t size, > return ret; > } > > +static unsigned int > +malloc_get_numa_socket(void) > +{ > + const struct internal_config *conf = eal_get_internal_configuration(); > + unsigned int socket_id = rte_socket_id(); > + unsigned int idx; > + > + if (socket_id != (unsigned int)SOCKET_ID_ANY) > + return socket_id; > + Strictly speaking, this affects more than control threads. socket_id == SOCKET_ID_ANY is possible if current thread is started on cores from 2 different sockets. See thread_update_affinity(), which calls eal_cpuset_socket_id(). As a consequence, this change affects lcores too like if a lcore is pinned on cores from two sockets like --lcores 0@(1,2) with core 1 on numa1 and core 2 on numa2 (giving this odd example on purpose). Previously, all allocations from this lcore would end up on numa0, regardless of memory availability. So this change fixes allocations for this odd setup too. > + /* for control threads, return first socket where memory is available */ > + for (idx = 0; idx < rte_socket_count(); idx++) { > + socket_id = rte_socket_id_by_idx(idx); > + if (conf->socket_mem[socket_id] != 0) > + return socket_id; > + } We could look at current thread cpu affinity to check to which sockets it is bound to (like what is done in eal_cpuset_socket_id()). But that would make the code rather complex and the setups in which it helps are surely even even more odd than what I mentionned above. Your proposed heuristic looks fine to me, let's go with it if nobody objects. > + > + return rte_socket_id_by_idx(0); > +} > + > void * > malloc_heap_alloc(const char *type, size_t size, int socket_arg, > unsigned int flags, size_t align, size_t bound, bool contig) > diff --git a/lib/eal/common/malloc_heap.h b/lib/eal/common/malloc_heap.h > index 3a6ec6ecf0..3a29d024b4 100644 > --- a/lib/eal/common/malloc_heap.h > +++ b/lib/eal/common/malloc_heap.h > @@ -33,17 +33,6 @@ struct malloc_heap { > char name[RTE_HEAP_NAME_MAX_LEN]; > } __rte_cache_aligned; > > -static inline unsigned > -malloc_get_numa_socket(void) > -{ > - unsigned socket_id = rte_socket_id(); > - > - if (socket_id == (unsigned)SOCKET_ID_ANY) > - return 0; > - > - return socket_id; > -} > - > void * > malloc_heap_alloc(const char *type, size_t size, int socket, unsigned int flags, > size_t align, size_t bound, bool contig); > -- > 2.30.2 > -- David Marchand