From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 784B6A0C47 for ; Wed, 3 Nov 2021 21:26:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 51E5040689; Wed, 3 Nov 2021 21:26:18 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 5935740689 for ; Wed, 3 Nov 2021 21:26:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635971176; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=O+zJkQkOzZT+VPdlPBcycn7BmorRn+l2bUuKfppR0HQ=; b=LXvX+U0FfnO9/2P1nU5/pD9Moxq0lfNlkSaUFmI6Qgs+G6UNbzEGfBmtHXQCzgJrRAMoAb URT0NC8pGCQ/7I5Xq80qn5qvTWsoZobZbeDpobtA0DiBAIGSmv3NofzwEehLrtcG0yOQvM BjgMZeKUvUVtgSuzFZcK8iq/ISPP0qQ= Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-217-R1yFtr2CMoGy50OwNUNx5w-1; Wed, 03 Nov 2021 16:26:15 -0400 X-MC-Unique: R1yFtr2CMoGy50OwNUNx5w-1 Received: by mail-lf1-f70.google.com with SMTP id c13-20020a05651200cd00b004001aab328aso1042155lfp.1 for ; Wed, 03 Nov 2021 13:26:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=O+zJkQkOzZT+VPdlPBcycn7BmorRn+l2bUuKfppR0HQ=; b=QVhLWBmd+J1W5ZoPDl+ifBfe6uZpijYDtMvLN0zaoKFpka4kEe1Iaz2bKJT427mRTY NkyX82gVgmpl8vRpxqWIkvylSx0ig7NawESpji7eOk7v+2h+Q30bb8T9zj0vzcSBR24n 1Z/y/7PDHVVcIMmGeZFrgCrD4EwMlRlsKPfvikDgCgWbMdpDB27ru9Ka8wlSv5F9rrYA DQavcKVUKHSCNqxnWQC5c7R12YNqzEoGgL2V7rQaI0TUOXg4i4HDJoHfk2xT/rCue6z2 rlYQwL4GZ4SBp8zonEuf4qETlZwvDYr7OOjM/0VG0CmgKpJrwUGFCL6cgAR9dfj6Crld 8y+w== X-Gm-Message-State: AOAM530Zji1kw0A/koybXZVP/i5SjXLcTyYUv7857WJl/HO14JxZQefX TdMIaQaRQNwalNHzOe8H3/0OsXmcSHGdw8sfm0LktYoGi4+z18Pp5Nlm4lBpa8v5/JdD/UJZyYo Eb/m3b1C2qVfl5GpV1vJRn3k= X-Received: by 2002:a2e:7f1b:: with SMTP id a27mr47706981ljd.55.1635971173920; Wed, 03 Nov 2021 13:26:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxGZyS2PqDdCFIYseSb9VG1/2/+Zucf0DqeaNhoCKPfz9aASd2HZ5B6Av3gxkrvzLrNWyIC0gVF+MNohx04ij8= X-Received: by 2002:a2e:7f1b:: with SMTP id a27mr47706952ljd.55.1635971173614; Wed, 03 Nov 2021 13:26:13 -0700 (PDT) MIME-Version: 1.0 References: <20211029094929.29864-1-olivier.matz@6wind.com> In-Reply-To: <20211029094929.29864-1-olivier.matz@6wind.com> From: David Marchand Date: Wed, 3 Nov 2021 21:26:02 +0100 Message-ID: To: Olivier Matz , Anatoly Burakov Cc: dev , Ilyes Ben Hamouda , dpdk stable Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH] eal: fix mem alloc from control thread if socket 0 is unused X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" On Fri, Oct 29, 2021 at 11:49 AM Olivier Matz wrote: > > From: Ilyes Ben Hamouda > > When using rte_malloc() from a control thread, the used heap is the one > from numa socket 0, which may not have available memory. > > Fix this by selecting the first socket which has available memory. > > Note: malloc_get_numa_socket() is only used from one .c file, so move > it there, and remove the inline keyword. > > Fixes: b94580d6887e ("malloc: avoid unknown socket id") > Cc: stable@dpdk.org > > Signed-off-by: Ilyes Ben Hamouda > Signed-off-by: Olivier Matz > --- > lib/eal/common/malloc_heap.c | 20 ++++++++++++++++++++ > lib/eal/common/malloc_heap.h | 11 ----------- > 2 files changed, 20 insertions(+), 11 deletions(-) > > diff --git a/lib/eal/common/malloc_heap.c b/lib/eal/common/malloc_heap.c > index ee400f38ec..6eff9a2284 100644 > --- a/lib/eal/common/malloc_heap.c > +++ b/lib/eal/common/malloc_heap.c > @@ -694,6 +694,26 @@ malloc_heap_alloc_on_heap_id(const char *type, size_t size, > return ret; > } > > +static unsigned int > +malloc_get_numa_socket(void) > +{ > + const struct internal_config *conf = eal_get_internal_configuration(); > + unsigned int socket_id = rte_socket_id(); > + unsigned int idx; > + > + if (socket_id != (unsigned int)SOCKET_ID_ANY) > + return socket_id; > + Strictly speaking, this affects more than control threads. socket_id == SOCKET_ID_ANY is possible if current thread is started on cores from 2 different sockets. See thread_update_affinity(), which calls eal_cpuset_socket_id(). As a consequence, this change affects lcores too like if a lcore is pinned on cores from two sockets like --lcores 0@(1,2) with core 1 on numa1 and core 2 on numa2 (giving this odd example on purpose). Previously, all allocations from this lcore would end up on numa0, regardless of memory availability. So this change fixes allocations for this odd setup too. > + /* for control threads, return first socket where memory is available */ > + for (idx = 0; idx < rte_socket_count(); idx++) { > + socket_id = rte_socket_id_by_idx(idx); > + if (conf->socket_mem[socket_id] != 0) > + return socket_id; > + } We could look at current thread cpu affinity to check to which sockets it is bound to (like what is done in eal_cpuset_socket_id()). But that would make the code rather complex and the setups in which it helps are surely even even more odd than what I mentionned above. Your proposed heuristic looks fine to me, let's go with it if nobody objects. > + > + return rte_socket_id_by_idx(0); > +} > + > void * > malloc_heap_alloc(const char *type, size_t size, int socket_arg, > unsigned int flags, size_t align, size_t bound, bool contig) > diff --git a/lib/eal/common/malloc_heap.h b/lib/eal/common/malloc_heap.h > index 3a6ec6ecf0..3a29d024b4 100644 > --- a/lib/eal/common/malloc_heap.h > +++ b/lib/eal/common/malloc_heap.h > @@ -33,17 +33,6 @@ struct malloc_heap { > char name[RTE_HEAP_NAME_MAX_LEN]; > } __rte_cache_aligned; > > -static inline unsigned > -malloc_get_numa_socket(void) > -{ > - unsigned socket_id = rte_socket_id(); > - > - if (socket_id == (unsigned)SOCKET_ID_ANY) > - return 0; > - > - return socket_id; > -} > - > void * > malloc_heap_alloc(const char *type, size_t size, int socket, unsigned int flags, > size_t align, size_t bound, bool contig); > -- > 2.30.2 > -- David Marchand