From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 672D5A00E6 for ; Tue, 14 May 2019 16:50:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C8FFE5F16; Tue, 14 May 2019 16:50:39 +0200 (CEST) Received: from hqmgw2.hq.eso.org (hqmgw2.hq.eso.org [134.171.42.202]) by dpdk.org (Postfix) with ESMTP id 8E17B5F16 for ; Tue, 14 May 2019 16:50:38 +0200 (CEST) X-IronPort-AV: E=Sophos;i="5.60,469,1549926000"; d="scan'208";a="48126624" Received: from mxroute01.hq.eso.org ([134.171.15.33]) by hqmgw2.hq.eso.org with ESMTP; 14 May 2019 16:50:37 +0200 Received: from mxadauth01.hq.eso.org (mxadauth01.hq.eso.org [134.171.42.78]) by mxroute01.hq.eso.org (Postfix) with ESMTP id F2426120002; Tue, 14 May 2019 16:50:37 +0200 (CEST) Received: from [127.0.0.1] (hq-000-kemp01.hq.eso.org [134.171.42.9]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: nbenes) by mxadauth01.hq.eso.org (Postfix) with ESMTPSA id EB3071003DE; Tue, 14 May 2019 16:50:37 +0200 (CEST) To: users@dpdk.org From: "N. Benes" Message-ID: Date: Tue, 14 May 2019 14:50:00 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: [dpdk-users] lcore stack location and NUMA nodes X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi all, is it ensured that the stacks of lcore threads with set affinity are located on the same NUMA node as the assigned CPU core? So far, I understand that e.g. `rte_eal_mp_remote_launch()` does not actually create threads but just communicates the function to execute to already existing threads (via pipes). These threads are created during initialisation of the EAL: https://git.dpdk.org/dpdk/tree/lib/librte_eal/linux/eal/eal.c#n1164 The threads execute a state machine `eal_thread_loop()` where the affinity is changed/set for the corresponding lcore/thread: https://git.dpdk.org/dpdk/tree/lib/librte_eal/linux/eal/eal_thread.c#n119 However, couldn't this lead to the stacks of some threads to be still located on the (different) NUMA node where `pthread_create()` was called from, i.e. possible decreased performance? I did not find an explicit relocation of thread stacks. (I also did not find anything related in the glibc/nptl code) One possible fix may be to change the affinity of the main/master thread before calling `pthread_create()` for the slaves and to reset the affinity of the master thread after all slave threads are created. Cheers!