From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50C7242BAC; Fri, 26 May 2023 16:44:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C4C2440DDA; Fri, 26 May 2023 16:44:28 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id 2C6C94068E; Fri, 26 May 2023 16:44:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685112267; x=1716648267; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=TanvbRWrXXDTQgwkAJEfd1oriaDTHfz753NpOdxBbps=; b=RYhStHhsnN8sjDaHDWZjHOoMJkGH8dfS/q6HdiZVRQBeAj2VCrUs6fCA DTn2KdA8Nr3xZcvzktrpKixYA1eZd0NC/863Gitri1ZaoyPXAD2nRS31b Xf6yAnGo3Q8dQPvrWoGeKRYHM0dwS/xybaC3Dhsb4WX6cpBmris5JnPMV zzuiH8UIb7aOrMdRavSuppMzxi0pFLQoLv7FDsBhKcy5RWDUXqREm4nz2 +A4Ggf/A1hIfDCgwlLpKzcT0f8oEGaI+Qf5yngEeXVZALB/2zqRpwRSLT nYQ8sGdBN/m+aWNSn4+hMomFQWBEiNLHS/O7JaNnKZ+ve5jCn62RdEHWU Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10722"; a="353056010" X-IronPort-AV: E=Sophos;i="6.00,194,1681196400"; d="scan'208";a="353056010" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 May 2023 07:44:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10722"; a="705235697" X-IronPort-AV: E=Sophos;i="6.00,194,1681196400"; d="scan'208";a="705235697" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga002.jf.intel.com with ESMTP; 26 May 2023 07:44:25 -0700 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Fri, 26 May 2023 07:44:25 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Fri, 26 May 2023 07:44:25 -0700 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (104.47.56.43) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Fri, 26 May 2023 07:44:24 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jpaGuSWm5XdLtmbUOIQzzYkoWGrNJqc7dMWsrwv0oqQvx9ETluPuEvl1EiXZic2dsMG/t5azDNLVYoRROqU1RPYXci6Ychz0BA4GWbA/7tBtz/Vqrq6JbSFU5I8I381b7msfaT7mEgFRfGz5UY6rtDlMVkepyY+4WEZkXUJI7B8PJDdRyLEzo/rjh05v2itKkWCLAqriJ0GmvMJIhBHV/MCQXIsw5O7RTrg38aZgOdaFfNggC5ucBMx/3hQ6Y4I2XVKW8fu/fPD+kT+DCG7NWVhNpHUzY8/SvtWd5Ac3rEuouOb3Ym2X7YIpnUmhRjXQvbYuQHy9rSuygDkuGvBMDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LKn0gmrIvZlwRO33YXH+o4JEp9RtUg53hyBEUHR1IWs=; b=evFaDRztNvgJrJceoRKo+lxAGRXnZgpXYTn8HQbYdCWijXAkcoxGnTZbbbTzmYXGKSSTuj/jlXL2NfUazrs2jXVh6ykAQpiYa6qA+569S9dUtBmvuYo77r4AQet+22UsNbxENaU2bHGQgbz8vrxT8mx92sBlqN+YVgSqINiC/QkU/M34tkSHSVeBOhIPnqwiE9tkG8XlKotaQ7izpOJh3b7R8PqW/YMipaMTOySbT5EB4cPNqp0cmeEabQSOF+uJpmu6fd9zEsepbl2Ml54gLW06+XgD/AmzJ9WIVkDopfkCZJt7K7O9xUzuNV2hzVNx1B9nMYdgYCXe8ZWJni4T+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by BL3PR11MB6363.namprd11.prod.outlook.com (2603:10b6:208:3b6::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.17; Fri, 26 May 2023 14:44:21 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718%5]) with mapi id 15.20.6433.017; Fri, 26 May 2023 14:44:21 +0000 Message-ID: <63f43dcf-0a03-d1f8-5120-39714cc712d9@intel.com> Date: Fri, 26 May 2023 15:44:15 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.11.0 Subject: Re: [PATCH v8] enhance NUMA affinity heuristic Content-Language: en-US To: Kaisen You , CC: , , , , , , References: <20230523025004.192071-1-kaisenx.you@intel.com> <20230526084535.374803-1-kaisenx.you@intel.com> From: "Burakov, Anatoly" In-Reply-To: <20230526084535.374803-1-kaisenx.you@intel.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO2P265CA0487.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13a::12) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|BL3PR11MB6363:EE_ X-MS-Office365-Filtering-Correlation-Id: 482969ed-d77e-4789-9686-08db5df7b13c X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HGbF6Q50M0EmeBdz6ucJlgI7Pgk5LhWU0u0WaHO/aAh7KQIxJWLHhrkGLfDR+TnsHxB8IKrRR4SHCbI3L9/Ys9FQHucdeLdgWmn+Ru3rpadTTrZelmABp/letPO0f0AgIKCIWM0VygCgcQAhc7bpZN/OFXRmk1KsijA0AzlQ06x2VwNKPggOVCm+LjX1wik/PQiUl98fj4H7iqdrjQt1iS5TLbQ182EbZVZ1bHfsYMoPXVbhwhHlWSJaDlaxmDPbYAgQiN21RgFmdMMTXLYsv4EN9gTQLESyGge9RDzXIvAx2z9TWYW++BOQqY1kI3r2DcstHtIAGfgMOq97Hlh76qAax1GTg+WjscWqikQJtg6reT9Yy6OyED36kewWRcE/yr3J/aN/+nHuehchoRmSLjhAUuvjke1rhBqBczBbpZhOHzlt4B3yP/bYuPOlRS6prAOOyh/wc5Fb8c1NmkT+IqwOpw1WuEfpGdPDJlxQWMkLqWXGZ16KKYjTwPySW3C8N3ZmurKe3wxQZAYwNc5Zgyyuz44UF9NAwFwXum9atgGkEeGBD0gPCDaN05AVzwuFHSWLfbLuLcPxdhnFBOdlduxt4HCnpvSEPxS1R+PEGjEHJMyaHfHXL8+VMk/6B0/xd2DP9MCrEUj5eqiZQw3lRw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(346002)(39860400002)(366004)(396003)(376002)(136003)(451199021)(31686004)(66476007)(66946007)(66556008)(478600001)(316002)(4326008)(86362001)(31696002)(36756003)(83380400001)(6506007)(26005)(6512007)(2616005)(186003)(53546011)(41300700001)(8676002)(8936002)(5660300002)(2906002)(6486002)(6666004)(38100700002)(82960400001)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cVNIT3BvWnhabVFSNGxRdXZKZklKK1Qxc09JVnVoWE0vVE5CWUwvV3dDMFR3?= =?utf-8?B?UjVBNVJ2aHFTQUl6d2NyRnJHSE0vVTg4emcybW1wZGJNMWNHYWdydWFEWWtp?= =?utf-8?B?RnBPeW9MQS92eEJPK2tYNlhJSngrVHRwTkpRNlNJUGpGMEdyTXgrcFI5V3lN?= =?utf-8?B?VGo4bE9YL21ONERNWTQ1M1NvVVRLb1dQVk9RNWM5NTRIdFAzNTFCSUUyM2dL?= =?utf-8?B?NWZIZmNLaTdrWlRqNWJ6ZERWNWUxM0dmeXVmT0FDNW4wYU9TdmltZ2RkY1U0?= =?utf-8?B?ODlGUEZISGoxOHFBSENVNUplNGd5Y3NWSDVDMjlWVDV2a2c4enBWZXB1enJN?= =?utf-8?B?QWZFRTNlWjJHVDVLZS9yS1JmQ2ZhdU1Ga2d0VEROQnVQWGo5bVJzVlUzUnp2?= =?utf-8?B?QTRnUjNnRWE0dkFtZndnSmQxWWY2VlA1YUZEN1NjeDg3Q2E4S2xBWHdlK09I?= =?utf-8?B?RXpxM0IwcmZUNE1oUDk0bGtwTG0ya1FzSlB0bU1zRWxWaldBUHNXcEI2RzVi?= =?utf-8?B?eDV6R205ckFmRlorTmNBWThRSHUrZ3JBZ21FbGhDdEVEcEljTzM2RDlQMkVu?= =?utf-8?B?Uzlhak5JaGVua3g2Z1AwUUNGa2I5UUkxU3ZHU2RxUmN2d0Y4amUzU1R0a2ZI?= =?utf-8?B?cUg3a0R2M3lncjVWL3ptSUFqZkxHb2JseituMnQ3eVE4aEVvVzNGdjBHWXNp?= =?utf-8?B?Nmh5TDRqdDlqd3RSL0ZhN0ZCb1VPemlUUEh0TzVFZUVNN0xZK3IyRnpIeksw?= =?utf-8?B?OTAyK3hVZUJvSEVjVFRyWDZldEZLU3RqQ3orR1pjR0RjNm1PYWdNdHpuc3B3?= =?utf-8?B?Z2hjUkc0SGQ0Z09nQXVxNkx5SU02QkszSmJJWE9FaEd1akVxNXFVSy9BOXMy?= =?utf-8?B?TzdIRHNGQkVlMzByVzhCWVNMaXRPZG5LNWdZVDlxTEZjc0VWMUhEYmtuUnhZ?= =?utf-8?B?RG1wejFEM0VhYmJDYlNRU0d3ekVJSWEwN1ZIT0RidlZZS3NEOElab2F5OWRu?= =?utf-8?B?NUpKR3JnMUt6d1REZndHNktmVVkwWXowWEVrRERCSVZYc0tneVNRTk50eUV6?= =?utf-8?B?TFh4VHVsVklCSnA0d0NaYWZudVVobFFGZWczZjAzMVNKNjd2dWdscmVsYlFZ?= =?utf-8?B?RGJkTDN4TDRKL3VGZS9XV2ljcGdwVEo4WS8rczRFcG4ycS93enNnYjhkd3lU?= =?utf-8?B?OHhmRXJiYWpUcXU4d3hla3hYM3VGeWZ0aEVYYW5GUERXa1BBTVU3emN6QTlx?= =?utf-8?B?NkhWbWkvSTZhY3R5Qks0ZmJGSU5OOE1wMlNZMUNDVlA4VjFWSmc4akJES3F5?= =?utf-8?B?bURXb2ZlWjJId1BEaTJBaDY0NDFIZ040WUt4TWdUNS9XSWNGOEdJMkZaZnc3?= =?utf-8?B?RVpVL1VKSGRJcWI3UHNSckxRY2lhSlU2d1dwT0dlSE9kamRrNitkQng1bFF1?= =?utf-8?B?b3lHTDYzTXRUOHVIT0dQN01ZVzY1UDYrY09kNW5XcCtESWQ2bVdnVXZPUklE?= =?utf-8?B?R3hBVTluczkyd1dDcko1N1lBQ05IcFVXUDFKUUQ2a1pONG5qcTc0MS95Zkl0?= =?utf-8?B?TEQxZDZJTUYvVFgvL0FudnN4UzVJeXhOL3k2dDJnUGhxV1BWQm9nNDd4bkhl?= =?utf-8?B?ZURRY21LMkozZnNYNUJnM0YzVTBwNUN0Ri9FNk9aSzR6SUNZRFg1SWJDcGVX?= =?utf-8?B?UVFCZXFSVXRHU2JHSndNY1hTWGpGRHZaTjZuNVVHaDdwbUVBU1I0K2g2QkpK?= =?utf-8?B?Y3pQTXVNY1lwSnVxdjJFS2JYampZUGtCL3FrSkNmWUx6Mk50dFFGZ1BPL1h1?= =?utf-8?B?M3BrWjNwdXhkT3FFU1RRS3VlOFdpTVd5NldxR0JjWUUyemxLaUF5SDd5RjFh?= =?utf-8?B?RkZVMlJyUXZSOTMwL1NadWNVa3RMSlk2KzhPcTdNUGdGaU8vdlo2YkNNRGJB?= =?utf-8?B?S0IzdU5IWmwveWlzaHlTOGIvZ3hGaTdiaTE0Y3VGMENzbWJGSko0cXhGS0Mx?= =?utf-8?B?aG5qcm5aOHFKNlgzTWNlZjRTakJoaHFTNG1pMDdQUzQrNlFkSG1lZXllb3My?= =?utf-8?B?NUl6cHkzSXBCQXFGNHF1RGV4UUxYeGovTmVCek1kL0xCbkp6dFNJSjF4dUtj?= =?utf-8?B?TFZVbDltdW9tUDVUOUgvSE8ySHByTHNMZjUwMURLRDUxakUzMW1rUCtlQWhx?= =?utf-8?B?TUE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 482969ed-d77e-4789-9686-08db5df7b13c X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 May 2023 14:44:21.7839 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OcXEn1u4etXi3BgTlCVm7xDzJ31vXAvHbEv6hdFQgucvgi5lc+kR3lqh/bL3ch7OEA/cAdhT8r2SNObAII0CO7ruKJZ7xtItIv4bInvfEvI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR11MB6363 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/26/2023 9:45 AM, Kaisen You wrote: > When a DPDK application is started on only one numa node, memory is > allocated for only one socket. When interrupt threads use memory, > memory may not be found on the socket where the interrupt thread > is currently located, and memory has to be reallocated on the hugepage, > this operation will lead to performance degradation. > > Fixes: 705356f0811f ("eal: simplify control thread creation") > Fixes: 770d41bf3309 ("malloc: fix allocation with unknown socket ID") > Cc: stable@dpdk.org > > Signed-off-by: Kaisen You > --- > Changes since v7: > - Update commet, > > Changes since v6: > - New explanation for easy understanding, > > Changes since v5: > - Add comments to the code, > > Changes since v4: > - mod the patch title, > > Changes since v3: > - add the assignment of socket_id in thread initialization, > > Changes since v2: > - add uncommitted local change and fix compilation, > > Changes since v1: > - accomodate for configurations with main lcore running on multiples > physical cores belonging to different numa, > --- > lib/eal/common/eal_common_thread.c | 4 ++++ > lib/eal/common/malloc_heap.c | 11 ++++++++++- > 2 files changed, 14 insertions(+), 1 deletion(-) > > diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c > index 079a385630..22480aa61f 100644 > --- a/lib/eal/common/eal_common_thread.c > +++ b/lib/eal/common/eal_common_thread.c > @@ -252,6 +252,10 @@ static int ctrl_thread_init(void *arg) > struct rte_thread_ctrl_params *params = arg; > > __rte_thread_init(rte_lcore_id(), cpuset); > + /* Set control thread socket ID to SOCKET_ID_ANY as control > + * threads may be scheduled on any NUMA node. > + */ > + RTE_PER_LCORE(_socket_id) = SOCKET_ID_ANY; > params->ret = rte_thread_set_affinity_by_id(rte_thread_self(), cpuset); > if (params->ret != 0) { > __atomic_store_n(¶ms->ctrl_thread_status, > diff --git a/lib/eal/common/malloc_heap.c b/lib/eal/common/malloc_heap.c > index d25bdc98f9..d833a71e7a 100644 > --- a/lib/eal/common/malloc_heap.c > +++ b/lib/eal/common/malloc_heap.c > @@ -716,7 +716,16 @@ malloc_get_numa_socket(void) > if (conf->socket_mem[socket_id] != 0) > return socket_id; > } > - > + /* We couldn't find quickly find a NUMA node where memory was available, typo: `find quickly find`, should probably be `quickly find` Can be fixed on apply. Reviewed-by: Anatoly Burakov > + * so fall back to using main lcore socket ID. > + */ > + socket_id = rte_lcore_to_socket_id(rte_get_main_lcore()); > + /* Main lcore socket ID may be SOCKET_ID_ANY in cases when main lcore > + * thread is affinitized to multiple NUMA nodes. > + */ > + if (socket_id != (unsigned int)SOCKET_ID_ANY) > + return socket_id; > + /* Failed to find meaningful socket ID, so just use the first one available */ > return rte_socket_id_by_idx(0); > } > -- Thanks, Anatoly