From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 654AE42B81; Tue, 23 May 2023 14:46:00 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E0EC340A80; Tue, 23 May 2023 14:45:59 +0200 (CEST) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id BCEFF40689; Tue, 23 May 2023 14:45:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684845958; x=1716381958; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=FEuAkgtrpl/dWY7BzvRgkrpfNq5c3oAuUqMqUXAWRiM=; b=lyxHoG2Ou3hkgSk8y1hpq4fQ25oyC37UIDIdVKUdXHqg6k1SZMrPqiTi B4qYESuyO5BtVrI1UEfQVr6cKttykYSwsKBIAHRoqXbdDmZczNSqpZoOF 8Ls7E0EKB0PlzzKwuvPSZym/AKaS+mZ1aU52ai2Am0vTUFOZXUMAQzk1X ew8hnKru83uzHJSUUa3m32La+VwzsoFgIWokrrnr+RbKjp6ix0/qs1QwF VWA8OBOiHUNBM4YY6Hp/A3UnxVHU+6QeLXFigPuef0TjzDpDKVfm4Ufhw wJMxHwW5S+vv68e18K0IkuPJaS20PeogUFk5prmJbHGQOjRmDfikAgW4f Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10719"; a="416683364" X-IronPort-AV: E=Sophos;i="6.00,186,1681196400"; d="scan'208";a="416683364" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 May 2023 05:45:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10719"; a="773823889" X-IronPort-AV: E=Sophos;i="6.00,186,1681196400"; d="scan'208";a="773823889" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmsmga004.fm.intel.com with ESMTP; 23 May 2023 05:45:55 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Tue, 23 May 2023 05:45:55 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Tue, 23 May 2023 05:45:54 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Tue, 23 May 2023 05:45:54 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Tue, 23 May 2023 05:45:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kmIMbmrnShSBVlDF8qc1bOaAoD4dJHuGISKeXK5Su9BkmlZ8/ZRk8wEOzP5DX3uTjoRCroNHjOlZ3+UIfycV0EmwxZyowr6RoNCF6F9c4oUokT636DQGKOAZUipZlz8chyarl6413qLptI83SWb4MK5ii56Zl8G/KAs6vZBcO8egSIUDb/1+FDyt7Y2ZWAzRUMaLNgqlL27bh5HzKOX1vne8goQb9dDogiCDxj6ZEmld3Y4BcuxCGg+20SWdXB0PXWgSxCUT298EUoiuJ8mzhuyk7SV6W4C3TFzCqCfesONQp1Qcd7HzjDLBi8dHDWQbkbR2Q4E139TFBwMDBDMtvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4TRBx/KNZuYc0gB6ievzGvz2MLhTSO66mt4BBQmAD0Q=; b=EcqQbSTZuhhsSK1mDlbGR1DtdctAsuyZ4q7vj0CzucjkOgHRvi/7cxOKYQKohRq2YMGU8Ni1nF8JazOm4PNS58wgKakm5GOpH0e9zdLGw7kW0x/yRO6xyWIvxldPiGPA/xTXdjEQNOlB/jz/10/wh1qhkfBwEJG7K5ZM49NW27ayFwDE0+K0dux8Unpyciy4DUaP9RRaBYNlDoFHQ1jEG2x16/JeKcp+pWA3FRhuV+7XM//F9M4D8U1qe/j2AZ5jqmYuf+F/cIS5tvZqLmWiHUuuiU5qZ8IhRliwG5CQptTahOGw+Kreg3fuRVvUdhmNjXZVrRcqzDt5/UVMwQNQmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by MN0PR11MB5985.namprd11.prod.outlook.com (2603:10b6:208:370::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.28; Tue, 23 May 2023 12:45:50 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718%5]) with mapi id 15.20.6411.028; Tue, 23 May 2023 12:45:50 +0000 Message-ID: <88ff7d8d-4911-9ce8-7d6c-f765740eacef@intel.com> Date: Tue, 23 May 2023 13:45:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.11.0 Subject: Re: [PATCH v7] enhance NUMA affinity heuristic Content-Language: en-US To: Kaisen You , CC: , , , , , , References: <20230425051649.1109428-1-kaisenx.you@intel.com> <20230523025004.192071-1-kaisenx.you@intel.com> From: "Burakov, Anatoly" In-Reply-To: <20230523025004.192071-1-kaisenx.you@intel.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: DU2PR04CA0360.eurprd04.prod.outlook.com (2603:10a6:10:2b4::28) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|MN0PR11MB5985:EE_ X-MS-Office365-Filtering-Correlation-Id: e7504dd3-2404-48bf-99a8-08db5b8ba313 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Uofz+BkmPxOk2fleOHnv3Y7O5iW+1Plhc7uPE2wzHpPuDj83WnbnMbDw3mDuko0zgMS36YB6zPOKykc6sx4xLBAyEgYyNQ9ZhvPdB73GjfVx2O6Zp12BwqDzBuAab5ymg9J9o6uleKKxumw00BeX957RfBc57kJ1vOCWB+uKFESGK1yEpcEGM+aAjzIOwKkskB0SdSsmTvTa/nB7qVeVoiZO72o7jjjhYAGF22zB+bOvsuUFCJCAwci0FVcGu+WWbAH5nvNso/SQFL+ygBec5YtB/SL3Z8ZdBpuHDK4dgJxtTPEPOGBBKWyv2m3WKerEwFFfyFC4NyLoG182WtEpHRdvH6DvT269zebGvoV0Ow0kLvuykB966i32GKY5plO0ruzAncvYLxv2MajSXSd5cKQZh8SagOM5/rmx1fFpHiomyHmOqYp+XQdmGwuYxZH0zTloD7Btezjv/U2ur2BN3xZe2zCffvCjg9CnJnFCBNHFbDusBdUVcYKFONE8Yy6pxRNUZuVUJrSztgApz7PmUHzfvBhD4xZG4bJnmdF/wJAenAug4+old4YnhU3OzDesD9lLFt/NuH82mmZFganIQqIPvvlLsEpgHMZnVPUhqSpnl50KcsbsZGHl9elvXOqOiij7wzwVtcvMgoPSU1ezIQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(396003)(136003)(39860400002)(366004)(376002)(346002)(451199021)(8676002)(8936002)(5660300002)(83380400001)(186003)(6512007)(6506007)(31696002)(2616005)(26005)(86362001)(53546011)(82960400001)(38100700002)(66946007)(6666004)(41300700001)(478600001)(66476007)(4326008)(6486002)(36756003)(66556008)(316002)(2906002)(31686004)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGFIdXVTaUwvQTdMR1J3WGN3ZkYrWDBVM0NwV0pya3Q1amxxR1VCd2JZQmJw?= =?utf-8?B?VzlhOHE2Yzh2RjlvNDM5MWNLNDQ4eU9hLzcvSGlRUzl5OUdWSk9uOEFKdDlZ?= =?utf-8?B?bEgzaHFBQnBza1FGZGFZMnZCYjg0c0NDSUVQM3YrbWdFR1hieGtnQUk3SzFW?= =?utf-8?B?Rnl3WjFvR1YrV3VBbXd4Y3I1L05Pc1NPNHlPRWl2ZksxbHJvNEE2YWFRQUZx?= =?utf-8?B?OGxvY3lHTXFxbkxmN3JpWnErS0htQUdiOVpxUGc2RURCYWU1ZnBOUE5FQXFn?= =?utf-8?B?NlpETENyN21ZRDZrY2dUc2pGOE0zTVBRblFXVjZJTmVMRlpyTUdjNXhJTmls?= =?utf-8?B?SU5lcFJjM2w0V0ZmTytFbHVUTndhMW5BZGJnQ3ArNGJLUDZYb2d1SGgyR2I1?= =?utf-8?B?M0hWemJwSkZVWEFXbzNuUzJMU0xrVXg1MGZ0eHlpamlGYkIzQ1RsN0U3NWQ1?= =?utf-8?B?UVRXSWROMVZxRTNVOUEzRWJEVmJTbzJuMXlCQThkMFNQQTErTSsva0NVNVgr?= =?utf-8?B?Q2NzQUJPMmRpNFdVZE1jZXpUcUJuYmtwNTRycE1rREpsRUZPaFpHOFlRdWRW?= =?utf-8?B?Q1BwV2hwQzVrT1Q5Q0llMG13SWZRWWxpL1NESzVHbTcxYnZZSUhGUzJkWXNK?= =?utf-8?B?TFkwSjYxMGdqUE1IVnR4QkRSRzdaY0JTbDhYcnJvOUVJTDZqMXdLSVBnczNT?= =?utf-8?B?VzAzQU9uQWlWd2R0VWZvdVpkZjlnS095aVk5Ym01cTJXR0tpYmZIU0M2VnVv?= =?utf-8?B?RFpEY3p3akVJamRKK0dtRE90bEJ4eWV0a1Q3YlBtN0FWNmc1Zmk0NndnRStD?= =?utf-8?B?eWxhT1VtM3FBdWk2YldoVkwxTlh4bEswR1dxTHBxOFpsQnp3NXREK1NzdjJX?= =?utf-8?B?aTFGK3dUYTU3cjJyL2p0Wm1xK2lDMWh4YVFjOEp2dUw4S0hvaFJzaGtyc1BN?= =?utf-8?B?dWI5TENNZWNha3hPUUhkZTVvTCtVdWpTVEQvTFlRTHE1cE9JeGNIcm82US81?= =?utf-8?B?NWpQVG1XZmNqN2N3R2FaSzlJY05zRHFMYmo2ektXMjI2Q25nZFJnd3NRL2JT?= =?utf-8?B?VERsZlk0b3lZWmgzVnliTHFqa1YxOEttMkczdnYvSVhvdUYwL3FubTF0OURl?= =?utf-8?B?ejQ0Y05QaW5HeGJWZDlyQ2tVSlMzMUI4bW9lZkIyaS9IWUdYaU0xaDNjUHpm?= =?utf-8?B?aWJqUTJYNzl4OHV4eFcrelZzQ3lTOHNHK0oxaEVJbUYxWnRXVDB6OWNHQkhq?= =?utf-8?B?dm5sZElxWngvQjNUU0VDVUw5UjBtMkVSdjN3NmhwZWUrOXRLQU5lMklPRXZ5?= =?utf-8?B?cWNLRXhDc1dzTmE0bGVKQkRLank3OGdRNFhaK2VyUjN2U0J4ME5GWDhNSXU5?= =?utf-8?B?Q2dvOE9ySWZNTnMydlVuYU9qeEo2UmdUOHdSZzkrTXVxY2I5WmtJR2ZjblVW?= =?utf-8?B?K2pZL1JDV2VaemtPVkNHQ3dzeFZDdXZQWGVwWmUzYjhNM050ZWdyMitOOFpR?= =?utf-8?B?TXRpK0hNYld3NFVRa1dFeEdLRkJadThueFVkdTgvZDh4aysxYTZoY2pPVnNm?= =?utf-8?B?TUNvWm52cGwxQ1dBbzByRnlSV3RBcmhFMWVPVWRTQ1JlUnpHZGRUUU1vRmdG?= =?utf-8?B?SU4wZEV1dGNnZVJCSUVmZFNlcTdpSlppMXBKWXhoZERrK2w0ck9jSUE5UUN0?= =?utf-8?B?VXdkbEdnUUFwT2Y3ZDIyZWFkWUN4NE9vaDl2VmROWjZTcHlrRnEyN0ZRS012?= =?utf-8?B?UUdYQUo0cnpyN1dPK3RGbjJTSVBVcXQzMUZQZTFxNmpjN1pxOGNFSTlnZlgw?= =?utf-8?B?UzVZSE45WHE0UFBBNGR3SzJsL1dFSTd6RXZZdUNWWmVpekZYaVpoN3p3MlZC?= =?utf-8?B?V1RLUUdxZWw0cFBic2ZRdnVKTjI3ejVKN0h1YlA4K3ljN0R0dXhybU82Z25v?= =?utf-8?B?bllLdVROMHE3MWZYQ2hLeFB6V1M0bkcvZGhDeUpjcngrMXc0anlaamlWWFF6?= =?utf-8?B?SGtQY2ZoanlYK0pVQjVFS1ZkaExnUWpwRDE1eCtrcjZMVFBvVEx5T09KSjVQ?= =?utf-8?B?MDBGZGZnT1AwUWVEc09aZzlod1F6cDVUMi9aa3NEZFdSSk82Wmw4eWhSZmQ1?= =?utf-8?B?UFBkSnNiNTBTMThRa21JL09PVzNzRDhLZ1ozY0hNd0ErZmJyNGJ4ZGxQT0RT?= =?utf-8?B?SXc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: e7504dd3-2404-48bf-99a8-08db5b8ba313 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2023 12:45:50.0328 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uPpkcV1sbtqjTsW98B5DutvNN4QmhgvUrAqjiGaP54HBv+ZdvCFslj8mevxMYuCkdPTvegMzHfisHmo8BIaDmmEy8B480tvPjicpVtNQl5k= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR11MB5985 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/23/2023 3:50 AM, Kaisen You wrote: > When a DPDK application is started on only one numa node, memory is > allocated for only one socket. When interrupt threads use memory, > memory may not be found on the socket where the interrupt thread > is currently located, and memory has to be reallocated on the hugepage, > this operation will lead to performance degradation. > > Fixes: 705356f0811f ("eal: simplify control thread creation") > Fixes: 770d41bf3309 ("malloc: fix allocation with unknown socket ID") > Cc: stable@dpdk.org > > Signed-off-by: Kaisen You > --- For the record, I still think that this is a solution for a problem that should be fixed elsewhere, because a DPDK lcore (even main lcore!) having a specific NUMA node affinity is one of the most fundamental assumptions about DPDK, and I feel like we're inviting problems if we allow lcores to have multiple NUMA node affinities. For example, if I run DPDK test app with the following command-line: --lcores "1@(1,29),2@(30)" The malloc autotest will fail because main lcore now returns -1 when we're calling `rte_socket_id()` from it. Correspondigly, any API's that use `rte_socket_id()` internally for various purposes (especially indexing arrays!) will now have to account for the fact that `rte_socket_id()` can just return -1 and it is not an exceptional situation. IMO if we want to keep this behavior, EAL should at least warn the user that a DPDK lcore was assigned SOCKET_ID_ANY on account of multiple NUMA nodes being in its cpuset. So, as an unrealted change (so, i'm not suggesting doing it in this specific patchset), I would suggest that `thread_update_affinity()` should warn about DPDK lcore being assigned socket ID like that. -- Thanks, Anatoly