From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 758BA42B81; Tue, 23 May 2023 15:56:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6A55640A80; Tue, 23 May 2023 15:56:14 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 1218E40689 for ; Tue, 23 May 2023 15:56:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684850172; x=1716386172; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=GpEcAf905EwFvnZXVJcgaMyQRong6ufX9SMqCGT1Ql4=; b=ZM+zhA2VHI4xb/YhOEi1vIlYjLjyASoEfSf8IK6BksWZZr5Z/x/ExfHd zIVIyq+68aBhtLI7BtUPcNb+/L1ZI/Dwc9cP+l8Ga27xDqxDVdeQwv/Uq SXM0VOkZiF3RNcjX+1vlkwTu9ANTPSbVWPOwFQY/oPowJ/Yfb8q1xpeUn LxsLrMB8NsDR9nr19ZV9xvsKQojDIdGA5Qj5Vfre3OSZDcCqVkRCVr+cv Oa9Is+76wje/z0ZM8eHLK9Cz8Rp8nRTjK9gNRwaXoI4LEcpxhyfexDuI0 iP853F0z+7gPY+LzUJFhZcHyz/bpyv0kGEJA08qa1u/RaXhX9pR1g89r4 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10719"; a="350751326" X-IronPort-AV: E=Sophos;i="6.00,186,1681196400"; d="scan'208";a="350751326" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 May 2023 06:55:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10719"; a="793730119" X-IronPort-AV: E=Sophos;i="6.00,186,1681196400"; d="scan'208";a="793730119" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by FMSMGA003.fm.intel.com with ESMTP; 23 May 2023 06:55:55 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Tue, 23 May 2023 06:55:55 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Tue, 23 May 2023 06:55:54 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Tue, 23 May 2023 06:55:54 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.170) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Tue, 23 May 2023 06:55:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RGj4lyUPnpdCK9PQDdmwSl03N+Z5HC+vJrIQx2H9MsRAYEmBA7Tha8cK6TUrwmMEAuRXI2aZNm7DaMHlvL6CpCWWJ0MkXLmGLkMiTMdBBgyA6fojxofqq3FxFU3PyYl/XyCn3/p1X2qW4g9U9FEQXNZOKRILi+lpXc8l7le+LwRtC1gbCbRaCfmYRd5LgWmZFbFvt8gBsiWpomWfzNNvZlt/qHI3SbJN6HSc0ec5Rkdlux7LJPa5fwtuHP/8asA4H8VGJzIARrcSp3sltyuJ6BypjGVjhyD2Fe20K+989KorRCulZjqo2Eq2E8RjRb/VLE4E06pFIEgOnVu4hxR19Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AFk6EEprmOWNFa68JZ3HlD6bRz1ilRVo35F/QzUkiVU=; b=HhCE2JfhU8f7upFtJv5fwSxfxEFWL/HKgqd6HSf6X0KOvzIUScnQCLV7vOdMEzviyQliarKlF44ZuDWvSe/ABaHzrk3k6U19i6qKgi+irkLXTv5w8FnzL3mGe6CdQjioyleT/3R946ZuSeQd8GSGuhC50KjyTGIn0nZSBkRNlWnOxqTf4soecS6CXvNk/rmalyzcIhEtVHOA4/8uKmN6hnVNqIMOuLh0JCjGyue/xCYp8cOmU2Bqdavk0gsM+pFGFv+RboXHqPWCy3sXu5jvPIZgExFFIU1pQ4ifDO7tG5i0IA4GeeD5e6dP73b7TV7lm6JmqpKW4iOsjlLZNU00bQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by DM4PR11MB5280.namprd11.prod.outlook.com (2603:10b6:5:38b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.28; Tue, 23 May 2023 13:55:53 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::b8f3:958:d2c5:2232]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::b8f3:958:d2c5:2232%7]) with mapi id 15.20.6411.028; Tue, 23 May 2023 13:55:53 +0000 Date: Tue, 23 May 2023 14:55:46 +0100 From: Bruce Richardson To: Anatoly Burakov CC: , , , Subject: Re: [PATCH v1 1/1] eal: warn user when lcore cpuset includes multiple sockets Message-ID: References: <0f77ce4515ff074f997f0068635b328060322d5a.1684847165.git.anatoly.burakov@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <0f77ce4515ff074f997f0068635b328060322d5a.1684847165.git.anatoly.burakov@intel.com> X-ClientProxiedBy: LO4P123CA0132.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:193::11) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|DM4PR11MB5280:EE_ X-MS-Office365-Filtering-Correlation-Id: c7c166a9-099e-422d-cc50-08db5b956c3c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zCq6W3ZG5bANhIza7i3ZL3arC5njIGXzZthfB7VX88/7vlY7wSr42HnoBPLBfOPUTLblizCtehe12BOwbaXPCe8bcZrJytUuXsHzN5iD8XPOGQqVUDkaEvdVeyL0Xux7AlBN2ZMp/x2TcWeFoHURgTGR9wZhYpECeDfO0ZK7k4UxEVA+VVcMFQlUhhM5teXa7DJMMZr0kSK2a3qvaEs1OzKKgEYZnFVWXib58Y4uJWMgootl2jSkVriS+S018QJBbymYSHoKiAjruUBYBchOpiAneHkju7GTjwTS2dNmFrebnSedbazlsGRvEme4tKtqzsFzBaV1aeTWFPlhs0MBEUv4ThZ+Gc5cB9BeUzSoBsxkG5yMF/040jct/Ormfwqdf9whAvb9Ljuxsly06JwjWw5/BEFWedN5iSePExMHtxeU+NcYlDy4JiYNAaqoz+H0gj2HcoYQKu/jwq2VftX4oOZ6JJGiyncVF5rIk3jT0HQ6fO7r+Lho2+3wGuxskqbeE1v2xGZdJ0oQmpdZmWGGr19RnbmWbCB9aEdVP72SgBkCUd0QxxvCQxZap2u9o8Zy X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(346002)(376002)(39860400002)(366004)(136003)(396003)(451199021)(83380400001)(478600001)(86362001)(38100700002)(186003)(2906002)(6636002)(4326008)(66476007)(66946007)(66556008)(6506007)(82960400001)(26005)(6512007)(44832011)(41300700001)(6486002)(6666004)(316002)(5660300002)(6862004)(8676002)(8936002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?d6xF9hgotvBw8/XJiJtM4fPiz5AaBlrld+8qF4vZFJvv4O7xzkWY+J3Hnf/J?= =?us-ascii?Q?8dlqETV/XnbIhY5AXEVwV8cpR682rWqHWyJuxZPfTMlGhzfJ+DaZ6sjAN5+h?= =?us-ascii?Q?MwPFpViazfGWtc/EuA995Av+LgW79OHxgBjDY7WFFwq+NdxWXqeP5XexaHu1?= =?us-ascii?Q?9pMHVG0nDnW0e+8XmM8DisqWVM+2QoII+OfR3+vJ7acj8I2uT0NT/PqY+Ug7?= =?us-ascii?Q?9YFF5XCkDdUd+ytbck1xpujAr5gtcu2hfa7n8HXnds4XgI1aHkYUD4NEtkgt?= =?us-ascii?Q?y7C0OALTksZlEbZxTSX5d7AEWXin1vt0BQOzarThmi1YlA/lhcqm/urTx7Wp?= =?us-ascii?Q?cGDuAMSag+6ob0yUOr60w1VYeyIhLXLPwsC8tnvfjwJSvU10dqLvVZh/pe5H?= =?us-ascii?Q?l74Lsaz/zcMurxoZvBw7FUPBZ5xY4y/0274LZUgQTDmPbWV4oWEnqG0vwCUw?= =?us-ascii?Q?On5N7RxMOw8YndHReIyPFCOT97uMYBz7TSh+huVbXzJGFcw119vL/PDkCJbr?= =?us-ascii?Q?/4HRTOmD5omX+RYvq22JjEKaE5YcOlT7yB2ri+UmY1Iqs0pKRwV7r5e8bDaA?= =?us-ascii?Q?VGl4FyMFwu0eRWRddseYzJI12wRLMmE/YnziKTDM0NukWm1PtdYhAeQoF6tt?= =?us-ascii?Q?ot3SIVepOI/d09I2A8r5BYAGJAJQHOv7QiW71WwfZ4YzOrbsZ9WUqFGmko1v?= =?us-ascii?Q?xZRy4LTjTX/HzSL/TRp48fnvDjVKQi5JGYgXFxe0zdI5NfYbyNUyAtGwyfFo?= =?us-ascii?Q?6ZVcx8sjbW6dywozTu9pT9cZafmar+HUtvrQu1Y8BP378MwZwTrTC1tOIHym?= =?us-ascii?Q?Hykj9MGTNjAaNl4+aiIZ9ST+JjkzeOvkusvMuSnS0EZ+RutR7t1+udtujGRg?= =?us-ascii?Q?6t/UKdL8EthpBdH0eTGXio/DZCsr36gMdraZBIt3vhaf5bACripcdBAdJwK5?= =?us-ascii?Q?kY1Jnj/BSdvUvFQ/vWtQN2Jyvbxkt+W5jWhmq8Anl7Y1WvkfaJxVtwa7hqTe?= =?us-ascii?Q?pwVb/DEneK25p/dSEDs+JN4zum53I+QJ2IFBUS8zfcORuJb2L2gimmKkpIFh?= =?us-ascii?Q?08aEzrqoSahf8nH87Y7r9D0hvn5wOlkmneHdPdfhAXWEv/fO5yB+b3BdNiCF?= =?us-ascii?Q?bKq9OfX1hKREkMSJqConkdDZTURKmtrGiDivpKJYqlm1+DTZoQMXD7+ERsm3?= =?us-ascii?Q?wWyYK/+xBjv3ocCOpRq30TTr5rLumGwLmOz0QyiRqc9jyM7wIBVGLCIogoBR?= =?us-ascii?Q?Cmm363uYdYjDiTvFqLNiHiCHpb1Fm5rkAh7yCIUfXL2Vxzgef34zZgCW6fQO?= =?us-ascii?Q?KRqbS4T5p7olnVQvaQbHij6JveXl4+mvNcwlh8gqk/J5UkxMvyg3X3REwX7/?= =?us-ascii?Q?R5FZqz5+yMuyJ4/jJPbzJQ9LlWNDKQBfi+xan3trNrAMj8XuzIj+c0d7JzzD?= =?us-ascii?Q?ZCVOK+Q1FEpPVOIGJ3O3vEOwOdE3XskCdE8ZyDWG319FuHgN96YaFFYhJdi/?= =?us-ascii?Q?Bu6b54+ZCAp4uCvX/hxCBxWYk8C7Md3E6aMIVOcDsySG1ZG+CJXI3mTzlPfi?= =?us-ascii?Q?qvLM6ou4rnIGVWd+BdTznHyFU4I/VPMf6TUNOMRl2q/OnXfAMzVd4Rzd9DFM?= =?us-ascii?Q?RA=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: c7c166a9-099e-422d-cc50-08db5b956c3c X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2023 13:55:53.0296 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wRgfe/GjBZ5ftZc52WebaJWE3hHtyacO1HHwpHKvX8cKWUQ04W7kYkKrmNF99ej1RHpQMxJAXnXeZEBbmL6BnypyhmEDIlVHSfkzx9eZwzs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5280 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, May 23, 2023 at 01:06:16PM +0000, Anatoly Burakov wrote: > Currently, it is allowed to specify a cpuset for lcores such that it > will include physical cores from different NUMA nodes. This has an > effect of setting `rte_socket_id()` for that lcore to SOCKET_ID_ANY, > so add a warning about it. > > Signed-off-by: Anatoly Burakov > --- > lib/eal/common/eal_common_thread.c | 20 +++++++++++++++++--- > 1 file changed, 17 insertions(+), 3 deletions(-) > > diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c > index 079a385630..46fd2aca1e 100644 > --- a/lib/eal/common/eal_common_thread.c > +++ b/lib/eal/common/eal_common_thread.c > @@ -65,10 +65,13 @@ static void > thread_update_affinity(rte_cpuset_t *cpusetp) > { > unsigned int lcore_id = rte_lcore_id(); > + int socket_id; > + > + /* find socket ID from cpuset */ > + socket_id = eal_cpuset_socket_id(cpusetp); > > /* store socket_id in TLS for quick access */ > - RTE_PER_LCORE(_socket_id) = > - eal_cpuset_socket_id(cpusetp); > + RTE_PER_LCORE(_socket_id) = socket_id; > > /* store cpuset in TLS for quick access */ > memmove(&RTE_PER_LCORE(_cpuset), cpusetp, > @@ -76,9 +79,20 @@ thread_update_affinity(rte_cpuset_t *cpusetp) > > if (lcore_id != (unsigned)LCORE_ID_ANY) { > /* EAL thread will update lcore_config */ > - lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id); > + lcore_config[lcore_id].socket_id = socket_id; > memmove(&lcore_config[lcore_id].cpuset, cpusetp, > sizeof(rte_cpuset_t)); > + > + /* > + * lcore_id is not LCORE_ID_ANY, meaning this is a DPDK lcore, > + * so having a valid NUMA affinity for this lcore is important. > + * However, if cpuset includes cores from multiple NUMA nodes, > + * the socket ID will be set to SOCKET_ID_ANY. Notify the user > + * about it if that happens. > + */ > + if (socket_id == SOCKET_ID_ANY) > + RTE_LOG(INFO, EAL, "DPDK lcore %u has NUMA affinity set to SOCKET_ID_ANY\n", > + lcore_id); > } > } While having a warning comment in the code is good, should we not also have a user visible warning to the user when the specific a corelist parameter value which includes the cross-socket scenario? /Bruce