From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31DAC4567F; Mon, 22 Jul 2024 12:12:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A9285432BC; Mon, 22 Jul 2024 12:12:17 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) by mails.dpdk.org (Postfix) with ESMTP id 5D329402CD; Mon, 22 Jul 2024 12:06:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1721642807; x=1753178807; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=mQ/yII00hQvLCyDg4v3509mMRFASpHBQNXIC/ReMOdw=; b=UjM3GfgGY8Qw0Cyunqmml6arXcx0bjryHqueT6qlif/xTgRNClDPsbSl kxnRixaPRvITR0FcNfVfHRur078Vvv/sL3r6bl5Hemujw1xn6T3AHRTaR as52A3o5BKxbXgQ2yy19LjX/D96MyfBJqSAeEU36HJUR2pHEhS4CNKoEm +5sfUD+gVzlRpRA/ayhuUWGvMD6HS4hK8lyAiCQJz2seaEvBP2CIlTOCl HHZFnqIKvZZvUJjXA0HnvinDTjoQLL6ax6poajWnBfevNoGz3PyDz/q4s SG4iqNqK9LAq/5oV1H8LG+FlvV5aJy6cveyU7LchHkVA1gHml7xF2Ub0M w==; X-CSE-ConnectionGUID: a5bf8louRQqsvjhBVPH6EQ== X-CSE-MsgGUID: 3YJY6F7hSxu95xYLxEMo0g== X-IronPort-AV: E=McAfee;i="6700,10204,11140"; a="23067269" X-IronPort-AV: E=Sophos;i="6.09,227,1716274800"; d="scan'208";a="23067269" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jul 2024 03:06:46 -0700 X-CSE-ConnectionGUID: 8XnYcHFKTfWdXuh5yYqslQ== X-CSE-MsgGUID: ad1Qa788TV2L4PI+jeswcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,227,1716274800"; d="scan'208";a="52456033" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orviesa007.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 22 Jul 2024 03:06:46 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 22 Jul 2024 03:06:44 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 22 Jul 2024 03:06:44 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 22 Jul 2024 03:06:44 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.40) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 22 Jul 2024 03:06:44 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UqbyLsKov4yHpLibmaWIUNB7ELlzL1L8/OIUEpl5wViLxADGk+ET6X1BzYX+X/v1RGoSoYjcVhsRX3Zfl4TWRIFfw6T6j2GzU8R02iTkRj2VQD5dO0AVvYE1FFHXEOTInMlNTroVGUWuZLEJDqoOgJ0hmb1O0cyHhnrzQ8uw6V4Gw503AUo6pkjj5SLoYAVZrVduTur23OCmmElkXHPSvfMVvp1nYj7XTHQTtfMA2HufLlqeVoHxboKsPA8/5BaBUjp7pZsQbTXusSzQ+faJP0vkJP6BEK/73s5HLzE1qpz9cEiu9MGhu5i5hOb1uyy5dbuGxR6o6ArkzUZXkS62xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pr3sapvSCyEg8ikUSCCiNNSuqk5Zo/9CCvzcv0pKsNg=; b=L+WoR/HMHkh39HVRNY6yxGMYYpg338ogHcqRpRS+q2eSR9QBhKMpLXjekeA/a7X8lP5i7ecwnDmUpSMMyorAWz744+5VouRMo8FAEEvKOGfxW3ZrfhpE2mCMy3C5bQtN3gOCdjquSswQ5mJmjT4LVgPHiSdBZgkAdgKSu3SphrBN8p6cguy72vmuLqQrSrJBRgvl6DM7r9gMYrPPl6JlJCkyghvaCA24CViv81ej7YNV33DJHs8l+/rT5dQ994zYXWP6GN7nd8TIvZiPhHRdUsDIhTPj40Obnil2zjzSF4Y8YQJPirmLeHMDv1rcHnET8O7cF+qLYjE6B1Ofofopqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by DS7PR11MB6198.namprd11.prod.outlook.com (2603:10b6:8:9a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.19; Mon, 22 Jul 2024 10:06:42 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::f120:cc1f:d78d:ae9b]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::f120:cc1f:d78d:ae9b%7]) with mapi id 15.20.7784.016; Mon, 22 Jul 2024 10:06:41 +0000 Date: Mon, 22 Jul 2024 11:06:36 +0100 From: Bruce Richardson To: Ferruh Yigit CC: , , Padraig Connolly Subject: Re: [PATCH] ethdev: fix device init without socket-local memory Message-ID: References: <20240711123500.483119-1-bruce.richardson@intel.com> <4f7e619a-0398-41cc-90a9-3c52b73d1c49@amd.com> <57b08bdb-5f45-4343-a8d2-598d66d82fe8@amd.com> <92bc58fd-4a77-464e-bc49-e012b428834f@amd.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <92bc58fd-4a77-464e-bc49-e012b428834f@amd.com> X-ClientProxiedBy: DB7PR05CA0027.eurprd05.prod.outlook.com (2603:10a6:10:36::40) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|DS7PR11MB6198:EE_ X-MS-Office365-Filtering-Correlation-Id: f1517f9f-87fa-410d-9785-08dcaa35fbf1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?oKdd46XnJaUl++j8TlHnvT9CFZMzNMNWYMOoA0GQIijpp7hSuzux4P7UhiwN?= =?us-ascii?Q?zYxPYuRtXaYVJZSG6Rr7MCe3rqdb/eXL0XOr9pvMo0lFTq6q5AteRKw67eCn?= =?us-ascii?Q?A9gl1hwjEwjf+RqxWGuFQMXLnMPRmI5pSUmCj6xIvA/69W4NpHEZF01t3FYp?= =?us-ascii?Q?vkJcdb6tNZz0aVSuayMKuo6e4Og3zjH4A9XvinVX/va22dugt/D3bl7peh4e?= =?us-ascii?Q?v8d0LOk+MDOcNlU31ZyFtLxrp7lZrQ+sqYvP/Tg5uYqdZZEFfglxFcecAonI?= =?us-ascii?Q?sLBWuuor35EAkjktTYcJJTRXbEte0D22EbqrGUl4BhHGbcxeglhL1kF0TjNp?= =?us-ascii?Q?IPDcYbsOyqCK3Q4a+ChdvQwqgm8wrhDbg2I6sns0ScjkraPrvYgrPx0gUPBA?= =?us-ascii?Q?xq4aSX03lCpBJ2Gl1d5rWBoq3vMfx4tYn+jIPf6M+S1S6JR25ju6+Hzo8jr8?= =?us-ascii?Q?+8iCEyBmXzAmsyBeniRN/1cDHab4OHl9WlRB4GAw4uKVqZW0+KrkjZaw6sEj?= =?us-ascii?Q?0G9tLfW1vv5Fz63KJcLiHZchX9BTzxxkdq2mE2R34XAldtd+6OmkZNeQB0Z8?= =?us-ascii?Q?UjRHnuwKZt1r3Bz5uvkZnGJ7G/HkuzRR3fc12r3xJ6swjC466dnDcWUv8hyu?= =?us-ascii?Q?YHWtcyXZi8VJlyGBNn9pa4+dRfaGbUNscHKDzCmmpQsDBx23igGFpVzOM7qV?= =?us-ascii?Q?E6v2+NnR2YTsmybOSv31Goc5EdjLeZ+jHcxjSY/3yJgSJWD550rD5iWckr2O?= =?us-ascii?Q?gwMjyyFtspynqF9HCglQsGfok+ZeZZWPSPZWR+05be4oO3DTvBmrDVrDQ4f5?= =?us-ascii?Q?B0LqbckZg2Pye+05nR3uQSeRAxP0CMZSAEgHOdv+25ZIqXc/t1f0EEqXVeop?= =?us-ascii?Q?tW5HEIjDsv7Uo9a1Z2jXgAIqqaW8WMyvPM3m9S6E/XAUSWPOZU4lz/kzYSIL?= =?us-ascii?Q?K0/IYReieSaoykVvs1y/uXSb4pANmdiohRN5atGEfu9tDmGEPGb8awoVa5hG?= =?us-ascii?Q?gskP+2JQ7dEwj5x3M/nbiDY3THl2PDC6ZbX54CKnzfIMtzCX3Ej9ZaOrg9OK?= =?us-ascii?Q?cNQWWDNWWsOW0sLOyeNirWw/LpET8YQEk6tiuZHHoq7baBwm8FdTVOh+RxND?= =?us-ascii?Q?Kxk2Y++Pi57Tub4IKe1YBg8H3OF1cPPhLi7QYZflc/1KMy3Ln+p4d893zPbv?= =?us-ascii?Q?OC1De+Tr+q34z6ghrAdOwL52sxGQwZSedXNVMgl5LE3WVDjo8mB6nV03vy0i?= =?us-ascii?Q?3IaZi+UBELUZrEwPn3kLXsMVBDyGEhmxjH8QrplNifE9+kVz/MZD7yc8Er1h?= =?us-ascii?Q?iVHfGzva3uaCFLW8vMrqec5uLOmemnmPyj5BCSYo1a8pGw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?aniIQAPuqvE5Zr6LoLK6U8F/a6i6JiZgXDQJAquySv5ImYFK/dLDtcAJIbdu?= =?us-ascii?Q?ieZYLkLHAp+MwaDsuLeug1jUhWYV/3D2LucXrOB7GJQdRhKlW0JZWdYN+xrN?= =?us-ascii?Q?Xl3EbfPVh4RmY3nIGW0FZDOF+Mtb7TyBN6kOI81pBRqRLSeYTTqGcUIBcyyi?= =?us-ascii?Q?RJRg/tlfXMp1sU9Hk9CTBe2RwG4Hf0Dvv/0khTZ3WEiDVRqKoYIz12FFHUZ4?= =?us-ascii?Q?+ZsZZE6WDvTTKzTJyrgPMXxuYvmvmLUYobhZwD3x/Zgsf/v5lNOjsrICv3ax?= =?us-ascii?Q?S4rjP1gb7T7PIJZFtPdRB3DeAB5QlfhlHuhym+QYlxKjN8i2QKLtnvnNwxgv?= =?us-ascii?Q?brx3XtttnsrvlzCfmTpVEaECn5/2lQ992/ImbvpnSPcYmAwwz8tYwDVGt1M4?= =?us-ascii?Q?9O+xeLHBHkpVTEdWh5auEhLeU3UHQWmSOESryKdLzdI371pEp74wtP48U7pj?= =?us-ascii?Q?YnRwS3ZnNmXafTsy4h0ncSaoiPb9mcaNz8JxiDJQnlU9Ml8RT6SPkAh8xTjp?= =?us-ascii?Q?i10Kn4E314Go3XdtfnlIv7azzZHRqnO0ZYNmkEmykBsBgzsKFYiwG1E9EN4y?= =?us-ascii?Q?Qv9zv4O6o4LGEoBNKLKucD7CRujvVOYSzO2cFKLc8Lke52Cx3WuHp6lO/fj+?= =?us-ascii?Q?y4b9wewJYIfwny9K+hd1cygTHprcTWODR1xQXgFM579HHdQKC2rePbzOJxxD?= =?us-ascii?Q?/LG7Jh4bZZYEEmG0HRe+jT5uIH4us09ueXZw7Hv69qx8vCx5GavayJqY1PUJ?= =?us-ascii?Q?0VnufgoObMWGY6PpJq3koBTX3dhI7gaa3MFt1GSWpfFOyJq98bq9tRfoSoeZ?= =?us-ascii?Q?OJwvAU5Y3yHkf2pQeRj2Uft+bv6Rrsiu0F04yF0nOAn30zlOf9xepan5YVhf?= =?us-ascii?Q?E8gvZKtKZN8xra3n0MvYkfV9XeFJmvvkaRcCH30ybOJU4RnwxyTC5/noN2AA?= =?us-ascii?Q?or2Jf9pMLYGHRFP1jxJ3LgRYeKCMlUdIVEDvodAvTI86M2C1WZcehZBoqlAv?= =?us-ascii?Q?4lb/s1+CW9f7AuXpPHa0QQSOFKRB0yhFxWNQU5YgITtCRymAt5j2Tin3Qfze?= =?us-ascii?Q?so4E634JrdwSN6PxxnKGNIpBzFwsIdbPCUKJViEw3ITZK2r30CcgRAt6RLuN?= =?us-ascii?Q?GZ8lOQi1/dou00veNtvBChsD7xAcAFTdsqRjGHSTZG0inTx92V/v3giYgpZz?= =?us-ascii?Q?2VAcngKUmmyFFEVjJ2X8QkOlwfhTs4JBXTCCwuOobgVBzXPHfkdUZgDPRV2O?= =?us-ascii?Q?1kOqW+RNULv5yCZxz4d1IgTecxm89tNIeIepmUuOhV9NpsoHt03gin2DEkjB?= =?us-ascii?Q?BGe0taaw4hqvUgf3KiKPdFiziXGM3bwvGNbpGfotEZV6fjcxSD6DnKo2KvYf?= =?us-ascii?Q?HO6UmdRb+S+5Siy3UYwBQHGzct4btZDBeYv8EzqkA/ydUdnTp6lHW2DFeIHT?= =?us-ascii?Q?2RzSZQ/AOECqg1wRkVD/T4qiFcbLNENGZVeRi5E+nGeU9HUbLMAww1R+HNoq?= =?us-ascii?Q?W3+E3Y0zeJ8T/hHIT/pkcnt4dQ4Q213a5ZT0t07mEY2iqXFVr7hjImjLoXS0?= =?us-ascii?Q?MFg61WbIMHypIXzuiRZvXdsDtgftwsmxLJMw1/4FGiBIkfGpKKMUbjEbvS4E?= =?us-ascii?Q?1w=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: f1517f9f-87fa-410d-9785-08dcaa35fbf1 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2024 10:06:41.8553 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lFxRYM109OdQ/NR8I6NDdPVj8E1s5a037sH3HhvUrXvi34R1zClXZVYl8OYz5xF8vG14dBjoEMSCh0eH4TKDkOB81f4WXGsrYIog5fDFRqw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB6198 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, Jul 21, 2024 at 11:56:08PM +0100, Ferruh Yigit wrote: > On 7/19/2024 5:10 PM, Bruce Richardson wrote: > > On Fri, Jul 19, 2024 at 04:31:11PM +0100, Ferruh Yigit wrote: > >> On 7/19/2024 2:22 PM, Bruce Richardson wrote: > >>> On Fri, Jul 19, 2024 at 12:10:24PM +0100, Ferruh Yigit wrote: > >>>> One option can be adding a warning log to the fallback case, saying that > >>>> memory allocated from non-local socket and performance will be less. > >>>> Although this message may not mean much to a new user, it may still help > >>>> via a support engineer or internet search... > >>>> > >>> > >>> Yes, we could add a warning, but that is probably better in the app itself. > >>> Thinking about where we get issues, they primarily stem from running the > >>> cores on a different numa node Since the private_data struct is accessed > >>> by cores not devices, any perf degradation will come from having it remote > >>> to the cores. Because of that, I actually think the original implementation > >>> should really have used "rte_socket_id()" rather than the device socket id > >>> for the allocation. > >>> > >> > >> Yes I guess private_data is not accessed by device, but it may be > >> accessed by cores that is running the datapath. > >> > >> This API may be called by core that is not involved to the datapath, so > >> it may not correct to allocate memory from its numa. > >> > >> Will it be wrong to assume that cores used for datapath will be same > >> numa with device, if so allocating memory from that numa (which device > >> is in) makes more sense. Am I missing something? > >> > > > > It depends on which you think is more likely for the polling cores: > > - they will be on the same socket as the device, but different socket to > > the main lcore. > > - they will be on the same socket as the main lcore, but different socket > > to the device. > > > > Personally, I'd suspect both to be unlikely, but also both to be possible. > > For the first scenario, I don't see anything being broken or affected by > > the proposed fix here, since priority is still being given to memory on the > > same socket as the device. It just opens up the possibility of scenario > > two. > > > > My comment was on suggestion to use "rte_socket_id()" rather than the > device socket id, > if both nodes have memory, memory should be allocated from the one where > device is in, because although device doesn't use private_data, polling > cores will and polling cores will be most likely in the same node with > device and memory, but we don't know main core is in. > So I think first try for memory allocation should be node where device > is in, which is the existing code. > > If node that has device doesn't have any memory attached, your change > enables this case, as already there is memory only in one node, it > doesn't matter if we check device node or main core node anyway. > > > Briefly, I am OK to current patch with a warning log in fallback, but > not to "rte_socket_id()" change. > Ack, makes sense. Done in V2 patch. Thanks for review. /Bruce