From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CF8B2429C1; Sun, 23 Apr 2023 08:52:12 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5AC1E4021D; Sun, 23 Apr 2023 08:52:12 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id A086140156; Sun, 23 Apr 2023 08:52:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682232730; x=1713768730; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=y/AZ0/y7/ybd4ObT7+s4egpKciY8/lRmg29wKRmdKm4=; b=N0xCVSt8EJaIFweKKywkpsjBzskDCwJMz8AIaRnfyzEofjZQrNCYzTMv ZiZwULmh64+SDttIU2OxST+HZ9W+AstxA/rbYx60DBI9LjmIG8ITtvDeU /hiCYtALCmzWGAFwV6LdRiHqgvIR0UN3U48p+bezXexefYPW2EXtrQ96F bLy3b4qDNRubermmSkGvTTM7u2fQwF5LFajU6JNRIOxOavOhRgyzR5bI5 tuUXk2d7pBBzccKM7m8vo8Vae6D2jR8/G6qX43dPX0swRUzfyfcOfbrAj sjywv20CpaGYV2Yo0vd76vRSVB8AD4/Z3ONchzXPeiT3VHzWsziLQTdG9 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10688"; a="411528101" X-IronPort-AV: E=Sophos;i="5.99,219,1677571200"; d="scan'208";a="411528101" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Apr 2023 23:52:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10688"; a="1022325094" X-IronPort-AV: E=Sophos;i="5.99,219,1677571200"; d="scan'208";a="1022325094" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmsmga005.fm.intel.com with ESMTP; 22 Apr 2023 23:52:09 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Sat, 22 Apr 2023 23:52:09 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Sat, 22 Apr 2023 23:52:09 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.172) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Sat, 22 Apr 2023 23:52:07 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a1muitd2uQMcfhBI/oc7KunSQJpUemdfslXeVXX0o5/NZMTwEeqs3fgI6gKF3Tl2SUeTu3aDdVvCtdEcaI4hb/RhbHFTqPWM5v2TJ1ACbbPWQdH6De66fnK0xYvKZdJLcSWAKwcW1bRnqpddGlDOOY7kdxgIh/LevuIyDzKPfYx4417LJWSMfJdWs+I57T0tVKZ+eDasb631NaHZtOi1BWJgJDXh5QPEYxkyMbj6FLeF/21UfZSJdFJDC6EQpl46wr9AhLQ8RbLu90Kzn6v4nFsj5EvQuvZkpQDbNjO5qM/GgOBDkRhvWkcGM23Le23WFTwNEkAeLc0Djwfjg/0Oxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U5T3lH7tktJZ88jXNF2SorszufUgl3WhYeDg43uygZM=; b=cr84Q7dwQwFQfXKWbo+Uic79+AQ/2fPBLj141NCtC/azEjbsdmfQz4Z+Yr/Vb4+Rg8lU3AYA5bIB144oz9hiWIdq/DMl+v+Bm8mJlGC5vpIYHGTXSoJmj9LAnChdUKUMs/kfQbDZRYPuH3cgZPhbHAUxaybhofRVwBIvgmC4Nh5aXc6b4IPAqryiFEFpdjaIBWG6kqmdBHtFkUIXKO0e0gUEVVLEtOsVF6ePKgN7lsWLY9+mTnwbLqbT874ZILWdKdkkgwqdHYeU0XVUN44LaiFW1+Arqzi0+Nm/issffIBis6h78sJEaaE4Wi/KAyUvtmPNarVz0FkI5l6Rfm/6eg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SJ0PR11MB6765.namprd11.prod.outlook.com (2603:10b6:a03:47b::10) by PH8PR11MB6802.namprd11.prod.outlook.com (2603:10b6:510:1ca::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.32; Sun, 23 Apr 2023 06:52:04 +0000 Received: from SJ0PR11MB6765.namprd11.prod.outlook.com ([fe80::d8de:a1e1:f598:6572]) by SJ0PR11MB6765.namprd11.prod.outlook.com ([fe80::d8de:a1e1:f598:6572%4]) with mapi id 15.20.6319.029; Sun, 23 Apr 2023 06:52:04 +0000 From: "You, KaisenX" To: Thomas Monjalon CC: "dev@dpdk.org" , "Zhou, YidingX" , "david.marchand@redhat.com" , "Matz, Olivier" , "ferruh.yigit@amd.com" , "zhoumin@loongson.cn" , "stable@dpdk.org" , "Richardson, Bruce" , "jerinj@marvell.com" , "Burakov, Anatoly" Subject: RE: [PATCH v5] enhance NUMA affinity heuristic Thread-Topic: [PATCH v5] enhance NUMA affinity heuristic Thread-Index: AQHZNf9KkwjFirxtKkqXm6ZbKOABl67QJeuAgADH7FCAGFkTAIAInSJQgDb0DOCACi2GgIACffvQgABijICAAwQmcA== Date: Sun, 23 Apr 2023 06:52:03 +0000 Message-ID: References: <20221221104858.296530-1-david.marchand@redhat.com> <3549930.R56niFO833@thomas> <3535353.LM0AJKV5NW@thomas> In-Reply-To: <3535353.LM0AJKV5NW@thomas> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR11MB6765:EE_|PH8PR11MB6802:EE_ x-ms-office365-filtering-correlation-id: 4ee2832f-6b49-45ce-806e-08db43c73f14 x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WY0qt4zkHryC/Fta36Crno7b9g+TKSPlTbrDXicrDJx/HriIPOr/00bhfQOcsTfJAtTgrJjXVtb1OrVTC1F7tECxutY5e6u6i9WCXJWPyTu5Zq7ksVVIFAMSCIEw+thc7B6+P3F5V34aLN2Zfy6eYXDKbe9I4BUi1UbmtJRFz7KKNNhqYvVsJF4GfbUgulsU56nEKCdysFBbp6QUmYbaNu1vnS3v62O/0XHDBY/LshnbTGFHKMez2rYWm0PgoEy/y0tQS6EEFGcZEjhS1EJ1hB8/5DpcUZ2a9eE7vnQrHQJOfjPdj8nRRCISzgVMcl9mzq+Sr+2t5dzl/PDHr2t+OuFKpwKI9pLdOvPQz+UsMSkz2BIX4B01R9P1KAWPxfTBBF8UzcZnJhex+P4zLQVAO6FcdBArPEZNeN7iyZl/5LF2u9FG2TxTfnI9ZFJ4ZvVQ1y6w5MaRdmXcO7lAtKCzHN81JQ0D051Yluy56gpvAYmJky5QhHVt7pDuqm2OKjxBbwxMZWhiwfnIbTyz/b7uaZsn+0H9detvJxSkUyCL/nhDLBXbVop5uIEGv24kKdDc1sZ2ygU5GbOKo/h4T9bj/X5wCOI7pKPLqTj2AMlgzk/alnLqzndBUhZCSlmxFIEi x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB6765.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(376002)(366004)(39860400002)(136003)(346002)(396003)(451199021)(8676002)(54906003)(8936002)(38070700005)(316002)(478600001)(66946007)(66476007)(66556008)(55016003)(64756008)(66446008)(6916009)(4326008)(76116006)(122000001)(82960400001)(41300700001)(2906002)(38100700002)(52536014)(5660300002)(86362001)(186003)(26005)(6506007)(9686003)(33656002)(53546011)(107886003)(7696005)(71200400001)(83380400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-2022-jp?B?WkF5bUdXell3MXI3Q2JCd09tK0V6c2JTT2h4V2dGSithWXpDQWpxQUVW?= =?iso-2022-jp?B?VTA0QzIyd3A1TFVtTE9SR09GcEZOTmdVVDhHZmk3d1piMFhJeU9BblhF?= =?iso-2022-jp?B?YUlPNVN0OXhneW4rSGlCcGJSTW5pZzJ3SFZiemdWTU5tejl4QkFpTS81?= =?iso-2022-jp?B?TXZKbDVTSmtmR3lFcm42NDVWS2FEaVVISUg0cnRsaW1QWSswV2ZiamJG?= =?iso-2022-jp?B?MmNwUUVUYW40WTErQU5oZE5RalBCUnlFU0hSVERRYUdKRUkwSytjZ3d1?= =?iso-2022-jp?B?UTFlWmxRanpJeTY2S05vSFVBWWpaNXI2WVVnbkpDa1dJajNLSmRYNWs2?= =?iso-2022-jp?B?Smx6aFNPMWZhaE4wYmUvaGI4VXFneHhpa2R2bWVyZGlCU1lCVlFEN3kw?= =?iso-2022-jp?B?R2xYYUY5cm5xNUVTVk1aNEZvMUFPamU3aUErVFB0M2hycU5lUHcvVEVB?= =?iso-2022-jp?B?TXhZdE5SWDEzWnJjYTY3NnBBTE1UT3Nzc1pIWjJ5YkpmaG1tTC9FWEJV?= =?iso-2022-jp?B?bEZOS052L2R0ZDVGZzBMTTJ2Y0NhamdtSWJ6OE5qRTAyRS9iRDJJOHRz?= =?iso-2022-jp?B?eTh1dlY1clZ4ZGFpR1U0UGNEaktTa01yZzE4SUltamFLcjNZN0xoUnJt?= =?iso-2022-jp?B?VWNEN2VCYk1ZdXVWTWxTWXh5dEJzUTUzdENtTUwwOWpMRTN5RzIzclZz?= =?iso-2022-jp?B?SFZnRjVYNEFEU3diQStCaVBhYzl5bkZYMk1RWUFFeTJiazR5SEhWVTUx?= =?iso-2022-jp?B?bUJjQ0FxWnZTZUxkSlJheElYenlHN1Q1T1JnVTdJeU1RSzVFUkhzNDdY?= =?iso-2022-jp?B?WUljbGRheWFKR0ZUa2JIMkJzdXI1ZUN3bEVTN1lpU2lMaXZRTk5MZ1lZ?= =?iso-2022-jp?B?ZGxXTnBuTGpOOXFPYkdvT3RwWDFDVWVlRzBFU25IRWMzazFxdWpkc3Mw?= =?iso-2022-jp?B?T2RsdXBDSDJ5TmFKYTBVQ2dzSkloQ0VVUUxYczZIWjFvZ2c3OFhQOCtX?= =?iso-2022-jp?B?Ui85WFFmSDNxT2VTVzZpT0F1Uzd3UGlCRDQ3OFZOYU10SHJTam16VjdB?= =?iso-2022-jp?B?RVlZMGV2N1E2MXdQeEVlRk1aTDV6MTl6SXpGS0IyQVFZQlVnY3EvcGx3?= =?iso-2022-jp?B?SjVEb3ZvSWsvaWhQcDZLK0VMYUJMOXlJWFU2eC9IS0hueXFkTGRtS29w?= =?iso-2022-jp?B?MTNJUWt0b2xlOEd5VCtmelhmV3g2N296ZzhFWjRGWE5RV2NvdnRJdjVK?= =?iso-2022-jp?B?WHlKYTd1dTQ4MGtEZnNlOUdtU1p4T2ErNXBZTlY4MkRlN1YxTTQwSUJ4?= =?iso-2022-jp?B?bFlxU1gvL1p4SXF0Nll4UHBPRXhHTFJ6S24rd1d3UFNZM2dCTmVnWkdY?= =?iso-2022-jp?B?emNDc3hVYWExbmRTdTJSRnI4Q0IwVUhUSzdzN3hqRDZma2JCb25scVNE?= =?iso-2022-jp?B?OFhOY2YvRWFPSkNRMzFaUnNFQ0REL3JvZXpSMHcrZ0FhNU1vQXRrcVRL?= =?iso-2022-jp?B?TVhIemdIcXZOSWd6UmdKWUc4c0s5OFVhbGlTZHBkdXQ5UHU5dEprZWMz?= =?iso-2022-jp?B?S2lQS003enRSaVhobmtRZ3pPeis3TENmN1Y1cGFDK2k0b1l1bGdBaU5r?= =?iso-2022-jp?B?YjhFMEYyZUNjejdvZmxoa0FjWkhGTjRvOCtJMUUvVTVBdmRWa2xDNzVq?= =?iso-2022-jp?B?NzU5YmdVT1dwOGdRMm1mWEJKVEQ3M3owMGFEY0xzNkxaaXFjQTB1UUl0?= =?iso-2022-jp?B?YmJ4dEYvNmwvRUNZd3I2aWNGQUNPMTNiSzVtZnBtVHBJT0hiRDNCaXNZ?= =?iso-2022-jp?B?QWtGS0FIR0J4akVWRTY3cGtCVmhGeWNVSW5sbzJQNjFLZG5OSjMzZGtT?= =?iso-2022-jp?B?bEFiZmFHTUQvZjdVSDg4algvOWxyK0tmZjdNTjNIWkxIU3NQcGhwMDJh?= =?iso-2022-jp?B?Ri8yWlJobzFPckd3N3RMQmF0U09LZ00rMkYweHBFZFFVT2xxZ1E5cmJu?= =?iso-2022-jp?B?UFNVYjg1RGY1aHJlMU54T3YycFpDS1E4aXFFaDdpK3kyVktFZVd1b1Br?= =?iso-2022-jp?B?ZFFvME1IYmdRMnVhcStvbDV5Z1BVZlJvRWlBcWszTVViMDloK2ZmVGxE?= =?iso-2022-jp?B?WDNhQStTVnNBanRaTXVqUXkybWFRRlhZeHVmc01aVU1DandVMnU2d1px?= =?iso-2022-jp?B?bE11U09GWHpTTlUwSThkczZhNy9PRGpZSS9ZMWdRaklvc0dNL0labDNX?= =?iso-2022-jp?B?Y2RLSjNaSmx5WCsyd3dtS01lVkxxOTBpQmI3NGZ5TGpxRWRITy9kV2Qz?= =?iso-2022-jp?B?cjlmcA==?= Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB6765.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4ee2832f-6b49-45ce-806e-08db43c73f14 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Apr 2023 06:52:03.8876 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: LLltDMdIe7rQ5n+20pBq1KdRo+ODfVg6CDrz8PAx2WhR00Z8+sYLl99whA5w0ZsioAxZNEOrdyxKEtYfnRWFPQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6802 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Thomas Monjalon > Sent: 2023=1B$BG/=1B(B4=1B$B7n=1B(B21=1B$BF|=1B(B 16:13 > To: You, KaisenX > Cc: dev@dpdk.org; Zhou, YidingX ; > david.marchand@redhat.com; Matz, Olivier ; > ferruh.yigit@amd.com; zhoumin@loongson.cn; stable@dpdk.org; > Richardson, Bruce ; jerinj@marvell.com; > Burakov, Anatoly > Subject: Re: [PATCH v5] enhance NUMA affinity heuristic >=20 > 21/04/2023 04:34, You, KaisenX: > > From: Thomas Monjalon > > > 13/04/2023 02:56, You, KaisenX: > > > > From: You, KaisenX > > > > > From: Thomas Monjalon > > > > > > > > > > > > I'm not comfortable with this patch. > > > > > > > > > > > > First, there is no comment in the code which helps to > > > > > > understand the > > > logic. > > > > > > Second, I'm afraid changing the value of the per-core variable > > > > > > _socket_id may have an impact on some applications. > > > > > > > > > > Hi Thomas, I'm sorry to bother you again, but we can't think of a > > > > better solution for now, would you please give me some suggestion, > > > > and > > > then I will modify it accordingly. > > > > > > You need to better explain the logic both in the commit message and > > > in code comments. > > > When it will be done, it will be easier to have a discussion with > > > other maintainers and community experts. > > > Thank you > > > > > Thank you for your reply, I'll explain my patch in more detail next. > > > > When a DPDK application is started on only one numa node, >=20 > What do you mean by started on only one node? When the dpdk application is started with the startup parameter "-l 40-59"= =20 (this range is on the same node as the system cpu processor).Only memory is= =20 allocated for this node when the process is initialized. >=20 > > memory is allocated for only one socket. > > When interrupt threads use memory, memory may not be found on the > > socket where the interrupt thread is currently located, >=20 > Why interrupt thread is on a different socket? The above only allocates memory on node1, but the interrupt thread is creat= ed on node0. Interrupt threads are created by rte_ctrl_thread_create() ,rte_ctrl_thread_= create()'=20 does NOT run on main lcore, it can run on any core except data plane cores.= =20 So interrupt thread can run on any core. > > and memory has to be reallocated on the hugepage, this operation can > > lead to performance degradation. > > > > So my modification is in the function malloc_get_numa_socket to make > > sure that the first socket with memory can be returned. > > > > If you can accept my explanation and modification, I will send the V6 > > version to improve the commit message and code comments. > > > > > > > Thank you for your reply. > > > > > First, about comments, I can submit a new patch to add comments > > > > > to help understand. > > > > > Second, if you do not change the value of the per-core variable_ > > > > > socket_ id, /lib/eal/common/malloc_heap.c > > > > > malloc_get_numa_socket(void) > > > > > { > > > > > const struct internal_config *conf =3D > eal_get_internal_configuration(); > > > > > unsigned int socket_id =3D rte_socket_id(); // The retu= rn value of > > > > > "rte_socket_id()" is 1 > > > > > unsigned int idx; > > > > > > > > > > if (socket_id !=3D (unsigned int)SOCKET_ID_ANY) > > > > > return socket_id; //so return here > > > > > > > > > > This will cause return here, This function returns the socket_id > > > > > of unallocated memory. > > > > > > > > > > If you have a better solution, I can modify it. >=20 >=20 >=20