From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD32F429C3; Sun, 23 Apr 2023 10:57:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A975D40A80; Sun, 23 Apr 2023 10:57:51 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 748E940A80; Sun, 23 Apr 2023 10:57:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682240269; x=1713776269; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=PQzA7nRcCIrFspYgqT+cdorU4LVkJvYasMzrlABDl4Q=; b=nNYrU3xgoFrfpsbyvDlaM/SaCx9pQ13XXhVHUTf1RAnJMk0qcXzGFDVY pyRXIdXOSdHh0CevSjEIdrkMi/FwKsbPI6VvfbbPiPsfbAnPVoXsqYcOJ IZK87Yl/vd4hVS/vwfw2Q2wKi5V+/3+pL87TA1uOuu3x9iVf285eAyEJm /eYpfBbgwePFO28L90h6i71yd2rwN3aGlq3ftFPnjpNr2BN2mshRlNEhO 6DeA8LGK4EU+JRaJs8J6JgWkGJokFiSx4PK7nCKTqykqwrV0JzriOYfvy r1gH4dW5I9pHU0G011wIpNHJ3O9RcLpXQn4/hCrMEd6xbV1Ga73U5swKp A==; X-IronPort-AV: E=McAfee;i="6600,9927,10688"; a="330471830" X-IronPort-AV: E=Sophos;i="5.99,220,1677571200"; d="scan'208";a="330471830" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Apr 2023 01:57:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10688"; a="867146405" X-IronPort-AV: E=Sophos;i="5.99,220,1677571200"; d="scan'208";a="867146405" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga005.jf.intel.com with ESMTP; 23 Apr 2023 01:57:48 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Sun, 23 Apr 2023 01:57:47 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Sun, 23 Apr 2023 01:57:47 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.168) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Sun, 23 Apr 2023 01:57:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=a8M5H2u399oyIn6AZnpOvs7C+aJ+yuc2QDA8TWNG912F+AOh+iAusOCgth1NpQhxdiXxrf+1Tg2eJn+wmg6GGxjwWn5HXNYqhHvqS0j785uteFQ+Lht4WhhJlVrm5c8FkMD+GvWg9HwIisyzi4SniI0cLuhqvvUiLpiuG3AqAPJyk0fXSXr7HcXtdkIvAc5XSmA+Xmu5Cp7fH86o9om/wPFzcVkjxA6qJyHAmyugINEBdz4/91gf3LYarDW7RQEL9oBxEy+9tR8eaZQFmj65RNtNj4TdcCyAZw7GalTt8AENdZSrrrPpbTP0nCAzvGVZbuQ8COMoxlel/mpEW6bNKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6n30wzAFokbplpp54+jJnwYPNgimbiX5W6keZ3em2sY=; b=iIYxYLWcUGceGy5DNNcM+/BLfaOw3yMA7pwFd3LLKNDqDgxmDm06Nzo0smtja23WuFaI6TMhOpi5bI4+gMD+UzFYHXvT4arGWOa/YQlqEPrKOypLGEIBg8b5tjIPSFYtZNoT4pVC7oD8ZuX1/pe2DIC8Nq46PvLwE9XWE2J2d08E3lNfFFMHFRJWO/7cB3SKXJFH+ALw6lSgAeSAcQ6srl5POwl6OCi2MHsO+vNXMWdXTww44X51cMfVDqKeXl1YzCHVpsgtGKnj/MnHDXXwMfI13dIXv1c9RpkA8+1NqOH24L475Nf8l4Ss5E6fN6qlYiviwdr9QSdXK6ELDnhYyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SJ0PR11MB6765.namprd11.prod.outlook.com (2603:10b6:a03:47b::10) by PH7PR11MB6955.namprd11.prod.outlook.com (2603:10b6:510:206::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.32; Sun, 23 Apr 2023 08:57:44 +0000 Received: from SJ0PR11MB6765.namprd11.prod.outlook.com ([fe80::d8de:a1e1:f598:6572]) by SJ0PR11MB6765.namprd11.prod.outlook.com ([fe80::d8de:a1e1:f598:6572%4]) with mapi id 15.20.6319.029; Sun, 23 Apr 2023 08:57:44 +0000 From: "You, KaisenX" To: "You, KaisenX" , Thomas Monjalon CC: "dev@dpdk.org" , "Zhou, YidingX" , "david.marchand@redhat.com" , "Matz, Olivier" , "ferruh.yigit@amd.com" , "zhoumin@loongson.cn" , "stable@dpdk.org" , "Richardson, Bruce" , "jerinj@marvell.com" , "Burakov, Anatoly" Subject: RE: [PATCH v5] enhance NUMA affinity heuristic Thread-Topic: [PATCH v5] enhance NUMA affinity heuristic Thread-Index: AQHZNf9KkwjFirxtKkqXm6ZbKOABl67QJeuAgADH7FCAGFkTAIAInSJQgDb0DOCACi2GgIACffvQgABijICAAwQmcIAAKWBg Date: Sun, 23 Apr 2023 08:57:44 +0000 Message-ID: References: <20221221104858.296530-1-david.marchand@redhat.com> <3549930.R56niFO833@thomas> <3535353.LM0AJKV5NW@thomas> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR11MB6765:EE_|PH7PR11MB6955:EE_ x-ms-office365-filtering-correlation-id: 607f71d9-7f63-4515-10c2-08db43d8cd8b x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Te/JYZMQtTR04vcxg+fXVJq4sTOIL+yX+p5y35n6i05W2Fk6j7pBeSrJ4hHyWLwXOk0dQab6Io2q00oomI93Xz6mBcCe6g8v81vk98ZpfCK/beRe5crEsLkkYhtkmi3ikltbaZ25NKVMv8uwxbwCVN1oBkur2+Ob6bPpq1K3okhltgKll+OC6YOu/eMJTB2fLk0ci5BqHPCyWHlNMBqHSIqBZ1OtyIHEvYIQyK/HTzo2P2sQlfrWigP7QsV8ioJc3u4GxehxMEqu4pJZHtcQzkx5AjIwJ1Y78CK30KmfgolEBdVNgfcgoA0xo2uz/hhFZTfXtVtzMVJ7XrRnDdJoL4QhFuF2zSdeuGbuEql4um7+LUERewI3/NtU9xVlasGm2HhD0yMYerBNvbIna5FFzj/dj69RjQTOGpLM7xu7VfW9kVWu21hEm0e/nW759wQYHonr3uzhY+cWY0UemlwqUv5hoCXR8YmHvMdiRa5E9raPm7r+4Sw3KpdRzFdtkeDugcpjYl9udHwuY0gCW9ompASm2gMsCJvlpw+mY5e8HkSVMN2Va7CCx5Y72QkpVy8mGDBf66bntpYQYaN8nCW5y1WEu1t2RLokwcEI/sMx/5EVJV8kc9gcHkUghg4IKE04 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB6765.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(346002)(366004)(136003)(39860400002)(376002)(396003)(451199021)(82960400001)(76116006)(7696005)(64756008)(66446008)(66556008)(66476007)(66946007)(71200400001)(2906002)(6506007)(53546011)(8936002)(55016003)(83380400001)(186003)(2940100002)(8676002)(41300700001)(26005)(9686003)(5660300002)(316002)(33656002)(52536014)(107886003)(38070700005)(86362001)(110136005)(54906003)(38100700002)(122000001)(4326008)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-2022-jp?B?YWxHUGphTmRtR1ltYzVUL0RNblZBd2NNUExScFdwSVI1NGdxWjk2dmRV?= =?iso-2022-jp?B?YTZZaHRRV01VR0wvUkhtVHdsdTJ4dmJzeWFES0lxL0RsS3hCYkdoR1d5?= =?iso-2022-jp?B?NE9SaXFQZzFLZHVVTlZwR3RuVzRWeXVXdVhlMXpNMk9yTmllbjR3OVBJ?= =?iso-2022-jp?B?K1pXMUF0Nk44eUprKzVJVHZTa3FWSjQydkZaUHl5Z201b2I5T0kyMFY3?= =?iso-2022-jp?B?NDVXUU9xU3A1N1lpN3NwaGV0TTRyTmdYU09iTFR1N3NkQ0VLWW5zWDA5?= =?iso-2022-jp?B?cmM4T0tEMm5UQjJRVUN5T2dmSHZZU3JKWGtkUitQVW1mL2dhc05PaHNK?= =?iso-2022-jp?B?RVkvcjYwZFlRbXEzS052T0lBUDFRY3d5T3Nma1ZHUVc3bTZlN1BXdjdm?= =?iso-2022-jp?B?NU5ldFhDalBWUTZyWXZXRVhSQkxtZFZRQkgrdXA4cjA4YXM5UzlwOGxM?= =?iso-2022-jp?B?RjlacTk4SjFvdVN1NUlmRW04QVd0NG5YOFJwSHBzODdoUSttbFdrNkhE?= =?iso-2022-jp?B?SGtKdElTbVB2MHU4NmlFeTFXSDlyVHJaZU4zNVIxd3Z0Q3JXclliMnBO?= =?iso-2022-jp?B?QnFDckJyYVM1YkM5VHVlbkZ5V0prenVMQUswM0ZkMVZDZW0vUmh4SFJF?= =?iso-2022-jp?B?WHlCYjBWWFBlUm5NWFBoV0lnTG9wTzNpT29hWC9ydEFvNFRHZG4wdDM1?= =?iso-2022-jp?B?ckY2VHNWSmIzRzdsSG43a3I3QmFjQy9VVnIwVVQ0bkpaS3daQ1ZJQm8y?= =?iso-2022-jp?B?QWxYMWxTVGZVSC9ESnJhU3N3TUcyZVZiT3pIUy9KRGxxTFA3ZHRTNS9q?= =?iso-2022-jp?B?c04rVUZ3ZmlFOCtvVXVhN3RSZ3RpNUt0dlFWdTF4aGFuZjNaWi9YZTNm?= =?iso-2022-jp?B?TTh5ZWJjNXU2aVNZU2E4ck5VWURNcEoyOXczSjlQVnozSHNscDdwd0JH?= =?iso-2022-jp?B?eEE0NWhwQU1yN3Q1SzJpOEZQM2pyeExPbE84OHhTQWJmeGp4aHV3Nm95?= =?iso-2022-jp?B?U3pJQzlPdFhMMW82aFNRTlRzQnNocDZCWXMxQk5PTFEyUjBTU1dXZFEw?= =?iso-2022-jp?B?UDZTenB4MUUyWDVwR1hoMUYvWUlVak0xRUhyNldLbjJmbXQzVXhFSnRY?= =?iso-2022-jp?B?NGZ1MER5d1ZGaSt6VStLdWRlTmJ3YUV2aTJzSEN4dHRwOVVhZVdRWFh1?= =?iso-2022-jp?B?TzQvaTE4c21Cb2Z3WjlqblZaU3U3WGNTMHJyd0o4TkRZWjVROHIrei9j?= =?iso-2022-jp?B?ZjJLaEpHaHdUYVpLTVlBemc2R1hzWEtZYXZmNERWbU9kRm9HMm1URjBG?= =?iso-2022-jp?B?aVJjTlI0M3p4RVBPOFF3SVBlbTRHR3l6TDNTTzlPcVB5YUhIMkhOUHJ0?= =?iso-2022-jp?B?bkRxOExLaUZ1cFcyVDlRS3psc3RZYkxnUkFHdmxpNEJlT2Exdi9PT28v?= =?iso-2022-jp?B?K2R1cVNQMzljalB5aVhVRHN3bmxUOU9RempSN0hvRko4NVVyOWpzZGx1?= =?iso-2022-jp?B?c1U4MlhRTWVDTVFrWTR2b0xiR2R3SnRyOXBMd1NGQ3haWkVXTVlGRG9P?= =?iso-2022-jp?B?NCtPb3dtd2x0NHNHaFFOcDBCY0gyOG0rTmJDV0NhTmlkZ05DbTU4TU8r?= =?iso-2022-jp?B?RkxzdXFqVzRxcmhRUTlkRGczOHc2NlZJZ0xnb1hCNnpab0l5c1ZBM0hw?= =?iso-2022-jp?B?dlY1TWhOakNjTXBLdllEQnJNU3B4ZE5COHl6a0U5bXhjMGNNZ2F6OXpV?= =?iso-2022-jp?B?UjZnaUgvUjhEYU00Nk85L3ZyaUhzajBQZFVhU0tJZll0MXJqT0ljZUZE?= =?iso-2022-jp?B?dGcyMFp4U20rRFlDSVdncUtVeVQvTk4yNHp6ZHVDNGZZcklSR0YzZHkr?= =?iso-2022-jp?B?eEF0TW5NbW5xdTVwTlRmYlhYdGF0aHR6b25tRWZuNWFPOTgvU0orQWJH?= =?iso-2022-jp?B?emE4WGFrVS9VN0o3amM5T0VEcThsT1N4OGtmaUZGSEM4T1QyWWk1Q09L?= =?iso-2022-jp?B?cUR1NG4xSFA4K0x6NG9lMlp1SDRQYzNNRWQvTXIrcHdXSGwyTFhRcWU1?= =?iso-2022-jp?B?b094VVBsdzVycnAwT0hFYm5PeGM5UUl5SWdBbTVPQ1NjY1lWYnZiSVRR?= =?iso-2022-jp?B?cjJTTFBMUzVINitqRFcvQmpGOWY1YnRGT05FajZmU1l6Yk1FOGpoTFE0?= =?iso-2022-jp?B?WGNzWWJtQ3QzcTZ3NjZhY3VacVRFdTIyRGhFOEpTL2ZhcnFkVXVwYkpp?= =?iso-2022-jp?B?dWo0MDdkTTFoSkFGNnBVcG1yc2ZpVkJnbmJPeWhmOHVnZ2tOWlZFZW81?= =?iso-2022-jp?B?eGxpZg==?= Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB6765.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 607f71d9-7f63-4515-10c2-08db43d8cd8b X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Apr 2023 08:57:44.3155 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8q0lZ9njLgeA9xsNFJLCPTqhZ0s6DEsNy5jJ6nsl6Ppb7FVGF2HV1R0rw7+2sPVKVp7dF2P+4nsn9KVBavfjKg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6955 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: You, KaisenX > Sent: 2023=1B$BG/=1B(B4=1B$B7n=1B(B23=1B$BF|=1B(B 14:52 > To: Thomas Monjalon > Cc: dev@dpdk.org; Zhou, YidingX ; > david.marchand@redhat.com; Matz, Olivier ; > ferruh.yigit@amd.com; zhoumin@loongson.cn; stable@dpdk.org; > Richardson, Bruce ; jerinj@marvell.com; > Burakov, Anatoly > Subject: RE: [PATCH v5] enhance NUMA affinity heuristic >=20 >=20 >=20 > > -----Original Message----- > > From: Thomas Monjalon > > Sent: 2023=1B$BG/=1B(B4=1B$B7n=1B(B21=1B$BF|=1B(B 16:13 > > To: You, KaisenX > > Cc: dev@dpdk.org; Zhou, YidingX ; > > david.marchand@redhat.com; Matz, Olivier ; > > ferruh.yigit@amd.com; zhoumin@loongson.cn; stable@dpdk.org; > > Richardson, Bruce ; jerinj@marvell.com; > > Burakov, Anatoly > > Subject: Re: [PATCH v5] enhance NUMA affinity heuristic > > > > 21/04/2023 04:34, You, KaisenX: > > > From: Thomas Monjalon > > > > 13/04/2023 02:56, You, KaisenX: > > > > > From: You, KaisenX > > > > > > From: Thomas Monjalon > > > > > > > > > > > > > > I'm not comfortable with this patch. > > > > > > > > > > > > > > First, there is no comment in the code which helps to > > > > > > > understand the > > > > logic. > > > > > > > Second, I'm afraid changing the value of the per-core > > > > > > > variable _socket_id may have an impact on some applications. > > > > > > > > > > > > Hi Thomas, I'm sorry to bother you again, but we can't think of > > > > > a better solution for now, would you please give me some > > > > > suggestion, and > > > > then I will modify it accordingly. > > > > > > > > You need to better explain the logic both in the commit message > > > > and in code comments. > > > > When it will be done, it will be easier to have a discussion with > > > > other maintainers and community experts. > > > > Thank you > > > > > > > Thank you for your reply, I'll explain my patch in more detail next. > > > > > > When a DPDK application is started on only one numa node, > > > > What do you mean by started on only one node? > When the dpdk application is started with the startup parameter "-l 40-59= " > (this range is on the same node as the system cpu processor).Only memory = is > allocated for this node when the process is initialized. > > > > > memory is allocated for only one socket. > > > When interrupt threads use memory, memory may not be found on the > > > socket where the interrupt thread is currently located, > > > > Why interrupt thread is on a different socket? > The above only allocates memory on node1, but the interrupt thread is > created on node0. > Interrupt threads are created by > rte_ctrl_thread_create() ,rte_ctrl_thread_create()' > does NOT run on main lcore, it can run on any core except data plane core= s. > So interrupt thread can run on any core. To avoid ambiguity, I'll explain my commet again. Interrupt threads are cre= ated by rte_ctrl_thread_create() , rte_ctrl_thread_create() doesn't control which n= ode the=20 thread it creates runs on, interrupt threads can run on any core except dat= a plane=20 cores. So interrupt thread can run on any core. > > > and memory has to be reallocated on the hugepage, this operation can > > > lead to performance degradation. > > > > > > So my modification is in the function malloc_get_numa_socket to make > > > sure that the first socket with memory can be returned. > > > > > > If you can accept my explanation and modification, I will send the > > > V6 version to improve the commit message and code comments. > > > > > > > > > Thank you for your reply. > > > > > > First, about comments, I can submit a new patch to add > > > > > > comments to help understand. > > > > > > Second, if you do not change the value of the per-core > > > > > > variable_ socket_ id, /lib/eal/common/malloc_heap.c > > > > > > malloc_get_numa_socket(void) > > > > > > { > > > > > > const struct internal_config *conf =3D > > eal_get_internal_configuration(); > > > > > > unsigned int socket_id =3D rte_socket_id(); // The re= turn value of > > > > > > "rte_socket_id()" is 1 > > > > > > unsigned int idx; > > > > > > > > > > > > if (socket_id !=3D (unsigned int)SOCKET_ID_ANY) > > > > > > return socket_id; //so return here > > > > > > > > > > > > This will cause return here, This function returns the > > > > > > socket_id of unallocated memory. > > > > > > > > > > > > If you have a better solution, I can modify it. > > > > > >