From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFA384240B; Wed, 18 Jan 2023 11:19:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 59764400D6; Wed, 18 Jan 2023 11:19:49 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2074.outbound.protection.outlook.com [40.107.243.74]) by mails.dpdk.org (Postfix) with ESMTP id 304BB4003F for ; Wed, 18 Jan 2023 11:19:48 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cGcZHGDXzBSqI9UqHZ4FtuBRuK7Mzbnv2HhXImc6Svmd8vMlFgaM4wqwtLfYX3T2EXPMUISLyuMLwlf5nIoZaWlCSwnzY9LgBcqDzZSwWb/urLpSFYCvQ5rr55np1uV2/T1Y6ku+BJ/dW4YRqmX0EiR4yO70va/GnjLcDJKW+nJY4sEgMKVFdi1npqHynA2WxPSCssK0plnzBbRGHpsVPGN9i2FQ6AAEYbFzXBAAOWLo2O2f19CZu6ir91jJtjCJPB1bZmGGKLBfsR3jXd2kUiyS9fkKvJHS9LR5RlzuwyUy2BLCQd7NQwvnHnldfa/rlI1jEShdWESNWSY/Mt//cQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jBWz2gI9WZ9PqGVajRC2CnAyrkMiO7+DCGb6TO2hT14=; b=dVsNpnFRjNtk+2XCeiefoNv8hImqZHFsWH2dRE9FgUyjOXv+iG7VNbETau6mTxtG6/PlhzMgemJYK4ZJunz5gvIqu1gfwKQ2OkFNlTn2pBzITQ4CvYdA4s4LSq4dPbFJJMVmHQ11FeJhXCnudbpjlmRJrO8lvKbRMcYZVMoq4Vc8BtsCj5+QAhOSuqdTQ94rXcBOudwf5eOlhordkthOGRV8pXqR1GMMQt/47V5k1ea87XM+shwD4uBfGs5Lq3hZX3LYU8l4fE3LkjvKWx6AjoYEXsjpx+MvEz/bYhgrX4k45V6TNfllk8hn34RZo32zLFFSUVXUi9YqDDvKyr92Nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jBWz2gI9WZ9PqGVajRC2CnAyrkMiO7+DCGb6TO2hT14=; b=ZetdgATegZoCUALGE2wT2fRBUHZLVxc2RrwACTDdLRFXY9hbkyiFL+OnLncBKsP5xmExgMkcvIjdYnN8HapxFxqyW+VAaUtghu+uiOkWS2QLU+mtCQUG8HpjYxxumJiEHMmbu5nuj8xhX1FufJIuO1R5OLbbDhRk+qC+TYC0K3A= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by DM6PR12MB4435.namprd12.prod.outlook.com (2603:10b6:5:2a6::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.24; Wed, 18 Jan 2023 10:19:46 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::4807:1f44:5e04:e05a]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::4807:1f44:5e04:e05a%8]) with mapi id 15.20.6002.024; Wed, 18 Jan 2023 10:19:46 +0000 Message-ID: <1af1292d-a03f-7467-c927-6c7f4ad0fc5c@amd.com> Date: Wed, 18 Jan 2023 10:19:40 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [PATCH] net/af_xdp: parse numa node id from sysfs To: "Du, Frank" , "Loftus, Ciara" Cc: "dev@dpdk.org" References: <20221212004857.2004805-1-frank.du@intel.com> <1cfc600f-32c0-e839-87af-3587f9255ce5@amd.com> Content-Language: en-US From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0105.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:191::20) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|DM6PR12MB4435:EE_ X-MS-Office365-Filtering-Correlation-Id: e7eb8408-0ddf-4317-fdd4-08daf93d85df X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YjS4GnVph9o3n0aH1Qal6TwpqS8nhnwXz7UGg5USTJQy7gT9BWFiJKSL5dIbesbCqT+Vu3Pi0jJ2HQopmoWLeibpQjL0HN9FkMwJZ4mcGSOynI/6S8eeBnxJicTY2AIU+50c7X8VxpMl5BMbCVv+1uZZKPUC49S4S+pjwMyUMaLOKl1JetB8UCkwdF/tgUMC/CGLmTskfB5DPgS5ovyYx0hFUMnc7tJDdOgbW1CcQ0MS+L/9zwX+XicdsaR7afT6ReXJ7C9QP6fhYuYVAVG7HYrY/xQxsHwOrn9bFBNxWJwHXbD0LmoSGqTFNjPey1zA5dl4NaL9BtHvStyCblROcYRBusaQot0LciUGO+FPyQ1cMVdIQ2/AZumQ4BrOYnspX/cVTHrt1kClaXt+vAEu/iZhcF+PP1bxvlf3y78DMaiU/umvf64p1ZcfZ6BzK0ZkegPH3fctTOJhtOATQh+UtHIu7LRfv7QG+StTpSNbGerjFxbN9nadjUDWR0rmMf1cOcOlBGANmcwe5qT5hwJMXdnglN1UZtk1mHZ/B4sPypo1m81HYOinQas2QPGFgeG321D2P2wAoR0PrHlz5cuxXojQRFEFvV5dziwe2CKxaNXcEn93Lf/giZbzJpSeONPufA+B1CTDpKtdv8rTyllgI/IsYQ8CZ2KaJrSmUxylyuXHpQz9eGykrBi/HcDkqdsjCWLRo71VddDQ1KYPQLLyXmcEIpVUwQcA0PsJrvxM6RU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(346002)(396003)(376002)(39860400002)(136003)(451199015)(38100700002)(31696002)(86362001)(8936002)(5660300002)(2906002)(44832011)(66946007)(66556008)(66476007)(4326008)(8676002)(41300700001)(2616005)(26005)(186003)(83380400001)(6512007)(53546011)(316002)(110136005)(6666004)(6506007)(6486002)(478600001)(36756003)(31686004)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V21PUzdqUXk4c3RlNm5zcU1xaWl6clNWOVhxcFlSdDlHVkF6L1lWQjFRTjhO?= =?utf-8?B?KyszRUZxb0FjTTY3WFJiRTc4eWlsS2ZWVWk1cDNNOXdEWEhBbmdZd2hPemZO?= =?utf-8?B?UU9jalhIbGNpTy9UY3F4Z05jRU9vU0lRZ21uRzlnUHhLa0U0WmZGL2ZCS2N5?= =?utf-8?B?dFozSTRaQVJhSXJZRUhudEJjdlZWbDlTaEFQK3BKcnFmSXJja3dJYVdYNlVx?= =?utf-8?B?UXFPYXptTUdpenVBR3dnMURObmdJZmFsLzBLSmtnbzVzejd2cHNXMndLU1dK?= =?utf-8?B?RTZWdHdaZnVXY05SN25JNFRrcW4xQ09rZ2JlTG1weXJOU2hmR1pGQVBHREFX?= =?utf-8?B?Q1V2Z25waXFHNnc5SWlnZ2ZrWm5veXM5SU5VTkFJUzc1LzdyckVLMStaZXhx?= =?utf-8?B?YVVicHJxSkNoclg3R0VGZkE4aGoxTlBuRVlJNVQrK2ZPUXp5d3JCOTVnUWIr?= =?utf-8?B?Y0wrMC8vRUhScFZiSGxjZlEreXVzaW4yZndLL245d0pEKzY3WlV2U2tnbkRM?= =?utf-8?B?UXN5aHYyOXdxYUxXbWxIWkVYSTgxNmkwdm45dnVXN3lpV1g1djVtd0svdVpY?= =?utf-8?B?THJwVHNveVlORXA5Y002SjZ2MnNJM0t0alg1SSt5cXVYbDMzN1dQeFZ3ZFR0?= =?utf-8?B?aXNmM1NPbVhxellpMHBwWGFUVU9qaGNrSnRjMzhabUVyNWpCcTVOcDM3emts?= =?utf-8?B?QUdSMFR3R0M3SFFscnBqOFJjV1ptNGREeU5mNXVrNktjdnpwSnUzRjZUcFU3?= =?utf-8?B?NkpSSDhoTmdWWjYrTUpSc1RMdzgzOVZPbVZNQVJMcm8rQ3Jkd2V6L1lmbFJ5?= =?utf-8?B?aXFpWmxpaXdGazJTVmtLWDBqYW9pdkhhcExJV3E4Y3E2WEhyMFNoYlFXWmlY?= =?utf-8?B?VkJJVjQ0ODhXbUlPY3NTaDRzTzdMbHRmYXV2YmNTWXA0aFVSVm5vQmxlazFp?= =?utf-8?B?UnpGTkU5T0VtUzJJOG1PWWpldVAwUWprdDJ1dmYrQ2V0ZGxIbGw0RXA4M2dW?= =?utf-8?B?eEJYNnc1bTZOdTlDcDRMRitrc2VWOHdsWFRRMU5YbXBJYWQreGdVUDV5Q3d4?= =?utf-8?B?MFFLb0FVOGJFWFF6cTRUZVJZTXg4L1FhWVIza3l2TkwySVJFSHhxNnJGLzll?= =?utf-8?B?cFVQelVUSFV3MllDUXk1N1lIb1RiNFJpa0dZU0lVWlVkbkkrbmJCL3VnVFJr?= =?utf-8?B?S0RPeHRQbzVjZUF1dDJQL05SWWc5SHV6NENMYkt6VjMwNDdCRE9vRFM1NVBl?= =?utf-8?B?SUVjODdhdnhTTzAwV1lieGRwNWtRNkNsbnZmK3RSUmdMY05UWVR6V1RJRSsx?= =?utf-8?B?Y3RwbnZSalNrYksrdDMrd0ZvNFVMTkdWb3BtOXZ1SlpkYzMrcld4Z3hFbkxP?= =?utf-8?B?WjVsNTdPYmdHK1A3N0k2MHBKZEZFcnhmMi8yYUk2WjE1NDV4aHpLN2JmUGkz?= =?utf-8?B?Sk1OUGhSb29waEhieGQ1MmV5RkpqV0VZUlJpcHdWZjNVU2V2T2t1R1p5L0Nj?= =?utf-8?B?eHZTSjE5cHRQMm5JZHA5T2NjazdQK2d2WW5zbEtUYVhaaFViSEJDUHZHZGVM?= =?utf-8?B?TjNoVUNCZE8xZFRvTzRsZ0Q1N0JQaWQ2NmNaOEUzUi9jdUVjNzdHSzBleDhq?= =?utf-8?B?bXRXNG0vM2REakswMlVNcFZvY3Y3RWlDb053NnJ6K2pydFdXTXpuQS9SSVMy?= =?utf-8?B?Nm00WTA0dWRFUjcrRGozcU93amdrMlV5ZElNUlB6Wi9EUXBINWYxbXRVWEJF?= =?utf-8?B?amkyOThrbmt0M1FPMzg3SGcyd2pZT1NPb0hLRUo3K0NxTTkyeHB1bnNaM21V?= =?utf-8?B?TGRuaHNhTzU2RVM3TzdaVGZnNWVCTWwrVmQ3MEpBOWZCb2lIR25JcWw0cUFM?= =?utf-8?B?TlZTSFpFUTY0Umh3RHc2ZW5hMG1UVW0xT3RPT2hWb0hTMERxcFRxNUZ1a21x?= =?utf-8?B?YVRON3RCWmNpVVdVSWFEc21DWklJdHhPektQT2xXOC9WejZTV25oZWp0NElH?= =?utf-8?B?MzB3SjBHM0psSFlnUVEyczQ2Y3dRTW5pU3luSkFEenI3ODBVaGE5S1RsdmY1?= =?utf-8?B?ZVp6c2ZHbVZ1djdKOWdCMWRmSVVGNGFFazBNd0I1TXhVTi8yV2lnV1prT1pS?= =?utf-8?Q?EXQzE+EZw2skQmvzkPvb2hvun?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7eb8408-0ddf-4317-fdd4-08daf93d85df X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2023 10:19:46.3216 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IRU5xUTY/FlzBsEw7y4pe/l5AgjRmTg2iBkpDq54JLq9kpO96tHBKrT2i1GF1nVi X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4435 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 1/18/2023 1:53 AM, Du, Frank wrote: > >> -----Original Message----- >> From: Ferruh Yigit >> Sent: Tuesday, January 17, 2023 5:14 PM >> To: Du, Frank ; Loftus, Ciara >> Cc: dev@dpdk.org >> Subject: Re: [PATCH] net/af_xdp: parse numa node id from sysfs >> >> On 1/17/2023 1:35 AM, Du, Frank wrote: >> >> Moved down, please don't top post. >> >>>> >>>> -----Original Message----- >>>> From: Ferruh Yigit, >>>> Sent: Monday, January 16, 2023 9:15 PM >>>> To: Du, Frank ; Loftus, Ciara >>>> >>>> Cc: dev@dpdk.org >>>> Subject: Re: [PATCH] net/af_xdp: parse numa node id from sysfs >>>> >>>> On 12/12/2022 12:48 AM, Frank Du wrote: >>>>> Get from /sys/class/net/{if}/device/numa_node. >>>>> >>>>> Signed-off-by: Frank Du >>>>> --- >>>>> drivers/net/af_xdp/rte_eth_af_xdp.c | 17 ++++++++++++++--- >>>>> 1 file changed, 14 insertions(+), 3 deletions(-) >>>>> >>>>> diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c >>>>> b/drivers/net/af_xdp/rte_eth_af_xdp.c >>>>> index b6ec9bf490..38b9d36ab5 100644 >>>>> --- a/drivers/net/af_xdp/rte_eth_af_xdp.c >>>>> +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c >>>>> @@ -39,6 +39,7 @@ >>>>> #include >>>>> >>>>> #include "compat.h" >>>>> +#include "eal_filesystem.h" >>>>> >>>>> #ifndef SO_PREFER_BUSY_POLL >>>>> #define SO_PREFER_BUSY_POLL 69 >>>>> @@ -2038,9 +2039,6 @@ rte_pmd_af_xdp_probe(struct rte_vdev_device >> *dev) >>>>> return -EINVAL; >>>>> } >>>>> >>>>> - if (dev->device.numa_node == SOCKET_ID_ANY) >>>>> - dev->device.numa_node = rte_socket_id(); >>>>> - >>>>> if (parse_parameters(kvlist, if_name, &xsk_start_queue_idx, >>>>> &xsk_queue_cnt, &shared_umem, prog_path, >>>>> &busy_budget, &force_copy) < 0) { @@ -2053,6 >> +2051,19 @@ >>>>> rte_pmd_af_xdp_probe(struct rte_vdev_device *dev) >>>>> return -EINVAL; >>>>> } >>>>> >>>>> + /* get numa node id from net sysfs */ >>>>> + if (dev->device.numa_node == SOCKET_ID_ANY) { >>>>> + unsigned long numa = 0; >>>>> + char numa_path[PATH_MAX]; >>>>> + >>>>> + snprintf(numa_path, sizeof(numa_path), >> "/sys/class/net/%s/device/numa_node", >>>>> + if_name); >>>>> + if (eal_parse_sysfs_value(numa_path, &numa) != 0) >>>>> + dev->device.numa_node = rte_socket_id(); >>>>> + else >>>>> + dev->device.numa_node = numa; >>>>> + } >>>>> + >>>>> busy_budget = busy_budget == -1 ? ETH_AF_XDP_DFLT_BUSY_BUDGET : >>>>> busy_budget; >>>>> >>>> >>>> Hi Frank, >>>> >>>> It looks reasonable to set virtual DPDK af_xdp device socket to actual >> underlying device socket. And as I checked quickly, it works as expected. >>>> >>>> But what is the impact and motivation of the patch? In other words why you >> are doing this patch and what output you are expecting as a result? >>>> Did you able to do any performance testing, and are you observing any >> difference before and after this test? >>>> >>> >>> Hi ferruh, >>> >>> Our application use rte_eth_dev_socket_id to query the socket that a NIC port >> connected, then allocate lcore/memory according to this affinity. >>> >>> The remote memory access is really slow compared to local. >>> >> >> As you observing any performance gain after change? If so, how much? > > Hi Ferruh, > > The NIC in our setup is on the socket 1, if our workload(both memory and cpu) are running on socket 0, it can get max 12g/s throughput on a single core. With this patch, the workload is running on the correct socket 1 cpus, it can get up to 16g/s on a single core. Thanks for clarification.