From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C336C43F31; Fri, 3 May 2024 17:27:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 84901402D1; Fri, 3 May 2024 17:27:51 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) by mails.dpdk.org (Postfix) with ESMTP id 1E9F6402D0 for ; Fri, 3 May 2024 17:27:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1714750069; x=1746286069; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=L3BUyyUzFFZggYVOQ76658uPrtoDPeXrr41sxLmBjZ4=; b=ktqQHyJo+/xPXHwgKDGsiI6sr7yNz5ellygRZ1zkoYYhqD3USdS/gPAz aD0/dSPRiJPqWXxj+aS5x3GdKOJOVhfCTtQ3cgEXqd7t6HZ6mq19MMIoX Hw3Os6K7Ad7BXut1OUZlycE1zSKsfuUnOKNj/WQls9CMG31JynKBq7HUY UrybW61sdJje6DmEpQpnTw8aEt3OcCCEkMaFi+teBmAr4sn94sRXKCmfl WZGYHMlVzHUBn682r5uRsAntgZdR/Tno6sxd52UNaoNq7+1oBjzrfDCgR wzz0ucKqiShnLVzHLXXEzGfjd1tjd12BzNWGDV2q3HcQw2955bOvkDZEO Q==; X-CSE-ConnectionGUID: KglrAmKOQk6PF/dpapY8Mg== X-CSE-MsgGUID: OAF0k61dRHOHt0BKytyjnA== X-IronPort-AV: E=McAfee;i="6600,9927,11063"; a="28091302" X-IronPort-AV: E=Sophos;i="6.07,251,1708416000"; d="scan'208";a="28091302" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 May 2024 08:27:48 -0700 X-CSE-ConnectionGUID: /C+OkOa0QmWAku39bXzv1w== X-CSE-MsgGUID: 23r+2n3mR+y+3W0ADyaDgw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,251,1708416000"; d="scan'208";a="50667965" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmviesa002.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 03 May 2024 08:27:48 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 3 May 2024 08:27:47 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 3 May 2024 08:27:47 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Fri, 3 May 2024 08:27:47 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.101) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Fri, 3 May 2024 08:27:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WsXoYF6RcmtMPczNs4rhPzrPzMUgqQ9idXi+3GNPdMYaHSb1MZ+R35dgzn5ZtEffWEFKAp2aH9uWI1ODmfkipNN4zrnGxIpw+jYKxYIlSWZeJkbZ29HdZ3ZC87tX5UqXfMQVlRo8zdza+GNU7r0yj9FuXVvNvdE1bzq4BqbS3yWAUGM0ZWrxhr9IJmuoGXx84XU4SFUcE2v0TlRJ+fX1dIUl/vL018H0vssCO0LvW3kVJ712+rMyi8PmNdHPssDiEx+i1atl0B8BGPWjSpeU1nws3pV5CmvpadTClnkwZzHXwrAbXibNG13b9lFs/kQ792odW7xSUkAWlz2gnUPa+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dDka4CQSWhD+VIlGbhNul+FqRtcFwE5ZoJymAvjRRXI=; b=BN4HU2bcCcDcuXfB3b0HALv67CgXyVhUVG1/CHm1KAZCLLDeQuSQOwV+Tj19NcE+oLz1INlB1FbuETt8XYO0eSWG0Hp6av+Z4uzvnJ1HrsDkYc7Mi+2TMh3X5m6BFlqXYf0QjKKIUkkJYVsA4X9o+Cml/qBZ8gIWDdwhTe+OWXS1xiTvicye9shydoeEikcPZtdxIwclh5Xu54NqjXFJbSjuKQtC9U2J8qTkiVBpEpnmagr0MN5q/+Sd8ADrT1azgKQvKvmegqSvLrI4dLTWNJcaTejPx1W9J7sQYbp/MREY4BLT2Nyc4zVujB/c9/kATuUgIDgrB0phrgkBxBvgmA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by PH7PR11MB6353.namprd11.prod.outlook.com (2603:10b6:510:1ff::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.28; Fri, 3 May 2024 15:27:43 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::487e:e20c:ad88:9c0f]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::487e:e20c:ad88:9c0f%7]) with mapi id 15.20.7544.023; Fri, 3 May 2024 15:27:43 +0000 Date: Fri, 3 May 2024 16:27:39 +0100 From: Bruce Richardson To: "Lombardo, Ed" CC: Dmitry Kozlyuk , "dev@dpdk.org" , Subject: Re: Need help with reducing VIRT memory Message-ID: References: <20240502230352.7bb1ef0d@sovereign> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: DUZPR01CA0064.eurprd01.prod.exchangelabs.com (2603:10a6:10:3c2::17) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|PH7PR11MB6353:EE_ X-MS-Office365-Filtering-Correlation-Id: c6b77ea8-c48f-4232-79b9-08dc6b8593b1 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|366007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?wQOspotmdQOeEpU1EEiyhSy3dRlQPGyNuYksnYylZeYm1K5dfDdZb66QXLBI?= =?us-ascii?Q?M5kEi33S5bykzugYX0sWnJ1IJumNQCyQuVM+lQln7zB/ipEpOw4fTmRf6/zx?= =?us-ascii?Q?mi8GpIKdSalLZbq40gMJQGf96KV8/UM1fjUv7assOhoET3Y8O78IgNw7kLiC?= =?us-ascii?Q?CabO+27FBXzav8IB0pyaPWjtPBynP9Uf74nWlVcXwyloUI2SYbmye+3FFEqF?= =?us-ascii?Q?qnA/fdHCYGujQWj706FgzPDprxRQONO8TrpKd7BiX7Ocu9Kd1+PB8b3DHg5M?= =?us-ascii?Q?JJCpz5p4m5yppmBq5eTG3PKDyXW7upEVjTmHSfSpQOlM+LTt2LQ/cjX65Gsw?= =?us-ascii?Q?aMB0YgJIfatb2IDe1WW9kwPEnwKtOwDEzfJisPkqFtlVv2XRa1s7ByZuwZbg?= =?us-ascii?Q?xd5h6hKCTnXB1ev1ahGYRRz+GY9f0ry2mlLWLZTCx5wjipTl8T1Iy0xeSY9L?= =?us-ascii?Q?wK2PE8n3nMyX3PRAjbhL0LF1mWsD2Ub8vfTFj6CYokArs01UnbKGfqLYq5q0?= =?us-ascii?Q?Y2xa3AicjimgOv6aZDlchH7lV6ju8A31vVmAVVoiEOj0Nb+ewjBbySv6o38n?= =?us-ascii?Q?+QLAWaEbCHTBdWJSdRZSzMTnCdEq1Bbn8TbUiRb0YWkOf0qC0ReYrKv3s4re?= =?us-ascii?Q?iyG/ItOqim+uE4W7rGzCNvbE9pBDI24+ECy51nMlzpZ18AyexNfuXCrNTm91?= =?us-ascii?Q?a6z9n/pjgiJi8C1G9KvRY7CEmI8sJ1mUb9GxxDuEZ6QYqf3W0C17Ex1jJ2Fj?= =?us-ascii?Q?YVbWx9PtDM+711DVpSWQTPWVxT1gO8t/p/wlVHMLt2kOZgpP4vbuTcutzqlI?= =?us-ascii?Q?CbYv3w5FK4rb7xWcYj1jL1ck0tukVpZcGgeT1yAyDeGTuOiINh7zXSg2nmKb?= =?us-ascii?Q?gHpMvq9vhJ9DS7xD6JWEs5kyz8jVUGruB6DDCixdh+Uen+vOu/O6LrBqH60n?= =?us-ascii?Q?Y/XNoGJyKbT6+ppN9R3WutuoaCUE3tXN/ga199q2i6fMsCPAOK+oN09wEg1V?= =?us-ascii?Q?TqrY9e1GsDCenx+qJ6BfkoDz5MuRsU+dNBZhIyOWoerciD96N/LH9CjICJKk?= =?us-ascii?Q?6vBCDybx6SFjsyZ0StbPhPWPVUGubPqmlpJFpMsnpE/pUEFlXAbYUwumgh/m?= =?us-ascii?Q?uhIU12jFOV8rEv4z8z+k1Aq/LJHBMeHM1nVtVtqKa5srvjNLDFJaYK/zJaVA?= =?us-ascii?Q?WSKKaofjfvwnZTPPEgTFHsyn50+JSooxKCZcvpR85eROvD4HJ6O66l1mlDhO?= =?us-ascii?Q?jOFXQJblssu3lZUa8SL9CAL04i0dMMrsBDo6bcb3dw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(1800799015)(366007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sEmK6Xp/9/wld08flaXq+BCNRdN9IrLBL68ZS7uj4RzKhZfj2jy9/ntqmCZB?= =?us-ascii?Q?B93Sx7ngA4hxooARb+25itJAn/70B/Mo1v3SAbqIyexf1Jz/LIMaG9FMOiuF?= =?us-ascii?Q?DIpw72uC64c9tL8G0pnWoYzftD0+LhkVFSqpB86IJ57zCx7Ld504Uxd7iKDV?= =?us-ascii?Q?66WjaGwO2f8Wg+BU+Cci6jU9sUr4FuTNER8tA+ZGiMZR2u3A7/RUzKX0QN9e?= =?us-ascii?Q?tkclLjsbnN/9VkPU7N9EDZDXGOLr3BNTlKvRqtDvSTKaCi1Zt5uHv4UEqPnZ?= =?us-ascii?Q?zRDYg7kuOelTs7nYOYN88afVWehbeSnanOebMSZLXKokWNs5nvRIQQ6WkNeU?= =?us-ascii?Q?n7gOYj7duVxr7rLHEA83qNt0VjLYzy0BtTT4CQnz9YtrWq41G3feDKpNbs5z?= =?us-ascii?Q?VI5Zbdxslix1fbV4ahsKLjhX/uLF8zwTm5b6SZARYH7Pt8LdlE723Ulxu9wM?= =?us-ascii?Q?BpYU8k8DebqpnHbYFFYyIg+oHyS/fzqFgp14glBFSELQJEayszj0e4SCW6Xe?= =?us-ascii?Q?VbwdEJC2Go/HIFRLoqiXxFJV5X4hkts7MEQg4Wcm1as2/UHKhi1os5qcyewr?= =?us-ascii?Q?TU1oOXZ0RpDtF5272PK1IUxCm77nBDnJ7U5OaqtbRFv6ISHouSo4cPd7Wddn?= =?us-ascii?Q?f3N4hkzqGAorYDd0+qV6PvR2x31k3tnK7l6wsVVoStk3Qo5JcpsS3R/S8Thn?= =?us-ascii?Q?dXyf35Ak+uUoqueA6lZw5KQ4St3h5ZOHUUqAH1X7jNHn6fM+7iUt1OZ+ZHHO?= =?us-ascii?Q?iFuWpSdYjf1ex2sTvuFsPZ6Yyjlkvo/PsMvJgUsvS7IlCciNW9VaObMj9JZz?= =?us-ascii?Q?TDRiqeMTu4MtLwS33cK0F9GkwJpI2FQxL6QrEuj2aSlAHll4A8ryqPjdLkS3?= =?us-ascii?Q?VJ59hGm3WJhQvlbfF2GIUp0th1IWs3gB7VzGlGVOkrcawr18mayCuOj/Z7Lh?= =?us-ascii?Q?uBpa4fqhTLUTl/iGUGC05NfgLyTp+RiG8gm0l8dp+dBqxzXYVU59ZpSmaRvj?= =?us-ascii?Q?VDjxFf1waekvvb0HhA7jPZXMCTQN3D9/4h8g5aX/Iq5VXFfNXdteDs1VHAmY?= =?us-ascii?Q?OiKGhTKgmCuNNuAwhDoz2LB9SbefQQCNFCCAB8Ki/MMB/Sgjb4+9St6+vSjn?= =?us-ascii?Q?R1I4MyVXl/z2u5YJNb99Vitn7268Sm2c/mDVhsntvNDLU1tSlyQNJbyF+OGs?= =?us-ascii?Q?wlgBbf0D7zJiE80Ka6Fv5/lcQTirFrKRrFx7S/jc8Flap8RzmjJYYTfJkAyb?= =?us-ascii?Q?5QV/zyfmlqGStJjeQKGdTVPSc+aZAiKc4wRqbPPzRhU4jt9cFhCiPb97DQbp?= =?us-ascii?Q?pFuoqcDe8RMO0QUq3qr18mgXHYXJK6qkoMjwPdTRHTWkCkIxwA0pggQPb50G?= =?us-ascii?Q?HdroYsTqqkIwXS7ECRZh+iVVQINd6aISTf0ltU+VIk8/pdOS15TZ8b4lzq7k?= =?us-ascii?Q?iTdGQJYmujL5h+dzF4Noc1eMJeAoXDFdQ7N2DsAiUGulH9zA88WLRAZ0Wm1x?= =?us-ascii?Q?eiWa3rRPvD4OnBTBn51olY6qWBQq86w65C4wUnncIcezgPjmtSzOVkb5DMtu?= =?us-ascii?Q?YYujUHvLGG26E3qAOb/nTZXrEbDefeBTK75woERCxV0ttLE26gXE7xY+2psz?= =?us-ascii?Q?3A=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: c6b77ea8-c48f-4232-79b9-08dc6b8593b1 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2024 15:27:43.4763 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: o3YjP3JBD5gTAYVbM8N4ZQMp8oR2vA4tzZJeaMQnOezCKUWiHHxP16MooyE05vJmiM8lGkcDgeTR1JWHgUc+g8D056aLaAkKwwri6vKxyZ8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6353 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, May 03, 2024 at 02:48:12PM +0000, Lombardo, Ed wrote: > Hi Dmitry, > I am not clear on the DPDK memory layout and how to tweak these #define values. > > #define RTE_MAX_MEMSEG_PER_LIST 8192 > #define RTE_MAX_MEM_MB_PER_LIST 32768 > #define RTE_MAX_MEMSEG_PER_TYPE 32768 > #define RTE_MAX_MEM_MB_PER_TYPE 65536 > > I want to limit how much DPDK grabs for memory, but grabs what it absolutely needs for our application. Hi, This is what DPDK does. What is being shown in the VIRT figures is the address space reservation DPDK has made, but not what memory it actually uses. Only sufficient hugepage memory to meet the demands of your app should be mapped by DPDK, the rest is unused address space that is not taking up any actual memory. Adding Anatoly to the thread, as he is probably best able to suggest what settings above to tweak to reduce this address-space allocation if it is a problem for you. If it is an issue, I'd be curious to know what the underlying cause is, in case there is something we need to fix. /Bruce > I don't want DPDK to plan for any hot-plug or dynamic adjustments of hugepages. Our configuration is static, we allocate hugepages, discover ports and initialize couple ring buffers and 1 Rx Q and 4 Tx Qs per port. Max of 4 ports. > My goal is to reduce our application VIRT memory by 80%. Not clear which config variables to adjust. > > What do you suggest, is there a formula? > > Regards, > Ed >