From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 19EAB432F3 for ; Fri, 10 Nov 2023 10:37:44 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0596540291; Fri, 10 Nov 2023 10:37:44 +0100 (CET) Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by mails.dpdk.org (Postfix) with ESMTP id 156B740268 for ; Fri, 10 Nov 2023 10:37:42 +0100 (CET) Received: by mail-lf1-f46.google.com with SMTP id 2adb3069b0e04-507b9408c61so2416391e87.0 for ; Fri, 10 Nov 2023 01:37:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699609061; x=1700213861; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=ESJaerymcEh/1JIdN6Kl08obZJ384nN77kH27rUlWyg=; b=CYcWzaKpKdmxhj+HHrgUE4YXHJLJi/vodNNglrQKUrDD4yOeeVsTFyDnCV4rDdvwKW llbTjaxFowrsHq2aRpuBeZzfDMrzPNgO/78INTmXaeuS/CZZ84o+3eaI0RbpdF+//awM wV7PUCueDPXm609CH2b7HDRQJBbQK3gFtSrAg0eyl0BsoNCmlS4GH6HCJQUVCaq+lOja TdtJZ8oo8L2BL74nVlamh/ciYOYcB67icSE90/kbcSlz/ecqP4YLJPaCTr+tmeeTJM3H TGvG6AJhEf94lBeoMaWIbKIsYKuTiBhQjxzEbdViruKDMJs3ISMnvV29svPU+d/5WbYG v8OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699609061; x=1700213861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ESJaerymcEh/1JIdN6Kl08obZJ384nN77kH27rUlWyg=; b=fCMM3GmdMDfqOy90FDYaroQOez+3wlpp6PppmEZMx16V6EbgfXTA9c68EDGsPjeh6o 20kxAhL4erpx2n0C2w+tCoHmQwCt9T+A+zOMoRe16PYrMhLteLgAWNp2XCioUmxWMGTI ssYQPhrCrFhLIlk+u8MSpwM8oWc52hM94ds6PAMrI/rClrm4bca5olgXy7PXGs6/ttYH SYnynS+yZ6HSR/+8+VeWIVuI3U1WHFMeTozwXgAUlF/MAib4tEy5ejRWFkggvbnQYw76 TjpzRgssk6+NP0cmjzhABX+2LFW1GV6SXqLOc+nKVnpt+Msn2fGm28n5UEy/g1uBQURb yvcA== X-Gm-Message-State: AOJu0YzJGeHBPWOJpW4f0NrLgTbK1IdcjIfgwHnvN3rCVDeE69h0evjc auw/D10HuuSel+W5LR3vYhE= X-Google-Smtp-Source: AGHT+IEy8n4AoXu8O0MmXTnHaNzDXbdDm/uFNk18IUTOxdkHoFLEkrrWi435INKrxGiYE6LCzfMg+A== X-Received: by 2002:a05:6512:3142:b0:509:45ed:1083 with SMTP id s2-20020a056512314200b0050945ed1083mr3542060lfi.40.1699609061194; Fri, 10 Nov 2023 01:37:41 -0800 (PST) Received: from sovereign (broadband-109-173-110-33.ip.moscow.rt.ru. [109.173.110.33]) by smtp.gmail.com with ESMTPSA id b14-20020a0565120b8e00b004fdd6b72bfdsm234567lfv.117.2023.11.10.01.37.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Nov 2023 01:37:40 -0800 (PST) Date: Fri, 10 Nov 2023 12:37:39 +0300 From: Dmitry Kozlyuk To: "Lombardo, Ed" Cc: users Subject: Re: DPDK 22.11.2 requires too much VIRT memory, how to reduce Message-ID: <20231110123739.21fbcb01@sovereign> In-Reply-To: <20231110123133.2adbd2cc@sovereign> References: <20231109020550.365e80bc@sovereign> <20231110123133.2adbd2cc@sovereign> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org 2023-11-10 12:31 (UTC+0300), Dmitry Kozlyuk: > Hi Ed, > > 2023-11-10 00:16 (UTC+0000), Lombardo, Ed: > > I finally finished testing all the options and found the VIRT value can be > > reduced from 66 GB to 16 GB with --legacy-mem setting in EAL init argument. > > Right. > By default, DPDK can use up to 64 GB of hugepage memory, > so it reserves 64 GB of VIRT (but does not map most of it), > RES should be low until the app actually allocates something. > In legacy mode, DPDK maps all available hugepage memory at startup, > in your case 16 GB, VIRT and RES should be close. > > > So I therefore had to increase the VM memory from 16 GB to 24 GB (instead > > of 80 GB without this setting). > > I don't understand why you have to do that. > Possible VIRT is not limited by available RAM. > DPDK should be able to reserve 64 GB of VIRT on a machine with 16 GB of RAM, > it will just be unable to map more than 16 GB (obviously). Sorry, I've sent the message early by mistake. > > I wonder what do we give up with this setting? Most importantly, in legacy mode DPDK will consume all available hugepages at startup and will not free them back to the system until the all is terminated. The default dynamic mode allocates and frees physical RAM on demand. Some advanced DPDK memory API don't work in legacy mode. > > > > All the other settings I tried and combinations of these had no impact > > (socket-limit=2048, single-file-segments, no-shconf, and no-telemetry) on > > VIRT memory. Right, they should not. DPDK assumes that VIRT reservation is almost free and unlimited. May it be that your system somehow limits it?