From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2698643F6C; Thu, 2 May 2024 22:03:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9812A402C5; Thu, 2 May 2024 22:03:56 +0200 (CEST) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by mails.dpdk.org (Postfix) with ESMTP id 45D65402B2 for ; Thu, 2 May 2024 22:03:55 +0200 (CEST) Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-51f25f87e58so1359568e87.2 for ; Thu, 02 May 2024 13:03:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714680234; x=1715285034; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=TuO5bFCxwcSOctWtT+Mwvp9a0cX6OjhpujrnJEOXr2I=; b=MZRlOywGKib8EdQJEdnb9jR/8jtcMzkvwL1+6T5ReM15yJ05ekP9JvSAEnsKxL/XFB rhs8OAo/sIQL7VWeWxmsocxq/ey8kEt83jTZQhuLrHX45vkI+x6wDJIuotSPPy0tWhPS vc7/BCaaDwiOauFHpKQhawnMwKatAOsLRJnIF3aYasBFPiS83WlaG1hV2zzwjAswWBi5 opGC1wI9IU6JBjndyHMhaljDo/pzsHKlT+iWSSOenTLM95pOvOa7ulGc+RVjeDbSoFsb Fm7AkZ83Ds5LXnMWQB4nPpnRKMfMLTJBYaYS7njoV8DMbsAg9clz1WQ7H2qlrnjvg6ks W50Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714680234; x=1715285034; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TuO5bFCxwcSOctWtT+Mwvp9a0cX6OjhpujrnJEOXr2I=; b=XmfF0IrHquu8N3toIvbQD4fTK5c8W8uiqIp3aItK6it1XDjJcbSYPLd3gkK90w+kS7 P9pJjSbXw1FMR73cmmCnLCuGHHfK2vyOcPKMu15EInY3wIQXY0OsMhpsohlkbetOYbp8 E9wqD8ju3FWIyfrOLzGSVGZ8W5jDhtcE0Mub6cUVjSyAfCrEAZgRDRfRQJuxXs2GXr1B T1AANpeTXK3iUrJWJS+yDCfGPOD+nn7z0Mq4q1ce04duZka5YQLvvmAsP6OLFsXIcRP0 3Ds6eZIvQTrLyy6TnszsLz7u9lbQmqgwdaL+M4PiDP9jm09NxWpbGZL/ojmRf8rTcE3D W+IA== X-Gm-Message-State: AOJu0YxLetmGdo2XeoO1IEeyLUSWIT4fIht7Ew3cq14tjauUrY2ByXAW v972/tt9JhHViSRtrl/8hE54qiKSgzlt/Z2YMp6x6kr/JlOsdkuB X-Google-Smtp-Source: AGHT+IF6avYrDPjxTMzPfO3FmIDIVM/TtrdCUx0utWgMbFPraFvtJ+OsewoFgNRFZYPEHtZOXD8x6Q== X-Received: by 2002:ac2:4959:0:b0:51a:bebb:690c with SMTP id o25-20020ac24959000000b0051abebb690cmr551077lfi.12.1714680234158; Thu, 02 May 2024 13:03:54 -0700 (PDT) Received: from sovereign (broadband-109-173-110-33.ip.moscow.rt.ru. [109.173.110.33]) by smtp.gmail.com with ESMTPSA id p25-20020ac246d9000000b0051b41844048sm280261lfo.285.2024.05.02.13.03.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 13:03:53 -0700 (PDT) Date: Thu, 2 May 2024 23:03:52 +0300 From: Dmitry Kozlyuk To: "Lombardo, Ed" Cc: "dev@dpdk.org" Subject: Re: Need help with reducing VIRT memory Message-ID: <20240502230352.7bb1ef0d@sovereign> In-Reply-To: References: X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Ed, I presume it's a revival of this thread: http://inbox.dpdk.org/users/CH3PR01MB8470C9675763E14954D6E3B88F292@CH3PR01MB8470.prod.exchangelabs.com/ 2024-05-02 19:05 (UTC+0000), Lombardo, Ed: [...] > My situation is as follows: > We were on DPDK 17.11.6 and upgraded to DPDK22.11.2 to support the Intel E810. Also upgraded from CentOS7 to Oracle 91. > In DPDK 22.11.2 the VIRT memory has skyrocketed and causes major issues with our application that runs on 16 GB of memory (virtual appliances and low end remote appliances). > > I have also tried DPDK 23.11 and 24.03 and I also see across the DPDK versions that the VIRT memory has increase by a factor of 5 over DPDK 17.11 > > Our application process shows VIRT memory is 7.6 GB with DPDK 17.11.6, > and with DPDK 22.11.2, 23.11.2 and 24.03 the VIRT memory is ~36.6GB > > Our application minimum configuration needs to run with 16 GB memory when DPDK is enabled. As explained in the linked thread, high reserved VIRT does not mean high memory demand. Running DPDK on a 16 GB machine is perfectly viable. Something in your app or setup conflicts with the high VIRT reservation. > Our application uses memory locking and rlimit settings to get optimal performance. Does it use address space limit (prlimit -v)? If so, this limit may be hit. With default build options, DPDK reserves at least 32 GB of VIRT (one list) per memory type (a combination of NUMA node + hugepage size). This is controlled by defines in : #define RTE_MAX_MEMSEG_LISTS 128 #define RTE_MAX_MEMSEG_PER_LIST 8192 #define RTE_MAX_MEM_MB_PER_LIST 32768 #define RTE_MAX_MEMSEG_PER_TYPE 32768 #define RTE_MAX_MEM_MB_PER_TYPE 65536 Try reducing those and rebuilding DPDK if the above is your case. Reducing these values will limit the amount of memory available to DPDK allocator in your app.