From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A38641C8B for ; Mon, 13 Feb 2023 18:17:46 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9132F41151; Mon, 13 Feb 2023 18:17:45 +0100 (CET) Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by mails.dpdk.org (Postfix) with ESMTP id 2DCDC40EE4 for ; Mon, 13 Feb 2023 18:17:44 +0100 (CET) Received: by mail-pl1-f175.google.com with SMTP id i18so5750775pli.3 for ; Mon, 13 Feb 2023 09:17:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=6Kz4NNqRsNDLqH/RU4B8i/xQmmXukCMATieeTpbdp54=; b=vYmGzXmRI88gg+1UYNoeJKgDsvpscXAnFzJMBEqC4/ctAcDIvlNdhqwn1+Ty+02Bs6 QhJ15sHj6oAX6HJQ4UKX+rP5/azs6T54rfCoa5W+NuhI8LSt+OrTCgyoe17WPbR+7pSC NZcqwt+RuPKIBjuVPBJ/F1mjVdSGTplSLl6BqX71WMUD2iyusmHrDlrFSO7SGHlvcRzH a0FTp/+YRC2lbGFMnAhn/AmmHXLVRkFvbjEcq0ucbWA2RgXVbDYyxwRoazNHstNDtFyA g4MeroEGs+uIuZ2LLvKCchWRlEwf+15umkttKoG5B+nfXPGcxW5pnDt7O9NJ23407VmT vEfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Kz4NNqRsNDLqH/RU4B8i/xQmmXukCMATieeTpbdp54=; b=KLUsfjJth5BfG+DZRr+t9ut73G9xBEj9H06s7cHDCNH8pmGXA4D+ghpGZsKSE3/Uzr xj8Q/gP2a4hwr2bHJJgi0pFJ67Fagu8R4bHOUvDI11jmDz7ZjrYf0Cfe2FT6DvgH/kYI x10+Z4wzEfIqY4c2Y5Gd4xwH0YjxlBwUxrPFnAQeTL2xafHjr5YiN0wXnxjLNdENL8i5 mNXm46UICTsiGuC6wdtFGJ1LklBgF9DcOW/DeZ9+V7JlDiuiGWQdkqDV0T+//abLKPOA kle131c9C3fA/J5wtrvXTBfg2r5Y+qkQZ0EBitPkDDxbjHkgVM4AKVE/RhqJ7zrBidHD 1qgg== X-Gm-Message-State: AO0yUKW8MJlEFqMXAbwfSxi/uOMeroThI1XefxApBoztq0ZliiWlv53J /oC8vOQY2PWv2dTGHlFzSKwkzA== X-Google-Smtp-Source: AK7set8MSg+NWskpIrXi7GIiRlCw48Lsq+bAIA7Yf+h3vbrTRWU3Qlsmc6J1qrO17Vtup3nPxd9OSA== X-Received: by 2002:a05:6a20:42a1:b0:be:c5de:7157 with SMTP id o33-20020a056a2042a100b000bec5de7157mr30201446pzj.53.1676308663243; Mon, 13 Feb 2023 09:17:43 -0800 (PST) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id x7-20020aa79187000000b005a84ef49c63sm8153770pfa.214.2023.02.13.09.17.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Feb 2023 09:17:42 -0800 (PST) Date: Mon, 13 Feb 2023 09:17:39 -0800 From: Stephen Hemminger To: Dmitry Kozlyuk Cc: Szymon Szozda , users@dpdk.org Subject: Re: Dpdk allocates more memory, than available physically (hugepages) Message-ID: <20230213091739.7f663fee@hermes.local> In-Reply-To: <20230213184614.3039b1af@sovereign> References: <20230213184614.3039b1af@sovereign> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Mon, 13 Feb 2023 18:46:14 +0300 Dmitry Kozlyuk wrote: > 2023-02-08 03:43 (UTC+0100), Szymon Szozda: > > Hey, > > I'm running dpdk on a machine with 64GB of RAM. It is configured, so 16GiB > > (16 x 1GiB chunks) of hugepage memory is reserved on boot. I was expecting > > dpdk to consume only those 16GiB, but it seems it gets more than 30GiB of > > virtual memory ( I base it on memory VSZ output of top command ). The > > machine is 1 NUMA, 1 NIC. I did some debugging and I do not see any logic > > which limits the memory consumption, basically it seems that > > eal_dynmem_memseg_lists_init() will allocate the same amount, no matter how > > much RAM is physically available. > > > > Is it expected? How to know that setup will not crash due to > > insufficient memory available? How to limit those memory consumption.by > > dpdk? > > Hi, > > DPDK always reserves a large chunk of virtual address space, > but this costs almost nothing and does not need to be limited. > Then DPDK maps and unmaps actual pages to those addresses as needed. > DPDK does not crash if it runs out of hugepages reserved in the system > but merely returns NULL from its allocation API (rte_malloc). > Real memory consumption can be limited with --socket-limit EAL option. > See also --socket-mem and -m to reserve hugepages at DPDK startup. Hugepages are mainly used for buffers and shared driver resources. The normal text and data are not in hugepages (unless you figure out how to use hugelbfs). Memory consumption limiting is best done with cgroups. But most applications die if they can't get the memory they need. You can see where memory is used with /proc/XXX/maps