From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3A6A3A00C5 for ; Mon, 6 Jul 2020 04:49:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A94C91D71A; Mon, 6 Jul 2020 04:49:28 +0200 (CEST) Received: from mail-vs1-f52.google.com (mail-vs1-f52.google.com [209.85.217.52]) by dpdk.org (Postfix) with ESMTP id C90C91D6C9 for ; Mon, 6 Jul 2020 04:49:27 +0200 (CEST) Received: by mail-vs1-f52.google.com with SMTP id o15so19785827vsp.12 for ; Sun, 05 Jul 2020 19:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6bijh3/SqvNi8TWPDZcj07tCe6wGzqAcgURriyIyQ1w=; b=YZsjCtuqffI9bDG/L45ymvkuzGuG260MWd+Il7N4wNexbATkuiQE0tqBYc743KKyMy tQ559kbilv9RvXskYVaVgj/qHqeM7uiB6dvIWtDCbzjKMqxyg5VjOefnPVhbYpi51oKx Hmev6TY4ivctt3tJhoxctm72l7cNhwrnqGsY2WnhzobU79F73AhGfuSBhVL51Vuj5lu2 Xs5XRpuEQjP/j9r04ULcC2pmKuzPPuQxIYek/9QDEm90VSn1yV0LYih7uEuvPD+I4Ht+ sZAEOKZTuQmft9KY7YzL61UrcndPjvTvvg3I8DqYkA38No0qhlTDR/YAbUuRm+dg2DaU b7Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6bijh3/SqvNi8TWPDZcj07tCe6wGzqAcgURriyIyQ1w=; b=ZnvwHgqlbu6duq5nxRL4DmKfgoWoyYW+Tk9Zh11u8qQq/Ol/ZNJv26kFD3vshMo3Gx +ZHtwVkqENsyZavJ/dqg7/fB66wfsZ4V0et+WOef6dJCfmqxkn+JuTLBIbtZh9PUlrEy Tz3Bmu69cAgyUpAQnuaEC+PClobFIAbg+pHoX/Uo6c0XHljIWRcm09DaeEgRV7SLtPYQ oQivz80B2sMcCma4P3VL85VHLcz9Wb3NwVmrWgTd6YvdxDJDXimOGQ5mPC5MJx76IGKH SnqdxBx5EvFHELzebnIcPFg4oFcWXKHOCOPmg7hkPP7/hgrJlNAwWNMcmnefc19b+91q d7aA== X-Gm-Message-State: AOAM531iNUdeauEFDkp05dMiGu5BTVxqLoQMdbLgjY9qyNVB4HIDOYXh SluRgDu/j3IQerOmmEGepdUQ/9ET6vYYpp4CE3gINg== X-Google-Smtp-Source: ABdhPJy2859A0+nHAf3Iej3tgpo7HvChYxZxXSzemEOPy+QYd9UF6v3Dcv+oOxdA4TX9oIPrKNbkZO5mhzYlXolWehA= X-Received: by 2002:a67:6546:: with SMTP id z67mr27513329vsb.169.1594003767054; Sun, 05 Jul 2020 19:49:27 -0700 (PDT) MIME-Version: 1.0 References: <8e4b96ea-cacd-3056-0120-595a7c3f30ea@gmail.com> In-Reply-To: <8e4b96ea-cacd-3056-0120-595a7c3f30ea@gmail.com> From: Li Feng Date: Mon, 6 Jul 2020 10:49:15 +0800 Message-ID: To: Valerii Malov Cc: users@dpdk.org Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-users] rte_eal_memseg_init(), virtual memory usage and coredump size X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi Valerii, You could upgrade your DPDK to the latest version. I have submitted a patch to fix this issue some months ago. Thanks, Feng Li On Mon, Jul 6, 2020 at 4:53 AM Valerii Malov wrote: > > Hi, > > > > I've noticed since upgrading to one of the recent DPDK versions (I think > it was DPDK 19), Linux application started to allocate rather big chunks > of virtual memory. This causes coredump file size to blow up, and > playing around with coredump_filter doesn't seem to help, since this is > not hugepages memory. > > > > > After digging through initialization process, seems like > rte_eal_memseg_init() is the culprit. It preallocates chunks for each > memory "type", until it hits one of the ceilings, in my case it's > default RTE_MAX_MEM_MB_PER_TYPE, which happens to be default 64 > gigabytes, so on 2 numa node system this results in nearly extra 128 > gigabytes of virtual memory. > > Surprisingly enough, in FreeBSD implementation of rte_eal_memseg_init() > virtual memory usage seems to be a concern due to coredump file size > specifically. > > Are there any ways to reduce the coredump file size apart from building > DPDK with reduced RTE_MAX_MEM_MB_PER_TYPE? Am I missing something in > system configuration and Linux is not supposed to dump those unused > virtual memory pages? This behavior is expressed on CentOS7, but I > *think* it was also reproducible on more modern distributions. > > Thanks in advance, > Valerii.