From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B721DA00C5 for ; Sun, 5 Jul 2020 22:53:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26EA91DA55; Sun, 5 Jul 2020 22:53:11 +0200 (CEST) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by dpdk.org (Postfix) with ESMTP id E4C901D9C0 for ; Thu, 2 Jul 2020 14:56:52 +0200 (CEST) Received: by mail-lf1-f45.google.com with SMTP id k17so3282461lfg.3 for ; Thu, 02 Jul 2020 05:56:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=VN8FMHAuMKJh59ma2kDGRRG8EUNX5A3ZxOryKpLs8kU=; b=DIUkSZ6XtodU9koTkehzgP4wuRcg2/OvNMT95PkU6kow5n6iACLOjGwJ337w8Pq5RH XSxuoKAoklBw9AELGida5sGnQbYmui2yHMFaf4wXbAdwqreOj6jEDo+i63UAtmyqm0sE XxF7A5DE62AFqt6yp9YduA/qG5QmqU01tBgjJAH6Icre3RRJ7XCZ/VNnkT1qBU1wynNz X1TvHsGSkwqJ2z7fs6vtlNv0vHWvRiv+nINEpkQy5lIZVEtksHm93N4r2cXRsPKTRcIq nTHFm5ntw4fP6EsdoaDb6+R/YwcplXAkMI+90rr6tXxsCW09c1f2nE1X/16jgM3FejMV jvAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:message-id:date:user-agent :mime-version:content-language:content-transfer-encoding; bh=VN8FMHAuMKJh59ma2kDGRRG8EUNX5A3ZxOryKpLs8kU=; b=P8wEQ8HaitZb4AXmsLE4RQijra3Rjgv5Q+Q0F2Yj+t+2YZHtRb8IsapEKJGmAOpMDs Ze7cglKyov6czlXDinN+aGKjrAKM9QCZ2vaOr8aoovfYbgr5MYEQ5X5R2nIl11x+1IH9 gKkEKgeXqqbK4QauY9m5ZZOynvRIJ/A7ngWInxKqikc8HovzzOCVAFFfRQaHgGAOOcuf XdPdkJI1oSq1NPyYjwT0LZNCyvBOYElpS4fTnJ8Ee6q2719K/riWJUNNOqu6HnvFxxGM xyRUCWN/11eHqIhGQwUYPloxAorIYJY1CDdxnRLH4/j78iV0O97rA0McjAK39vHhV27B wwBQ== X-Gm-Message-State: AOAM5312u4vQHXz7/1l5hlzNIgZwv5xiz8yKrSaPfieBRPfM2IZ3J3xj BSHvkDP5C0MBu4/2ltMF1RUZHv6TyJU= X-Google-Smtp-Source: ABdhPJwtKwz4vRWpJA3c74HiG/3ILAOE+RHjw4Kz2sjQl6wL5+aDk3k6brEQjphLJESSxb3ypIZpeA== X-Received: by 2002:a05:6512:3226:: with SMTP id f6mr18220702lfe.180.1593694611831; Thu, 02 Jul 2020 05:56:51 -0700 (PDT) Received: from [192.168.31.44] ([46.32.73.2]) by smtp.gmail.com with ESMTPSA id w19sm1728069ljm.120.2020.07.02.05.56.50 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 02 Jul 2020 05:56:51 -0700 (PDT) From: Valerii Malov To: users@dpdk.org Message-ID: <8e4b96ea-cacd-3056-0120-595a7c3f30ea@gmail.com> Date: Thu, 2 Jul 2020 15:56:50 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 05 Jul 2020 22:53:09 +0200 Subject: [dpdk-users] rte_eal_memseg_init(), virtual memory usage and coredump size X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, I've noticed since upgrading to one of the recent DPDK versions (I think it was DPDK 19), Linux application started to allocate rather big chunks of virtual memory. This causes coredump file size to blow up, and playing around with coredump_filter doesn't seem to help, since this is not hugepages memory. After digging through initialization process, seems like rte_eal_memseg_init() is the culprit. It preallocates chunks for each memory "type", until it hits one of the ceilings, in my case it's default RTE_MAX_MEM_MB_PER_TYPE, which happens to be default 64 gigabytes, so on 2 numa node system this results in nearly extra 128 gigabytes of virtual memory. Surprisingly enough, in FreeBSD implementation of rte_eal_memseg_init() virtual memory usage seems to be a concern due to coredump file size specifically. Are there any ways to reduce the coredump file size apart from building DPDK with reduced RTE_MAX_MEM_MB_PER_TYPE? Am I missing something in system configuration and Linux is not supposed to dump those unused virtual memory pages? This behavior is expressed on CentOS7, but I *think* it was also reproducible on more modern distributions. Thanks in advance, Valerii.