From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from alln-iport-3.cisco.com (alln-iport-3.cisco.com [173.37.142.90]) by dpdk.org (Postfix) with ESMTP id 16DB45A5E for ; Tue, 20 Oct 2015 19:05:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=27047; q=dns/txt; s=iport; t=1445360711; x=1446570311; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=TpoeHtd1PGcD/Lz5vJxO+YIlThOjkIzNXDp4pXrdacg=; b=WqeghayrRdolVsbKHPDrTuitFAkxtgtdfzwbp7QimbGfayZGrNqR7Um6 JXWZwGdioRmy1Lr2zJSyAynrs2JJt7LcC0Bj9Bw7HEvJBVE1V8PVyaNuy KIZAzJK5hdAK3gSWWlryuxC1gGSQz7a24jduZ/Jwp2EkG8SYtg+wI7xiv w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0D4AQD4cyZW/5xdJa1dgmlNgUMGuXiEIQENgVqGHgKBOzgUAQEBAQEBAX8LhC4BAQQnUhACAUAHBzIUEQIEDgWIMMQMAQEBAQEBAQECAQEBAQEBAQEBAQEYim+GEweELgWHO4cHh2IBjR6cIAEfAQFCghEdgVVyhB4kH4EGAQEB X-IronPort-AV: E=Sophos;i="5.17,707,1437436800"; d="scan'208,217";a="199416642" Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by alln-iport-3.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Oct 2015 17:05:09 +0000 Received: from XCH-ALN-004.cisco.com (xch-aln-004.cisco.com [173.36.7.14]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id t9KH59lj003345 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Tue, 20 Oct 2015 17:05:09 GMT Received: from xch-rcd-004.cisco.com (173.37.102.14) by XCH-ALN-004.cisco.com (173.36.7.14) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Tue, 20 Oct 2015 12:04:50 -0500 Received: from xch-rcd-004.cisco.com ([173.37.102.14]) by XCH-RCD-004.cisco.com ([173.37.102.14]) with mapi id 15.00.1104.000; Tue, 20 Oct 2015 12:04:50 -0500 From: "shesha Sreenivasamurthy (shesha)" To: "dev@dpdk.org" Thread-Topic: [PATCH v2] mem: command line option to delete hugepage backing files Thread-Index: AQHRC1jVXvzHm9nhFUW3n7KImjqHHJ50erYA Date: Tue, 20 Oct 2015 17:04:50 +0000 Message-ID: References: <1445335331-9050-1-git-send-email-shesha@cisco.com> In-Reply-To: <1445335331-9050-1-git-send-email-shesha@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.24.228.81] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] [PATCH v2] mem: command line option to delete hugepage backing files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 17:05:11 -0000 When an application using huge-pages crash or exists, the hugetlbfs backing= files are not cleaned up. This is a patch to clean those files. There are = multi-process DPDK applications that may be benefited by those backing file= s. Therefore, I have made that configurable so that the application that do= es not need those backing files can remove them, thus not changing the curr= ent default behavior. The application itself can clean it up, however the r= ationale behind DPDK cleaning it up is, DPDK created it and therefore, it i= s better it unlinks it. Signed-off-by: Shesha Sreenivasamurthy > --- lib/librte_eal/common/eal_common_options.c | 12 +++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/linuxapp/eal/eal_memory.c | 39 +++++++++++++++++++++++++++= +++ 4 files changed, 54 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/co= mmon/eal_common_options.c index 1f459ac..5fe6374 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -79,6 +79,7 @@ eal_long_options[] =3D { {OPT_MASTER_LCORE, 1, NULL, OPT_MASTER_LCORE_NUM }, {OPT_NO_HPET, 0, NULL, OPT_NO_HPET_NUM }, {OPT_NO_HUGE, 0, NULL, OPT_NO_HUGE_NUM }, + {OPT_HUGE_UNLINK, 0, NULL, OPT_HUGE_UNLINK_NUM }, {OPT_NO_PCI, 0, NULL, OPT_NO_PCI_NUM }, {OPT_NO_SHCONF, 0, NULL, OPT_NO_SHCONF_NUM }, {OPT_PCI_BLACKLIST, 1, NULL, OPT_PCI_BLACKLIST_NUM }, @@ -722,6 +723,10 @@ eal_parse_common_option(int opt, const char *optarg, conf->no_hugetlbfs =3D 1; break; + case OPT_HUGE_UNLINK_NUM: + conf->hugepage_unlink =3D 1; + break; + case OPT_NO_PCI_NUM: conf->no_pci =3D 1; break; @@ -856,6 +861,12 @@ eal_check_common_options(struct internal_config *inter= nal_cfg) return -1; } + if (internal_cfg->no_hugetlbfs && internal_cfg->hugepage_unlink) { + RTE_LOG(ERR, EAL, "Option --"OPT_HUGE_UNLINK" cannot " + "be specified together with --"OPT_NO_HUGE"\n"); + return -1; + } + if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) !=3D 0 && rte_eal_devargs_type_count(RTE_DEVTYPE_BLACKLISTED_PCI) !=3D 0) { RTE_LOG(ERR, EAL, "Options blacklist (-b) and whitelist (-w) " @@ -906,6 +917,7 @@ eal_common_usage(void) " -h, --help This help\n" "\nEAL options for DEBUG use only:\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" + " --"OPT_HUGE_UNLINK" Unlink hugepage backing file after in= italization\n" " --"OPT_NO_PCI" Disable PCI\n" " --"OPT_NO_HPET" Disable HPET\n" " --"OPT_NO_SHCONF" No shared config (mmap'd files)\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/comm= on/eal_internal_cfg.h index e2ecb0d..84b075f 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -64,6 +64,7 @@ struct internal_config { volatile unsigned force_nchannel; /**< force number of channels */ volatile unsigned force_nrank; /**< force number of ranks */ volatile unsigned no_hugetlbfs; /**< true to disable hugetlbfs */ + volatile unsigned hugepage_unlink; /** < true to unlink backing files */ volatile unsigned xen_dom0_support; /**< support app running on Xen Dom0*/ volatile unsigned no_pci; /**< true to disable PCI */ volatile unsigned no_hpet; /**< true to disable HPET */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/ea= l_options.h index f6714d9..745f38c 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -63,6 +63,8 @@ enum { OPT_PROC_TYPE_NUM, #define OPT_NO_HPET "no-hpet" OPT_NO_HPET_NUM, +#define OPT_HUGE_UNLINK "huge-unlink" + OPT_HUGE_UNLINK_NUM, #define OPT_NO_HUGE "no-huge" OPT_NO_HUGE_NUM, #define OPT_NO_PCI "no-pci" diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linu= xapp/eal/eal_memory.c index ac2745e..2b86428 100644 --- a/lib/librte_eal/linuxapp/eal/eal_memory.c +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c @@ -786,6 +786,37 @@ copy_hugepages_to_shared_mem(struct hugepage_file * ds= t, int dest_size, return 0; } +static int +unlink_hugepage_files(struct hugepage_file *hugepg_tbl, + struct hugepage_info *hpi, + unsigned num_hp_info) +{ + unsigned socket, size; + int page, nrpages =3D 0; + + /* get total number of hugepages */ + for (size =3D 0; size < num_hp_info; size++) + for (socket =3D 0; socket < RTE_MAX_NUMA_NODES; socket++) + nrpages +=3D internal_config.hugepage_info[size].num_pages[socket]; + + for (size =3D 0; size < num_hp_info; size++) { + for (socket =3D 0; socket < RTE_MAX_NUMA_NODES; socket++) { + for (page =3D 0; page < nrpages; page++) { + struct hugepage_file *hp =3D &hugepg_tbl[page]; + if ((hp->size =3D=3D hpi[size].hugepage_sz) && + (hp->socket_id =3D=3D (int) socket) && + hp->final_va !=3D NULL) { + if (unlink(hp->filepath)) { + RTE_LOG(WARNING, EAL, "%s(): Removing %s failed: %s\n", + __func__, hp->filepath, strerror(errno)); + } + } + } /* foreach page */ + } /* foreach socket */ + } /* foreach pagesize */ + return 0; +} + /* * unmaps hugepages that are not going to be used. since we originally all= ocate * ALL hugepages (not just those we need), additional unmapping needs to b= e done. @@ -1289,6 +1320,14 @@ rte_eal_hugepage_init(void) goto fail; } + /* free the hugepage backing files */ + if (internal_config.hugepage_unlink && + unlink_hugepage_files(tmp_hp, used_hp, + internal_config.num_hugepage_sizes) < 0) { + RTE_LOG(ERR, EAL, "Unlinking hugepage backing files failed!\n"); + goto fail; + } + /* free the temporary hugepage table */ free(tmp_hp); tmp_hp =3D NULL; -- 1.9.1