[dpdk-dev] [PATCH v8] mem: command line option to delete hugepage backing files

[dpdk-dev] [PATCH v8] mem: command line option to delete hugepage backing files

[Shesha Sreenivasamurthy]

When an application using huge-pages crash or exists, the hugetlbfs
backing files are not cleaned up. This is a patch to clean those files.
There are multi-process DPDK applications that may be benefited by those
backing files. Therefore, I have made that configurable so that the
application that does not need those backing files can remove them, thus
not changing the current default behavior. The application itself can
clean it up, however the rationale behind DPDK cleaning it up is, DPDK
created it and therefore, it is better it unlinks it.

Signed-off-by: Shesha Sreenivasamurthy <shesha@cisco.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
---
 lib/librte_eal/common/eal_common_options.c | 12 +++++++++++
 lib/librte_eal/common/eal_internal_cfg.h   |  1 +
 lib/librte_eal/common/eal_options.h        |  2 ++
 lib/librte_eal/linuxapp/eal/eal_memory.c   | 32 ++++++++++++++++++++++++++++++
 4 files changed, 47 insertions(+)

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index c614477..4e73b85 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -79,6 +79,7 @@ eal_long_options[] = {
 	{OPT_MASTER_LCORE,      1, NULL, OPT_MASTER_LCORE_NUM     },
 	{OPT_NO_HPET,           0, NULL, OPT_NO_HPET_NUM          },
 	{OPT_NO_HUGE,           0, NULL, OPT_NO_HUGE_NUM          },
+	{OPT_HUGE_UNLINK,       0, NULL, OPT_HUGE_UNLINK_NUM      },
 	{OPT_NO_PCI,            0, NULL, OPT_NO_PCI_NUM           },
 	{OPT_NO_SHCONF,         0, NULL, OPT_NO_SHCONF_NUM        },
 	{OPT_PCI_BLACKLIST,     1, NULL, OPT_PCI_BLACKLIST_NUM    },
@@ -718,6 +719,10 @@ eal_parse_common_option(int opt, const char *optarg,
 		conf->no_hugetlbfs = 1;
 		break;
 
+	case OPT_HUGE_UNLINK_NUM:
+		conf->hugepage_unlink = 1;
+		break;
+
 	case OPT_NO_PCI_NUM:
 		conf->no_pci = 1;
 		break;
@@ -841,6 +846,12 @@ eal_check_common_options(struct internal_config *internal_cfg)
 		return -1;
 	}
 
+	if (internal_cfg->no_hugetlbfs && internal_cfg->hugepage_unlink) {
+		RTE_LOG(ERR, EAL, "Option --"OPT_HUGE_UNLINK" cannot "
+			"be specified together with --"OPT_NO_HUGE"\n");
+		return -1;
+	}
+
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) != 0 &&
 		rte_eal_devargs_type_count(RTE_DEVTYPE_BLACKLISTED_PCI) != 0) {
 		RTE_LOG(ERR, EAL, "Options blacklist (-b) and whitelist (-w) "
@@ -891,6 +902,7 @@ eal_common_usage(void)
 	       "  -h, --help          This help\n"
 	       "\nEAL options for DEBUG use only:\n"
 	       "  --"OPT_NO_HUGE"           Use malloc instead of hugetlbfs\n"
+	       "  --"OPT_HUGE_UNLINK"       Unlink hugepage files after init\n"
 	       "  --"OPT_NO_PCI"            Disable PCI\n"
 	       "  --"OPT_NO_HPET"           Disable HPET\n"
 	       "  --"OPT_NO_SHCONF"         No shared config (mmap'd files)\n"
diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h
index e2ecb0d..292013c 100644
--- a/lib/librte_eal/common/eal_internal_cfg.h
+++ b/lib/librte_eal/common/eal_internal_cfg.h
@@ -64,6 +64,7 @@ struct internal_config {
 	volatile unsigned force_nchannel; /**< force number of channels */
 	volatile unsigned force_nrank;    /**< force number of ranks */
 	volatile unsigned no_hugetlbfs;   /**< true to disable hugetlbfs */
+	unsigned hugepage_unlink; /** < true to unlink backing files */
 	volatile unsigned xen_dom0_support; /**< support app running on Xen Dom0*/
 	volatile unsigned no_pci;         /**< true to disable PCI */
 	volatile unsigned no_hpet;        /**< true to disable HPET */
diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h
index f6714d9..745f38c 100644
--- a/lib/librte_eal/common/eal_options.h
+++ b/lib/librte_eal/common/eal_options.h
@@ -63,6 +63,8 @@ enum {
 	OPT_PROC_TYPE_NUM,
 #define OPT_NO_HPET           "no-hpet"
 	OPT_NO_HPET_NUM,
+#define OPT_HUGE_UNLINK        "huge-unlink"
+	OPT_HUGE_UNLINK_NUM,
 #define OPT_NO_HUGE           "no-huge"
 	OPT_NO_HUGE_NUM,
 #define OPT_NO_PCI            "no-pci"
diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index ac2745e..3a9d190 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -786,6 +786,30 @@ copy_hugepages_to_shared_mem(struct hugepage_file * dst, int dest_size,
 	return 0;
 }
 
+static int
+unlink_hugepage_files(struct hugepage_file *hugepg_tbl,
+		unsigned num_hp_info)
+{
+	unsigned socket, size;
+	int page, nrpages = 0;
+
+	/* get total number of hugepages */
+	for (size = 0; size < num_hp_info; size++)
+		for (socket = 0; socket < RTE_MAX_NUMA_NODES; socket++)
+			nrpages +=
+			internal_config.hugepage_info[size].num_pages[socket];
+
+	for (page = 0; page < nrpages; page++) {
+		struct hugepage_file *hp = &hugepg_tbl[page];
+
+		if (hp->final_va != NULL && unlink(hp->filepath)) {
+			RTE_LOG(WARNING, EAL, "%s(): Removing %s failed: %s\n",
+				__func__, hp->filepath, strerror(errno));
+		}
+	}
+	return 0;
+}
+
 /*
  * unmaps hugepages that are not going to be used. since we originally allocate
  * ALL hugepages (not just those we need), additional unmapping needs to be done.
@@ -1289,6 +1313,14 @@ rte_eal_hugepage_init(void)
 		goto fail;
 	}
 
+	/* free the hugepage backing files */
+	if (internal_config.hugepage_unlink &&
+		unlink_hugepage_files(tmp_hp,
+		internal_config.num_hugepage_sizes) < 0) {
+		RTE_LOG(ERR, EAL, "Unlinking hugepage files failed!\n");
+		goto fail;
+	}
+
 	/* free the temporary hugepage table */
 	free(tmp_hp);
 	tmp_hp = NULL;
-- 
1.9.1

Re: [dpdk-dev] [PATCH v8] mem: command line option to delete hugepage backing files

[Sergio Gonzalez Monroy]

On 28/10/2015 22:04, Shesha Sreenivasamurthy wrote:
> When an application using huge-pages crash or exists, the hugetlbfs
> backing files are not cleaned up. This is a patch to clean those files.
> There are multi-process DPDK applications that may be benefited by those
> backing files. Therefore, I have made that configurable so that the
> application that does not need those backing files can remove them, thus
> not changing the current default behavior. The application itself can
> clean it up, however the rationale behind DPDK cleaning it up is, DPDK
> created it and therefore, it is better it unlinks it.
>
> Signed-off-by: Shesha Sreenivasamurthy <shesha@cisco.com>
> Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
> ---
v3:
- Fix typo in comments

v2:
- Update function X return value
>   lib/librte_eal/common/eal_common_options.c | 12 +++++++++++
>   lib/librte_eal/common/eal_internal_cfg.h   |  1 +
>   lib/librte_eal/common/eal_options.h        |  2 ++
>   lib/librte_eal/linuxapp/eal/eal_memory.c   | 32 ++++++++++++++++++++++++++++++
>   4 files changed, 47 insertions(+)
>
>
Patch looks good!

Just a couple of things for the next time ;)
You might be aware of them, but it doesn't hurt to remind them:
- When sending new version, use --in-reply-to to the last version of the 
patch sent, it's easier to have all patches on the same thread (if your 
email client supports it)
- Also when sending new versions it's useful to add what has changed 
from the previous to the new version.
   (add such info after the three dashes as shown above)

Cheers,
Sergio

Re: [dpdk-dev] [PATCH v8] mem: command line option to delete hugepage backing files

[Thomas Monjalon]

2015-10-28 23:23, Sergio Gonzalez Monroy:
> On 28/10/2015 22:04, Shesha Sreenivasamurthy wrote:
> > When an application using huge-pages crash or exists, the hugetlbfs
> > backing files are not cleaned up. This is a patch to clean those files.
> > There are multi-process DPDK applications that may be benefited by those
> > backing files. Therefore, I have made that configurable so that the
> > application that does not need those backing files can remove them, thus
> > not changing the current default behavior. The application itself can
> > clean it up, however the rationale behind DPDK cleaning it up is, DPDK
> > created it and therefore, it is better it unlinks it.
> >
> > Signed-off-by: Shesha Sreenivasamurthy <shesha@cisco.com>
> > Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>

> Patch looks good!
> 
> Just a couple of things for the next time ;)
> You might be aware of them, but it doesn't hurt to remind them:
> - When sending new version, use --in-reply-to to the last version of the 
> patch sent, it's easier to have all patches on the same thread (if your 
> email client supports it)
> - Also when sending new versions it's useful to add what has changed 
> from the previous to the new version.
>    (add such info after the three dashes as shown above)

+1

Other things:
Some lists like EAL options are alphabetically sorted.
When adding a new line, please try to keep the order.

Applied with orders fixed, thanks.