From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76]) by dpdk.org (Postfix) with ESMTP id 408F8C374 for ; Thu, 22 Oct 2015 18:03:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4706; q=dns/txt; s=iport; t=1445529802; x=1446739402; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=QA/VVDTl/f1npl61vDM575RekodF/loDXKvB1N51B1g=; b=DztYBbhCaZBK9WztDf73xW0trMAvmvMFlVuyC1Sw8AMfJyeVIxL5Pd+D F74ggxOqrnLEvGegG3PH0mJGJd5ZFeG3cwf+MwJi61Hv8nYwXwlkq5Nj5 ZTSNZqKM+ILStqyrAWTIdVlldC3+l6EFh6ERUHNqoTc30WaglXroJ2Ln7 U=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0D1AQCbBylW/5hdJa1egzaBQwa5d4QhAQ2BWYYdAoFFOBQBAQEBAQEBgQqELgEBAQQnUgwEAgEIEQMBAgEuMh0IAgQOBYgwxUABAQEBAQEBAQEBAQEBAQEBAQEBGopvgQaFDQcGhCgBBIc9hweHZwGNHYFYh2OOe4NvAR8BAUKCER0WgT9yAYU8gQYBAQE X-IronPort-AV: E=Sophos;i="5.20,183,1444694400"; d="scan'208";a="40012104" Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-5.cisco.com with ESMTP; 22 Oct 2015 16:03:21 +0000 Received: from XCH-ALN-005.cisco.com (xch-aln-005.cisco.com [173.36.7.15]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id t9MG3LIG027236 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 22 Oct 2015 16:03:21 GMT Received: from xch-rcd-004.cisco.com (173.37.102.14) by XCH-ALN-005.cisco.com (173.36.7.15) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Thu, 22 Oct 2015 11:03:00 -0500 Received: from xch-rcd-004.cisco.com ([173.37.102.14]) by XCH-RCD-004.cisco.com ([173.37.102.14]) with mapi id 15.00.1104.000; Thu, 22 Oct 2015 11:03:00 -0500 From: "shesha Sreenivasamurthy (shesha)" To: Sergio Gonzalez Monroy Thread-Topic: [dpdk-dev] [PATCH v3] mem: command line option to delete hugepage backing files Thread-Index: AQHRDBvgbQ3X6oaEs0+1a6gRQb16gJ51/8QAgAB4hACAARD/AIAAA08A Date: Thu, 22 Oct 2015 16:03:00 +0000 Message-ID: References: <1445419101-19690-1-git-send-email-shesha@cisco.com> <20151021163422.GA10344@bricha3-MOBL3> <5628A390.9080602@intel.com> In-Reply-To: <5628A390.9080602@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.24.233.20] Content-Type: text/plain; charset="Windows-1252" Content-ID: <44E99D3DBE2E7444A5A931C516F5E7EB@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH v3] mem: command line option to delete hugepage backing files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 16:03:22 -0000 Sergio, Your comment regarding remap_all_functions is correct and can be fixed by unlinking in remap_all_hugepages() too. However, regarding you comment that =B3unmap_unneeded_hugepages=B2 will fail =8B in the unmap_unneeded_hugepages() we do not unlink if final_va is equal to NULL guarded by RTE_EAL_SINGLE_FILE_SEGMENTS. My testing did not catch as RTE_EAL_SINGLE_FILE_SEGMENTS was set. Is there any reason why we should not skip unlinking if final_va is null always (removing ifdef RTE_EAL_SINGLE_FILE_SEGMENTS) ? However, if you think having a separate function is better, I am all for it. -- - Thanks char * (*shesha) (uint64_t cache, uint8_t F00D) { return 0x0000C0DE; } -----Original Message----- From: Sergio Gonzalez Monroy Date: Thursday, October 22, 2015 at 1:51 AM To: Cisco Employee Cc: "dev@dpdk.org" , Bruce Richardson Subject: Re: [dpdk-dev] [PATCH v3] mem: command line option to delete hugepage backing files On 21/10/2015 17:34, Bruce Richardson wrote: > On Wed, Oct 21, 2015 at 04:22:45PM +0000, shesha Sreenivasamurthy >(shesha) wrote: >> When an application using huge-pages crash or exists, the hugetlbfs >> backing files are not cleaned up. This is a patch to clean those files. >> There are multi-process DPDK applications that may be benefited by those >> backing files. Therefore, I have made that configurable so that the >> application that does not need those backing files can remove them, thus >> not changing the current default behavior. The application itself can >> clean it up, however the rationale behind DPDK cleaning it up is, DPDK >> created it and therefore, it is better it unlinks it. >> >> Signed-off-by: Shesha Sreenivasamurthy >> --- >> lib/librte_eal/common/eal_common_options.c | 12 ++++++++++++ >> lib/librte_eal/common/eal_internal_cfg.h | 1 + >> lib/librte_eal/common/eal_options.h | 2 ++ >> lib/librte_eal/linuxapp/eal/eal_memory.c | 30 >> ++++++++++++++++++++++++++++++ >> 4 files changed, 45 insertions(+) >> > >> +static int >> +unlink_hugepage_files(struct hugepage_file *hugepg_tbl, >> + unsigned num_hp_info) >> +{ >> + unsigned socket, size; >> + int page, nrpages =3D 0; >> + >> + /* get total number of hugepages */ >> + for (size =3D 0; size < num_hp_info; size++) >> + for (socket =3D 0; socket < RTE_MAX_NUMA_NODES; socket++) >> + nrpages +=3D internal_config.hugepage_info[size].num_pages[socket]; >> + >> + for (page =3D 0; page < nrpages; page++) { >> + struct hugepage_file *hp =3D &hugepg_tbl[page]; >> + if (hp->final_va !=3D NULL && unlink(hp->filepath)) { >> + RTE_LOG(WARNING, EAL, "%s(): Removing %s failed: %s\n", >> + __func__, hp->filepath, strerror(errno)); >> + } >> + } >> + return 0; >> +} >> + >> /* >> * unmaps hugepages that are not going to be used. since we originally >> allocate >> * ALL hugepages (not just those we need), additional unmapping needs >>to >> be done. >> @@ -1289,6 +1311,14 @@ rte_eal_hugepage_init(void) >> goto fail; >> } >> =20 >> + /* free the hugepage backing files */ >> + if (internal_config.hugepage_unlink && >> + unlink_hugepage_files(tmp_hp, >> + internal_config.num_hugepage_sizes) < 0) { >> + RTE_LOG(ERR, EAL, "Unlinking hugepage backing files failed!\n"); >> + goto fail; >> + } >> + > Sorry for the late comment, but... > > Rather than adding a whole new function to be called here, can the same >effect > not be got by adding in 2/3 lines like: > if (internal_config.hugepage_unlink) > unlink(hugetlb[i].filepath) > > at line 409 of eal_memory.c where were have done our final mmap of the >file. > [You also need the same couple of lines for the 32-bit special case at >line 351]. > It would be a shorter diff. > > /Bruce If you wanted to avoid the extra function call, I might be cleaner to just unlink all files when doing unmap_all_hugepages_orig. My two cents: I think it would be easier to read/debug having a function that "unlinks files" instead of unlinking files at different points in map_all_hugepages. Unfortunately the proposed approach does not work for all cases: - If we have single file segment, map_all_hugepages does not get call a second time, instead we call remap_all_hugepages - If we use options -m or --socket-mem, because unmap_unneeded_hugepages does not expect files already unlinked, it will fail when trying to unlink unneeded hugepage files. The current patch would work as we only unlink after unmap_unneeded_hugepages. Sergio