From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <shesha@cisco.com>
Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76])
 by dpdk.org (Postfix) with ESMTP id 408F8C374
 for <dev@dpdk.org>; Thu, 22 Oct 2015 18:03:22 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
 d=cisco.com; i=@cisco.com; l=4706; q=dns/txt; s=iport;
 t=1445529802; x=1446739402;
 h=from:to:cc:subject:date:message-id:references:
 in-reply-to:content-id:content-transfer-encoding: mime-version;
 bh=QA/VVDTl/f1npl61vDM575RekodF/loDXKvB1N51B1g=;
 b=DztYBbhCaZBK9WztDf73xW0trMAvmvMFlVuyC1Sw8AMfJyeVIxL5Pd+D
 F74ggxOqrnLEvGegG3PH0mJGJd5ZFeG3cwf+MwJi61Hv8nYwXwlkq5Nj5
 ZTSNZqKM+ILStqyrAWTIdVlldC3+l6EFh6ERUHNqoTc30WaglXroJ2Ln7 U=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A0D1AQCbBylW/5hdJa1egzaBQwa5d4QhAQ2BWYYdAoFFOBQBAQEBAQEBgQqELgEBAQQnUgwEAgEIEQMBAgEuMh0IAgQOBYgwxUABAQEBAQEBAQEBAQEBAQEBAQEBGopvgQaFDQcGhCgBBIc9hweHZwGNHYFYh2OOe4NvAR8BAUKCER0WgT9yAYU8gQYBAQE
X-IronPort-AV: E=Sophos;i="5.20,183,1444694400"; d="scan'208";a="40012104"
Received: from rcdn-core-1.cisco.com ([173.37.93.152])
 by rcdn-iport-5.cisco.com with ESMTP; 22 Oct 2015 16:03:21 +0000
Received: from XCH-ALN-005.cisco.com (xch-aln-005.cisco.com [173.36.7.15])
 by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id t9MG3LIG027236
 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL);
 Thu, 22 Oct 2015 16:03:21 GMT
Received: from xch-rcd-004.cisco.com (173.37.102.14) by XCH-ALN-005.cisco.com
 (173.36.7.15) with Microsoft SMTP Server (TLS) id 15.0.1104.5;
 Thu, 22 Oct 2015 11:03:00 -0500
Received: from xch-rcd-004.cisco.com ([173.37.102.14]) by
 XCH-RCD-004.cisco.com ([173.37.102.14]) with mapi id 15.00.1104.000; Thu, 22
 Oct 2015 11:03:00 -0500
From: "shesha Sreenivasamurthy (shesha)" <shesha@cisco.com>
To: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Thread-Topic: [dpdk-dev] [PATCH v3] mem: command line option to delete
 hugepage backing files
Thread-Index: AQHRDBvgbQ3X6oaEs0+1a6gRQb16gJ51/8QAgAB4hACAARD/AIAAA08A
Date: Thu, 22 Oct 2015 16:03:00 +0000
Message-ID: <D24E5244.2405D%shesha@cisco.com>
References: <1445419101-19690-1-git-send-email-shesha@cisco.com>
 <D24D0953.23D9B%shesha@cisco.com> <20151021163422.GA10344@bricha3-MOBL3>
 <5628A390.9080602@intel.com>
In-Reply-To: <5628A390.9080602@intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.24.233.20]
Content-Type: text/plain; charset="Windows-1252"
Content-ID: <44E99D3DBE2E7444A5A931C516F5E7EB@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3] mem: command line option to delete
 hugepage backing files
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Oct 2015 16:03:22 -0000

Sergio,
  Your comment regarding remap_all_functions is correct and can be fixed
by unlinking in remap_all_hugepages() too. However, regarding you comment
that =B3unmap_unneeded_hugepages=B2 will fail =8B in the
unmap_unneeded_hugepages() we do not unlink if final_va is equal to NULL
guarded by RTE_EAL_SINGLE_FILE_SEGMENTS. My testing did not catch as
RTE_EAL_SINGLE_FILE_SEGMENTS was set. Is there any reason why we should
not skip unlinking if final_va is null always (removing ifdef
RTE_EAL_SINGLE_FILE_SEGMENTS) ?

However, if you think having a separate function is better, I am all for
it.
--
- Thanks
char * (*shesha) (uint64_t cache, uint8_t F00D)
{ return 0x0000C0DE; }


-----Original Message-----
From: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Date: Thursday, October 22, 2015 at 1:51 AM
To: Cisco Employee <shesha@cisco.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>, Bruce Richardson
<bruce.richardson@intel.com>
Subject: Re: [dpdk-dev] [PATCH v3] mem: command line option to delete
hugepage backing files

On 21/10/2015 17:34, Bruce Richardson wrote:
> On Wed, Oct 21, 2015 at 04:22:45PM +0000, shesha Sreenivasamurthy
>(shesha) wrote:
>> When an application using huge-pages crash or exists, the hugetlbfs
>> backing files are not cleaned up. This is a patch to clean those files.
>> There are multi-process DPDK applications that may be benefited by those
>> backing files. Therefore, I have made that configurable so that the
>> application that does not need those backing files can remove them, thus
>> not changing the current default behavior. The application itself can
>> clean it up, however the rationale behind DPDK cleaning it up is, DPDK
>> created it and therefore, it is better it unlinks it.
>>
>> Signed-off-by: Shesha Sreenivasamurthy <shesha@cisco.com>
>> ---
>>   lib/librte_eal/common/eal_common_options.c | 12 ++++++++++++
>>   lib/librte_eal/common/eal_internal_cfg.h   |  1 +
>>   lib/librte_eal/common/eal_options.h        |  2 ++
>>   lib/librte_eal/linuxapp/eal/eal_memory.c   | 30
>> ++++++++++++++++++++++++++++++
>>   4 files changed, 45 insertions(+)
>>
> <snip>
>> +static int
>> +unlink_hugepage_files(struct hugepage_file *hugepg_tbl,
>> +		unsigned num_hp_info)
>> +{
>> +	unsigned socket, size;
>> +	int page, nrpages =3D 0;
>> +
>> +	/* get total number of hugepages */
>> +	for (size =3D 0; size < num_hp_info; size++)
>> +		for (socket =3D 0; socket < RTE_MAX_NUMA_NODES; socket++)
>> +			nrpages +=3D internal_config.hugepage_info[size].num_pages[socket];
>> +
>> +	for (page =3D 0; page < nrpages; page++) {
>> +		struct hugepage_file *hp =3D &hugepg_tbl[page];
>> +		if (hp->final_va !=3D NULL && unlink(hp->filepath)) {
>> +			RTE_LOG(WARNING, EAL, "%s(): Removing %s failed: %s\n",
>> +				__func__, hp->filepath, strerror(errno));
>> +		}
>> +	}
>> +	return 0;
>> +}
>> +
>>   /*
>>    * unmaps hugepages that are not going to be used. since we originally
>> allocate
>>    * ALL hugepages (not just those we need), additional unmapping needs
>>to
>> be done.
>> @@ -1289,6 +1311,14 @@ rte_eal_hugepage_init(void)
>>   		goto fail;
>>   	}
>>  =20
>> +	/* free the hugepage backing files */
>> +	if (internal_config.hugepage_unlink &&
>> +		unlink_hugepage_files(tmp_hp,
>> +			internal_config.num_hugepage_sizes) < 0) {
>> +			RTE_LOG(ERR, EAL, "Unlinking hugepage backing files failed!\n");
>> +		goto fail;
>> +	}
>> +
> Sorry for the late comment, but...
>
> Rather than adding a whole new function to be called here, can the same
>effect
> not be got by adding in 2/3 lines like:
> 	if (internal_config.hugepage_unlink)
> 		unlink(hugetlb[i].filepath)
>
> at line 409 of eal_memory.c where were have done our final mmap of the
>file.
> [You also need the same couple of lines for the 32-bit special case at
>line 351].
> It would be a shorter diff.
>
> /Bruce
If you wanted to avoid the extra function call, I might be cleaner to
just unlink all files when
doing unmap_all_hugepages_orig.
My two cents: I think it would be easier to read/debug having a function
that "unlinks files" instead
of unlinking files at different points in map_all_hugepages.

Unfortunately the proposed approach does not work for all cases:
- If we have single file segment, map_all_hugepages does not get call a
second time, instead we call
   remap_all_hugepages
- If we use options -m or --socket-mem, because unmap_unneeded_hugepages
does not expect files
   already unlinked, it will fail when trying to unlink unneeded
hugepage files.

The current patch would work as we only unlink after
unmap_unneeded_hugepages.

Sergio