From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EF4F4A0352; Tue, 8 Feb 2022 18:05:32 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DCC6341141; Tue, 8 Feb 2022 18:05:32 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 8DD884111B for ; Tue, 8 Feb 2022 18:05:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644339930; x=1675875930; h=message-id:date:subject:to:references:from:in-reply-to: content-transfer-encoding:mime-version; bh=WStiYviFi3xoCUOqSbxMNRAxFfQoH+cuozQJ+1uAw6o=; b=GaFCk4LMsd5uFjPXNiZRLbX8ragbddHlNBowol9igMpY7RYJTkN7Gzw6 egrdShS0v63KYu0zAjwykOcW8Yfyf7+YN8/9ZM3inoD7DNBXeZqzF2Tcv NlEci8e2v9VDl8x7IEpPLihlexVPhhLPVtn37wwfj5gTs+vO9T89fS2tU YX9yTzc7SIJ4KJAYG/E0/1NC86K6/tNnbA8r1pYGuvdiJLm1lphj9rEP3 B1CqfflEWUudpLRRBIrXorR1KGRHNiq4AymSpuFbG+jNCYSgg6NjclJWG VsWPiUV7RY0sRvxZmnnatCW8B4uVqahol9f54USoPRjqiszt33L4ylWwb A==; X-IronPort-AV: E=McAfee;i="6200,9189,10252"; a="236404272" X-IronPort-AV: E=Sophos;i="5.88,353,1635231600"; d="scan'208";a="236404272" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2022 09:05:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,353,1635231600"; d="scan'208";a="621968445" Received: from orsmsx605.amr.corp.intel.com ([10.22.229.18]) by FMSMGA003.fm.intel.com with ESMTP; 08 Feb 2022 09:05:28 -0800 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 8 Feb 2022 09:05:27 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Tue, 8 Feb 2022 09:05:27 -0800 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.170) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Tue, 8 Feb 2022 09:05:27 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fYz0t5OvrFd9MCpciM1PBYaU3a29oAsqrSYTYXq5hcF3jH0wfKF9j/ac/j94Oj4Qzxp8fHprZ00GaQNZ9sOGRYMuQMtWIVsQMfzspmXWBVF8J4g5IC1Cgp1ashPWa00Y9ICHJ2CQWL7hc+A2MYeB9tmT+13jeMUOi14de/mwi8oDz2Wg5wdGYq2OdNEu3aMHKu+qE9C+6kYNefZoiDTKRd3wCVgT8CqumkhkAlHk6lcYX3aIkujArJZh5lVw/6SCt/HG2zWw06uDteVd+cJdiUYjSSsmNZW71mQj9giGNMVZMJm3PctmbLbhQvzl+Hp5Rh5GRZjvztP7AupS9q5hDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sRicla6E4dwZmuVj+bMMLbbPvbym2OJDKy34k5/pLac=; b=ClFUApCTHU0iGxEJumH3DV4JFnHtsXLZ/TRqz1mfi4gf3oaSwPZL3SXszt+JNLM0XnOasnUC647k/GOqA+fYGDwQZH/R5yVJk8SpgKE49goR7fmd8QGN7103FgO1PfHno2mHSCTMFB/JJBwfcKU2d2thmVHm9MRGYUjLgZyHrbmArInfKlJE1hMcqx3iOvvWATpPQ3HR4B3eeWGBKMVDI4ul+z3LGMMXrnVjfC4EsB1fdsN6MUKLH7HKtHyUffop+fUDMO347fgwUav5AjcnHnrYHS9L4B8ukDzJO5uGwjOc6K5yYFHN948lXPyKfVcWE941OMIFbo0eHWqWM+GjtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) by DM5PR1101MB2250.namprd11.prod.outlook.com (2603:10b6:4:4d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.14; Tue, 8 Feb 2022 17:05:25 +0000 Received: from PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::ad52:b9b6:8147:b4d5]) by PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::ad52:b9b6:8147:b4d5%4]) with mapi id 15.20.4951.019; Tue, 8 Feb 2022 17:05:25 +0000 Message-ID: Date: Tue, 8 Feb 2022 17:05:20 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.5.1 Subject: Re: [PATCH v3 5/6] eal/linux: allow hugepage file reuse Content-Language: en-US To: Dmitry Kozlyuk , References: <20220119210917.765505-1-dkozlyuk@nvidia.com> <20220203181337.759161-1-dkozlyuk@nvidia.com> <20220203181337.759161-6-dkozlyuk@nvidia.com> From: "Burakov, Anatoly" In-Reply-To: <20220203181337.759161-6-dkozlyuk@nvidia.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0202.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a5::9) To PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 756315cd-0670-423b-e2ec-08d9eb2532fc X-MS-TrafficTypeDiagnostic: DM5PR1101MB2250:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QUII7muJyti2Qws0u+V8DMaeSmbBJxG2afPEoIODA0zQPx+k1eyzcO9ev6eqFarGpeGzuAJqDJBpcMKieKwhATYcyMohC/xc1sV1iXeD5ResQ/QkA7ijEUQBZgUtN6zuKo7A3LKbAzEITF+HpP76SZHqBJ1Ff3mM5La3Tsq7Y/+1kImdJRLeN3ROrj7rq+cAH3CrzHJ5adyO7/rlvHY17XjQ9eVlA8yKMb1laCGmbL/v5sM+jrZT+KdjdwwoLM5AIEHV65J5gieNl31zN18SQUMfeuJy8BGfKRXcBuw9nbK25Fuy4OIDXDK4CHoQMPvgMwg1rP4Z8OaHpXlGRE+ZFwk7NQpXwf4X74cp68qY0SNymH+t4W+7GnUXLvUYwcoHo2k2BY47hNhtNpprNJyn3ETxIHFoXedKJ7WGpMIPHFXbDoKD3oBX5/H6jOXasvy946Hc3sMd8KtAWwWsAmVevAC65yKr9h2TFdtD53ObpfZKDeyWwV0ypYvp3IC0+kY/Hf+ufwHofjT+DHXrZP6sgTYoLYwmTD7d8fQwULZMwSnky4YwkVNzpzm88EyIIBLaTk1ejwGf9+5uDSwqsJQ7203fEtjHiCQayabwR/simKhEzGoxopSyUeX2jdFUt8P5lj+v9OdxI0KMA0um1+na+gUcrbBYZzXPCsMeXDh1rcsBbIH9Vf4wscS/PwL97DixOYY9tv393v88TNj6nEeRmsO6PBoivMhooacnfQUvzQI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5093.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(82960400001)(36756003)(31686004)(5660300002)(31696002)(2616005)(38100700002)(316002)(26005)(186003)(508600001)(66556008)(66476007)(8676002)(86362001)(2906002)(6486002)(6666004)(6506007)(6512007)(53546011)(83380400001)(66946007)(8936002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V2lFTUZNQTgrcmthWmhpUnZ3WCtOcEJmZXpOVzZPYzhyS0duSGRoTm9IdWRE?= =?utf-8?B?R0xxRVU0NWlQLzBvSGlXOGZ0dHNDQjZKZ0NiU1lWelhVa0kxbGpVMEJsbnFr?= =?utf-8?B?NXJ1VUV3RkdDT21aKzY1VVZsbmtJZGJ5V0RVTTF2N3BOS1I3RUUwZzFvSjlY?= =?utf-8?B?WTg4NDliUEorckE4c0x3WEtiVjdUSkZvRXBmQThIS1UvMk5KOVM5OE8wemN3?= =?utf-8?B?Um9TR0hLWHZzUTN2akxXZldwQmhCZ29weTIxYjAvRDg0MUhmUnJIOHFLK2g2?= =?utf-8?B?RTIvNm51d25IZjVNUHlJUnZYYVNvamp1ckVjanhNSnl6dER3TnZKZ2toTC9z?= =?utf-8?B?d1M3NHV6YnV5OVZZSHpMeVJmNFFDSXZhUjVhckl0Zy9Qa1NnSmRSallBNDhW?= =?utf-8?B?SnF6Z1VoY2ZYM0FJQktVMCtza0FURzlwNU1WTG1Hc1F6dXBVeTE4WUpzdlFN?= =?utf-8?B?aHRIWW9pdmtpcUp2L1I0TTdlR3ZzUk4rdHR3eXpHYTh0N0xSa1dmMXVoNW82?= =?utf-8?B?bnB1cVU0L05wTW9wRnpRUTFwejIwZDJLc2JBazErVnNIOERlOWRiUjJ5L1pl?= =?utf-8?B?NHFENGlqUmF5UWc3RDN4ZzZMZU1RQ3dXZjN2dU9jaXZmckRXTUZOL29LbGRa?= =?utf-8?B?dDdhb1BWOTBONEtiSVFoZE5zYXR3TFJnRUlBRXNydlZ2NjdOSlNPb2E4NjE5?= =?utf-8?B?dnRteVF0NHRlVEFLUVdlSTRNcFBjb0FRWGZyYUdyV21YSm94TW1lRWs3NFA4?= =?utf-8?B?ZFg4cVFGMFU4OXV3WnBzTzdScGNZWEk2SkViYjhrY2FNU3hjam4wcWhQUXN5?= =?utf-8?B?T3pBNnd6aVpHU2NFQkdOYVMxZUxoVUpYVEVkL29Pb2RHeEhjT2dlZWRlRFV1?= =?utf-8?B?ZjVwMC9UdENYeDVMTWUwaGpLTHdlTkVSdzlXNFhaRmI3dEpVTWFiVE4zb0dx?= =?utf-8?B?THZUWi9tWXV1Sy9nZzVyODRrYVdTajVaNjRxQkpSWUo2L2oyTkFkVjRQNTNk?= =?utf-8?B?MTJDYkY3c0VIbW85U3ZmV29QQTJ2VnA0VWE1UzRxb0lQcGszNHB0QUh0cnQ4?= =?utf-8?B?SkRXcFNvMEQvSXc0cFNCeTNvMHNlTkRldERSeVhXLytWQ09mWDRjTlA4b2Mw?= =?utf-8?B?cExlQVZxaFdpMTIvbzVHcXROM2lPaVludVpwNE1wSnlqVTVHMGprNnB4MExi?= =?utf-8?B?L1RrS0NpYzNKaGN1Vi9WZTZlbVU1bnVjaTJOMFNPUzJLaExKTDdGZFA5RlRX?= =?utf-8?B?K1RBU29vay9POFZUaGQ5QkhPM1MwK2h1ei9ya2VMb1FlT09tSUZwOHpLUTZt?= =?utf-8?B?OUNrWDhybkxRQU0xUUtaNXZoR2ZpcmZQNDA0bFpIVDFoU0MxQnZEZkZkYWFk?= =?utf-8?B?QWVYKytiTFNzdnRQd3M2N2RwalRVdGVpdXpEODB1eFJHbitZNzZnRzhySi84?= =?utf-8?B?RGd6NXlha1VqaURyYi9EakxrMUlTdE9pSUl5ZkRjSlpDNUI3SElEUHpNSWlq?= =?utf-8?B?aFpvMENWejE4NjF6ZDE5STNqWjFqNWlYT3RyRDZXRmFyMURFZmlremQrZ2p4?= =?utf-8?B?KzBEa1FTODVjKzBsa3YyM205RlYzbkt4OERONkxRdzBXRDJuNG5DTkg2MFRQ?= =?utf-8?B?VTE5ak1wQmhOSGN2cHBENmFSaTd1UEFwYWxQL3d3bTZNZWdyNExwc2lwTzZl?= =?utf-8?B?c2d0Mno0R05FQVpOeHhzZVZFcEo2dStmQWNvM3orS2c0M0xNdHhWdVdqTG1T?= =?utf-8?B?aEJyRTlZUXF0bGgrOWwxbFJ1T3B0MlVORWVMbHpoQVB4OGJNVWQ1SDc2RCtG?= =?utf-8?B?T0Zlc3F6amdyZHdyUzdEVThoTHpDTVovTzRQcFVaS0Vhb1hBeGNJRXlCWlZs?= =?utf-8?B?N1ZtSFpNcFBuaW9mWWYzUXhlOFV6aTRzbGxMRE9uWk0zbXJBVS90MHV4MHFX?= =?utf-8?B?cTh3Z3RsTFd0bVhJV3hoWnBCVVg0NTFiSSsxL0Q4dHlNaFd6a1VITlE3TlM4?= =?utf-8?B?L3lPSTVsS0gzS1M0ZlpNQ2JPUHRYdmREOG5Fd0pVWTFZNVFpV3p0QU5rWHQv?= =?utf-8?B?NGN2NnF0QVlmSVgyamg3UzRyRk9pKyswTXJiWmZtOFYyUWdGcmN6ZUlTRnBN?= =?utf-8?B?TnEzK2ZXa014QVNXbkx3cytya0VBYzdDSmdubFFVYU9xQmJWY3pLVzM5TkNj?= =?utf-8?B?ZVEzS0dXVkN6N2RKNm1DS2YvWHBrNDE4c1dPQ05OdkRMSWh2MVJTVFlvSFRD?= =?utf-8?B?U051VnNtQ3c3Mk5MUlgrR2Y4QllRPT0=?= X-MS-Exchange-CrossTenant-Network-Message-Id: 756315cd-0670-423b-e2ec-08d9eb2532fc X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5093.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2022 17:05:25.3886 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 886dhQUw6Bsg8rrR/d9KkEOl5V/Aac0FGlwP8LEDPUlIOa5CND6xV8FgsE5M7FXza2y+5bAHQD7Y+2EENlmLD8xgdNck05MxstqV53EnOg8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1101MB2250 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 03-Feb-22 6:13 PM, Dmitry Kozlyuk wrote: > Linux EAL ensured that mapped hugepages are clean > by always mapping from newly created files: > existing hugepage backing files were always removed. > In this case, the kernel clears the page to prevent data leaks, > because the mapped memory may contain leftover data > from the previous process that was using this memory. > Clearing takes the bulk of the time spent in mmap(2), > increasing EAL initialization time. > > Introduce a mode to keep existing files and reuse them > in order to speed up initial memory allocation in EAL. > Hugepages mapped from such files may contain data > left by the previous process that used this memory, > so RTE_MEMSEG_FLAG_DIRTY is set for their segments. > If multiple hugepages are mapped from the same file: > 1. When fallocate(2) is used, all memory mapped from this file > is considered dirty, because it is unknown > which parts of the file are holes. > 2. When ftruncate(3) is used, memory mapped from this file > is considered dirty unless the file is extended > to create a new mapping, which implies clean memory. > > Signed-off-by: Dmitry Kozlyuk > --- > - while(dirent != NULL){ > + while (dirent != NULL) { > /* skip files that don't match the hugepage pattern */ > if (fnmatch(filter, dirent->d_name, 0) > 0) { > dirent = readdir(dir); > @@ -345,9 +357,15 @@ clear_hugedir(const char * hugedir) > /* non-blocking lock */ > lck_result = flock(fd, LOCK_EX | LOCK_NB); > > - /* if lock succeeds, remove the file */ > + /* if lock succeeds, execute callback */ > if (lck_result != -1) > - unlinkat(dir_fd, dirent->d_name, 0); > + cb(&(struct walk_hugedir_data){ > + .dir_fd = dir_fd, > + .file_fd = fd, > + .file_name = dirent->d_name, > + .user_data = user_data, > + }); Off topic, but nice trick! Didn't know C allowed for this. Otherwise, LGTM Reviewed-by: Anatoly Burakov -- Thanks, Anatoly