From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 623E4A0351;
	Wed, 19 Jan 2022 22:12:31 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 4B65841229;
	Wed, 19 Jan 2022 22:12:31 +0100 (CET)
Received: from NAM11-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam11on2047.outbound.protection.outlook.com [40.107.220.47])
 by mails.dpdk.org (Postfix) with ESMTP id 30BB0411F3
 for <dev@dpdk.org>; Wed, 19 Jan 2022 22:12:29 +0100 (CET)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=BXmw7XHQk9QOFGaPxNPpuO0ez6afTbVh2q369wP+o0AeP70uUcdRxdTUxFQ6P4jHpiGU63MGZGMxdQzZec/rZWGvC9kKJHDIOzqZ7JKr+JGvzQRaafGAdZr1qLW3m+8X84gAR/hyJtipBHAZhFwoZzY8JhG645aZRP1ULpR0H5RftBaFEfySddRoAqGBA4HWfn+6RaB7uEPbtDmGq89q6F5n1SD1jr240gPttoGXUr8guBk2ahM1lWPZLsKBFHshx4Aurd0+ZRY6C4cik9v9sc0ttVcF/59ikobUKgepJUf+lQTQQOnranGHeKO8BP8EVpFHzJXy2eTthJsIYrHbKg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=fqXq/pzhRtH8vxisk2vKmw2xIF2Nk2g81mKezra6xvw=;
 b=cpMGGp+FTdW2k5WqeF76zrLqJntV2sLjlxGgqnpC81iDeNZvRkW7jL+msNff5MOPGaRwCXvKGYkbzGjb9vf0Vi3xV+LJ4QwevgG1gzVkl0woQILQoJ2xyxQh8GHuIOUJlaLCQI/kBXEAHlNtIFpLRapBiwDXcZOkBMT4NmrErWCI2ZB+ubfdYjQV+l4RdZ8Z8igR6DdEnq2OJ2meONkSbus0YaoMx2mPPNKrl8VLAjfw6qeCKqGEFIT1Ca2U/5XwugECndkFk1iBe2pUkcOmf+KJrgwA3JuxuLyWIBTrJRW3ibqFd1mqalSlh1nJFdvmD7CmpcloectDxZcqYKjg2A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none;
 dkim=none; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;
 s=selector2;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=fqXq/pzhRtH8vxisk2vKmw2xIF2Nk2g81mKezra6xvw=;
 b=Qo03XncjFoaLjLwM1Dg2YStB0bTY5hqL/ME2rq3HFxtS4H6RxcondJnVRcMkm0lFG9y74E5ketXRiY1mzT5+ekSDL1cfezraiPRODQSDHPpe3x9yIjFahmwLEP4vFTKkojoow3bPbVx46cpom42JjVHwXF65Ol0qAGr6Tw2aXnrdiTcvVl06tLW7eN93jSnyRJn6VIrcS1O9Og9PoZSOFpXFq8ses4EYBykvscpK1LLOqp2fom06oTc21I63cyJGcmme4YUAQotWCxO0DC7zX+PkJf7FfeV6ogzVfd5sfnjYHx50cNL2VYW0WbZx7J+3vKKbcrax+ywT3O4S7/mf7Q==
Received: from BN8PR12MB2899.namprd12.prod.outlook.com (2603:10b6:408:6b::20)
 by CH2PR12MB4309.namprd12.prod.outlook.com (2603:10b6:610:a4::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.8; Wed, 19 Jan
 2022 21:12:27 +0000
Received: from BN8PR12MB2899.namprd12.prod.outlook.com
 ([fe80::6447:52d7:9c7b:8e1a]) by BN8PR12MB2899.namprd12.prod.outlook.com
 ([fe80::6447:52d7:9c7b:8e1a%6]) with mapi id 15.20.4888.014; Wed, 19 Jan 2022
 21:12:27 +0000
From: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
To: Bruce Richardson <bruce.richardson@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, Anatoly Burakov
 <anatoly.burakov@intel.com>, Slava Ovsiienko <viacheslavo@nvidia.com>, David
 Marchand <david.marchand@redhat.com>, "NBU-Contact-Thomas Monjalon
 (EXTERNAL)" <thomas@monjalon.net>, Lior Margalit <lmargalit@nvidia.com>
Subject: RE: [PATCH v1 0/6] Fast restart with many hugepages
Thread-Topic: [PATCH v1 0/6] Fast restart with many hugepages
Thread-Index: AQHYC3lrsVp4wrrpE06VyRQHDd2jCaxnav+AgAE214A=
Date: Wed, 19 Jan 2022 21:12:27 +0000
Message-ID: <BN8PR12MB28994C19A2177CCC7820A11BB9599@BN8PR12MB2899.namprd12.prod.outlook.com>
References: <20211230143744.3550098-1-dkozlyuk@nvidia.com>
 <20220117080801.481568-1-dkozlyuk@nvidia.com>
 <YeWcDZ8y3voGoS1t@bricha3-MOBL.ger.corp.intel.com>
In-Reply-To: <YeWcDZ8y3voGoS1t@bricha3-MOBL.ger.corp.intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nvidia.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: da8edcd1-00bf-4239-146a-08d9db906567
x-ms-traffictypediagnostic: CH2PR12MB4309:EE_
x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
x-microsoft-antispam-prvs: <CH2PR12MB4309F247C1B3BF16F92EC4C6B9599@CH2PR12MB4309.namprd12.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: Uw2nIAEvYcmvNyxIydnS0XVfEoW25TM+HdvnYRCeJsDb7Xi/DR9PcsGa40nA3N9jxyenIstdkfxR/YgAcdfTh1ZcvO4NT8rX9Y99OvZoUf6/wwY/0QbrobIJ4iCyEZIqx1GHmqsmw11HrPff+bI8zux7N9XO3xwVJd/rDQjpbNKXEU4/TG7lRgxjTYNgHnW/nG4xMhOXRZrxuFka9IAVXnliJicmngR/ckd5nmEyi1TN02lmE0gOot+78slZ6xnlXsTnLBTpe6Fzp3EfuQAYJ3ZqTiCjX87LQTJLXjcKYoF3fP6f2ukIettzvcYsrJ0TzxaZJcOr3UGTMyOz6rz+iKuJ5/eaXUsDk0wBkpivCHRiA2dVVHumcmpKarOeJL4R+2CpWAMDLWjxz/drg+BGuW8oBopm5cylnQBQ3qZNuW0af50sUQRNTx8AO4/2WKnzCZ68N9sAqqqyuxc3QlLkTixdT7ZeY55fcth5tdGlYCTG2plX6a8UABUcjW8KXxxBthmT3mszbasLLfM0ErhOH4+hCf12GDbT/Km4QKCVIxRgc4HALcWXay8uYWlAe0zao8eLpjxk2msqQ0hbGpBx6hvXTWSYk2ijrdCW4l41NSEBrxRhhhyidSCCraHzeHMChlqBGPrEusk7ROBXXyB4vqFkZFHAwLHbGUMxLYVq+RWPfE7FC07dC91OMTV7Hsjd/i7QuoOicqNHRRPGdamS4g==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN8PR12MB2899.namprd12.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(366004)(6506007)(33656002)(7696005)(4326008)(86362001)(9686003)(6916009)(38100700002)(2906002)(54906003)(71200400001)(316002)(8936002)(107886003)(66556008)(66946007)(122000001)(66476007)(64756008)(76116006)(66446008)(38070700005)(508600001)(8676002)(83380400001)(26005)(52536014)(5660300002)(186003)(55016003);
 DIR:OUT; SFP:1101; 
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?v6Tj7li9Cl7Erhjd9cCGr3fXAjQ61G/PHaqPQLSBqy1BGLS7EfABmV5KLARk?=
 =?us-ascii?Q?4KimpjSAsG4SAYn7OwhjS+eYaYm/uXsVSHNp4SzmtgZJtwpWdacHcJRZOozU?=
 =?us-ascii?Q?fd6XkxOJEdCMifL2MDFCKM+kNkTelem3zlcqa50ybJVKMNsLcySe+FFjv8E9?=
 =?us-ascii?Q?QT3yPhyb4N8bX5+OspODa9xZXZdXLD5Ezv8zgZSTC+tzaX+IQMOPW/knS+NO?=
 =?us-ascii?Q?4lDb5kejE0cywF2zPQfEHT3yJdkRCf3XIaTzbi3anwL2L6npatfNFdkMPIe1?=
 =?us-ascii?Q?Db/v4jtiBqXNGvLiHrfRHWUdPYqozdpQtkqrnrPgd4X1mnESCfdq1ESxOrQ9?=
 =?us-ascii?Q?RlJiAIrzOtgejF+RpVwwQ2Pt92Jf5CYf3qVkIWNBTsYdT+8BGLSXHsQSDr2l?=
 =?us-ascii?Q?lwPuce/+unLDNZKAFrOEH9xeK9OI3x0Bcjk+CrhsDxS+NSYfNoUPw+mRqz2P?=
 =?us-ascii?Q?l3fxcRgjgrPkrDQS0Z6oW2LRCEFHIC6/JG3b/DgeLz8UsMfiGrrZ/Pk5Exdt?=
 =?us-ascii?Q?b89NtzPTf5a7EaF17I63flcQre2A+p+2woJYaaue+Lb855geCx3zMH2ZPw6X?=
 =?us-ascii?Q?dFNMJuy/63JYZ7V8zv1FBeBavGIbqxrd268ffHArMMqTMZCOIE9SZmjgErwL?=
 =?us-ascii?Q?Cso6sh+11O8GsdsPqUPhu/5mzhYACnk02KPe/fqj6jXu5pfAdNM1aax0MBtV?=
 =?us-ascii?Q?ZtEHXox9bqWQdzC3FJ3G78uSw2FbectKV07614GAA/+Mu4Qpn5KJTxy3vKKj?=
 =?us-ascii?Q?Vjs8sBbVK7fOt4VPefo037dT/Jo/9ssXAjNd9utUGO/tsGcdSzeYWKHsB2NH?=
 =?us-ascii?Q?eN7WIRjj6bujgEhoF5JgvfPkMpR38wY1uSc25PCHjBvqGzkgnec1VzD0a04d?=
 =?us-ascii?Q?2aCvy7Chi5IUF3kbomGzE15kHJOTA2L0NtYuvYmfndO5CLyNsN+Q0/WU3wXu?=
 =?us-ascii?Q?agJyRj/F8AQzc8XgZ27IBxXHgZtrIl9V5CmoKML+1NN/EkgviPlVaAFH/nsT?=
 =?us-ascii?Q?PpXqlwhVuxKi9Au8jPkjwZ1iaUQRtsJ6JiorzbuH+wbCm+zDmxx9fG3pydL1?=
 =?us-ascii?Q?GcFvXkkA+uX6CziRr18bsmodtULucZjQxDHn+DBsesMYCJR2j3RInRdf28/M?=
 =?us-ascii?Q?wEpWWOrjUhQrP0zzbqBKi8MtbgmSUhsAJTYv9OFqmP7tDpd42wXRz68/RSuU?=
 =?us-ascii?Q?aejTrcKy/PUJAfPkRHzlZYie1IrwesXlExNbxmeBEDWpXZczTTD+Y5Gbkp5I?=
 =?us-ascii?Q?v9Cg+Z9g3cI6MC7uCtjJ4e4uHZm39JqGevk6rQ+thg6W1npb29hbl8/6EqQL?=
 =?us-ascii?Q?csDIqnlmTcWWkUXBOfTuOT8AU4VbUzbUocVqgTDNdriZhX8xF60esOAhDzG9?=
 =?us-ascii?Q?tntWIZGTw2GesxkIz1Sc2ivZijczbVO0A80pQMd5EspE//NR7WEf+LsuykA+?=
 =?us-ascii?Q?SGGMUYrqccOJkK7zRY4VHqPrNHMgSVniPRHFnh2U3hh4VhotsjOG388xnQ7v?=
 =?us-ascii?Q?TbKgNJNiz4rhiqFAOpJu1IpIxVC1+R8eae94SX3rHjkqedrdIGFo6AFWMxWR?=
 =?us-ascii?Q?rmjHP75so+AzqqUjHudmfezwcJfSUbXbIcyN1VZHNgsgoc0inLD86SOKgOTo?=
 =?us-ascii?Q?6A=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB2899.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: da8edcd1-00bf-4239-146a-08d9db906567
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2022 21:12:27.1882 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: UGwyZw6n+2yEg6+saoXGCPh+9WpnzI+BjRKc4iBhgmKG3IBwjBqP4jN9njZhHVrnGobjt+Fq1MvRNpGV6OAAFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4309
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

Hi Bruce,

> From: Bruce Richardson <bruce.richardson@intel.com>
> [...]
> this seems really interesting, but in the absense of TB of memory
> being
> used, is it easily possible to see the benefits of this work? I've
> been
> playing with adding large memory allocations to helloworld example and
> checking the runtime. Allocating 1GB using malloc per thread seems to
> show
> a small (<0.5 second at most) benefit, and using a fixed 10GB
> allocation
> using memzone_reserve at startup shows runtimes within the margin of
> error
> when run with --huge-unlink=3Dexisting vs huge-unlink=3Dnever. At what
> size of
> memory footprint is it expected to make a clear improvement?

Sorry, there was a bug in v1 that completely broke the testing.
I should've double-checked
after what I considered a quick rebase before sending.
Version 2 can be simply tested even without modifyin the code:

time sh -c 'echo quit | sudo ../_build/dpdk/app/test/dpdk-test
	--huge-unlink=3Dnever -m 8192 --single-file-segments --no-pci
	2>/dev/null >/dev/null'

With --huge-unlink=3Dexisting:
real    0m1.450s
user    0m0.574s
sys     0m0.706s	(1)

With --huge-unlink=3Dnever, first run (no hugepage files to reuse):
real    0m0.892s
user    0m0.002s
sys     0m0.718s	(2)

With --huge-unlink=3Dnever, second run (hugepage files left):
real    0m0.210s
user    0m0.010s
sys     0m0.021s	(3)

Notice that (1) and (2) are close since there is no reuse,
but (2) and (3) are differ by 0.7 seconds for 8GB,
which correlates with 14 GB/sec memset() speed on this machine.
Results without --single-file-segments are nearly identical.