From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 623E4A0351; Wed, 19 Jan 2022 22:12:31 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4B65841229; Wed, 19 Jan 2022 22:12:31 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2047.outbound.protection.outlook.com [40.107.220.47]) by mails.dpdk.org (Postfix) with ESMTP id 30BB0411F3 for ; Wed, 19 Jan 2022 22:12:29 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BXmw7XHQk9QOFGaPxNPpuO0ez6afTbVh2q369wP+o0AeP70uUcdRxdTUxFQ6P4jHpiGU63MGZGMxdQzZec/rZWGvC9kKJHDIOzqZ7JKr+JGvzQRaafGAdZr1qLW3m+8X84gAR/hyJtipBHAZhFwoZzY8JhG645aZRP1ULpR0H5RftBaFEfySddRoAqGBA4HWfn+6RaB7uEPbtDmGq89q6F5n1SD1jr240gPttoGXUr8guBk2ahM1lWPZLsKBFHshx4Aurd0+ZRY6C4cik9v9sc0ttVcF/59ikobUKgepJUf+lQTQQOnranGHeKO8BP8EVpFHzJXy2eTthJsIYrHbKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fqXq/pzhRtH8vxisk2vKmw2xIF2Nk2g81mKezra6xvw=; b=cpMGGp+FTdW2k5WqeF76zrLqJntV2sLjlxGgqnpC81iDeNZvRkW7jL+msNff5MOPGaRwCXvKGYkbzGjb9vf0Vi3xV+LJ4QwevgG1gzVkl0woQILQoJ2xyxQh8GHuIOUJlaLCQI/kBXEAHlNtIFpLRapBiwDXcZOkBMT4NmrErWCI2ZB+ubfdYjQV+l4RdZ8Z8igR6DdEnq2OJ2meONkSbus0YaoMx2mPPNKrl8VLAjfw6qeCKqGEFIT1Ca2U/5XwugECndkFk1iBe2pUkcOmf+KJrgwA3JuxuLyWIBTrJRW3ibqFd1mqalSlh1nJFdvmD7CmpcloectDxZcqYKjg2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fqXq/pzhRtH8vxisk2vKmw2xIF2Nk2g81mKezra6xvw=; b=Qo03XncjFoaLjLwM1Dg2YStB0bTY5hqL/ME2rq3HFxtS4H6RxcondJnVRcMkm0lFG9y74E5ketXRiY1mzT5+ekSDL1cfezraiPRODQSDHPpe3x9yIjFahmwLEP4vFTKkojoow3bPbVx46cpom42JjVHwXF65Ol0qAGr6Tw2aXnrdiTcvVl06tLW7eN93jSnyRJn6VIrcS1O9Og9PoZSOFpXFq8ses4EYBykvscpK1LLOqp2fom06oTc21I63cyJGcmme4YUAQotWCxO0DC7zX+PkJf7FfeV6ogzVfd5sfnjYHx50cNL2VYW0WbZx7J+3vKKbcrax+ywT3O4S7/mf7Q== Received: from BN8PR12MB2899.namprd12.prod.outlook.com (2603:10b6:408:6b::20) by CH2PR12MB4309.namprd12.prod.outlook.com (2603:10b6:610:a4::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.8; Wed, 19 Jan 2022 21:12:27 +0000 Received: from BN8PR12MB2899.namprd12.prod.outlook.com ([fe80::6447:52d7:9c7b:8e1a]) by BN8PR12MB2899.namprd12.prod.outlook.com ([fe80::6447:52d7:9c7b:8e1a%6]) with mapi id 15.20.4888.014; Wed, 19 Jan 2022 21:12:27 +0000 From: Dmitry Kozlyuk To: Bruce Richardson CC: "dev@dpdk.org" , Anatoly Burakov , Slava Ovsiienko , David Marchand , "NBU-Contact-Thomas Monjalon (EXTERNAL)" , Lior Margalit Subject: RE: [PATCH v1 0/6] Fast restart with many hugepages Thread-Topic: [PATCH v1 0/6] Fast restart with many hugepages Thread-Index: AQHYC3lrsVp4wrrpE06VyRQHDd2jCaxnav+AgAE214A= Date: Wed, 19 Jan 2022 21:12:27 +0000 Message-ID: References: <20211230143744.3550098-1-dkozlyuk@nvidia.com> <20220117080801.481568-1-dkozlyuk@nvidia.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: da8edcd1-00bf-4239-146a-08d9db906567 x-ms-traffictypediagnostic: CH2PR12MB4309:EE_ x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Uw2nIAEvYcmvNyxIydnS0XVfEoW25TM+HdvnYRCeJsDb7Xi/DR9PcsGa40nA3N9jxyenIstdkfxR/YgAcdfTh1ZcvO4NT8rX9Y99OvZoUf6/wwY/0QbrobIJ4iCyEZIqx1GHmqsmw11HrPff+bI8zux7N9XO3xwVJd/rDQjpbNKXEU4/TG7lRgxjTYNgHnW/nG4xMhOXRZrxuFka9IAVXnliJicmngR/ckd5nmEyi1TN02lmE0gOot+78slZ6xnlXsTnLBTpe6Fzp3EfuQAYJ3ZqTiCjX87LQTJLXjcKYoF3fP6f2ukIettzvcYsrJ0TzxaZJcOr3UGTMyOz6rz+iKuJ5/eaXUsDk0wBkpivCHRiA2dVVHumcmpKarOeJL4R+2CpWAMDLWjxz/drg+BGuW8oBopm5cylnQBQ3qZNuW0af50sUQRNTx8AO4/2WKnzCZ68N9sAqqqyuxc3QlLkTixdT7ZeY55fcth5tdGlYCTG2plX6a8UABUcjW8KXxxBthmT3mszbasLLfM0ErhOH4+hCf12GDbT/Km4QKCVIxRgc4HALcWXay8uYWlAe0zao8eLpjxk2msqQ0hbGpBx6hvXTWSYk2ijrdCW4l41NSEBrxRhhhyidSCCraHzeHMChlqBGPrEusk7ROBXXyB4vqFkZFHAwLHbGUMxLYVq+RWPfE7FC07dC91OMTV7Hsjd/i7QuoOicqNHRRPGdamS4g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN8PR12MB2899.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(6506007)(33656002)(7696005)(4326008)(86362001)(9686003)(6916009)(38100700002)(2906002)(54906003)(71200400001)(316002)(8936002)(107886003)(66556008)(66946007)(122000001)(66476007)(64756008)(76116006)(66446008)(38070700005)(508600001)(8676002)(83380400001)(26005)(52536014)(5660300002)(186003)(55016003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?v6Tj7li9Cl7Erhjd9cCGr3fXAjQ61G/PHaqPQLSBqy1BGLS7EfABmV5KLARk?= =?us-ascii?Q?4KimpjSAsG4SAYn7OwhjS+eYaYm/uXsVSHNp4SzmtgZJtwpWdacHcJRZOozU?= =?us-ascii?Q?fd6XkxOJEdCMifL2MDFCKM+kNkTelem3zlcqa50ybJVKMNsLcySe+FFjv8E9?= =?us-ascii?Q?QT3yPhyb4N8bX5+OspODa9xZXZdXLD5Ezv8zgZSTC+tzaX+IQMOPW/knS+NO?= =?us-ascii?Q?4lDb5kejE0cywF2zPQfEHT3yJdkRCf3XIaTzbi3anwL2L6npatfNFdkMPIe1?= =?us-ascii?Q?Db/v4jtiBqXNGvLiHrfRHWUdPYqozdpQtkqrnrPgd4X1mnESCfdq1ESxOrQ9?= =?us-ascii?Q?RlJiAIrzOtgejF+RpVwwQ2Pt92Jf5CYf3qVkIWNBTsYdT+8BGLSXHsQSDr2l?= =?us-ascii?Q?lwPuce/+unLDNZKAFrOEH9xeK9OI3x0Bcjk+CrhsDxS+NSYfNoUPw+mRqz2P?= =?us-ascii?Q?l3fxcRgjgrPkrDQS0Z6oW2LRCEFHIC6/JG3b/DgeLz8UsMfiGrrZ/Pk5Exdt?= =?us-ascii?Q?b89NtzPTf5a7EaF17I63flcQre2A+p+2woJYaaue+Lb855geCx3zMH2ZPw6X?= =?us-ascii?Q?dFNMJuy/63JYZ7V8zv1FBeBavGIbqxrd268ffHArMMqTMZCOIE9SZmjgErwL?= =?us-ascii?Q?Cso6sh+11O8GsdsPqUPhu/5mzhYACnk02KPe/fqj6jXu5pfAdNM1aax0MBtV?= =?us-ascii?Q?ZtEHXox9bqWQdzC3FJ3G78uSw2FbectKV07614GAA/+Mu4Qpn5KJTxy3vKKj?= =?us-ascii?Q?Vjs8sBbVK7fOt4VPefo037dT/Jo/9ssXAjNd9utUGO/tsGcdSzeYWKHsB2NH?= =?us-ascii?Q?eN7WIRjj6bujgEhoF5JgvfPkMpR38wY1uSc25PCHjBvqGzkgnec1VzD0a04d?= =?us-ascii?Q?2aCvy7Chi5IUF3kbomGzE15kHJOTA2L0NtYuvYmfndO5CLyNsN+Q0/WU3wXu?= =?us-ascii?Q?agJyRj/F8AQzc8XgZ27IBxXHgZtrIl9V5CmoKML+1NN/EkgviPlVaAFH/nsT?= =?us-ascii?Q?PpXqlwhVuxKi9Au8jPkjwZ1iaUQRtsJ6JiorzbuH+wbCm+zDmxx9fG3pydL1?= =?us-ascii?Q?GcFvXkkA+uX6CziRr18bsmodtULucZjQxDHn+DBsesMYCJR2j3RInRdf28/M?= =?us-ascii?Q?wEpWWOrjUhQrP0zzbqBKi8MtbgmSUhsAJTYv9OFqmP7tDpd42wXRz68/RSuU?= =?us-ascii?Q?aejTrcKy/PUJAfPkRHzlZYie1IrwesXlExNbxmeBEDWpXZczTTD+Y5Gbkp5I?= =?us-ascii?Q?v9Cg+Z9g3cI6MC7uCtjJ4e4uHZm39JqGevk6rQ+thg6W1npb29hbl8/6EqQL?= =?us-ascii?Q?csDIqnlmTcWWkUXBOfTuOT8AU4VbUzbUocVqgTDNdriZhX8xF60esOAhDzG9?= =?us-ascii?Q?tntWIZGTw2GesxkIz1Sc2ivZijczbVO0A80pQMd5EspE//NR7WEf+LsuykA+?= =?us-ascii?Q?SGGMUYrqccOJkK7zRY4VHqPrNHMgSVniPRHFnh2U3hh4VhotsjOG388xnQ7v?= =?us-ascii?Q?TbKgNJNiz4rhiqFAOpJu1IpIxVC1+R8eae94SX3rHjkqedrdIGFo6AFWMxWR?= =?us-ascii?Q?rmjHP75so+AzqqUjHudmfezwcJfSUbXbIcyN1VZHNgsgoc0inLD86SOKgOTo?= =?us-ascii?Q?6A=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB2899.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: da8edcd1-00bf-4239-146a-08d9db906567 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2022 21:12:27.1882 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: UGwyZw6n+2yEg6+saoXGCPh+9WpnzI+BjRKc4iBhgmKG3IBwjBqP4jN9njZhHVrnGobjt+Fq1MvRNpGV6OAAFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4309 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Bruce, > From: Bruce Richardson > [...] > this seems really interesting, but in the absense of TB of memory > being > used, is it easily possible to see the benefits of this work? I've > been > playing with adding large memory allocations to helloworld example and > checking the runtime. Allocating 1GB using malloc per thread seems to > show > a small (<0.5 second at most) benefit, and using a fixed 10GB > allocation > using memzone_reserve at startup shows runtimes within the margin of > error > when run with --huge-unlink=3Dexisting vs huge-unlink=3Dnever. At what > size of > memory footprint is it expected to make a clear improvement? Sorry, there was a bug in v1 that completely broke the testing. I should've double-checked after what I considered a quick rebase before sending. Version 2 can be simply tested even without modifyin the code: time sh -c 'echo quit | sudo ../_build/dpdk/app/test/dpdk-test --huge-unlink=3Dnever -m 8192 --single-file-segments --no-pci 2>/dev/null >/dev/null' With --huge-unlink=3Dexisting: real 0m1.450s user 0m0.574s sys 0m0.706s (1) With --huge-unlink=3Dnever, first run (no hugepage files to reuse): real 0m0.892s user 0m0.002s sys 0m0.718s (2) With --huge-unlink=3Dnever, second run (hugepage files left): real 0m0.210s user 0m0.010s sys 0m0.021s (3) Notice that (1) and (2) are close since there is no reuse, but (2) and (3) are differ by 0.7 seconds for 8GB, which correlates with 14 GB/sec memset() speed on this machine. Results without --single-file-segments are nearly identical.