From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FBFCA0C53; Wed, 25 Aug 2021 13:47:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E745E40041; Wed, 25 Aug 2021 13:47:32 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id D95864003D for ; Wed, 25 Aug 2021 13:47:30 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10086"; a="278520206" X-IronPort-AV: E=Sophos;i="5.84,350,1620716400"; d="scan'208";a="278520206" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2021 04:47:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,350,1620716400"; d="scan'208";a="473819154" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by orsmga008.jf.intel.com with ESMTP; 25 Aug 2021 04:47:29 -0700 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Wed, 25 Aug 2021 04:47:29 -0700 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Wed, 25 Aug 2021 04:47:28 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10 via Frontend Transport; Wed, 25 Aug 2021 04:47:27 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.175) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.10; Wed, 25 Aug 2021 04:47:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WlFNm6VBp0GQXmkDxp0LdPDoEP2zfjP5oflU6ABDNDhEn4y/zeWFl4JIpSSfkrra4lv+eIdGdS3Qx3jAXyE6BwSKGQoXDuYZNETxhU4iloaVejqZK+XnAAX0HPBDc58WjdKUk6+NDB7bAGSjez8Bzu5DIZ0VPGF2Oo4lfCwjd/kmx9dm1A4blFSEQUjc4OVCbfVVRdp4fVc0YlQtG6QmhTLi1Q7vlVp8eR+QWkkVVT1VYUasQ9RWe/cIF4sZcnmojYSCOxN43jd7xEC9W4WxwUDMhnyp5EbPdBE6brCDrjzRCXvfM5x4taNWg3HKibgjzM2u+9pd7L5N+J/1uSC8xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bOCgcYNAVcJ1Wa6u+H8Z1XXvgJe2fL+RzPEjFcUcf+I=; b=AzMXAbULrEI22vBuAMFCevbzWGJgio1tdn76YSTMVHGsSn31cLa4bElA6ddo8c5DymnT8+0aG6R+z+kkrd6kNcZZYqfUnQIwz0RJB22eMN0pX5FzuAM/9JfOdzfDk0SbAsAhybtIfXnw1mPxEktJ+deob9AjX9TC+EUchaIxIySh+B9lU/cgg10Ja0py0MQLO21y/aV9KiUKdjnlvB2LwYPNtSO4W5S05GQjK36IznyZT/50rB7aumVykyJKFGTiBV0L/zk9Cp7aCd8eChLE7ZOGwU7j0K/OmmIp/vu7T7G1CqedjYU9Siybdq9VphOT9REwvBSiGczIZa5DvN35XQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bOCgcYNAVcJ1Wa6u+H8Z1XXvgJe2fL+RzPEjFcUcf+I=; b=bI3busJPRkpghLb21cLlwYsaO90kcqBVlQcn0Kgk1u4ND2qC78BcAB9UB/sXJNdmdGkt9LFxXSDNXcK5WKpUEFFcRHWq8KfP6lI7gGPqhAJ3VHZmlm9l4T4chAf0o3gCBxIwEa8p4QyXpdaNBVXFdNGa52ABr5viM5cnnZbSN58= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) by PH0PR11MB4997.namprd11.prod.outlook.com (2603:10b6:510:31::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19; Wed, 25 Aug 2021 11:47:25 +0000 Received: from PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::5193:dc97:b903:aaaa]) by PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::5193:dc97:b903:aaaa%5]) with mapi id 15.20.4436.027; Wed, 25 Aug 2021 11:47:25 +0000 To: Thomas Monjalon , Dmitry Kozlyuk CC: "dev@dpdk.org" , "Xueming(Steven) Li" , , References: <1730403.MaGYrl2sIR@thomas> From: "Burakov, Anatoly" Message-ID: <915867f6-f558-e2be-0c19-d52b313d032e@intel.com> Date: Wed, 25 Aug 2021 12:47:19 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.13.0 In-Reply-To: <1730403.MaGYrl2sIR@thomas> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: DB8PR09CA0023.eurprd09.prod.outlook.com (2603:10a6:10:a0::36) To PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.1.12] (212.17.34.161) by DB8PR09CA0023.eurprd09.prod.outlook.com (2603:10a6:10:a0::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.18 via Frontend Transport; Wed, 25 Aug 2021 11:47:24 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 85968af8-a27a-4b91-b541-08d967be1b62 X-MS-TrafficTypeDiagnostic: PH0PR11MB4997: X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +MRG47h4UysXp4TEIFFyaN9nKBCqGqh/NBVidVnxbDlMhFWpvXitrWuQUOXF6G5yIg2hNqBnyYmHkvaKw2a4+eaZPlHcGmDXebVhvBoenHpavire4y8V2oKt4xFOS72Tv1dRfJKA+Ij6DT572pkBQXHlCtPaDlIQUng3JN4dYuaaWfqoe4xC0dHA5lnehkTYKtPa9ly3DXTBp0RAzy65mdc/psvizrluPn8QuhvlwfvgKeCpHkjTd4aSmtcoYiNBKQqDkcDlxehF/I1yaP0E58W4RRrUyzm99j8GyBOQODFVJ86tTrHLFQNMl3gMmjHqsYzZq0RSG83ymUJB4CYstjpVjSmxAv5rKk/YVeUfv8/iF0SGxGSQlJS4NG5pJxBut5dXksjXCRom9TYsyJMZeqaFUId0x/Rnp/0XbmqNnRZvEG7//wdk2fA64TipABWNh3ll50RIyIX13cAHtYqhPOdVVPHSVBLO3AI1HAQtgDKXZDqaA1rPqLERavRdXkXDJ+x5tpVYI/wgSQNKut1F7JhzjRFMTGUTh02jjDtoiJEnO6VHCofxA0zd6mvRCvEtd7vbKAYQEaKB7YzWNKrmT34mG1+/v+iAR8G8w+9rMdyzx9ok9mpQtHtDPwvYIgO9vm/EbfV+6BLNnEMATzWYphv6H/DU9OKmioaUtU16SjI6ViX9yZhYz1wxpTdEA91rh71QIUZ/s8JezVrWwMVVwNUxoxLe8FtaUDVhvNdmm1s= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5093.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(39860400002)(396003)(136003)(376002)(16576012)(66946007)(4326008)(107886003)(66476007)(316002)(6666004)(31686004)(54906003)(110136005)(66556008)(5660300002)(83380400001)(2616005)(2906002)(956004)(8676002)(6486002)(478600001)(26005)(53546011)(31696002)(8936002)(38100700002)(36756003)(86362001)(186003)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TVhsaFFMU1NQVmgzZnBWVS9QdFh2cTVvUnFmZ0JLazk5clB4N0ZKWW9aUW0x?= =?utf-8?B?Vm40ZDJGT3l2R1ZiMGtyKzZlWE03TGZEejNUK2NjMHNqVW5XU1I2Yitxd3BU?= =?utf-8?B?dHJOWTh1OFZ1cThPUWZidENMRHZkdTdud1pOTTNYSmt4VzIrcmUrSW1WZURB?= =?utf-8?B?UnphblM4NGVpaEVEQjJoNFRpYUFxZjVpVldzeUJXeVh4eEl0THZoMnBvVXdM?= =?utf-8?B?aGk3ODI2WkthbnIrU1dYT1ZRMEVHamxiSElHaDdkbGNZd0EzMmN4TWFiQmJN?= =?utf-8?B?WDQ0MWsvL0VVUERkYmxITFU2ZkZCblJWUERVMHgvRWx2SmlTSlZWTFIzNFpu?= =?utf-8?B?c09WUDVQd1M1Y0hEWU05SXY5U0NJbjNtcEo2cmlNMzVyY2NTdFdRMURiS29G?= =?utf-8?B?L09DSVZqYzczbk0wK0IwSFcvV2dtRFRCVWRIdjhTMTNHMjY4TnN2YUFwU040?= =?utf-8?B?K2hDQTlPWFlWelVYZkt1RGlwaXZIK2EycVNvMnJFNlMwOGkvb0luSTZtSlRq?= =?utf-8?B?U3pMTjBGQ3JLWjlBSklXaWRnNlVVdkRZNUZqeEYyOG1EU1NsRWlNc0xkNnky?= =?utf-8?B?K1FTQ1ZuTTV5M0dDS1FXK1BYdzBlQnVkYWQvTVY4NENPM2JjQm5KNmgveWJ3?= =?utf-8?B?dnNhSjlRWm1QUDVlMzl1VWJGaG1LaStEM0VXQ1QzRXhMUHJOYUVDVmc4YjhW?= =?utf-8?B?TDdXQmloZWdEVmxPL0V0R0p2WERtRzE0RXlnVkU1SmVLcFhMUUcxbDNOazY1?= =?utf-8?B?RGVpaCthK2tWZVluZGZKRnE3bWJnVSt1azB5UVF6Q0l4cG5yZU56YXRRdGt5?= =?utf-8?B?U1pGa1B2ZW5HdkFnbFRQT3hCVVlCUWt2QUlDUXEwZDhjUmppdHZaQXRlMUJF?= =?utf-8?B?MkdpcG9OTGVod0hpUGJrcDBtZE00d00za3ZyQzFCOXlrbGNDZ1hjQXFvZWlP?= =?utf-8?B?ZDhNaURIdjY0c0JpcWpVZS8wd0FtK2YyME8zRndsSW9sN1pYNldzamVBaTFC?= =?utf-8?B?dDVKWWJMNTZaUUJLbWx3M05NSXZEa1dONDRNM1h0dm5DbG16amVwRGdmd2Mv?= =?utf-8?B?cVhPNGVBbDgweUkxY2FVdnNoRzhEM1VVWVhlQzRrK0h5cytaMnJlYTNqYnY1?= =?utf-8?B?ZGlkVG9xRWV1bEFpdE1abEVZcVpTS1lRL1pXM1pEOE9UWmR2RnpYVFZQbTBn?= =?utf-8?B?NnhaenJPUXk2THEvNmx0S2F6aU1kTnZWWTFJZVlhdEtVR0s4SXZtakt2MHhZ?= =?utf-8?B?eWpzU01zeU5VUTdsMXVGa2ZCZXVwQWtUTWFmV09FNmFpRUlzb2hzMEJSM2lO?= =?utf-8?B?THB0ZnRoWkpvajVONXB5dE1XY3NCbjdKMktvQkxvNkhrU3lzcXp5a25HTXhx?= =?utf-8?B?Kzk0Z2ZNQ00xMElGMXVKQXYvV3dEdFdIUDhNdEpxSDR4VjdGdCszREZtY3Bt?= =?utf-8?B?dmg4em1NOXNFcFZyd3pxZWk0cGRQVCtYSnhiYWhha2NYL2g3TDZkc0RtRlY3?= =?utf-8?B?NCtseDBpZlNJbXdrcVdmZW8wbTdKUitqQk84ME44S2ZpUXNsSm40Z3hOTkwx?= =?utf-8?B?RVNVbVdXbHFwa0pCbnpqcHNCQlJBeU1uRzVWRkhJRlNhRWgvRzlpNlFhN21z?= =?utf-8?B?MVFuZ0kvYmhwYTMyMGRvOWdwempxTjJkOVFMM2dOeXRYYTVPd3pvYzFXS2h2?= =?utf-8?B?d3Y0aHd4WVEvK0w2eTV1L1d2ak44dlV1VG9Zbzh1NzBzbnNBb0w3ZzFHQW50?= =?utf-8?Q?rVb5oV6jb1MvM2vX0KklHzjBqp7HKI5h3dhpiEO?= X-MS-Exchange-CrossTenant-Network-Message-Id: 85968af8-a27a-4b91-b541-08d967be1b62 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5093.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2021 11:47:25.3633 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bx41+h2guhQSN9ihY3mfmkTObYcxJBSH9gbaxzo15qOeOf25PcRlPee7ltCscpCCKK/joinCfP6DCRdTpA3pwKbRRpSP9jXUJwTQGQIhLgY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4997 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] Zeroing out memory on free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 25-Aug-21 8:26 AM, Thomas Monjalon wrote: > 24/08/2021 12:58, Dmitry Kozlyuk: >> Hello, >> >> Me and Xueming are wondering why DPDK clears the memory on free >> and not only when it's explicitly requested (rte_zmalloc). >> >> It's been so for a while: >> >> commit ea0bddbd14e68fb42d9774bc3543e51b510e48d3 >> Author: Sergio Gonzalez Monroy >> Date: Tue Jul 5 12:01:15 2016 +0100 >> >> mem: zero out memory on free >> >> Since commit fafcc11985a2, memzones are not guaranteed to be zeroed out. >> This could potentially cause issues as applications might have been >> relying on the allocated memory being zeroed out. >> >> On init all allocated memory is zeroed by the kernel, so by zeroing out >> memory on free, all available dpdk memory is always zeroed. >> >> Fixes: fafcc11985a2 ("mem: rework memzone to be allocated by malloc") >> >> Signed-off-by: Sergio Gonzalez Monroy >> >> At present this explanation doesn't look satisfying: >> 1. Memzone API does not promise they will be zeroed out. >> Memzones are mostly used for DMA, so their content will likely be overwritten. >> 2. If application assumptions are wrong, DPDK should not try to fix it. >> Memory manager poisons memory in debug mode to detect such errors. >> >> Zeroing memory is quite slow, but in many cases unneeded. >> It looks attractive to only do it in rte_zmalloc(). >> AFAIK what prohibits moving memset() there unconditionally >> is that the kernel already gives us cleared pages, >> so the first allocation of each piece of memory would do unnecessary work. >> This can be solved by giving the user a choice in EAL options > > No I don't think it should be workarounded in EAL options. > >> or with more elaborate tracking of memory dirtiness in MM. > > Looks a good idea. > >> Are there any other reasons why clearing is done the current way? > > I think the only reason was to avoid doing reset for the first usage. > You're right it is not optimal when reusing memory > without any zeroing need. > You said it yourself: zeroing out memory is quite slow. This was pretty much done for performance reasons - memory is released way less often than allocated, and we get memory from the kernel that is *always* zeroed out in case of hugepages, so we don't need to zero them out initially. So, in essence, this allows us to always keeps memory zeroed out without worrying about doing that for every allocation. -- Thanks, Anatoly