From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EF63DA04FF; Wed, 27 Apr 2022 17:50:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9D93040E78; Wed, 27 Apr 2022 17:50:11 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 2751040691; Wed, 27 Apr 2022 17:50:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1651074610; x=1682610610; h=message-id:date:subject:from:to:cc:references: in-reply-to:content-transfer-encoding:mime-version; bh=7vOYXzlH4+tlLlTxXLqhw5+IW43LbJLVIPMDqwo4H0s=; b=PFvIriSnQkSqlPxAIlFgZZs+wisz33rLZTQWnxfELUvD7pVkFVuMUBQE a9AurJXz9bniqCdG1ncmy0LHq6HPSuzOK/X0mAVPJluqtci85I/yKxW6x Yg1gBG0VbRyFJcZtg9LBowCIbI8q/gVH5JwDvoIwhDs1CGyf1P8zqQI6X eaAMCj2a+ohSfd1aRYQ6Sakwfd6JqiulkNzuqUY7KLsTV/U6LPZgU/CFD TraUTd3hOgHvVR2TwciSaMrvRSwfvNW89nMgh0vW5P1WfaxvNi/8nLXwx ZRo/JHXozm4Qn8COnaiv2JeURYWTU0XW54x/KBBDdUnr3fzEcMMFV+3Ff A==; X-IronPort-AV: E=McAfee;i="6400,9594,10330"; a="245891408" X-IronPort-AV: E=Sophos;i="5.90,293,1643702400"; d="scan'208";a="245891408" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2022 08:32:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,293,1643702400"; d="scan'208";a="730857132" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by orsmga005.jf.intel.com with ESMTP; 27 Apr 2022 08:32:12 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 27 Apr 2022 08:32:11 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 27 Apr 2022 08:32:11 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 27 Apr 2022 08:32:11 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.107) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 27 Apr 2022 08:32:10 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Bu4mM8fHWYUX8xagnqlaN4ZaoEAm/q02rXKqCMbRAUhYLyTDlm/1avw9WyeOYOqcqgKw2pItnin8NkLpLE4yfgx7EighBsYjCACZnAoRaz41ImSwnV61Q3eNfyY5F/sjIz5Qf8hczoGmBdhIVvYyXZ4cAiUTzNY3ARusPIr/JYuBOpaBWQt9sHRM9ejudn+wsllsG+otCyaiLrzFmoDkmwcIBApOrUSAUwhCi2OEiecD0ejgGT5VZcDe3msAq2w4mQU3dEBPVznNjSK3OdYm7s95ZC3TdLqwuNMDue6wxPGlJyOdYmCuYChnxw8ieEbKA4593HlJ2asTpSu3cjIpFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WkSD40BzI3/ER7lRb4Li6gAovAYWRBSPh7NOP9XF7yQ=; b=I34DcZHBjDEppuoCidfp7Gw5u+vuNoAwyMEMPmCDJxlaAERsLRi3vFOSpDKEtaucWd5BXcR+rEFcUhgtAGbDEj1A7nLnx+zjL/pwUakFN/araDr/J4mCzslo2MHO6TOoSeZwiTbvWpNF/V9qj8oDicVhzqtMHVkFNF7b7asTPKRa9vBQu3HEC+xcnZ+/cZTnNjFL8Wqo3CAn5fTfkkmWVQi1/ibMKzCiw1q0BLzdhkmwJE6hWKpjxjtVDyu5PGXHrGYLCZM417SOSjXQHOBstF4vZHjh+Hlg1kb/d7P2IliqwrRVGvvNpR+BbGNd8KMZt2HxDOk2w94POsgcAsTt7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) by SN6PR11MB2733.namprd11.prod.outlook.com (2603:10b6:805:58::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.12; Wed, 27 Apr 2022 15:32:09 +0000 Received: from PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::28af:8ec5:1817:1af1]) by PH0PR11MB5093.namprd11.prod.outlook.com ([fe80::28af:8ec5:1817:1af1%3]) with mapi id 15.20.5186.021; Wed, 27 Apr 2022 15:32:09 +0000 Message-ID: <3d6e93d8-e9f6-6d69-c542-4e274e410a58@intel.com> Date: Wed, 27 Apr 2022 16:32:03 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.8.1 Subject: Re: [PATCH 2/3] mem: fix ASan shadow for remapped memory segments Content-Language: en-US From: "Burakov, Anatoly" To: David Marchand CC: dev , "Mcnamara, John" , "Dmitry Kozlyuk" , dpdk stable , "Xueqin Lin" References: <20220415173127.3838-1-david.marchand@redhat.com> <20220415173127.3838-3-david.marchand@redhat.com> <419bb7fc-cb04-10cf-a40a-5dba39323f9e@intel.com> <249b5c3e-513b-af96-399a-01d60ff93d26@intel.com> <36f82439-a251-d095-ca9b-c14fe4a32430@intel.com> In-Reply-To: <36f82439-a251-d095-ca9b-c14fe4a32430@intel.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO2P123CA0034.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::22) To PH0PR11MB5093.namprd11.prod.outlook.com (2603:10b6:510:3e::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d63eb5d6-f2a2-4554-2ee4-08da28631790 X-MS-TrafficTypeDiagnostic: SN6PR11MB2733:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SX1wgRRsj9McU8kX8O2ijCIdA/hhcXVlAmTjd1dP2MNkkrfXdeK7KyGdJdJchBe1x7SzyllxjsRga2b6Jjbk3DpEKACPyLeI+6rgLKrZf5e5cMSwaAN94KZr75QwdyebdaDegbt0uXQb2K41LwqhI+ePTAADkP8vpxNfzsQoI9BjYYc+yGCCueaiT8V9HuWRHRNF2LIG7P40QcVRGWUeI6VWBmz1AxI6k3Ur+LkuSsMXTn3TkKNCbxrpid98hWRDK+JfrqJqcNp8Ewa3qr+t4DG8uznhzs+QVXPI/LCeKtin9Xvn+RCfq/NLsXB/2pS/ks13eL0ik9GTg3AUA1W0MYB6WlkaYVNlLmt2njqRx0eUiI1NbP8FJLeHzH3kUcUtk15jpoUjoeVV6MRtGHTLzI/HXgr7qpaQBlFFMaOmHJH8AuI8fUWR0Mk0E1+aK+SwOObekJVV9YNi+UyTcxDfr2cphJpCNCGQZQC/NHuHK1hd4wwHPT68tZ46n3CMottWDmfSuHEwWPBT4sgThJrBMv5HgKddB+1jeBUMrEkwnoqeHIYIRb2fkjV5giigeZKXVyxsdWIo9UV58PaKFJmo0mEo/YSiF5gOkRbRUW6bsnIX+HBDkwfUeT9KJJbftj+vX4PBGCIplu6a62kneSOAI/3fUFwh1zcqfexPwBcAbQXK8uSR2ABht7R5fUo3+w/FwFVv+CikI10DzI6dND4j9LJsiFMgYbOOwkWRxSK9cH4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5093.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6486002)(508600001)(83380400001)(107886003)(2616005)(31696002)(86362001)(6512007)(53546011)(26005)(6506007)(6666004)(82960400001)(38100700002)(186003)(316002)(2906002)(66946007)(66556008)(5660300002)(36756003)(6916009)(54906003)(31686004)(8936002)(66476007)(4326008)(8676002)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SUhqQmgyMjJxd0VYRnFQOWVTMGJPUGRpd010dFl4UktHOG9Pa3g2THdiNjFw?= =?utf-8?B?UjdtUHNrVzZzck43aFB2YVNvaFlZSmZIcnJSMndmUlB3NTI3ZWJ1aFEwUUtY?= =?utf-8?B?S2RCY01vbVJsUDUrekJreWdxZnk0S3MvOTVrTkRDMXl6eVFxVXRrK0ZOQlFw?= =?utf-8?B?eVoxREczM1VGSGZTMGZXV2N0b1JmMW1RdEpyb214OWFJQnN5T0dlVU5WSzBG?= =?utf-8?B?QkpOaVZPK3o2SWpvVjN4dkhrdStMK1hDdGJVQ1ZWZmcweEdZNmRTTXVNTEpU?= =?utf-8?B?eFlKaEgwRXBtdjNHeFQwZUhkbkZqMm5Jdi9aZzRiaTRpR0cwNG9JWWxzMEtj?= =?utf-8?B?Q2tEb2pQN0Z4N2loTEtueDd4WWcvenYyNlZ4cWRKcEVjcEd0NnFLdWNaNGVv?= =?utf-8?B?RTFIN0twL215eDBqZEZ6TWxNa1pBbC9XSnNOYTdHYU9uWW53QlUwQjZDTmFt?= =?utf-8?B?cTVNOHc0SDU3NHF1RFhWYVFGY2xVVGw1Q2JqVE11b3NwT2poazd0UVVzNWsw?= =?utf-8?B?MDJvQUs0YkRCVmdHa1A4bFo0LzlnWUVmd3JSUlNkWmRVbHl1ek84ZW9HMy8r?= =?utf-8?B?ZDdDbGxaOVJMM2NWRVRzSkdsVkh4SVU0M202SGFyd2svRDM1eEw2SWxmS3BZ?= =?utf-8?B?eGdaZmhUamhSb28wR2pSNFZkaVl4NnQ2Z2NLdnZoWkZsM1ZQeHRIdmdXQURp?= =?utf-8?B?Y2RBNm44TXcxbHd1Y3ZqeFVZS2I3Z001cElaMS85K3l0ZWIrQlNpbnAyK2VK?= =?utf-8?B?NVpmNkxORWNZK3BpVUt2NjEyR1pSOFlxN1lGbGRrZVFUMVJNS0tyeUZiM1hv?= =?utf-8?B?ekR5OWNMUDZ1aTliQ1JSek04SnRjdHRzN2o4KzZGUDF1UXg0L3hRNnFDVCtr?= =?utf-8?B?SUFUTUtNR01BSVovbENWYXBMN3htZVZjSEJtaUNNZmRXSXMyMSs3d3FQZkEr?= =?utf-8?B?NlBSMkN2MUgyVjQ2UVo5dWdpYnBMb2NlU3NIVkZIQW95U1pvN1ZxUnJCNzJw?= =?utf-8?B?aUdjQ3JGTlFsYWVFcGJ3N2E2aEZUT3pNbUVpVG9yTTAxTzdiY0FPRFAxaVkv?= =?utf-8?B?VjdXRUF1VzB1bGpZeTdmVkZIdWQ5dGtMeDFoZ2pvN0JoQTF3TXNOMXY3YlNi?= =?utf-8?B?cTNMV2JUTFdudHJoZDlVL0cyWjVjaFlneUJuMlFRUGQxT0RCYVFJWUJ5T0ZQ?= =?utf-8?B?VE9kU0RSM0hhQkZkWEV5SVJuRGRuUnRUVVQzMVZ0QTBrczhjVVpxZkNDU2Zn?= =?utf-8?B?VS9hY25pa3pEUXc3TnJpWFV2bEJ4dUJrdG14OFhUU0s2d1BNb1hPdnNDdm1U?= =?utf-8?B?dHVBM0hlN2ZML0VKYkFtemtMUlJha2ovcnFWNFJ6OHBKclBUYlJuaUtRdm8v?= =?utf-8?B?ckFEdktoRklEUGU3MURxM3NFWlhTbTNXSGJwdHQyNnZCQWhBNFRudUtFTlUx?= =?utf-8?B?VmxHNzJBN3Y1QlJNbDVueVJ2U01PYXgxUFo2bmxPQUVFVmpkZlZtOTJGTGNG?= =?utf-8?B?ZGxmemxlQ1gxU2RCNG94dEtoQmhqVUNyc2ZxU3ByQWwveXNxWHg4dlI5SUJL?= =?utf-8?B?NG1sSi9sTzJsZVk2bEhNbnByUW1KS3NXZ3NOYzY2MWVFSzVOQjUvZVdETW56?= =?utf-8?B?SnBRZThXQTFBQWQxVkhjL2k2KzhTQURwUU9CMFI4cnBrTHNoSVJHV1pXSVJq?= =?utf-8?B?SjJIOGJDdDM1dHkyS29sdE9VVExDUTF3QlUxYTlMeGRKUGJtWVVNK2VpYkFl?= =?utf-8?B?OWoxUThnbk1wUk9RV21tVnExN1lmTzlWVU9FNzkxTmZjWWx5cytrQnRZTW1x?= =?utf-8?B?c1pxUUVTSm04anlOc3lBcmNyMUZuN3JtUzhOb0cvQ3JoSFV4ZHp0RXhJTzBQ?= =?utf-8?B?UE5rcHJHZWxzdWI1MjI1VWgyYlRYdDZvU1BpQm5IdVFLbFB3RTZaOUV3VXlw?= =?utf-8?B?VCtzaG5nemU5RGN6cVduQmxuLzlmYWwrZ0VWSGdiNno4RGNaVFg1aGNHUnpZ?= =?utf-8?B?aWJ2U0txUG4wNEtib0xRUno3eEYvRHE5Z1pFS2dsV0pqbGVlbWhPSjNBdnJP?= =?utf-8?B?TWxwY21LamEwR0M1cC9zNU1oQ01xOTljVGUxb3ZZS3lOblFiWG5LNCtVdnE3?= =?utf-8?B?VkMrSllRdWlIMW5wcHRIR05PckNBSlR3Vld2YmRuOXV2UUs1dTliMWNIbHBM?= =?utf-8?B?V1VhSk9TY044WnlLTmZCRFJFNDFXRm1lUkZRdVJXZXZBdk55RUdmZkEzYTc0?= =?utf-8?B?dG1wVnpHOHVQQkNEcEx1NThhVXhRM2xwZHlDYndVT01GalEwaW1ldmpZL3Bw?= =?utf-8?B?TE42ZHdmbmJWb3BIQkc0ZHhqYTdjVmVCTEh0c2w5WUFEemo5djdPaXVjbk8z?= =?utf-8?Q?Z3LUtdeWTV6s6Fjg=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: d63eb5d6-f2a2-4554-2ee4-08da28631790 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5093.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2022 15:32:09.1513 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SUAcXRgugJwucIvmWM17VpHkGw543PE7WMhl7r01DjcSQ3Lx5yW5j5Fxd1NlwrXHpWTWs8HoE+xBPfZlS+4CZ/0QtZYznRvfAgj2S6/HsQ0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2733 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 26-Apr-22 5:07 PM, Burakov, Anatoly wrote: > On 26-Apr-22 3:15 PM, David Marchand wrote: >> On Tue, Apr 26, 2022 at 2:54 PM Burakov, Anatoly >> wrote: >>>>> @@ -1040,9 +1040,25 @@ malloc_heap_free(struct malloc_elem *elem) >>>>> >>>>>           rte_mcfg_mem_write_unlock(); >>>>>    free_unlock: >>>>> -       /* Poison memory range if belonging to some still mapped >>>>> pages. */ >>>>> -       if (!unmapped_pages) >>>>> +       if (!unmapped_pages) { >>>>>                   asan_set_freezone(asan_ptr, asan_data_len); >>>>> +       } else { >>>>> +               /* >>>>> +                * We may be in a situation where we unmapped pages >>>>> like this: >>>>> +                * malloc header | free space | unmapped space | free >>>>> space | malloc header >>>>> +                */ >>>>> +               void *free1_start = asan_ptr; >>>>> +               void *free1_end = aligned_start; >>>>> +               void *free2_start = RTE_PTR_ADD(aligned_start, >>>>> aligned_len); >>>>> +               void *free2_end = RTE_PTR_ADD(asan_ptr, >>>>> asan_data_len); >>>>> + >>>>> +               if (free1_start < free1_end) >>>>> +                       asan_set_freezone(free1_start, >>>>> +                               RTE_PTR_DIFF(free1_end, free1_start)); >>>>> +               if (free2_start < free2_end) >>>>> +                       asan_set_freezone(free2_start, >>>>> +                               RTE_PTR_DIFF(free2_end, free2_start)); >>>>> +       } >>>>> >>>>>           rte_spinlock_unlock(&(heap->lock)); >>>>>           return ret; >>>>> >>>> >>>> Something like that, yes. I will have to think through this a bit more, >>>> especially in light of your func_reentrancy splat :) >>>> >>> >>> So, the reason splat in func_reentrancy test happens is as follows: the >>> above patch is sorta correct (i have a different one but does the same >>> thing), but incomplete. What happens then is when we add new memory, we >>> are integrating it into our existing malloc heap, which triggers >>> `malloc_elem_join_adjacent_free()` which will trigger a write into old >>> header space being merged, which may be marked as "freed". So, again we >>> are hit with our internal allocator messing with ASan. >> >> I ended up with the same conclusion. >> Thanks for confirming. >> >> >>> >>> To properly fix this is to answer the following question: what is the >>> goal of having ASan support in DPDK? Is it there to catch bugs *in the >>> allocator*, or can we just trust that our allocator code is correct, and >>> only concern ourselves with user-allocated areas of the code? Because it >> >> The best would be to handle both. >> I don't think clang disables ASan for the instrumentations on malloc. > > I've actually prototyped these changes a bit. We use memset in a few > places, and that one can't be disabled as far as i can tell (not without > blacklisting memset for entire DPDK). > >> >> >>> seems like the best way to address this issue would be to just avoid >>> triggering ASan checks for certain allocator-internal actions: this way, >>> we don't need to care what allocator itself does, just what user code >>> does. As in, IIRC there was a compiler attribute that disables ASan >>> checks for a specific function: perhaps we could just wrap certain >>> access in that and be done with it? >>> >>> What do you think? >> >> It is tempting because it is the easiest way to avoid the issue. >> Though, by waiving those checks in the allocator, does it leave the >> ASan shadow in a consistent state? >> > > The "consistent state" is kinda difficult to achieve because there is no > "default" state for memory - sometimes it comes as available (0x00), > sometimes it is marked as already freed (0xFF). So, coming into a malloc > function, we don't know whether the memory we're about to mess with is > 0x00 or 0xFF. > > What we could do is mark every malloc header with 0xFF regardless of its > status, and leave the rest to "regular" zoning. This would be strange > from ASan's point of view (because we're marking memory as "freed" when > it wasn't ever allocated), but at least this would be consistent :D > I've been prototyping a solution for this, but I keep bumping into our dual usage of ASan: ASan doesn't differentiate between allocator-internal accesses, and user code accesses. Therefore, we can't either, so either we start marking areas as "accessible" even when they shouldn't be (such as unallocated areas that correspond to malloc headers), or we only use ASan to mark user-available areas and forego its usage inside the allocator entirely. Right now, the best I can think of is the combination of approaches discussed earlier: that is, we mark all malloc element header areas as "available" unconditionally (thereby sacrificing part of the protection ASan provides us - because we can't prevent ASan from complaining about accesses from inside the allocator without losing our ability to detect cases where user accidentally accesses a malloc element), and we also mark unmapped memory as "available" (because writing to it will trigger a fault anyway). I haven't yet figured out the cleanest solution (we miss asan zoning for headers somewhere), but at least i got func reentrancy test to pass :D -- Thanks, Anatoly