From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CF9254263D; Tue, 26 Sep 2023 07:30:34 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3E733402E8; Tue, 26 Sep 2023 07:30:34 +0200 (CEST) Received: from mgamail.intel.com (unknown [198.175.65.58]) by mails.dpdk.org (Postfix) with ESMTP id 77F1440271 for ; Tue, 26 Sep 2023 07:30:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695706232; x=1727242232; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=uBIaEW/dv9s24eyzqDke5WXr5TTZWpyEDkPVfckzios=; b=jrGdnvK7Pw8NA5de9Pk+hrEbm+zLaXTQJXAUA9XcXCDY8inw+vSqA9kZ brG1npZTsrFztu7hUXTEyqjZMlGiUoKq9WP4tNQWf4xSE2+iQ9PsydHJ9 1UmqvXKALlXW12MdKrrdE6hoa3oOD8SRdSOoxbBFXh+D4zgZW6SWGEtrk FCdU5XCWHzZCoBEwnIL30mShORd38r4LY24DqE8pjlPF7OCUPIspH5X54 Ewi71CIguTgbz+rFI4WNeOiI7dF9og7ws56k40/K72T8r6sA3u8h+8Mfr fHvSc3sprFOEagaIP/SZz1MAJG7iVrolaVFYZ7Dwt7b2Wzl5F1naqfslO A==; X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="133302" X-IronPort-AV: E=Sophos;i="6.03,174,1694761200"; d="scan'208";a="133302" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2023 01:03:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10843"; a="891577606" X-IronPort-AV: E=Sophos;i="6.03,174,1694761200"; d="scan'208";a="891577606" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by fmsmga001.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 25 Sep 2023 01:01:59 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Mon, 25 Sep 2023 01:03:00 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32 via Frontend Transport; Mon, 25 Sep 2023 01:03:00 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.173) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.32; Mon, 25 Sep 2023 01:03:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RzacY8w8gQVKRuEpBwCyC2BY31/ZZfcUmecnd+UT9w7jCQPoc9jInyHnQ2YcfR7vXCNIBP4HKeP5Mjz91f9CMTU9NgrFA09b6BmUo6eoxLIv0vF98pAKsa8yo/iEJF9iIP8cGF1xlPsdtQ8c6WTRzVbQTLwswy6pxufLQolcxdP4KIlp+OeU52EAD2dEqufqrM8185F/JC6CVZ51NYJeIWXc84z2b7HlqxfCI0qmYMRF+/XzLN9MJhgzKFBYujnxb9z0ugwg1rkCYMBU5A8wkpIPaA/15BEshF5/EgUMjSzlt7G+EKhgOfPHkVbQrcUf17fr4lxxQjzxeapYxPO/Cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pcJplTvTGBFDUsJF/pTtpIzkxXCb8W13z1TANTBpDGI=; b=O3dUPrac+rElYp/ExnrrRUS09bur1jZea72PpD4+e25SxbUfrcz7+C5wznd7S9RtThF80Kvgv+x9UYCI1OaAUQvRV50TnJG0rU8aLFcsaasMO7+AdYnhsHI0Eu7lJSXp6siRN9yhZ5PAKJOWSHjLL0L7xiZ7RgoFrXm1DSv3DF7SN769iLWw2cWhwBtWNY3SXPDvVWPgdA0kGnazRS4nbU28ckXV1kY7nDEpLsEC2ClP7rrZeUsBYiHbsjNNtsjO1XTqZZ5zzulyf/DHfkakWHtD3V7IFChOA8nWX6Z7Hw7FXrk5cto73PI/44HerD1mE0hSrds+Ep0nSmi58gPVdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by DS0PR11MB8051.namprd11.prod.outlook.com (2603:10b6:8:121::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.21; Mon, 25 Sep 2023 08:02:57 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347%5]) with mapi id 15.20.6768.029; Mon, 25 Sep 2023 08:02:57 +0000 Date: Mon, 25 Sep 2023 09:02:51 +0100 From: Bruce Richardson To: Thomas Monjalon CC: David Marchand , , Aaron Conole , Ferruh Yigit , Subject: Re: [PATCH 0/1] make file prefix unit test more resilient Message-ID: References: <20230914104215.71408-1-bruce.richardson@intel.com> <11510039.CDJkKcVGEf@thomas> <4113725.VLH7GnMWUR@thomas> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4113725.VLH7GnMWUR@thomas> X-ClientProxiedBy: DB8P191CA0002.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:130::12) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|DS0PR11MB8051:EE_ X-MS-Office365-Filtering-Correlation-Id: e80e6352-ff75-45a9-cdd0-08dbbd9dd424 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mzbgP8TTyd2s7BHrcJl+p5ZDoobW3/pW+wfFhIkPyl5JRGJ33/yb9yIFt3AiIndu68rgbYU1mC7ZzJq7N7adXt7h67Qw3MTl2H/TjZg5oNWrdwpzJpnNrMdhPZlFs7qeW9/EoySEse/KkLS1J1YIrUudnXBc4gxOOJPDMGiiPV6vVGYxLNWI1vRj+EZCwW4eBe1XdlBO0l7uj3OOxanedYuwAEYI2cmgX46rTIXhhQyGy+cKwC9D6uofNF1CgbkD0LkfMXZwVVM1qJKmafJhj9/4EzXYzk6y3d/8PxBgIXCPNITDR2i5o3FbqzL1JNSOoS0I16sT/Dnkp2ePqzvqEdubD4XYzQbiy/dHroXG5V7olz7oOFBZIHAm6LkU2LmxI7H4vfAOyX7azCT9KpNVO3aO5W3QxoFxL14RqoZB8ybL+upe2i2/UQm62Z3TdgAWJ/I7ebqai07qcW2flaACBsbFBfuoHt9HORavkG5TekZyV2kwkjQy4BrHOky6Ve7j7uc4XORQJCCjfBTXKN5buEsCb32iZxTqG+2BRQ32GFCD7Xnd1t9z9LXlz8yoRGekkDwUaLHTQnt+1AXTv+dc7Pimr9ENTNmCiZ6SkcarM1DuJqyitEjl696V6T8j9z3b X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(396003)(39860400002)(366004)(346002)(376002)(136003)(230922051799003)(186009)(1800799009)(451199024)(26005)(966005)(5660300002)(6666004)(83380400001)(8936002)(8676002)(2906002)(478600001)(44832011)(4326008)(38100700002)(82960400001)(6512007)(41300700001)(316002)(6916009)(6506007)(66946007)(66556008)(66476007)(6486002)(107886003)(53546011)(86362001)(54906003)(67856001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QlpDV3hTY1phWk5zSFVsOVJ4VG9oUlRoU2tsSUNhQXUyQzg2bHBpNmcyRUVX?= =?utf-8?B?NlZnSm5yWUJEbVZVWEhXdTNaV0hPRGhiNVVOdWQ3c1E0R3pUT1BSaEg0eU9Q?= =?utf-8?B?NDU1OXlQR21jcHVKYUxsc09hcnIrSm9KbVlGZ2d3YWpqcHFkVzRkWEgxb0hH?= =?utf-8?B?TzlHMlM0TTFHTlljNUg3bEZvT3hvZVZsN0VXazNlbnRrTE4rV05kQlBiVy83?= =?utf-8?B?K3hRdTR3UE9taTJ6QXZMOTI3dlJSRVZZaENoNFhzd3BZUWFiMEttWmx2Nkta?= =?utf-8?B?dVA4bU5temU1ZEQ4eXpDV3MyU2RoK3RmZWdMaUtzTnNlUFNqT2ZwMi9MN012?= =?utf-8?B?NXd3L3RNNktUWDRXeXdGc0JUQjJsL0hscWwzYU9kMTRXY1hYL2RzcndNcXVC?= =?utf-8?B?dDVkMDU1U1Vib240SEl5d3RIRjlJOHhKZHBWaWVxZnJaZVZTRDR2NVlISEtm?= =?utf-8?B?eGxUTjluN3Z3eDBvSGdFa3FQU1lVdnRVOHR4bXl5SndXLzdhcHd1MVU1Sm5z?= =?utf-8?B?N3drK1lhRE9rUW03RDBkaGQ3NlI1c1Q2RVN3b3U1bnhEY1poWFFzVG9qalVW?= =?utf-8?B?WGNFRzg4OHJLcVVSSGtldFJFdUVPcFhhQWNFYzFLblBKeXpQaDBVSTA2RVN3?= =?utf-8?B?ZE5TbytOUjhHaUgvZWtGcldZSHhzeC9QeUZrbWZQbVNFdTg4UldjcDcxZG9T?= =?utf-8?B?ZW1OK3M5S0ptSStRQ2dlSHNqSHhkNFh4R2c3NHZ6SzRNbFRGMkd5bWNWdXhp?= =?utf-8?B?dGJvYXE5RndRc2VYL2JOMzhoZlJad3RLL3drbXJjQXRBN3Y4OXM4QnJMTWtl?= =?utf-8?B?dVE1NWRYSW1zbG5UZGVzN1NWLzRiQWROY08yNFdUNm44RnlSN2hpQWJOUitM?= =?utf-8?B?OTFhVWQrS1pQRWxWY3Npdm1EM3ErNXRrR0dCK0I3d0QydG1HbkxhWjNjNWln?= =?utf-8?B?RnNrZUJzRlA2Ukh0UlBOczEwTTZZRUlUMzI1WW55S2ZzTmhobzRIMlFTUEZa?= =?utf-8?B?eXRlNFRTek4wRThhWGR6Z0tiVTJqZGdIaWdhZVZ1S3ozNGtRcGE1d2JGeTJi?= =?utf-8?B?bVJrV1l0QmVld2tCZ0tWbkJnTXVRdzM4UVd1TElaT3ZhTlBFanZjSDVQNHY2?= =?utf-8?B?WkgySXo3RGVmaGszU3VPWGlCbzNXbXRncEYvZGxYcUgyWFVCRmFoU2hobGdY?= =?utf-8?B?T2xibDEzaG9LQ1hLdUkzem95bnZzUVBvN3EwamNWNmZ6ZGNhN2NHR1ArOEVM?= =?utf-8?B?VloyMEtoL0JySFkxdEpiSGFtZVMzSlRvWjJ0VlVXejlVWi9zSGhVR2hNWEVi?= =?utf-8?B?N0hNVGt5ZUJIN29TZXEzcDM3cC84cnRwd3NDQm1MQzJlblBBcjQ4NEVZb3U4?= =?utf-8?B?MVRNVktPY0tmeW1ZZ0NPbFRHME9nOHNlcndrQ3IxajAzNmFVRXpHemxMQ2hY?= =?utf-8?B?SFZiSkNWSUNmdFU0SFBndVdSZTdBUEM4YWJIdy9KRlhKRVFXajlJOU9nMjQ3?= =?utf-8?B?Nkpob2VNWGYwR0RQUWhWL0hreUdlL21zUFdwZ2pkb0lXTmpJbm1CK1NSODBl?= =?utf-8?B?bXkvYXhTTVRpZmdJVzZaY2pWRStaVUFyTSt6L0U2dGtHK3U4U1BKbUNwUHI2?= =?utf-8?B?MXA2WU41b1Z2Q0lHcW5pZ21JdXc1eDN5VDAxQjNBbHk1cGQzVTJCREJybVF0?= =?utf-8?B?Z2FLVml3NGc4ZW5hUFNpdmtTZGl5RWhOZmtGZVE1d2ltdDZFVjd5ZnhEZXF1?= =?utf-8?B?ZlUvb3piRklsYlk5WjZTVER1RHNNWmpGQ3hRRm1YelU1OEtmN29manBUbjVI?= =?utf-8?B?bHNaSEx0RHR6dGJ1Y0RGSzZBbnZaYTVlM2ZVTDJ6aWhESXU3QTJGbWhqVGRL?= =?utf-8?B?d3Z5czRQL0VKWS9EaDB3ZnhBb0EvWEVscy9FUGJRVngyTkY2eEV0Mk1FNDN5?= =?utf-8?B?L1FkU1dTdzVyNzVKdjlOL3pxZnNNL2g5N1lpb3IvSGc3dzZCOGF4L2kvZjJx?= =?utf-8?B?aW1JMjVMZ05RSHBCZTgyc3dOK2ZRaW8zZ1R2YTZPdWNaZFZRYjJQQ0l3bUZx?= =?utf-8?B?MHZaL3pGelZVdWxuQ1hhR0NPRll4WjdIZTFheDByNFVqMGdpalVNUVNiTUpI?= =?utf-8?B?cGk2bWlIYXV3OFB3bHEvaEFLbTVTOHduRXZSaGoxWWxEVjRLVlluTjlrVWli?= =?utf-8?B?OWc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: e80e6352-ff75-45a9-cdd0-08dbbd9dd424 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2023 08:02:57.1752 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: f+OKkuG807WYNm+sy1Vjw07Jy5zFknmmLBYD1bxTxl/SLht4QY3nBI/zIoa6sE5LEdooqkwiwo/iMvEn6N+9GTCioqTBJugaaqDWyObGSKQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB8051 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, Sep 23, 2023 at 10:21:04AM +0200, Thomas Monjalon wrote: > 22/09/2023 15:23, Bruce Richardson: > > On Fri, Sep 22, 2023 at 02:57:32PM +0200, Thomas Monjalon wrote: > > > 20/09/2023 12:09, Bruce Richardson: > > > > On Wed, Sep 20, 2023 at 12:00:08PM +0200, David Marchand wrote: > > > > > On Thu, Sep 14, 2023 at 12:42 PM Bruce Richardson > > > > > wrote: > > > > > > > > > > > > When examining the IOL testing failures for patch series [1], I observed > > > > > > that the failures reported were in the eal_flags_file_prefix unit test. > > > > > > I was able to reproduce this on my system by passing an additional > > > > > > "--on-pci" flag to the test run, since the log to the test has errors > > > > > > about device availability. Adding the "no-pci" flag to the individual > > > > > > > > > > Something is not clear to me. > > > > > > > > > > While I understand that passing "no-pci" helps avoiding the issue (as > > > > > described below), I have some trouble understanding this passage > > > > > (above) with "--on-pci". > > > > > > > > That's a typo for no-pci. When I ran the test on my system with the main > > > > process using no-pci, I was able to reproduce the issue seen in the IOL > > > > lab. Otherwise I couldn't reproduce it. > > > > > > > > > How did you reproduce the issue? > > > > > > > > > > > > > > > > test commands used by the unit tests fixed the issue thereafter, > > > > > > allowing the test to pass in all cases for me. Therefore, I am > > > > > > submitting this patch in the hopes of making the test more robust, since > > > > > > the observed failures seem unrelated to the original patchset [1] I > > > > > > submitted. > > > > > > > > > > > > [1] http://patches.dpdk.org/project/dpdk/list/?series=29406 > > > > > > > > > > > > Bruce Richardson (1): > > > > > > app/test: skip PCI bus scan when testing prefix flags > > > > > > > > > > > > app/test/test_eal_flags.c | 20 ++++++++++---------- > > > > > > 1 file changed, 10 insertions(+), 10 deletions(-) > > > > > > > > > > Iiuc, the problem is that the file_prefix unit test can fail if any > > > > > DPDK subsystem forgets to release some memory and some hugepages are > > > > > left behind at the cleanup step. > > > > > Passing --no-pci as you suggest hides issues coming from PCI drivers. > > > > > > > > > > This is something I tried to fix too, with > > > > > https://patchwork.dpdk.org/project/dpdk/list/?series=29288 though my > > > > > fix only handles a part of the issue (here, the ethdev drivers). > > > > > > > > > > Another way to make the file prefix more robust would be to remove the > > > > > check on released memory, or move it to another test. > > > > > > > > > I actually think the test is a good one to have. Also, taking in your patch > > > > to help with the issue is a good idea also. > > > > > > > > I'd still suggest that this patch be considered anyway, as there is no need > > > > to do PCI bus scanning as part of this test. Therefore I'd view it as a > > > > harmless addition that may help things. > > > > > > I'm hesitating. > > > This test is checking if some memory is left, and I think it is sane. > > > If we add --no-pci, we reduce the coverage of this check. > > > > > > Now that the root cause is fixed by David in ethdev > > > (https://patches.dpdk.org/project/dpdk/patch/20230821085806.3062613-4-david.marchand@redhat.com/) > > > we could continue checking memory freeing with PCI drivers. > > > So I tend to reject this patch. > > > > > > Other opinions? > > > > > No objection to this patch being rejected if not necessary. > > > > However, I'd question if the normal case is actually checking for freeing > > memory in PCI drivers. I suspect that in EAL cleanup we delete all files we > > use, irrespective of whether the mappings are still in use. Then when the > > process exits the hugepages will be completely freed back - even if some > > components leaked memory. I believe this case is checking for correct EAL > > cleanup of hugepage files, not for any memory leaks, and in that regard > > omitting some components should make no difference. > > You're right, that's why I'm hesitating. > Fortunately it helped to discover a memory leak. > Do we want to add a new specific test for memory leaks, > or is it OK to have it in this one? > Not really sure. I'd tend towards saying that special memory leak checkers like valgrind are better to use than trying to detect them in unit tests directly. However, not an expert in this area. /Bruce