From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61F3CA0032; Fri, 22 Apr 2022 11:18:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 07F2A4067C; Fri, 22 Apr 2022 11:18:19 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 4F3F54067B for ; Fri, 22 Apr 2022 11:18:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650619097; x=1682155097; h=message-id:date:subject:to:references:from:in-reply-to: content-transfer-encoding:mime-version; bh=47cPttCq4rfqhJ+alxp77utt340NMfiPQD4v6hhUhx8=; b=a823RaXxPSrtAE9ZOGpEnhC0utxgzIqDjXS6E/SXRYCjwfjEO6vyuCdE REEK03KLBs0360IkXLzqt12+3sm9dUZJ79cE8OcR5pnlpqMQTApKdnjI8 foxn1RrDoEgfJeSXItQMHZMcu18v/yPZZWoRRA3zVB/iIc4bbvQg3FjY7 sFSFzAQJa5nZQAeM/Ol5cwJ89QOyDkTSAeHTtKFy6G4edDjZi6OnujB3W sutrcGMjnBF05wD3rmVfYUl5ivq+JywA6SxRGv/gAw0Y4cKEpGAX2tkpK BvBZCViENvYRsqM4L6yjdml49mRZmVHM6yRs4zLpxgIKm3mxV8WZ5zwkA A==; X-IronPort-AV: E=McAfee;i="6400,9594,10324"; a="262226650" X-IronPort-AV: E=Sophos;i="5.90,281,1643702400"; d="scan'208";a="262226650" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Apr 2022 02:18:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,281,1643702400"; d="scan'208";a="805926316" Received: from fmsmsx605.amr.corp.intel.com ([10.18.126.85]) by fmsmga006.fm.intel.com with ESMTP; 22 Apr 2022 02:18:16 -0700 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 22 Apr 2022 02:18:15 -0700 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 22 Apr 2022 02:18:15 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Fri, 22 Apr 2022 02:18:15 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.103) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Fri, 22 Apr 2022 02:18:15 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ntep2FN/d3TXdJBzwRU3DK0hiuLpundmKteuJOkje0A6dTRk149T7Ej83sdRKUgTZOOn+K8GuEwu/hCcToBEzptK3nW3FyxU3IVR3yv3h9gLn7VNTdcaELdGs8uuGH+e7KAMz+klvvb5Ilxv5QT/IhiNNA9/s9FqlJTsN1Vs4gBqX8XkbaES+lPl2e7ZrFn3y718vYtkUhszO0kOUq1ZEd0vw66MGEwb02Dm1yG3vjfVWXLZuNeaa3yVLQuBRTFdLoyjbyvVHz7Wa/Z4qRcHJtuR1nDP4O59w6J1h7N2SYPYXlREPVkV61lB4a9KWsxa/3yL7Ma4KrUr6apQ7aKShA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lgZ0kb6w9fsZChv7cuUZv1aAJbw/AqApbT7106kdfF0=; b=fDFKdMENkGH+he7AkEK4gV2s7MJ8rgik/ojFJ/ocoOVTodBInci11eaFt4aItk4skZUwQALa2wRBKdTarOrwjPuePqXFUcSUSgSZ8w6UbDX3c3ycs4sZBxoaOcseOEvpCFOvDSqsrHGzQ6j6m6KZWSIPqO9x/soSXK+NJqe4luGWukwkzJLqDlbu8cEWWrmxTRNopOF9eZmCNkeZtIkOck6rKh953NRhfERH8GqVrobCMV1PvAPN22wPih3DcA8ds6SkrM6GYrLXta++d4HGFYqhehCQA+J2fqAfKBkwHOVJk21h7ntXw/lRDrtYp8a5fYMh3Vr9OXxs0jcYx7fu8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) by MN2PR11MB4111.namprd11.prod.outlook.com (2603:10b6:208:138::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.15; Fri, 22 Apr 2022 09:18:13 +0000 Received: from MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::d55d:28c1:bfab:3dd]) by MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::d55d:28c1:bfab:3dd%5]) with mapi id 15.20.5186.013; Fri, 22 Apr 2022 09:18:13 +0000 Message-ID: <50309690-64f6-7390-278f-21e3cf950527@intel.com> Date: Fri, 22 Apr 2022 10:18:07 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [RFC] eal: add bus cleanup to eal cleanup To: =?UTF-8?Q?Morten_Br=c3=b8rup?= , References: <20220419161438.1837860-1-kevin.laatz@intel.com> <98CBD80474FA8B44BF855DF32C47DC35D86FE1@smartserver.smartshare.dk> From: Kevin Laatz In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86FE1@smartserver.smartshare.dk> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0501.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1ab::20) To MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b12b2472-7f66-449c-b5e5-08da244106cd X-MS-TrafficTypeDiagnostic: MN2PR11MB4111:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QyEFBFPC4lFAuNwQOwYiK6wzBfiqGCUqGr8G351yExX/H14Tk6Zo5nCtNwXDLBsqTdeNSepaVDFwflrrlb66KMbSnoG7Yld77m1cjj+O+/5rBaG549hsKkhkpGy51XcxneCDo+HFCOuiBYkDFTXVMnf3L/JWT3lN/BNc7RuH0A65f/95tXC0Abo31OYNUkqhCIMjA5FYNOPlRcbnhFXtchvLP3hUBr+xn67TyWf2IkLyI8r/Y6XMdHB9ss+bQ3njvP802PJR5CJlEc5q4AoeSTnlQb5Vf4D73LSAxefj0dPj3pzYJG61IpNTPO4I8YL675vZiJ1iNHh8RCy1JGWvFJQ9TZTKKDWE+qKuFHeQ1TY4Um7TRkXbGvq4t3AGmX9JE5/fdDgBAv6CCx/3EQ+iAETO6kh5Zq/UClGC3wgPKIHFRfzkc/3LBdATclbQgSl+LHp3QUEskSe5ROfzwDmO2SJrzNWRqLFBMNPVWBbPobJcZEHP73ZewHK5FVFAjrbhjTwkNWUiDdB1KJK9rgExvCKi0R1CDsm7jk+C5p/6Lq0WVk5dk694xBnGl6FHAGHsydEcYjqAeNrKnnKtvGgLdcse8+dWYVdlS5k2QUGZ/B79Oq49vvV8ygcFMxTJF11HbukhTEOKWh33bzf6lvcT5uUoYewX1khD8493J9pwcmn6O5VLe6KiOFuUMzq1YXStjIy7yYaE3IlWyebg8hcl+JyklO8rfDIonGKsdfqZo+U= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW4PR11MB5872.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(186003)(66574015)(6512007)(26005)(36756003)(6666004)(6506007)(82960400001)(66556008)(66476007)(8676002)(2616005)(508600001)(66946007)(2906002)(53546011)(8936002)(316002)(86362001)(31686004)(6486002)(83380400001)(44832011)(5660300002)(38100700002)(31696002)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eDJaUHJQMHpqTUl0VmFDeWRZMkx3TUNFRS9FWWJOTkk5TVpmQWlvdk1mK1JM?= =?utf-8?B?d0dLNUtCeU8vY3ZZWTVoenRMdFNGT3pJT1dWanFCeGZTeEhxMDkxNHlVRDF6?= =?utf-8?B?ajNHVEpSWk9HZGNsQWRhZ09tQUN6QmZCZUtGRC9DZVVBK2pyZy8zelhDS3k0?= =?utf-8?B?VzFuQit5b3RzMzdaNVphTVY3U1ZHYUFSK1dUUG9rL2xCUGlzSG5GMTNjUElj?= =?utf-8?B?clhiQWZ4Q2hYQmJBUXV1T2JHcERkQnJQcFd3SisrYktVY0l4RUxxRkZhNDJR?= =?utf-8?B?T214cEtObzNMMDlXdEw1R1dXYjRPdGFLYVo5U3dtalVOc3pPelhPMFhOUlVp?= =?utf-8?B?d1ZUTCtHL29aejk0b0pPLzdCcW1ORlIrOGZrcHRnV0w5UUtPZU5IQkRUbjBC?= =?utf-8?B?S1J3REJwV2tvMmdOaVluSFZkZjROSktEdW1VbDh1K3NZYzV5R2wycEJybWFp?= =?utf-8?B?a2tKYVBFWm9aWTF6dTZzbkJPWmxPWGVvTmZrNDdBZVB1UVRNVFo4YlkrVDlH?= =?utf-8?B?SDlUTFR0Z3hjeXR1TFJUc1RrZXlaTHFvWkgxd3lYamhHemo2TWV6ZUtTbG52?= =?utf-8?B?MjhrcGw0V3I0Zk04K0ZxUm1PNXlKK2o4T2ZKRHNmT05sSUJQSUttTmUxa2U4?= =?utf-8?B?WDdxbzZlSzVueEVldkxBbFBiaXhoazhoU2VLWXhIZ0ZwWG51SUEzWGZxVUtz?= =?utf-8?B?RDBIWGxCMFZLcFBSWVpOOGdDWWtndHdoZys1OHZyaGQ0RGx2M0tPc2Y3aW5o?= =?utf-8?B?aHpxcWlRazRZUTNjUThTWmJSU2s0UnFtTjI3S1BTUURYZnA2d0R4ZVhBTVVp?= =?utf-8?B?RDI5c1dlNk9xUDIvQlJJbTZ1TjlBTWYrd3M2dWw5aW5WaVZLMDVib1piQXFp?= =?utf-8?B?Rld6ak5wT21sWGE2b3RWdVNtcVNCVzRDRG5nWTd5d2tFakpsQ2VUd2RKaFJ3?= =?utf-8?B?amkzN1ZqTGs3QkUrS3BuQ0N4WFZ2V2phNW1Takkzb2hxdlUxVWc3QnNlSng2?= =?utf-8?B?c2J1RkVyQTBCWHA2S0lqeUl1THMvSCtWbmRHczdTUWRpc3BQbXJEWGthYnV1?= =?utf-8?B?MEh3c29lOVhleDBmREJ3SHpBNzlqL0VJaEg2RUFYeXJJQVZmSEhVa2kxeHFU?= =?utf-8?B?cHJaRmk1QitrSlllQkJxU29JeDdyWTVKM00zT1lTaCtsS3k4UUMzVDdQdjlO?= =?utf-8?B?NjRteWlXcDVQVTVKeElkNVJON29MeURJMGlXVzdlTEJmSnZ5eWI5eHMwdDJX?= =?utf-8?B?TDJXTGlFb05JZHd2Y1B3SzBZbTdxanVlcEMzbFp6NmRabENha01VRm5wSkVE?= =?utf-8?B?Vm91bDBUNkR3NHBlU3BORE1pWnVPbktDK0xFTVVLbkFRUDBIRFJEalZ0YnBv?= =?utf-8?B?STkxWGZISEZUVjBJc21CN3ZLQ3RuOURsbmpRNFE2RzRUOFhGbXByWG1FV2lD?= =?utf-8?B?YXpWT2VsOWUrZmlxTWpkamkrTTRQd1VFS3A5MWRrRHl3QlVXQm5lZCtuUXB3?= =?utf-8?B?ZjkzSlpVU3pOWC9lVEJ1M3J4aGdPczI3ZFpjTnpuaG1ZM3FmYm95dUtPYjhj?= =?utf-8?B?R3V5Q2pOeTFxd29CQjZEa0dVbGY4SC9lWGtsa2ZVRkxHTHI1bzd0U0h3OHI2?= =?utf-8?B?RGliK1FjMElsb1NwbmtQTlFzUzN1WWIzU3Urai9mYzdNR0M5QmRrUXZNY2Vn?= =?utf-8?B?SlhJZFRYeE5STkdBcU9YUHBYcXhWRWJzV1gvMHZBM3hXQXRsbnhobHVVbEs3?= =?utf-8?B?WFQzK2x4a215eUtvZC9TbjlhVS82cUhCbFpMeGlPMGIrQnJOd0ZkWENURmhF?= =?utf-8?B?TG5RWGR3K0V0T1dlTjVpajlBMHB4a1p5ek9EUXE4YWgrekwrN3lwMnhnVEU5?= =?utf-8?B?N2xTNFJsVnRyUjVHdjM1Ui9SOGtpTW01MGlBYTRUWjZEOWRqcjRseFplcUd5?= =?utf-8?B?dHMxN2hqVmVnQUdJb0NnclJoMmpSU0lBdEtDZ1ZGcEFtS0V1ellMeDhsNHln?= =?utf-8?B?T3RTbnFaQ05CSS91MWRlODlTWk42WlZNMm1tUXpqR1R0MDdPeTdRSWcreHc2?= =?utf-8?B?Q3JBT2lsNGw4cFJRVnZla25MeGRRZ2pieUszMUMwV3F6MHFGWk5OWSswYzJx?= =?utf-8?B?YUovVFk3Tk14THlKNzRtaW8wWnJzWTRzWndmSWJ6b2xzcHRsUUNnbzNlbUVP?= =?utf-8?B?emtLdFBDdnlSU1RXSkVZdkhOYUVTUU5rWmpqUGtzbUpaSkZVSmRLWmtBNjls?= =?utf-8?B?NDRxb0MxZmdMbS9WTWtpNmhDOVJNQUUxTElMNExrRDhWRnNoTm50Mk1zdkhX?= =?utf-8?B?dzN3QUNhY0tHclJJY0hrYy9Rek9MSDFsOGxUeTlZanlGU3RkMUpTWlRqZStm?= =?utf-8?Q?7WS6pubuCpRbDlKI=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: b12b2472-7f66-449c-b5e5-08da244106cd X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5872.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 09:18:13.4621 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GR0pqbKOl6K9DYZA1GXhzCklo74sCREzSX3iFJEZe51jgX7i69GaEGJDiy5DGE0INMpc537tBxoTz8jn7lqvIQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4111 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 20/04/2022 07:55, Morten Brørup wrote: >> From: Kevin Laatz [mailto:kevin.laatz@intel.com] >> Sent: Tuesday, 19 April 2022 18.15 >> >> During EAL init, all buses are probed and the devices found are >> initialized. On eal_cleanup(), the inverse does not happen, meaning any >> allocated memory and other configuration will not be cleaned up >> appropriately on exit. >> >> Currently, in order for device cleanup to take place, applications must >> call the driver-relevant functions to ensure proper cleanup is done >> before >> the application exits. Since initialization occurs for all devices on >> the >> bus, not just the devices used by an application, it requires a) >> application awareness of all bus devices that could have been probed on >> the >> system, and b) code duplication across applications to ensure cleanup >> is >> performed. An example of this is rte_eth_dev_close() which is commonly >> used >> across the example applications. >> >> This RFC proposes adding bus cleanup to the eal_cleanup() to make EAL's >> init/exit more symmetrical, ensuring all bus devices are cleaned up >> appropriately without the application needing to be aware of all bus >> types >> that may have been probed during initialization. >> >> Contained in this RFC are the changes required to perform cleanup for >> devices on the PCI bus during eal_cleanup(). This can be expanded in >> subsequent versions if these changes are desired. There would be an ask >> for >> bus maintainers to add the relevant cleanup for their buses since they >> have >> the domain expertise. >> >> Signed-off-by: Kevin Laatz >> --- > [...] > >> + RTE_LOG(INFO, EAL, >> + "Clean up PCI driver: %s (%x:%x) device: >> "PCI_PRI_FMT" (socket %i)\n", >> + drv->driver.name, dev->id.vendor_id, dev- >>> id.device_id, >> + loc->domain, loc->bus, loc->devid, loc- >>> function, >> + dev->device.numa_node); > I agree with Stephen, this message might as well be DEBUG level. You could argue for symmetry: If the "alloc" message during startup is INFO level, it makes sense using INFO level for the "free" message during cleanup too. However, the message probably has far lower information value during cleanup (because this driver cleanup is expected to happen), so I would degrade it to DEBUG level. Symmetry is not always the strongest argument. I have no strong preference, so I'll leave it up to you, Kevin. Thanks for the feedback. +1, will change to debug for v2. > > [...] > >> @@ -263,6 +275,7 @@ struct rte_bus { >> const char *name; /**< Name of the bus */ >> rte_bus_scan_t scan; /**< Scan for devices attached to >> bus */ >> rte_bus_probe_t probe; /**< Probe devices on bus */ >> + rte_bus_cleanup_t cleanup; /**< Cleanup devices on bus */ >> rte_bus_find_device_t find_device; /**< Find a device on the bus >> */ >> rte_bus_plug_t plug; /**< Probe single device for drivers >> */ >> rte_bus_unplug_t unplug; /**< Remove single device from >> driver */ > Have you considered if modifying the rte_bus structure in /lib/eal/include/rte_bus.h breaks the ABI or not? I've looked into this and have run test-meson-builds with ABI checks enabled. The output of those checks flagged some potential breaks, however I believe these are false positives. The output indicated 2 potential breaks (in multiple places, but the root is the same) 1. Member has been added to the rte_bus struct. This is flagged as a sub-type change, however since rte_bus is only ever reference by pointer, it is not a break. 2. Offset of members changes in 'rte_pci_bus' and 'rte_vmbus_bus' structs. These structs are only used internally so also do no break ABI. Since the ABI checks do flag the addition, I will add an entry to the abignore for the v2. > > > Overall, this patch is certainly a good idea! > > On the condition that modifying the rte_bus structure does not break the ABI... > > Acked-by: Morten Brørup >