From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id C71F6A04FF;
Tue, 24 May 2022 17:22:05 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id CE72342BA1;
Tue, 24 May 2022 17:22:01 +0200 (CEST)
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
by mails.dpdk.org (Postfix) with ESMTP id 67F204281F
for <dev@dpdk.org>; Tue, 24 May 2022 17:21:59 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
t=1653405719; x=1684941719;
h=message-id:date:subject:to:cc:references:from:
in-reply-to:content-transfer-encoding:mime-version;
bh=RwGJBV4O32BhwNC2+cgn1qlw0CJZ7RnsGoNt5JzWGPU=;
b=WgdP9eyG0ThqU5/RRpPOtJNdzcOnjiYCL0Y+ZOWF41I/DpSqg1O+9fb5
Ef8wJxLIHVDUiHZhcMBbM5Kcn/l/PbofbuKkO48Itekrfb06+GomvBsni
s/DRDBsBZZvekGCX1LhPTorubZBOORCXXBNpuHo6mkPohqjxIcMaecmCN
fH9A/qNCbxBs7/GlfYe8DvcGJ23Wcgp/niskEhN5xzB13DJnpHx8FWtgn
6OpVDldQ1MVS/VLNaXAkWFTsU9TkpdR9yJn4wfsMzvRmLYZVrWlcJvouv
lRaSgckKj262v3BZf+h58e6sJW5lmqjd7h9EkvRudLboFlzreFdGtQk9A A==;
X-IronPort-AV: E=McAfee;i="6400,9594,10357"; a="255624015"
X-IronPort-AV: E=Sophos;i="5.91,248,1647327600"; d="scan'208";a="255624015"
Received: from fmsmga008.fm.intel.com ([10.253.24.58])
by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
24 May 2022 08:19:54 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.91,248,1647327600"; d="scan'208";a="629927052"
Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16])
by fmsmga008.fm.intel.com with ESMTP; 24 May 2022 08:19:54 -0700
Received: from orsmsx604.amr.corp.intel.com (10.22.229.17) by
ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2308.27; Tue, 24 May 2022 08:19:53 -0700
Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by
orsmsx604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2308.27 via Frontend Transport; Tue, 24 May 2022 08:19:53 -0700
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.42) by
edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.2308.27; Tue, 24 May 2022 08:19:53 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=QC3MVVFoXaCCL8KRCmYuzTLxK2oVT+ENZGzcW8WNaHzuqFJ1RY7zAy+ZGjlF5oSdviyic/lbsYYJZ97an1pPh2l7na15EAFONVP4U3Cq3Y1Hd9v46nDf5Ez5GDlcy6NrFixO4uXUQU4gIaOtjAvtiZn63XYLYOnsnazKp8hWNWtHeGWPS7r2iNcJwyyXAr/fsA/YLrnKpqZO1PpqJXkF+Yr0tRc5hjr/IRFBTGbAxeuFiBubjR23deQ8QvqNeiFNRfaDh057bP2iY55q2ikM8AwAOjv8vPTB7rD9cUPy4GhHAfai+y8ETnxL1Jf6XHMK1PXY4FMI9PwETCeUtXzynQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=wijZRAGRPUadlbzMViHIWITZApzzaQc2eNG9IbJKwv0=;
b=lgyyIJ81DLQ5y2FeCAGZwW8O2gXe56wzXNi844a0GYtE+i/9TuYUhmyqcXd0Yf5zzveWQ4zAHbwSFoS4U1WRAzleC4u6DDG2/hk9A5qmvLuJR/eXhdFLQaBMJmngalOaoIJU38O7ya6Xt3XFwfYJRoHvR6f4IXyYI9OwLQsBAtj5cJEKqkpJUnKduQgzeBSdYpuCAkb8ruCEf7IbjG2PK08SY+C5pcKd1n4hNCWDsHTGLRu133nvudZ22v0RGh94KDu7jagHzyo63MOZXtf2UxF9MHur8uK0Tr0vPwDNuHnw2XSSZe0kktsny8SpymSvBsb5mkHNGAgF9qkJSMy4vQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
Authentication-Results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=intel.com;
Received: from MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14)
by DM6PR11MB4105.namprd11.prod.outlook.com (2603:10b6:5:5::30) with
Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.5273.15; Tue, 24 May 2022 15:19:52 +0000
Received: from MW4PR11MB5872.namprd11.prod.outlook.com
([fe80::d55d:28c1:bfab:3dd]) by MW4PR11MB5872.namprd11.prod.outlook.com
([fe80::d55d:28c1:bfab:3dd%5]) with mapi id 15.20.5273.023; Tue, 24 May 2022
15:19:52 +0000
Message-ID: <3bd7ebff-5abf-9627-70d0-9459a95ea51f@intel.com>
Date: Tue, 24 May 2022 16:19:46 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.8.1
Subject: Re: [PATCH v4] eal: add bus cleanup to eal cleanup
To: Bruce Richardson <bruce.richardson@intel.com>
CC: <dev@dpdk.org>, =?UTF-8?Q?Morten_Br=c3=b8rup?= <mb@smartsharesystems.com>
References: <20220419161438.1837860-1-kevin.laatz@intel.com>
<20220524092501.419761-1-kevin.laatz@intel.com>
<YoyngJG8A3yu0yU7@bricha3-MOBL.ger.corp.intel.com>
From: Kevin Laatz <kevin.laatz@intel.com>
In-Reply-To: <YoyngJG8A3yu0yU7@bricha3-MOBL.ger.corp.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: LO4P123CA0372.GBRP123.PROD.OUTLOOK.COM
(2603:10a6:600:18e::17) To MW4PR11MB5872.namprd11.prod.outlook.com
(2603:10b6:303:169::14)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 2acd0ed8-2861-4fa4-9b0d-08da3d98d98d
X-MS-TrafficTypeDiagnostic: DM6PR11MB4105:EE_
X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
X-Microsoft-Antispam-PRVS: <DM6PR11MB4105F6E6C85292D61E170A11FCD79@DM6PR11MB4105.namprd11.prod.outlook.com>
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: kNRMIfxT+f0yqDrlNEjFhhgF8GtyO6odbySI0puVM1dS6tnwehuGVahLB/PkpUsaUAZC4AABYSPCcLKBsFctQpldcWVbjQ6MJcqETbVI/c0AGI2XjI0obOCwo7w9jUjYwzdcng2dl9HaDrLdzYSIo8rkYZK+6Mz8BTeDdWX2IeULDcAEHb7f0XQ/fTl4pPAzz2tCdvg0ibybHeez/r1RtT363kvzuvnT8k7gu7Dq2+UEQVEnDLtflNiZhF5w+8a+WKabsAfrkzxYrEr3R4xx3ZVjwPLpsaowChtuNDY0g8RqBMi41DMDJhkOQbk3vVNxOIglbN0YRBtrx1V2uEVFEFDWJh9lplDlk0WByv1Y9cL+zp/ev40qYSxg1uxUzn8dglHxeOC5ohyKbf/JC3yS/l2p2/NNkFq7kFyqg6DDIY9qU+R5o2VB6d9dcytT8bbWJ9wsEWAPNOPIlf0THS7ZnVmgGlP/gCoKCKv4eHGREPNQXWXjwpFzZQcVmOvvnaYG7VMS55uM367nHzR7n9F7IHERU0913w3xpy2k7zt6DMScym607w/0Zkd0ZEPLqS+/hscZ1s5rzruxx2VN24QfBAOod5WgD28UncN4mEJ25n96CxN3Iw3lyGyv4RBGS4/rPhdwITV1Z6d0wTKMPOStRB+PItpxsHdGAnozJEo/ACdD3wW7M77wLmYibctqIiMd7WgXzdoc76WfRl5NVnvuHUX0BmavczH3vPgh0FY0bn2psN5NAt5kMa8NP3I9SFGv+DPPLR4ysDHQiGMmyqI2ig==
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:MW4PR11MB5872.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230001)(366004)(38100700002)(508600001)(37006003)(6636002)(53546011)(82960400001)(6486002)(83380400001)(316002)(86362001)(5660300002)(8936002)(31696002)(6512007)(6506007)(186003)(4326008)(44832011)(66476007)(6666004)(31686004)(66556008)(66946007)(36756003)(26005)(2906002)(2616005)(8676002)(66574015)(6862004)(87944003)(45980500001)(43740500002);
DIR:OUT; SFP:1102;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TDNNbWhrYUc2azZCa0VpSmVQUDdDOERCb0dBRlpOZDZnWU4xQ29XbjJnSUhF?=
=?utf-8?B?Q3BXU0xLTzd4VmViZmlWUFphc09GRGJSRWlZTjhDYkhpLyt5OC8rNmlSOW1J?=
=?utf-8?B?SktIcEs0YzlvRmszdG5MRkNtNWdDNnpTRG1tOVlETUI2eXNWby9nbWFjUWNL?=
=?utf-8?B?YlZvUEcwMTcxQ3NhYTRlbTMzUno2RUk5VGs2ZkRWczBvSnNCMXk5b3N6dVpo?=
=?utf-8?B?SXVMODEyM3hoUDFFeUpnVEZPL01keFpIM1lGVlNickNUaTVZNW9NY1FHWUls?=
=?utf-8?B?R2JLQW0vVjNlTDN1dmRJUE5wZjlFb1lYbXhxb01pTFpTaW5FVFBzYUhWMnc5?=
=?utf-8?B?OEMzZ1ptdGhlNENQQ2lFZVRJVHR0QlJYODNQTW14cnNCMEdSWWVobEEzVTdE?=
=?utf-8?B?Siszb0cxYUg2dTVleVRKVW5aMllrTjNDV2NVSm14eDdmR3gzdHU5Z05lQWY5?=
=?utf-8?B?MDA4cUQzcUdrbW5zUjBSUXFiOHh5anB5MU1iVjlFMStNR0ZhaGNGQ2EwZUxQ?=
=?utf-8?B?dGRhR2NQSkJJOUlMTTc4U1JQemhmdmpDV3VPc1VyR2VBV2FtcjlvM09wc2dp?=
=?utf-8?B?RmtjUm9kTXhCRVZyaHQvVkRhWEdkTE5SeVV0WmdycFBaVGVBYlI4amc0MjFC?=
=?utf-8?B?eXpqaDlHWldZT1U2TjZJNVVKQ0lMdVZ2ZzFQNzVnVUtLSkNsbG1oNXNpN2Mx?=
=?utf-8?B?amozRmFzTUhkMEx3WHU0enBPYnBBdUZjNXZQNWpST090eVlod3dFRDFNdkNL?=
=?utf-8?B?eGZQVmtBRHFJSzM5SHpSdHNnb29vMjZXWXNNbU5zNzdkd3Zvc0pET0d2aDg4?=
=?utf-8?B?azRmcWlKZ1pudk9WT0twSi91aTl5WnVNV1hmZlYrcXNGWUlFTHJGM214T0JT?=
=?utf-8?B?RlJDYU0vS3JpRCtGUGxJZlNzQmVQblBoV3pTZ0NhSFhINGl6bUtyaXc4M1Za?=
=?utf-8?B?OTVtam9vakxleVRCQnpWNkJLQnZ6NlhySDFnSGlubkhuOWhkZHZ0ZVFwbW92?=
=?utf-8?B?WWhzNDNJYVk2VWp4cDZGR3JsWFRIV3V2NjBXTEtuSm0xT3hITnc4bnNJbmxa?=
=?utf-8?B?QXZXbkFySVoxZHV5TytEQTNjVVRSbTRrRzdHc3dOUlpCZVl1MzVPMkYrbnAr?=
=?utf-8?B?RHBLeUczeE54Yi9LV3pueDIra3h3R0wwZXFUT3dQazB5T3ZqWEZtaEZrM3lr?=
=?utf-8?B?MTFsN0tsdytnUDBJRGFOdENrdWlpYytkVllNWFJWODAzeGsyVU1QcHRTMFFY?=
=?utf-8?B?RFc4Vms5VzBjRm5pUEUvZng2Z1NTNGtDMngzZWd0UnVzOUExa0VDQVpPNnJy?=
=?utf-8?B?MkxWNCs4VzVqV1ArS3RieThQcXhpdk5pY1Y0NUhJUy9TNXpqU1dsV3I0RGgv?=
=?utf-8?B?dWtBbE1SY0Zjd3BWcDFsdG03blByMFlQaDNNQ3N4U2FUU242UU1xdWVVcTJ3?=
=?utf-8?B?aVFEbHhWZVNwSnJSRmdQOHZrL295aXVzaWFqSGU3cHB0MnMyRndLLy9kZEND?=
=?utf-8?B?ZEpCRCtNY0x1TDlFa2RRakk0M2JHMnQyK2dpWVE3eXlwWU5LeWVEVDg4Vk54?=
=?utf-8?B?OFdDUWgxKzk0T0pmTzRjRG13cnIwTFVMTisyZnFYWUZ2Z2QzelNaYTZTaDFZ?=
=?utf-8?B?V1J5TllhUW1TRE1qUFdjdENvYjFQR1J1YlNIL0I1TC9LbjkwTVVCSW11WkVi?=
=?utf-8?B?aU9BK1RoSFdSZEI5S213ejVNa1dMYVNnb1M4S01hZEhtKzlmZnJoK3FNaFVv?=
=?utf-8?B?ZWVTWEptakZRaHVJcUQwZTd4WFR2MWVacEZvZkZpN0QzVmRhUmpzN2c0M0J3?=
=?utf-8?B?NFZtQkhrVVpEb0xLd1VhbjQ5TUZCaDk4dlJHRlIrbnFhWXZWQ1A5bXZaUmZY?=
=?utf-8?B?VldwZHhOZzVQM1Vhb3RKS2RzdWdES1B5T3pYMnJKbjhqNENmaDhVWTB5WmJ4?=
=?utf-8?B?d2FLeXpLQzVRTzg1NUNsQVJkUWt6Sm5KdW1rNmpZZnZnNlg0SG93UysvM25a?=
=?utf-8?B?NW8yTVVhUVJqWmFZQjA4bWlZOGgwUkpBL0lIUVA3M2FnNGNoOTBFZnVvaW5h?=
=?utf-8?B?SzNtcW1CZ0xjVFlIK0h2VjJucWNveDF4VVBNZldkNU9HYUttK3BkaExIeGwy?=
=?utf-8?B?ZjFmUDE0TFoyV25PMUh3Z01VbFpzNUZNelNua09jUEIxODY0UE00SXNtWWlJ?=
=?utf-8?B?YkhkRXdzNC8vRzd6UEx2NGJVcWEvdmJqV0ZzRFgwOHhodWtFYnNnQVM5cTVt?=
=?utf-8?B?LzB1U2FwWC8vSWdDdlBGcVE0ZnBUSXNTYVVTSEllbVAveGtVUVYxWTd4SmRa?=
=?utf-8?B?MlBabHdWMVFIbGVPOURTalJYamFrcXBvbGpvQVJ0REV0SUpMdnNmZVJjakxB?=
=?utf-8?Q?rP5qizd7atEy7j/g=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2acd0ed8-2861-4fa4-9b0d-08da3d98d98d
X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5872.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2022 15:19:52.2793 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5FaPk7O97G0/qE5gKePP2L8E9mfvDbHaTP8OfzuNT51SDH7SmlaOq7gL+9nlGuIoqCtnjWVqkrveGOTPfnJ3bA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4105
X-OriginatorOrg: intel.com
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
On 24/05/2022 10:38, Bruce Richardson wrote:
> On Tue, May 24, 2022 at 10:25:01AM +0100, Kevin Laatz wrote:
>> During EAL init, all buses are probed and the devices found are
>> initialized. On eal_cleanup(), the inverse does not happen, meaning any
>> allocated memory and other configuration will not be cleaned up
>> appropriately on exit.
>>
>> Currently, in order for device cleanup to take place, applications must
>> call the driver-relevant functions to ensure proper cleanup is done before
>> the application exits. Since initialization occurs for all devices on the
>> bus, not just the devices used by an application, it requires a)
>> application awareness of all bus devices that could have been probed on the
>> system, and b) code duplication across applications to ensure cleanup is
>> performed. An example of this is rte_eth_dev_close() which is commonly used
>> across the example applications.
>>
>> This patch proposes adding bus cleanup to the eal_cleanup() to make EAL's
>> init/exit more symmetrical, ensuring all bus devices are cleaned up
>> appropriately without the application needing to be aware of all bus types
>> that may have been probed during initialization.
>>
>> Contained in this patch are the changes required to perform cleanup for
>> devices on the PCI bus and VDEV bus during eal_cleanup(). There would be an
>> ask for bus maintainers to add the relevant cleanup for their buses since
>> they have the domain expertise.
>>
>> Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
>> Acked-by: Morten Brørup <mb@smartsharesystems.com>
>>
> Thanks for the non-RFC versions. Some comments inline.
>
> /Bruce
>
>> ---
>> v2:
>> * change log level from INFO to DEBUG for PCI cleanup
>> * add abignore entries for rte_bus related false positives
>>
>> v3:
>> * add vdev bus cleanup
> Missing v4 update note.
> Please reverse the order here, so it goes from newest to oldest, so those
> of us tracking the patch can just look at top entry.
Ack
>> ---
>> devtools/libabigail.abignore | 9 +++++++++
>> drivers/bus/pci/pci_common.c | 32 ++++++++++++++++++++++++++++++++
>> drivers/bus/vdev/vdev.c | 23 +++++++++++++++++++++++
>> lib/eal/common/eal_common_bus.c | 18 ++++++++++++++++++
>> lib/eal/include/rte_bus.h | 23 +++++++++++++++++++++++
>> lib/eal/linux/eal.c | 1 +
>> 6 files changed, 106 insertions(+)
>>
>> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
>> index 79ff15dc4e..3e519ee42a 100644
>> --- a/devtools/libabigail.abignore
>> +++ b/devtools/libabigail.abignore
>> @@ -56,3 +56,12 @@
>> ; Ignore libabigail false-positive in clang builds, after moving code.
>> [suppress_function]
>> name = rte_eal_remote_launch
>> +
>> +; Ignore field inserted to rte_bus, adding cleanup function
>> +[suppress_type]
>> + name = rte_bus
>> + has_data_member_inserted_at = end
>> +
>> +; Ignore changes to internally used structs containing rte_bus
>> +[suppress_type]
>> + name = rte_pci_bus, rte_vmbus_bus, rte_vdev_bus
>> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
>> index 37ab879779..7d0c49f073 100644
>> --- a/drivers/bus/pci/pci_common.c
>> +++ b/drivers/bus/pci/pci_common.c
>> @@ -394,6 +394,37 @@ pci_probe(void)
>> return (probed && probed == failed) ? -1 : 0;
>> }
>>
>> +static int
>> +pci_cleanup(void)
>> +{
>> + struct rte_pci_device *dev = NULL;
>> + int ret = 0;
>> +
>> + FOREACH_DEVICE_ON_PCIBUS(dev) {
>> + struct rte_pci_addr *loc = &dev->addr;
>> + struct rte_pci_driver *drv = dev->driver;
>> +
>> + if (loc == NULL || drv == NULL)
>> + continue;
>> +
>> + RTE_LOG(DEBUG, EAL,
>> + "Clean up PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n",
>> + drv->driver.name, dev->id.vendor_id, dev->id.device_id,
>> + loc->domain, loc->bus, loc->devid, loc->function,
>> + dev->device.numa_node);
>> +
>> + ret = drv->remove(dev);
>> + if (ret < 0) {
>> + RTE_LOG(ERR, EAL, "Cleanup for device "PCI_PRI_FMT" failed\n",
>> + dev->addr.domain, dev->addr.bus, dev->addr.devid,
>> + dev->addr.function);
>> + rte_errno = errno;
>> + }
>> + }
>> +
>> + return ret;
>> +}
> This function returns the status of the last remove call only, which is
> probably not what we want. Better to make "ret" a local variable inside the
> loop and have a new variable in function scope called "error", initialized
> to zero. Then where you assign rte_errno you can also assign error to -1
> and return error from the function at end.
Will fix for v5.
>> +
>> /* dump one device */
>> static int
>> pci_dump_one_device(FILE *f, struct rte_pci_device *dev)
>> @@ -813,6 +844,7 @@ struct rte_pci_bus rte_pci_bus = {
>> .bus = {
>> .scan = rte_pci_scan,
>> .probe = pci_probe,
>> + .cleanup = pci_cleanup,
>> .find_device = pci_find_device,
>> .plug = pci_plug,
>> .unplug = pci_unplug,
>> diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c
>> index a8d8b2327e..eca1a3d536 100644
>> --- a/drivers/bus/vdev/vdev.c
>> +++ b/drivers/bus/vdev/vdev.c
>> @@ -569,6 +569,28 @@ vdev_probe(void)
>> return ret;
>> }
>>
>> +static int
>> +vdev_cleanup(void)
>> +{
>> + struct rte_vdev_device *dev;
>> + int ret = 0;
>> +
>> + TAILQ_FOREACH(dev, &vdev_device_list, next) {
>> + const char *name;
>> + struct rte_vdev_driver *drv;
>> +
>> + name = rte_vdev_device_name(dev);
>> + if (vdev_parse(name, &drv))
>> + continue;
>> +
>> + ret = drv->remove(dev);
>> + if (ret < 0)
>> + VDEV_LOG(ERR, "Cleanup for device %s failed\n", rte_vdev_device_name(dev));
>> + }
>> +
>> + return ret;
>> +}
> Same comment for ret as above.
>
>> +
>> struct rte_device *
>> rte_vdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
>> const void *data)
>> @@ -627,6 +649,7 @@ vdev_get_iommu_class(void)
>> static struct rte_bus rte_vdev_bus = {
>> .scan = vdev_scan,
>> .probe = vdev_probe,
>> + .cleanup = vdev_cleanup,
>> .find_device = rte_vdev_find_device,
>> .plug = vdev_plug,
>> .unplug = vdev_unplug,
>> diff --git a/lib/eal/common/eal_common_bus.c b/lib/eal/common/eal_common_bus.c
>> index baa5b532af..046a06a2bf 100644
>> --- a/lib/eal/common/eal_common_bus.c
>> +++ b/lib/eal/common/eal_common_bus.c
>> @@ -85,6 +85,24 @@ rte_bus_probe(void)
>> return 0;
>> }
>>
>> +/* Clean up all devices of all buses */
>> +int
>> +rte_bus_cleanup(void)
>> +{
>> + int ret;
>> + struct rte_bus *bus;
>> +
>> + TAILQ_FOREACH(bus, &rte_bus_list, next) {
>> + if (bus->cleanup == NULL)
>> + continue;
>> + ret = bus->cleanup();
>> + if (ret)
>> + RTE_LOG(ERR, EAL, "Bus (%s) cleanup failed.\n", bus->name);
> Is this error message needed, if the individual buses all print out their
> own failure logs? Probably harmless enough.
>
>> + }
>> +
>> + return 0;
> Do you want to pass back up the fact that a bus cleanup failed rather than
> always returning 0?
Will change this and review the need for the logging.
>
>> +}
>> +
>> /* Dump information of a single bus */
>> static int
>> bus_dump_one(FILE *f, struct rte_bus *bus)
>> diff --git a/lib/eal/include/rte_bus.h b/lib/eal/include/rte_bus.h
>> index bbbb6efd28..42da38730f 100644
>> --- a/lib/eal/include/rte_bus.h
>> +++ b/lib/eal/include/rte_bus.h
>> @@ -66,6 +66,18 @@ typedef int (*rte_bus_scan_t)(void);
>> */
>> typedef int (*rte_bus_probe_t)(void);
>>
>> +/**
>> + * Implementation specific cleanup function which is responsible for cleaning up
>> + * devices on that bus with applicable drivers.
>> + *
>> + * This is called while iterating over each registered bus.
>> + *
>> + * @return
>> + * 0 for successful cleanup
>> + * !0 for any error during cleanup
>> + */
>> +typedef int (*rte_bus_cleanup_t)(void);
>> +
>> /**
>> * Device iterator to find a device on a bus.
>> *
>> @@ -277,6 +289,7 @@ struct rte_bus {
>> /**< handle hot-unplug failure on the bus */
>> rte_bus_sigbus_handler_t sigbus_handler;
>> /**< handle sigbus error on the bus */
>> + rte_bus_cleanup_t cleanup; /**< Cleanup devices on bus */
>>
>> };
>>
>> @@ -317,6 +330,16 @@ int rte_bus_scan(void);
>> */
>> int rte_bus_probe(void);
>>
>> +/**
>> + * For each device on the buses, perform a driver 'match' and call the
>> + * driver-specific function for device cleanup.
>> + *
>> + * @return
>> + * 0 for successful match/cleanup
>> + * !0 otherwise
>> + */
>> +int rte_bus_cleanup(void);
>> +
> This is in a public header file, so it's visible to users, but it's not in
> the version.map file, so not actually usable by anything other than EAL. We
> need to decide whether to make this function explicitly public (in which
> case it probably needs to be marked as experimental and added to
> version.map), or to decide its for EAL use only, in which case move the
> defintion to a private header file e.g. eal_private.h.
The intention is to only call this in eal_cleanup() since all
applications should call it when closing, so I think its reasonable to
move it to a private header.
Will move in v5. Thanks for reviewing.
>
>> /**
>> * Dump information of all the buses registered with EAL.
>> *
>> diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c
>> index 1ef263434a..27014fdc27 100644
>> --- a/lib/eal/linux/eal.c
>> +++ b/lib/eal/linux/eal.c
>> @@ -1266,6 +1266,7 @@ rte_eal_cleanup(void)
>> vfio_mp_sync_cleanup();
>> #endif
>> rte_mp_channel_cleanup();
>> + rte_bus_cleanup();
>> /* after this point, any DPDK pointers will become dangling */
>> rte_eal_memory_detach();
>> eal_mp_dev_hotplug_cleanup();
>> --
>> 2.31.1
>>