From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8ABDAA055B; Fri, 3 Jun 2022 16:36:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 744A040694; Fri, 3 Jun 2022 16:36:19 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id B47744021E for ; Fri, 3 Jun 2022 16:36:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1654266977; x=1685802977; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=A9iO1wYUdW1Cn9EzOd8FzUJD5U12vylucxFsy/3djm8=; b=KSbgIuqXd+OmEVJjO9pvnRaCblntmIgFVpS0DRDFZKd3uLwalnfhiq5y nkk/L3XhzaRLZTS/vawFIgx53upaRT/03YtH3yNy7PlNBITJCilgSaL2d 9Z6r/qVFZH00x5KANV3bcSYmHhOQ3dgfVlhunQTUvFsyUuQ2EzKpr8hjm upTFSYenURZ7ETrc0EijHT3DQ/KgirQat/4hcBA2ro9gBaXYeCrfAK03N RQAuTIHmBnnYMCjJhiHZDZrz5EjI1sWNw4iWcVawsBkxdAmw0r0ffkyb+ Y4JrP8R+E8Xhe6xfo1qdduyqxgntHJXyvBCWSOjuONUARH5U787aQYdSm w==; X-IronPort-AV: E=McAfee;i="6400,9594,10366"; a="256124637" X-IronPort-AV: E=Sophos;i="5.91,274,1647327600"; d="scan'208";a="256124637" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Jun 2022 07:35:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,274,1647327600"; d="scan'208";a="578044509" Received: from fmsmsx605.amr.corp.intel.com ([10.18.126.85]) by orsmga007.jf.intel.com with ESMTP; 03 Jun 2022 07:35:32 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 3 Jun 2022 07:35:31 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Fri, 3 Jun 2022 07:35:31 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.171) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Fri, 3 Jun 2022 07:35:31 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Cf4c2pynzjbh63Jr0VCSntntZaTYCTnihzht+tlXiOh2fMlWjkMUOKz6W20neXYTVkIa1J7pixK5YsQ2WK2L5PXXnmY9AQrET0aw3+Td4l4lj9lmqXMVLYHOYWqAS8ceU1yu7jM/yrpQpGrOaMEL25tPwSNRaBQiD5GV6akuzzLjeSHUjvniN/6TnhNSGi+y/OJIA2AHxt4McsncIO3bxHMRcxZxQx0tsRhvuqWOTl1o5QR2zTNNnjz1dNjdYy1krU5T90DnUg5ixoC/vWhz1gBMj74kvvM33ffDcUXkXEsRztrh9V6JcQWw3S0HxlhiEEEousYxu5fTQ8sjYveyEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XxeBD0g869chga+mYeKMud7Flf4wh8tS89GjbIYAUCo=; b=VgygEdNHqv3kh/xst+auElc/AiyGXHx9zx9VA9B37wknTnPx/HkusGnARABbhcW9pmihLFUAAHPPhXG44IsoEwu0EfV8urkhCfcDakPqRnOkW3jr495JTEMfC3j8sKjy7i9TSY254GVy1R+NpUwR+IHmCvLfUvDSvfm4YsCS+hvD4/NdLFqeyWH6I4BDlONlRSpgWkkfjy6/LryrKOminrMG21eZepw3/5/pEIMiTUjjBcGIiBYmQbvmYBEBdQfO87/Q3bfYnQ4+d1JkyxHD/5CNSZj/i3GY3sdgVSwtO6YxhjHaZ6i4Dc7bSfogWsIbksCl+n5jwrQQnHZqwwiuHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) by BN7PR11MB2547.namprd11.prod.outlook.com (2603:10b6:406:af::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.12; Fri, 3 Jun 2022 14:35:26 +0000 Received: from MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::382e:55b8:41a6:6eff]) by MW4PR11MB5872.namprd11.prod.outlook.com ([fe80::382e:55b8:41a6:6eff%6]) with mapi id 15.20.5314.012; Fri, 3 Jun 2022 14:35:26 +0000 Message-ID: <2ea583db-8738-492f-4119-ff4dff50de5c@intel.com> Date: Fri, 3 Jun 2022 15:35:21 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [PATCH v6] eal: add bus cleanup to eal cleanup To: "lihuisong (C)" , CC: =?UTF-8?Q?Morten_Br=c3=b8rup?= References: <20220419161438.1837860-1-kevin.laatz@intel.com> <20220601170234.11100-1-kevin.laatz@intel.com> From: Kevin Laatz In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO2P265CA0463.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a2::19) To MW4PR11MB5872.namprd11.prod.outlook.com (2603:10b6:303:169::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 38a24f56-b14e-4e30-38dc-08da456e4c8f X-MS-TrafficTypeDiagnostic: BN7PR11MB2547:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lhvtO6oyzHEHjH9zn/Zn/pGSbtufujKaZt28oFQi56ogMpqm9yHepbrz9S+4nE+Rpo56EHb91USf0a4RvPqW6RjqGEE58uBT0MfvdOVeabqM4iLHSw31L8jIWnVRyIpgQ+9bVr23/AN5MXswMGTn0fl5M1EJ45wrmfATW/X345jr8Pxwq/5FVhy4m/jH4ARy3XcXdTkLev/SrgYAFFXbSKoC8WkP1hw+U3kTDx3yuFNbUyEDxtrVReaWUL1EfeeoKePCWcpXo3FFgureesenQgLe97qmVPA/uSXZT4SSRBwsj2I/Mh/r6pNh3hOIKyA62MwOZjzccc4ZnjAlr6eGPRav6cOg9D9TEJrlcPdEeY/mq3TRcnT0W49MK5D+R0eY+XI82Jkhl1tdzz/YaYoZfBa30xZYQc9L0bbY1ET9toa8zb5Z8QkzgkxzTfWOdT7Ri4Mo2Z6yYri5SDQkJVdyFRwCEuf5w4lbfkR0GbBDNB/pdvp3Cxf5zZgd01j4gW1NZSHxNdX2Qfh49JeWnxRd9grc3JrYIc2W6i+vLLbtYIVKzvS/6OpkNcElF6eQo3TpOhFPHCqb2IFaL0Alyb9DQ40SRQKRLlrv/wLlSLTAMrDC0oL4/Ic41k9DkDo94ksjh6t2xYmshPLRd4lB4osbcWQORSicGQipcMymboC6Xcdd77tlVJbDx+gEj/GvwUp8s8/tZlVUQ2X9SOPSl8AuB1D0QbDIgmUUtJb3SNomhT0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW4PR11MB5872.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6666004)(6486002)(2906002)(38100700002)(508600001)(6506007)(186003)(82960400001)(316002)(31686004)(5660300002)(66476007)(66574015)(66556008)(83380400001)(86362001)(31696002)(6512007)(26005)(4326008)(44832011)(8676002)(53546011)(2616005)(8936002)(66946007)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ekZrbnZhS2VtNFdwRTN2ZWV4ekNaRXpmL0lHWnp3Sk1MN3RqNHdqRmYyemFw?= =?utf-8?B?K21tQmFFVDh5RitzRy9Hbm8rVEZHZkk3RTNmcWszYnE5MFg5TzNzOWpVcElh?= =?utf-8?B?SUFQZWVyMHM3ME5sRXBGY3p6ZUFsSDBGNURKV3g1Q3pDNDlHNTlFbmxmMjR0?= =?utf-8?B?UGpmV1EzTjh6a0hmZGJrb3RUSVJWaEMycDdwSnBTMGpFVi9PclFrUWU1bUNt?= =?utf-8?B?MUZLdlZMWGwwVUFqVDJFdEVpVzI5YkRKRGtjVWMxeFRxMHZORkUwaTVEbjU0?= =?utf-8?B?U3lkVjdqYWloVUFtZDR2cFZURktqQkdLV0YwSHNGSkRhemVnT1lhSThqT2x2?= =?utf-8?B?SG90VHpFMFV6K3V2UStpejkwQjRZc1gxZHhSeTN4Q2N6eHhSNDI2djIyMUlU?= =?utf-8?B?Ky9sR0hhajd1eGNEWjV4VU5tcTQyTzhEM1FpN0hwNWZkMWt5ZjlyT09yZCtN?= =?utf-8?B?YWFqQ3I2NXRnSzFZVXpZR1lBVjhEdTVocjdMQ3JGS0dVYXBsMVpYek9ab2gx?= =?utf-8?B?L1RnNmJkSnFnRmNFV3FVeEpFOC9VN1Z4YmdYMGowL1JEMlNwWWVDZHV5a2U1?= =?utf-8?B?UDFEV0RpRHpHVHdhUEJaSkFlTDdSRTdscFg5aDMwd1MrRjVIcjdZbW92K0Rj?= =?utf-8?B?Mi8wNXhNbWdSVlR2YnlyRmQxWWwwSEdJWGsvY0xxcGd1TmFRV2F2eWNuK3Bi?= =?utf-8?B?dVp5ZFkrelFiYmNSUnlkWUxLMW9zYXZPak5JcjhnRFRTR29VbUhRVTdFNjF5?= =?utf-8?B?dlViS1FHWTRhQ0FYVUdoRVRETm5sYXF5d3VnektYczBLMHhlenh3cUJ5NWdB?= =?utf-8?B?Y3JhbkMwZzRWNlFjVUVhaW1wNmo5UHVTbXlRSmRKeU1CRENjWmpPbDEvZWox?= =?utf-8?B?SEdEZGlORTkrL1pNQ1MzN25LaVBDTTdaWnlabE5CSEVzTXJ1NzBxMnhQeFVO?= =?utf-8?B?eGJEQ1BKbjlnL2wzcmErcFdSeU9sU3BUSzdSaDRmeHdTQnVZM2hTZHVkcjRa?= =?utf-8?B?dS9LMTlQYWc5bmJYd3FGdktGNFZQbisxeVVTOHB6THFOVmJDVU85V0NZN1Ro?= =?utf-8?B?bURsbWJKR0hyOG1NK0plN2E2dHNta0FEK2RXaXU0S2dpRlJGcENLSlNsc0lU?= =?utf-8?B?OWEvRmc4ZlNqT1NoL0VBQUtzRGlMV3g0MkplQmVFZHlXTXZIQmx0RzExWC9V?= =?utf-8?B?Sk93WmpIaXRHanYvMmFlbm9IWjRxN1hNdFlOMTJmQnptOWxVeCtFMnhDd1lv?= =?utf-8?B?eTA1em5TcmtBMjZEMmVLSmRITlY0YkhJQXdxTEFOTi92Mk9QZWxPeVVPcHR1?= =?utf-8?B?bXNhMk9DV3p0dzFrbk9hUlBUQVltd09TWGtQK1psYTltR1c4bkNOd1U0VGdI?= =?utf-8?B?eFNTMW9LNk9NYnFkeDJzQXJObi9Jd2thbHpOdGh6NG1MNWVnM1FmYXR6cEhM?= =?utf-8?B?UklnZ0M1d2NvbGltL1RqYUs1MjJObU9Wc3p6MVhpbHhuV2VmVzN6Wm9Falh2?= =?utf-8?B?RzY2Tkh2N285Q0ZHLzlpZEo2d25WL0FuV0RXNXVLdUc2S0EyS1h6aWdYeXY0?= =?utf-8?B?Z2pKdElQbVpiK2NEdWVFUmkxanBvZi9kZHhnbW9HSisyYWMyZ3F3MHZURUVz?= =?utf-8?B?SWFmM2twektIMzBoem1lMzBBSjdienZiQVBERlVlRFlQSWd6UmNVV205bkVw?= =?utf-8?B?aEFIQ2lqUFJyMVJTcUQwQm1pekk2M280c3Evc0x5VXNFbXRGNWF5VFRwbmVj?= =?utf-8?B?N2svUjlUVnNZdFFJQ3JnZzVtNXBoYUoxN20yUFI5MnljeVRxbVF3UXppMUo3?= =?utf-8?B?TWh5dS8xeXl0NXQ2V242cnU0NzNZWXNmakdCeUt0bEcrdDYxNFA5VkZSQ0RE?= =?utf-8?B?N2lFNU16Ry9kL2ZVK0t1RDRDM3g1MWl3YWtaVVR4SEErN2s2ZDNWZ1p4dEpy?= =?utf-8?B?L1NDR2ZqdUZ4ZVIyWlMzQ3dnQy9aNEZtSFp3TGZiK1MrT0gram1RYU1uZitP?= =?utf-8?B?ZFdwR0ZBSlAyb21IOW1GeDZpQlRCdVppQk1ackdSbjF0bHE0UWgyd1lZeExv?= =?utf-8?B?MXpvR3VEbVY4ZEM2d3BxRlNLUXRaTGtWNGZpZFRJa1Z4U3lPdTMyZG4xTThV?= =?utf-8?B?eENqckFEZHowSHArVHVKMzFGSDJoOWNKZzNpM0UwM0NKQlNPWXNOVTZQY0Ur?= =?utf-8?B?REZxTFFGZkE1Yk9BbzlBRmxWK20vOUIzZXJqMXBoVDdTRlZpejFCRXlwWThP?= =?utf-8?B?bzBSUHp4V2YzOWJhRzF5VUhxYXlpejZ4enBYQnhydm9OWEgxQ05Yd1hYaW13?= =?utf-8?B?a1JHREtRMXRjaXNpRUFReGhpR1dIMlc4dTNMMjAxSkdKR1ZnTUR5QT09?= X-MS-Exchange-CrossTenant-Network-Message-Id: 38a24f56-b14e-4e30-38dc-08da456e4c8f X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5872.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2022 14:35:26.1707 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bvKz9GrR9cO3/pB+GhuRS2cV5jHGD2e4zFgYWfqmIN1ldIdHYqZHaDM8Q9MR0HQOgcaUEIhiianMUdiAh1Wb/w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR11MB2547 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 02/06/2022 03:06, lihuisong (C) wrote: > Hi Kevin, > > 在 2022/6/2 1:02, Kevin Laatz 写道: >> During EAL init, all buses are probed and the devices found are >> initialized. On eal_cleanup(), the inverse does not happen, meaning any >> allocated memory and other configuration will not be cleaned up >> appropriately on exit. >> >> Currently, in order for device cleanup to take place, applications must >> call the driver-relevant functions to ensure proper cleanup is done >> before >> the application exits. Since initialization occurs for all devices on >> the >> bus, not just the devices used by an application, it requires a) >> application awareness of all bus devices that could have been probed >> on the >> system, and b) code duplication across applications to ensure cleanup is >> performed. An example of this is rte_eth_dev_close() which is >> commonly used >> across the example applications. >> >> This patch proposes adding bus cleanup to the eal_cleanup() to make >> EAL's >> init/exit more symmetrical, ensuring all bus devices are cleaned up >> appropriately without the application needing to be aware of all bus >> types >> that may have been probed during initialization. >> >> Contained in this patch are the changes required to perform cleanup for >> devices on the PCI bus and VDEV bus during eal_cleanup(). There would >> be an >> ask for bus maintainers to add the relevant cleanup for their buses >> since >> they have the domain expertise. >> >> Signed-off-by: Kevin Laatz >> Acked-by: Morten Brørup >> >> --- >> v6: >> * add bus_cleanup to eal_cleanup for FreeBSD >> * add bus_cleanup to eal_cleanup for Windows >> * remove bus cleanup function to remove rte_ prefix >> * other minor fixes >> >> v5: >> * remove unnecessary logs >> * move rte_bus_cleanup() definition to eal_private.h >> * fix return values for vdev_cleanup and pci_cleanup >> >> v4: >> * rebase >> >> v3: >> * add vdev bus cleanup >> >> v2: >> * change log level from INFO to DEBUG for PCI cleanup >> * add abignore entries for rte_bus related false positives >> >> --- >>   devtools/libabigail.abignore    |  9 +++++++++ >>   drivers/bus/pci/pci_common.c    | 24 ++++++++++++++++++++++++ >>   drivers/bus/vdev/vdev.c         | 24 ++++++++++++++++++++++++ >>   lib/eal/common/eal_common_bus.c | 17 +++++++++++++++++ >>   lib/eal/common/eal_private.h    | 10 ++++++++++ >>   lib/eal/freebsd/eal.c           |  1 + >>   lib/eal/include/rte_bus.h       | 13 +++++++++++++ >>   lib/eal/linux/eal.c             |  1 + >>   lib/eal/windows/eal.c           |  1 + >>   9 files changed, 100 insertions(+) >> >> diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore >> index 79ff15dc4e..3e519ee42a 100644 >> --- a/devtools/libabigail.abignore >> +++ b/devtools/libabigail.abignore >> @@ -56,3 +56,12 @@ >>   ; Ignore libabigail false-positive in clang builds, after moving code. >>   [suppress_function] >>       name = rte_eal_remote_launch >> + >> +; Ignore field inserted to rte_bus, adding cleanup function >> +[suppress_type] >> +        name = rte_bus >> +        has_data_member_inserted_at = end >> + >> +; Ignore changes to internally used structs containing rte_bus >> +[suppress_type] >> +        name = rte_pci_bus, rte_vmbus_bus, rte_vdev_bus >> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c >> index 37ab879779..8b132ce5fc 100644 >> --- a/drivers/bus/pci/pci_common.c >> +++ b/drivers/bus/pci/pci_common.c >> @@ -394,6 +394,29 @@ pci_probe(void) >>       return (probed && probed == failed) ? -1 : 0; >>   } >>   +static int >> +pci_cleanup(void) >> +{ >> +    struct rte_pci_device *dev = NULL; >> +    int error = 0; >> + >> +    FOREACH_DEVICE_ON_PCIBUS(dev) { >> +        struct rte_pci_driver *drv = dev->driver; >> +        int ret = 0; >> + >> +        if (drv == NULL || drv->remove == NULL) >> +            continue; >> + >> +        ret = drv->remove(dev); > All devices, such as, compressdev, ethdev and dmadev, on the bus are > released here. > However, the rte_pci_device or rte_vdev_device on the bus allocated > during EAL init > are not yet released. Why not free these devices here? v7 sent with this change added, thanks. /Kevin