From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46D4C41DE6; Mon, 6 Mar 2023 12:00:47 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2457540ED6; Mon, 6 Mar 2023 12:00:47 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2047.outbound.protection.outlook.com [40.107.93.47]) by mails.dpdk.org (Postfix) with ESMTP id 4CFA740A8A for ; Mon, 6 Mar 2023 12:00:46 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L0W9FOPokwc6GI6j232LgUgsru7F3c2xGB3MmzIoHwGOxpdTr+04d15+SIAMa84FkAgs86kAP+Q5V5q5zhKpBSEO5VmNafqoOFw50etGC1GKxyPrlKvCD6sWSzI4s5LE1SVeFL2ohbudG3nAiOLgkg0V9/PWR0VR9fJbSm+zeEelrEroresSSsAIcxREMotZX0PxVdb8TugRik9BXHv+oi0ZD0EsNiRZRZOQUpJvZXmc9Q+NEzCIxmqzBeWPRfwGlTnWJGcsNxr93TDSxnO5AmDsvFbg4BqD/CjRa6WK3KCfXMxukFyKNd65BxgzGHP15YOC/JesMMMMMPLOB3mTZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mix6LP3WhSAKZilO0nuI9WHVq7YprYn/wxNAfft18bI=; b=Y6qf4ji01MPYjzC1huuBV+gcJc7P8jxdhMJrgNDXSKBnbMSzULIJBfhxHz97P7Fj7wRhBQ0s3tQu3R8/fvAGa+o9bzn0ASbwZEBs1U5Ke6hp7GDcsmJo4hl/ms9juxNCWDs1ogbS2eoJXelpf6JAKcUqBtrOnBN6yuo6rMQPwsxVw7kXtXvNjr4xcUcjKB+DQq/WTf/CRX6B6kWbrnIdZfSKmCiMO9X2/KQZg0/0gd+K9lL2FOVkfczN2iWQ6rBMRQtgDdfzRwxd72Brj6iovZmEqV7NYUFc2kGxLyeKc6PHlaX4rFH9D7pNz50jxoss3vDTiLaxSrPxGtsjhFbmww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mix6LP3WhSAKZilO0nuI9WHVq7YprYn/wxNAfft18bI=; b=aMvO+F1A15pcIyxgQef7sshHy8CZU+QdIySsAwv7hND/e67GfJ+vPIq6/xdwSuzS95A6e840AKh8nMzQRzO3fUaBx85DPTqfVJwCua+lyJxKj1HKicNKgzLmlgzx0YRpPFhgAOCAIfb3b2yzo+00z1HeU+LXDyrnVue4BOZHSK4= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by BY5PR12MB4917.namprd12.prod.outlook.com (2603:10b6:a03:1d1::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.28; Mon, 6 Mar 2023 11:00:43 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::dd5a:8a5c:f493:9640]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::dd5a:8a5c:f493:9640%5]) with mapi id 15.20.6156.028; Mon, 6 Mar 2023 11:00:43 +0000 Message-ID: <0effdaa9-5045-0635-775c-6e1eda0d5dba@amd.com> Date: Mon, 6 Mar 2023 11:00:36 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Content-Language: en-US To: Konstantin Ananyev , Thomas Monjalon , Andrew Rybchenko References: <20230301030610.49468-1-fengchengwen@huawei.com> <20230301030610.49468-2-fengchengwen@huawei.com> <0f387ca1eee34a7f92745de7b59a71a1@huawei.com> <18c5b676-ae72-e646-89af-d6cd636d923f@yandex.ru> <5b40049f-6f22-fc3f-b13e-da1793c2fd1a@amd.com> <9092fdae9c1d4c53a00a8f23eb1129ec@huawei.com> From: Ferruh Yigit Cc: "dev@dpdk.org" , fengchengwen Subject: Re: [PATCH 1/5] ethdev: fix race-condition of proactive error handling mode In-Reply-To: <9092fdae9c1d4c53a00a8f23eb1129ec@huawei.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LNXP265CA0043.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:5c::31) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|BY5PR12MB4917:EE_ X-MS-Office365-Filtering-Correlation-Id: a38636fb-db1a-452f-b7ab-08db1e3207bf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e0awUvRuYuiZOceuWQU+8fMXO51WYYxcMA+htlvAFVpuTp0W4xEooSSPxwOghglWVR0M3FIJVEwHV/OdWzkiUK+/3p5EGiHy2r+Ypn1XikCxHDICwe1gt1NC2lXQv0BaxcQTuh54BveSIaclYWlY26JsNZU+7t8ymsv9AQj5xO+LcYgaxwMk5SqxeHHWxE5JwOkE0TuyGi8uW6aKT2iIrr/WHDuEzFt0duzTy3rCQCpactVt7N4gV5a7NtbLFzb2orJkGER+ObYz4uztLUpVSjsMMZwTFz9M8YLGSdc1RxMQvthkeARs0AgA1QN2p4j9X2f+/EKHvNZ/OAnrSWu7w/J2id7vclczAiftR0ud7Fb+2xWi0YH/7Je6eigSs6X9ckmeADsKdEdfaVW0roXnUhfvG2D0pX09q+nb/8x8KqcyPd9RWCChFqtsKaraBv3Ki0kfzQSolzlttkOtOcBLw3GNRmnESs5v5Wvu1gN/3OuxFlsfIFpo+qMXB6zUQdmWBxKSQ4cFrZ3VWTyyxJ/XZZDeQXaZ86i1J/7a5c32I2g1oD53Di23UVjrK9EUNNRpcQjGYH9wD1fN4vfZguYHNeDeMQNOCMmCw/VD4E373jEFqGuBlfbDPMOVyrnA4gPdUESUpOBURU/QTCABGkCrdAuy6SukGCx+HW455TB5+OzrOmu4PM7Tz6UCdsrknQMwPXOFiPPWdmsZrIe7lWNEjYDG76DO2fmxLS/rxYXYx7g= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(39860400002)(346002)(376002)(136003)(366004)(396003)(451199018)(110136005)(54906003)(36756003)(6486002)(966005)(316002)(83380400001)(86362001)(31696002)(31686004)(186003)(2906002)(44832011)(5660300002)(30864003)(66556008)(8936002)(66946007)(41300700001)(66476007)(4326008)(8676002)(478600001)(53546011)(6512007)(6506007)(26005)(2616005)(6666004)(38100700002)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?czk4Wng5NWQ2Z2NoK3lFeE4wYmhmRms2VUt1UHBLNC9uZG9NQUoweFNtdENz?= =?utf-8?B?YkFxR282WEllYk5LTnkyYzNQaFNISVBRL2NPcE53azFOVkMrK28xeVRaSFA3?= =?utf-8?B?MlVMVzhINW1KMGxVOE5rRGhWcnFnMVpuWVdmTHp0dUVjRmdPVWxYR01YYThv?= =?utf-8?B?NUU0TVdWWkhyWGJLTzY4ODZVQS9PWU1lWlRZT3FqYW9ZeDcwbmU3UUZsU1lJ?= =?utf-8?B?SHd5N2hzQnBtbWFTMzkyb3c5Ukt2YXM1WWpUbUhKd0tZcVUrMCtEelpDTGJl?= =?utf-8?B?TFhRTkViUEhDUTRZTTJodHpJZFQ5cnRDSFpmamkvRkcwT2pDSmRqd2V4NFd0?= =?utf-8?B?UnpsSkJTMTVDRzJrWUw2c1lhU1dwRkJINTRPQ0g1Q01vK0JBMjFGb082L2Z4?= =?utf-8?B?Q1o5a3V3ZTIrMUZYWXE2Tmo2WkZPUnpNTldueVFXem9GOElnaUIrQ2FCdFYr?= =?utf-8?B?T0hrYmlQWkNyL3B5QkdYbG5ESWtnUzVXYlQvc0FzTnFKWmdWYmJOODJOT3Ns?= =?utf-8?B?V1p3WmNUSFJQamlLdit3M2tQRkVNdkhqVFhvOXZxdEVoTk94elhjQTBneTZU?= =?utf-8?B?RVpMeENMVVlaUUxJdlhrK1FQaE9LNVRpd1lFOWtRbEQyRmFnZlRRdGZCRTlD?= =?utf-8?B?U05ucFdJK25UVXV0YklCanVjdU1HZ29PdUtPcDBKRVB2Z1YxUzRHQkdIWVBt?= =?utf-8?B?Ykw5T0c0Z2RwTTNVZmZJelFLeG8vZytpdml4SG4yMFhiZ2txK0JiTVo0R3k4?= =?utf-8?B?cy8yQUVVTzJrUC8vejNNbHlPR0dBZkFleDllR3J0ZFp4akhoRThmaU1hY09H?= =?utf-8?B?bVJtYWdYc1FJZGoydU8rbC83Y0pkajVYcjBkZHRnZmZTc0hDVHBZM1BOc0FF?= =?utf-8?B?aW5Mbk5vNnAxZjlxamNCNWxvbHUreCt6RjhlTUdBYTdpcEw0T0M3aHhJdWdI?= =?utf-8?B?S2cwcUh4azlCUTI2Y1dhQ3JvRFFxOUxnNTF4L2U0YmJRZVhjQ2Y3QVBhanhI?= =?utf-8?B?NmdWM2ljaEtRTHE3cXVQaDFJZnVOaTMzK0xGd1VEQnF0SjR1VTNqeWZsL2FO?= =?utf-8?B?Tm4yMVVxUUx0elJxdUUzR2V6ZEJwd1BtZFQzbWdja0dBVUk3OUFhNTdYcVZP?= =?utf-8?B?d3p5U0dPaThSUkxBRnlsVXRsNTR2QjdzUEROWTNRNm8wdkwxSDd1WGZGNE92?= =?utf-8?B?RWRNWmxpMmd4UHBFQWJsRUNiZDNobFB2STJXb2cyY2VUQVZpa1BFT0ROaUtj?= =?utf-8?B?ckp3QXNGSm1mdjhlQjdlZjlLRkJFMTlyWG0rMkZhVktHekw0ZEtVNkdnNWZH?= =?utf-8?B?d0pWODcrWVJRdExnaHNSMmdzOFByT3ZqcGdQL0RGcUplbGE2eHo4SEFydUEx?= =?utf-8?B?Q1lsRCsxWG1LWFdKRW5vay9obUhTdnFnYmlHZkJtS0g3S1Y0RUdFQ1kzN0dN?= =?utf-8?B?K25wRWxuRWxGYWFrVG42NGRINWRIbDVXdGZCYzhOR0d2YVJNTGo3ZWhackNI?= =?utf-8?B?bE03VlFQSDhjTFJ1YnFXaC82SkhzVm80SE9nRksyVGxtZFl5VzFSalhHNmc0?= =?utf-8?B?K2pCRzJpWUNzcFZ6a2xzZlQwb0JvdTQ4Yzl6UDdvTlloYWJrZ2xmeWc2djcr?= =?utf-8?B?U2F2OEk4aGd5WDdaY3Q5MVJyelRyWHgySnBxMjRSdytzQmFwY3NxYUdHdFIv?= =?utf-8?B?cEtBUjJROUNyMUZ6RDA3Qk9TVXV3L3dIZUx4RGl3c0JwdUQ4akdMV2RjR0dw?= =?utf-8?B?UmhYWUFWMVdCT21TSy85WDNWSmJrNmp4NU9CVVp0UWQ3dXZhTVJpSFZ0NTYr?= =?utf-8?B?VUY5a1NNVldveVBVQW8yQjd3QXFsbGtrSUlTUHVNd0FiTmFKdVltNkRDVWd6?= =?utf-8?B?VDQrU25IN2wyejNuNGJGVllOQStmaVFYN3FTdW82ZkFxN0RIdjA2bGJBbnJE?= =?utf-8?B?VFk1cFNSRVBwWGlMOXFXTDNTY2NxOGdIOVRKYTV6TnhoTFFmTjNGbXRBQnp0?= =?utf-8?B?cVpDRnZCWER4RmdsNlhHUFJYcExHVVVRQ2xUZm85S2xLeUhDekpLUUpvY2Q5?= =?utf-8?B?dEZBTDFVNHV5VE0vbjhMeW50NERwM2s0eDV6MFRSZ0NPRE8rWTd4TVlUTTA0?= =?utf-8?Q?VBx1OLR5ze+EynyH73+SgNIwO?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a38636fb-db1a-452f-b7ab-08db1e3207bf X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2023 11:00:43.2448 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dFjY3PrNd3lM0LZKZZvQHpXGyAmK6FdO2xTo0xPWx0Q+y+M3H2qsYeKOSX2iImiv X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4917 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 3/6/2023 10:22 AM, Konstantin Ananyev wrote: > > >>>>>> In the proactive error handling mode, the PMD will set the data path >>>>>> pointers to dummy functions and then try recovery, in this period the >>>>>> application may still invoking data path API. This will introduce a >>>>>> race-condition with data path which may lead to crash [1]. >>>>>> >>>>>> Although the PMD added delay after setting data path pointers to cover >>>>>> the above race-condition, it reduces the probability, but it doesn't >>>>>> solve the problem. >>>>>> >>>>>> To solve the race-condition problem fundamentally, the following >>>>>> requirements are added: >>>>>> 1. The PMD should set the data path pointers to dummy functions after >>>>>>     report RTE_ETH_EVENT_ERR_RECOVERING event. >>>>>> 2. The application should stop data path API invocation when process >>>>>>     the RTE_ETH_EVENT_ERR_RECOVERING event. >>>>>> 3. The PMD should set the data path pointers to valid functions before >>>>>>     report RTE_ETH_EVENT_RECOVERY_SUCCESS event. >>>>>> 4. The application should enable data path API invocation when process >>>>>>     the RTE_ETH_EVENT_RECOVERY_SUCCESS event. >>>>>> >>>> >>>> How this is solving the race-condition, by pushing responsibility to >>>> stop data path to application? >>> >>> Exactly, it becomes application responsibility to make sure data-path is >>> stopped/suspended before recovery will continue. >>> >> >> From documentation of the feature: >> >> `` >> Because the PMD recovers automatically, >> the application can only sense that the data flow is disconnected for a >> while and the control API returns an error in this period. >> >> In order to sense the error happening/recovering, as well as to restore >> some additional configuration, three events are available: >> `` >> >> It looks like initial design is to use events mainly inform application >> about what happened and mainly for re-configuration. >> >> Although I am don't disagree to involve the application, I am not sure >> that is part of current design. > > I thought we all agreed that initial design contain some fallacies that > need to fixed, no? > Statement that with current rte_ethdev design error recovery can be done > without interaction with the app (to stop/suspend data/control path) > is the main one I think. > It needs some interaction with app layer, one way or another. > >>>> >>>> What if application is not interested in recovery modes at all and not >>>> registered any callback for the recovery? >>> >>> >>> Are you saying there is no way for application to disable >>> automatic recovery in PMD if it is not interested >>> (or can't full-fill per-requesties for it)? >>> If so, then yes it is a problem and we need to fix it. >>> I assumed that such mechanism to disable unwanted events already exists, >>> but I can't find anything. >>> Wonder what would be the easiest way here - can PMD make a decision >>> based on callback return value, or do we need a new API to >>> enable/disable callbacks, or ...? >>> >>> >> >> As far as I can see automatic recovery is not configurable by app. >> >> But that is not all, PMD sends events to application but PMD can't know >> if application is handling them or not, so with current design PMD can't >> rely on to app. > > Well, PMD invokes user provided callback. > One way to fix that problem - if there is no callback provided, > or callback returns an error code - PMD can assume that recovery > should not be done. > That is probably not the best design choice, but at least it will allow > to fix the problem without too many changes and introducing new API. > That could be sort of a 'quick fix'. > In a meanwhile we can think about new/better approach for that. > -rc2 for 23.03 is a few days away. What do you think to have 'quick fix' as modifying how driver updates burst ops to prevent the race condition, for this release? And plan a design update for the next release? >> >>>> I think driver should not rely on application for this, unless >>>> application explicitly says (to driver) that it is handling recovery, >>>> right now there is no way for driver to know this. >>> >>> I think it is visa-versa: >>> application should not enable auto-recovery if it can't meet >>> per-requeststies for it (provide appropriate callback). >>> >> >> I agree on above, we are saying similar thing in different perspective. > > Ok, that's good we are on the same page. > > >> >>> >>>> >>>>>> Also, this patch introduce a driver internal function >>>>>> rte_eth_fp_ops_setup which used as an help function for PMD. >>>>>> >>>>>> [1] >>>>>> http://patchwork.dpdk.org/project/dpdk/patch/20230220060839.1267349-2-ashok.k.kaladi@intel.com/ >>>>>> >>>>>> Fixes: eb0d471a8941 ("ethdev: add proactive error handling mode") >>>>>> Cc: stable@dpdk.org >>>>>> >>>>>> Signed-off-by: Chengwen Feng >>>>>> --- >>>>>>   doc/guides/prog_guide/poll_mode_drv.rst | 20 +++++++--------- >>>>>>   lib/ethdev/ethdev_driver.c              |  8 +++++++ >>>>>>   lib/ethdev/ethdev_driver.h              | 10 ++++++++ >>>>>>   lib/ethdev/rte_ethdev.h                 | 32 >>>>>> +++++++++++++++---------- >>>>>>   lib/ethdev/version.map                  |  1 + >>>>>>   5 files changed, 46 insertions(+), 25 deletions(-) >>>>>> >>>>>> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst >>>>>> b/doc/guides/prog_guide/poll_mode_drv.rst >>>>>> index c145a9066c..e380ff135a 100644 >>>>>> --- a/doc/guides/prog_guide/poll_mode_drv.rst >>>>>> +++ b/doc/guides/prog_guide/poll_mode_drv.rst >>>>>> @@ -638,14 +638,9 @@ different from the application invokes recovery >>>>>> in PASSIVE mode, >>>>>>   the PMD automatically recovers from error in PROACTIVE mode, >>>>>>   and only a small amount of work is required for the application. >>>>>> >>>>>> -During error detection and automatic recovery, >>>>>> -the PMD sets the data path pointers to dummy functions >>>>>> -(which will prevent the crash), >>>>>> -and also make sure the control path operations fail with a return >>>>>> code ``-EBUSY``. >>>>>> - >>>>>> -Because the PMD recovers automatically, >>>>>> -the application can only sense that the data flow is disconnected >>>>>> for a while >>>>>> -and the control API returns an error in this period. >>>>>> +During error detection and automatic recovery, the PMD sets the >>>>>> data path >>>>>> +pointers to dummy functions and also make sure the control path >>>>>> operations >>>>>> +failed with a return code ``-EBUSY``. >>>>>> >>>>>>   In order to sense the error happening/recovering, >>>>>>   as well as to restore some additional configuration, >>>>>> @@ -653,9 +648,9 @@ three events are available: >>>>>> >>>>>>   ``RTE_ETH_EVENT_ERR_RECOVERING`` >>>>>>      Notify the application that an error is detected >>>>>> -   and the recovery is being started. >>>>>> +   and the recovery is about to start. >>>>>>      Upon receiving the event, the application should not invoke >>>>>> -   any control path function until receiving >>>>>> +   any control and data path API until receiving >>>>>>      ``RTE_ETH_EVENT_RECOVERY_SUCCESS`` or >>>>>> ``RTE_ETH_EVENT_RECOVERY_FAILED`` event. >>>>>> >>>>>>   .. note:: >>>>>> @@ -666,8 +661,9 @@ three events are available: >>>>>> >>>>>>   ``RTE_ETH_EVENT_RECOVERY_SUCCESS`` >>>>>>      Notify the application that the recovery from error is successful, >>>>>> -   the PMD already re-configures the port, >>>>>> -   and the effect is the same as a restart operation. >>>>>> +   the PMD already re-configures the port. >>>>>> +   The application should restore some additional configuration, >>>>>> and then >>>>>> +   enable data path API invocation. >>>>>> >>>>>>   ``RTE_ETH_EVENT_RECOVERY_FAILED`` >>>>>>      Notify the application that the recovery from error failed, >>>>>> diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c >>>>>> index 0be1e8ca04..f994653fe9 100644 >>>>>> --- a/lib/ethdev/ethdev_driver.c >>>>>> +++ b/lib/ethdev/ethdev_driver.c >>>>>> @@ -515,6 +515,14 @@ rte_eth_dma_zone_free(const struct rte_eth_dev >>>>>> *dev, const char *ring_name, >>>>>>       return rc; >>>>>>   } >>>>>> >>>>>> +void >>>>>> +rte_eth_fp_ops_setup(struct rte_eth_dev *dev) >>>>>> +{ >>>>>> +    if (dev == NULL) >>>>>> +        return; >>>>>> +    eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); >>>>>> +} >>>>>> + >>>>>>   const struct rte_memzone * >>>>>>   rte_eth_dma_zone_reserve(const struct rte_eth_dev *dev, const char >>>>>> *ring_name, >>>>>>                uint16_t queue_id, size_t size, unsigned int align, >>>>>> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h >>>>>> index 2c9d615fb5..0d964d1f67 100644 >>>>>> --- a/lib/ethdev/ethdev_driver.h >>>>>> +++ b/lib/ethdev/ethdev_driver.h >>>>>> @@ -1621,6 +1621,16 @@ int >>>>>>   rte_eth_dma_zone_free(const struct rte_eth_dev *eth_dev, const >>>>>> char *name, >>>>>>            uint16_t queue_id); >>>>>> >>>>>> +/** >>>>>> + * @internal >>>>>> + * Setup eth fast-path API to ethdev values. >>>>>> + * >>>>>> + * @param dev >>>>>> + *  Pointer to struct rte_eth_dev. >>>>>> + */ >>>>>> +__rte_internal >>>>>> +void rte_eth_fp_ops_setup(struct rte_eth_dev *dev); >>>>>> + >>>>>>   /** >>>>>>    * @internal >>>>>>    * Atomically set the link status for the specific device. >>>>>> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h >>>>>> index 049641d57c..44ee7229c1 100644 >>>>>> --- a/lib/ethdev/rte_ethdev.h >>>>>> +++ b/lib/ethdev/rte_ethdev.h >>>>>> @@ -3944,25 +3944,28 @@ enum rte_eth_event_type { >>>>>>        */ >>>>>>       RTE_ETH_EVENT_RX_AVAIL_THRESH, >>>>>>       /** Port recovering from a hardware or firmware error. >>>>>> -     * If PMD supports proactive error recovery, >>>>>> -     * it should trigger this event to notify application >>>>>> -     * that it detected an error and the recovery is being started. >>>>>> -     * Upon receiving the event, the application should not invoke >>>>>> any control path API >>>>>> -     * (such as rte_eth_dev_configure/rte_eth_dev_stop...) until >>>>>> receiving >>>>>> -     * RTE_ETH_EVENT_RECOVERY_SUCCESS or >>>>>> RTE_ETH_EVENT_RECOVERY_FAILED event. >>>>>> -     * The PMD will set the data path pointers to dummy functions, >>>>>> -     * and re-set the data path pointers to non-dummy functions >>>>>> -     * before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS event. >>>>>> -     * It means that the application cannot send or receive any >>>>>> packets >>>>>> -     * during this period. >>>>>> +     * >>>>>> +     * If PMD supports proactive error recovery, it should trigger >>>>>> this >>>>>> +     * event to notify application that it detected an error and the >>>>>> +     * recovery is about to start. >>>>>> +     * >>>>>> +     * Upon receiving the event, the application should not invoke any >>>>>> +     * control and data path API until receiving >>>>>> +     * RTE_ETH_EVENT_RECOVERY_SUCCESS or RTE_ETH_EVENT_RECOVERY_FAILED >>>>>> +     * event. >>>>>> +     * >>>>>> +     * Once this event is reported, the PMD will set the data path >>>>>> pointers >>>>>> +     * to dummy functions, and re-set the data path pointers to valid >>>>>> +     * functions before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS >>>>>> event. >>>>>> +     * >>>>>>        * @note Before the PMD reports the recovery result, >>>>>>        * the PMD may report the RTE_ETH_EVENT_ERR_RECOVERING event >>>>>> again, >>>>>>        * because a larger error may occur during the recovery. >>>>>>        */ >>>>>>       RTE_ETH_EVENT_ERR_RECOVERING, >>>>>>       /** Port recovers successfully from the error. >>>>>> -     * The PMD already re-configured the port, >>>>>> -     * and the effect is the same as a restart operation. >>>>>> +     * >>>>>> +     * The PMD already re-configured the port: >>>>>>        * a) The following operation will be retained: (alphabetically) >>>>>>        *    - DCB configuration >>>>>>        *    - FEC configuration >>>>>> @@ -3989,6 +3992,9 @@ enum rte_eth_event_type { >>>>>>        *      (@see RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP) >>>>>>        * c) Any other configuration will not be stored >>>>>>        *    and will need to be re-configured. >>>>>> +     * >>>>>> +     * The application should restore some additional configuration >>>>>> +     * (see above case b/c), and then enable data path API invocation. >>>>>>        */ >>>>>>       RTE_ETH_EVENT_RECOVERY_SUCCESS, >>>>>>       /** Port recovery failed. >>>>>> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map >>>>>> index 357d1a88c0..c273e0bdae 100644 >>>>>> --- a/lib/ethdev/version.map >>>>>> +++ b/lib/ethdev/version.map >>>>>> @@ -320,6 +320,7 @@ INTERNAL { >>>>>>       rte_eth_devices; >>>>>>       rte_eth_dma_zone_free; >>>>>>       rte_eth_dma_zone_reserve; >>>>>> +    rte_eth_fp_ops_setup; >>>>>>       rte_eth_hairpin_queue_peer_bind; >>>>>>       rte_eth_hairpin_queue_peer_unbind; >>>>>>       rte_eth_hairpin_queue_peer_update; >>>>>> -- >>>>>   Acked-by: Konstantin Ananyev >>>>> >>>>>> 2.17.1 >>>>> >>>> >>> >