From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A212141DF3; Mon, 6 Mar 2023 09:56:13 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8961D40A8A; Mon, 6 Mar 2023 09:56:13 +0100 (CET) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2053.outbound.protection.outlook.com [40.107.95.53]) by mails.dpdk.org (Postfix) with ESMTP id 34F944067B for ; Mon, 6 Mar 2023 09:56:12 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GK3/4lqMe0Z7EIIoX/44IXQzyZnkDu0z73kuhAl85TlteeFkwzv0clYM3j7gPqwdJvbnWQqh2XJV6L3RQM2rcHX1hS9QJXDGVBtkQh2UlpVEvVjDckQ3UmjR/NVijje71FanZrGOWXrIzV+ONULu3itP8h0omeBa7HsqgEm1ZpPKR98fOGR5Kc6FXEO16wrPsrdDuMBLeYByXq6TKFIQmyLvMQjCAWxccJAP5ZtCrHomPfSIQK2LNpwY8eZf16+bH64tfv9PGIzpUtLCHu6d9SpK3nTPGtLWrNKaQLXI3qd4NXHX1TSa5bd7cwOabiOOS2Mk2UCLZNVXTeZHyzdxkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WywQfDCn5quuoDYoQNcE8hpPIH/8zNOIkugY9tBy2VQ=; b=BphTRNbJB1V5WgC9PKlG2ipJ7GQ3681ktJO/D0JvUV9bLuXzj4dGyQq6xZPJrPi2xoVEucKmpLr2KP4aBJ0SpK8QIhwzu+oKO2C0KIcy543VvOgLtFDdhmS4zUc3A8ZFiLABP2oMN5GKnS9aHKnD7iDw4z/tksQZzJ8Eu6ooWFboeVe0DeY5kHDSKOPJP1WoX7nI7nZ2FTJdoV5uMQT7AUxOl4NrV2YqgswR0fAzBd62zLt+CLauVXjq9O48oqqztK4/GnfhIZioayKBpHzDz/roDeQOOe9aEUzUrOihbImt06y3MJbrEnnMTjHnK1CA4WtzUdikosCQOK0SSMD6aA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WywQfDCn5quuoDYoQNcE8hpPIH/8zNOIkugY9tBy2VQ=; b=C6cOUUk0MQAXgtSGhvGb21JcoTh4J4yyQkvAaj0UY/UnCvg28sJyV1TFKIg61BvLFEpGqPVzefEFbTUfStGwb4/M24mJ1l1KTUSA5D0M/3cZeQ+B1XSMDzAIM4ejEIt9+kdgMyya1qnjN8Y3zSJ4VH8cNXQ5wUtVpHTkR13KEyg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from MN2PR12MB4301.namprd12.prod.outlook.com (2603:10b6:208:1d4::22) by BY1PR12MB8445.namprd12.prod.outlook.com (2603:10b6:a03:523::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.28; Mon, 6 Mar 2023 08:56:09 +0000 Received: from MN2PR12MB4301.namprd12.prod.outlook.com ([fe80::80ae:e5ed:4fa7:2ad7]) by MN2PR12MB4301.namprd12.prod.outlook.com ([fe80::80ae:e5ed:4fa7:2ad7%8]) with mapi id 15.20.6156.028; Mon, 6 Mar 2023 08:56:09 +0000 Message-ID: <5b40049f-6f22-fc3f-b13e-da1793c2fd1a@amd.com> Date: Mon, 6 Mar 2023 08:55:02 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Content-Language: en-US To: Konstantin Ananyev , dev@dpdk.org References: <20230301030610.49468-1-fengchengwen@huawei.com> <20230301030610.49468-2-fengchengwen@huawei.com> <0f387ca1eee34a7f92745de7b59a71a1@huawei.com> <18c5b676-ae72-e646-89af-d6cd636d923f@yandex.ru> From: Ferruh Yigit Subject: Re: [PATCH 1/5] ethdev: fix race-condition of proactive error handling mode In-Reply-To: <18c5b676-ae72-e646-89af-d6cd636d923f@yandex.ru> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P123CA0048.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:152::17) To MN2PR12MB4301.namprd12.prod.outlook.com (2603:10b6:208:1d4::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR12MB4301:EE_|BY1PR12MB8445:EE_ X-MS-Office365-Filtering-Correlation-Id: f5c3fccb-c02a-4fc6-050f-08db1e20a0ff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iiaFmwTmNxsdh/qXhpiV0o+74qgBGdO43uhi3t3fThh13QYsiT6jUBTF97S1d1oL5pfMt4E5b4/VvQIg51pDTUFyq6dMlGdvhP7oXKRxAETNVx+UICkGjBwL9MPx7gHynwTHH9TVpYzoYojuVz/rcI5RlFYAvH2drkISfnVdKlxfuOlalXhY295Lu8ryVeuSCDymyoHeBQeHccc9PrLf357zUnphy22/2kSeMuMSR7AitaH6MmWDz5esBl6HrxODrItL9tUNnQ2RyNJZAmdH2qG9teg4eBcNRJT6nukqJKK5Uwc0cD7HCYZ8cXDpe9lHZVsAOlJMaYAl3rHTe4TzhKZyi9agghOEWv9IDdswmerQYWT3N5qR5JVEkeoCEZXHoaEOHRsysmQsHtJKeyEdHOhHHwuo1VyaBzJI+X230gnRDUAwe9Ffm3wM7CbMYJb/GdgIUqxbssAVa3M8PaRae+IMOTCSfJynl90AW4kOC9GsKZVLwS7niPMZmTv6WaEdXiwbiAKViEYMUoybCXwIICtnEwCQqY9ppbVZzagwAsKgaYX1mY4f2lrAMFyiXDRRy6gTg5/3XZWTY8iVVSfFzwgHldLrJdBpF/YOfCtmDJdPgrr4LZSzX8WIJstIf7kTrSA/GOVxlzC8I5gY/eGsoG48ufj0/MFvg1/HJWrrsDeq0WxCj8CXdrE7CW7pw4f0tufvzt3MbCO/lcWi3dB/tWF3ElnI5GZNjp9bQUCDIow= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB4301.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(136003)(366004)(39860400002)(396003)(376002)(346002)(451199018)(186003)(38100700002)(8936002)(66556008)(66476007)(8676002)(41300700001)(66946007)(44832011)(2906002)(5660300002)(30864003)(478600001)(6666004)(2616005)(53546011)(6512007)(26005)(6506007)(6486002)(966005)(316002)(36756003)(86362001)(83380400001)(31696002)(31686004)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NlZoVXFEcjQxRXBDNllLRWdEQ0F1SHlzZGFoNEZvUVVFb0l2MTJ3RFFNUlR6?= =?utf-8?B?SkZ5ZkM4bWVuUzlPRC9LYW1EV0pLM0FKQk9CMm41Z1pRTE42UmpKK1FFc1Yz?= =?utf-8?B?YzIweFdHTmlDN2E3UDZ5VVE3SjJBRjRVd2FTbnlDMXNVaVprby83VVhRMkI5?= =?utf-8?B?T0lPNlRtOTNVK3c0TlZ6UTJSOTFIbFRrU1d0NENjZWR3cVBNSmlaL2kyWGFq?= =?utf-8?B?RGtLUCs3SkZMVnFyYU9qS1BEcWFRa0dLekZsK01vamE2bC92MWFBV081T1VU?= =?utf-8?B?NElKWGtFV3dJRlJWdHE1clppem9wMFQwSFBQc2RxY0xkMkM0VlluTFVCcjlu?= =?utf-8?B?U25DZStGcmppeHFxMTBkbEFjZStqaDZYZzJoWU51a1J6b2NrdUNubFJYYStW?= =?utf-8?B?ZWZCTGJRSzE0QW0xMU9LZkxsVFBuMUJEc05JTkZhRlluRUVScU5TRmw1em5o?= =?utf-8?B?dUQ2U1FIeXVQWVovbHBDME4rVVd0NzhaNDNLNWtsUU9UVGZjejN1bHhRaFBp?= =?utf-8?B?RmhvT2FHdmQ3N1FJcU8zV2dFWDlCYjAwaUZvZ2hPTlJybVlzWHhsWndhUHVy?= =?utf-8?B?cEdJb29xOG16b3VXZzgzUHhvQnVHL0NIV2ZBSDg4MjZ2Sjd3RWVNeXUrb1Ir?= =?utf-8?B?RFQyNURka3c5MlNOUmJEZHY5WGpaaW9kSDZEK202M1NXcE02NFZKZE9IRVRX?= =?utf-8?B?YVo4bWdUUktUcEZyNU56U0RiVUZEd2lVTDVjVWp6d3JTVnZYbGhRdHZlVUU3?= =?utf-8?B?aE9EL1E0aEZBbHkrYU9FUTdoL2JSRTIxR3laQktFL1lyREdJTkxFNkF2VmY4?= =?utf-8?B?cTE0Y2RnNG9oTkgzL29mcFM4MFpVSGlDNzVkWG9DTkRudktSMFd0TzMwYSti?= =?utf-8?B?R1BkWFh3dm42WGFwYk5VdklrN1dmN0g4ZnllNkUrbWFvZXZ6d1FLTHZjUVRm?= =?utf-8?B?Q2pUL0d6ZTh5MEkwenAyLzJkeDNJTWlPSVFJNTRLcE5XcHdrczNrbUs1QW83?= =?utf-8?B?SVg0dEZRWkdBTHU2R3hUTkFJLzd5OXhZS3J0TklMVzA2ZnFaK2xabDltelUw?= =?utf-8?B?WlpXWERIR1RRSm13aVB4d3N2czlUWXZUWUxJRDcwY3dReFlOa1hvWmJDNCtz?= =?utf-8?B?M0VrQk1aa2dmODgxWHV3czRwSzR1Z3ppU01BN0ppQlp2ZnBGTWl2QmVqZG1i?= =?utf-8?B?OE4wVDRjaTVRYWg3cmVBZEJlSlFiU3UwdHB4S3FjTG1zZ09YZ2YzR1NpSWtl?= =?utf-8?B?UzRidWtqQkVtV0F0RUZZa1pMbWlrNHFPZ3hzdFRhamRydVE2M0cwSHNyMzk0?= =?utf-8?B?dGJydEcxM29BYzlXVDdkT1p4YmdhSXR3RnpoQnRjTkdxNHdMUzZoYXV3NTJC?= =?utf-8?B?Z2ZoSk1ZaDZiMjU0QUYrWkUrZzhjbWx5dVl4R0tUaTNjQW1jelQ5ZkpJaWY0?= =?utf-8?B?SGdsK2RMYjBveEY2bTRkRU5ubGUxU2Y3OU4yNlhuWXJ0bUx6aUVueVIwdUtS?= =?utf-8?B?ckNmS2NTc3VOTVZ6QldsYnlBQ0xVWFQ1WlZsc0ljV3ZWRU9IWTVHOVZ5SE4x?= =?utf-8?B?ZmJNZmN5N3F0U01DRHQ3d2x0RHBwN1lQckpIQ2FCeGN1dWxoTUdsNktJcUxN?= =?utf-8?B?aEFJQ1ZkNE9CQ2VCYUhJMmRXM0N0WEZxOXJPS3RwRGF3WDJ6RSttODVwSEdv?= =?utf-8?B?ZmFZT2ZhSE1rNVZuNHhxYVFCU3hPTmdXZUZ0OTBhL3U0ZzlSeVdUQitZVUtt?= =?utf-8?B?Q2FrR2Y5MHpVd2hpQmRiSlZFMzQ1N2pNOVlWaVFQdU5RczZQZFQ3U0x2MEc3?= =?utf-8?B?SE16dGJxK2xmS1dYMWJwdnFkam9FR0dBNWxWVjVod0ljVmFyV0lkcU02cjR0?= =?utf-8?B?dEhkeUwrYml1cC9zZEh1ZzdjdzZ3SzRhSmhnQ0xjUDkrQ3hJODRJeEs3aURt?= =?utf-8?B?VHJYU0l5VjJpME5jbW9wMmxRRjlEaHBsQXVFMHpMMU0yUFRTaUtMRmJiOWFQ?= =?utf-8?B?T2dONXp3QTZqQ21UdzNKMGxmS2twcmh2bWovS2Rob3IydkVoNW5HMVZnbE1h?= =?utf-8?B?RDJOeHRHOVlIVSthdnRaTk9zMmxFKzlVSkkzd0h4U0RncnNsck51aVpoRHp0?= =?utf-8?Q?4FIfiOC37Lx/JCMrttylWMmwC?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f5c3fccb-c02a-4fc6-050f-08db1e20a0ff X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4301.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2023 08:56:09.5204 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IE3WG5+cuDcirB5O4AlnTf91b8No29s2bRZsfRwRoU5N+F+lR56WAR6buKQAPyeD X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR12MB8445 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 3/5/2023 2:53 PM, Konstantin Ananyev wrote: > 03/03/2023 16:51, Ferruh Yigit пишет: >> On 3/2/2023 12:08 PM, Konstantin Ananyev wrote: >>> >>>> In the proactive error handling mode, the PMD will set the data path >>>> pointers to dummy functions and then try recovery, in this period the >>>> application may still invoking data path API. This will introduce a >>>> race-condition with data path which may lead to crash [1]. >>>> >>>> Although the PMD added delay after setting data path pointers to cover >>>> the above race-condition, it reduces the probability, but it doesn't >>>> solve the problem. >>>> >>>> To solve the race-condition problem fundamentally, the following >>>> requirements are added: >>>> 1. The PMD should set the data path pointers to dummy functions after >>>>     report RTE_ETH_EVENT_ERR_RECOVERING event. >>>> 2. The application should stop data path API invocation when process >>>>     the RTE_ETH_EVENT_ERR_RECOVERING event. >>>> 3. The PMD should set the data path pointers to valid functions before >>>>     report RTE_ETH_EVENT_RECOVERY_SUCCESS event. >>>> 4. The application should enable data path API invocation when process >>>>     the RTE_ETH_EVENT_RECOVERY_SUCCESS event. >>>> >> >> How this is solving the race-condition, by pushing responsibility to >> stop data path to application? > > Exactly, it becomes application responsibility to make sure data-path is > stopped/suspended before recovery will continue. > >From documentation of the feature: `` Because the PMD recovers automatically, the application can only sense that the data flow is disconnected for a while and the control API returns an error in this period. In order to sense the error happening/recovering, as well as to restore some additional configuration, three events are available: `` It looks like initial design is to use events mainly inform application about what happened and mainly for re-configuration. Although I am don't disagree to involve the application, I am not sure that is part of current design. >> >> What if application is not interested in recovery modes at all and not >> registered any callback for the recovery? > > > Are you saying there is no way for application to disable > automatic recovery in PMD if it is not interested > (or can't full-fill per-requesties for it)? > If so, then yes it is a problem and we need to fix it. > I assumed that such mechanism to disable unwanted events already exists, > but I can't find anything. > Wonder what would be the easiest way here - can PMD make a decision > based on callback return value, or do we need a new API to > enable/disable callbacks, or ...? > > As far as I can see automatic recovery is not configurable by app. But that is not all, PMD sends events to application but PMD can't know if application is handling them or not, so with current design PMD can't rely on to app. >> I think driver should not rely on application for this, unless >> application explicitly says (to driver) that it is handling recovery, >> right now there is no way for driver to know this. > > I think it is visa-versa: > application should not enable auto-recovery if it can't meet > per-requeststies for it (provide appropriate callback). > I agree on above, we are saying similar thing in different perspective. > >> >>>> Also, this patch introduce a driver internal function >>>> rte_eth_fp_ops_setup which used as an help function for PMD. >>>> >>>> [1] >>>> http://patchwork.dpdk.org/project/dpdk/patch/20230220060839.1267349-2-ashok.k.kaladi@intel.com/ >>>> >>>> Fixes: eb0d471a8941 ("ethdev: add proactive error handling mode") >>>> Cc: stable@dpdk.org >>>> >>>> Signed-off-by: Chengwen Feng >>>> --- >>>>   doc/guides/prog_guide/poll_mode_drv.rst | 20 +++++++--------- >>>>   lib/ethdev/ethdev_driver.c              |  8 +++++++ >>>>   lib/ethdev/ethdev_driver.h              | 10 ++++++++ >>>>   lib/ethdev/rte_ethdev.h                 | 32 >>>> +++++++++++++++---------- >>>>   lib/ethdev/version.map                  |  1 + >>>>   5 files changed, 46 insertions(+), 25 deletions(-) >>>> >>>> diff --git a/doc/guides/prog_guide/poll_mode_drv.rst >>>> b/doc/guides/prog_guide/poll_mode_drv.rst >>>> index c145a9066c..e380ff135a 100644 >>>> --- a/doc/guides/prog_guide/poll_mode_drv.rst >>>> +++ b/doc/guides/prog_guide/poll_mode_drv.rst >>>> @@ -638,14 +638,9 @@ different from the application invokes recovery >>>> in PASSIVE mode, >>>>   the PMD automatically recovers from error in PROACTIVE mode, >>>>   and only a small amount of work is required for the application. >>>> >>>> -During error detection and automatic recovery, >>>> -the PMD sets the data path pointers to dummy functions >>>> -(which will prevent the crash), >>>> -and also make sure the control path operations fail with a return >>>> code ``-EBUSY``. >>>> - >>>> -Because the PMD recovers automatically, >>>> -the application can only sense that the data flow is disconnected >>>> for a while >>>> -and the control API returns an error in this period. >>>> +During error detection and automatic recovery, the PMD sets the >>>> data path >>>> +pointers to dummy functions and also make sure the control path >>>> operations >>>> +failed with a return code ``-EBUSY``. >>>> >>>>   In order to sense the error happening/recovering, >>>>   as well as to restore some additional configuration, >>>> @@ -653,9 +648,9 @@ three events are available: >>>> >>>>   ``RTE_ETH_EVENT_ERR_RECOVERING`` >>>>      Notify the application that an error is detected >>>> -   and the recovery is being started. >>>> +   and the recovery is about to start. >>>>      Upon receiving the event, the application should not invoke >>>> -   any control path function until receiving >>>> +   any control and data path API until receiving >>>>      ``RTE_ETH_EVENT_RECOVERY_SUCCESS`` or >>>> ``RTE_ETH_EVENT_RECOVERY_FAILED`` event. >>>> >>>>   .. note:: >>>> @@ -666,8 +661,9 @@ three events are available: >>>> >>>>   ``RTE_ETH_EVENT_RECOVERY_SUCCESS`` >>>>      Notify the application that the recovery from error is successful, >>>> -   the PMD already re-configures the port, >>>> -   and the effect is the same as a restart operation. >>>> +   the PMD already re-configures the port. >>>> +   The application should restore some additional configuration, >>>> and then >>>> +   enable data path API invocation. >>>> >>>>   ``RTE_ETH_EVENT_RECOVERY_FAILED`` >>>>      Notify the application that the recovery from error failed, >>>> diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c >>>> index 0be1e8ca04..f994653fe9 100644 >>>> --- a/lib/ethdev/ethdev_driver.c >>>> +++ b/lib/ethdev/ethdev_driver.c >>>> @@ -515,6 +515,14 @@ rte_eth_dma_zone_free(const struct rte_eth_dev >>>> *dev, const char *ring_name, >>>>       return rc; >>>>   } >>>> >>>> +void >>>> +rte_eth_fp_ops_setup(struct rte_eth_dev *dev) >>>> +{ >>>> +    if (dev == NULL) >>>> +        return; >>>> +    eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); >>>> +} >>>> + >>>>   const struct rte_memzone * >>>>   rte_eth_dma_zone_reserve(const struct rte_eth_dev *dev, const char >>>> *ring_name, >>>>                uint16_t queue_id, size_t size, unsigned int align, >>>> diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h >>>> index 2c9d615fb5..0d964d1f67 100644 >>>> --- a/lib/ethdev/ethdev_driver.h >>>> +++ b/lib/ethdev/ethdev_driver.h >>>> @@ -1621,6 +1621,16 @@ int >>>>   rte_eth_dma_zone_free(const struct rte_eth_dev *eth_dev, const >>>> char *name, >>>>            uint16_t queue_id); >>>> >>>> +/** >>>> + * @internal >>>> + * Setup eth fast-path API to ethdev values. >>>> + * >>>> + * @param dev >>>> + *  Pointer to struct rte_eth_dev. >>>> + */ >>>> +__rte_internal >>>> +void rte_eth_fp_ops_setup(struct rte_eth_dev *dev); >>>> + >>>>   /** >>>>    * @internal >>>>    * Atomically set the link status for the specific device. >>>> diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h >>>> index 049641d57c..44ee7229c1 100644 >>>> --- a/lib/ethdev/rte_ethdev.h >>>> +++ b/lib/ethdev/rte_ethdev.h >>>> @@ -3944,25 +3944,28 @@ enum rte_eth_event_type { >>>>        */ >>>>       RTE_ETH_EVENT_RX_AVAIL_THRESH, >>>>       /** Port recovering from a hardware or firmware error. >>>> -     * If PMD supports proactive error recovery, >>>> -     * it should trigger this event to notify application >>>> -     * that it detected an error and the recovery is being started. >>>> -     * Upon receiving the event, the application should not invoke >>>> any control path API >>>> -     * (such as rte_eth_dev_configure/rte_eth_dev_stop...) until >>>> receiving >>>> -     * RTE_ETH_EVENT_RECOVERY_SUCCESS or >>>> RTE_ETH_EVENT_RECOVERY_FAILED event. >>>> -     * The PMD will set the data path pointers to dummy functions, >>>> -     * and re-set the data path pointers to non-dummy functions >>>> -     * before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS event. >>>> -     * It means that the application cannot send or receive any >>>> packets >>>> -     * during this period. >>>> +     * >>>> +     * If PMD supports proactive error recovery, it should trigger >>>> this >>>> +     * event to notify application that it detected an error and the >>>> +     * recovery is about to start. >>>> +     * >>>> +     * Upon receiving the event, the application should not invoke any >>>> +     * control and data path API until receiving >>>> +     * RTE_ETH_EVENT_RECOVERY_SUCCESS or RTE_ETH_EVENT_RECOVERY_FAILED >>>> +     * event. >>>> +     * >>>> +     * Once this event is reported, the PMD will set the data path >>>> pointers >>>> +     * to dummy functions, and re-set the data path pointers to valid >>>> +     * functions before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS >>>> event. >>>> +     * >>>>        * @note Before the PMD reports the recovery result, >>>>        * the PMD may report the RTE_ETH_EVENT_ERR_RECOVERING event >>>> again, >>>>        * because a larger error may occur during the recovery. >>>>        */ >>>>       RTE_ETH_EVENT_ERR_RECOVERING, >>>>       /** Port recovers successfully from the error. >>>> -     * The PMD already re-configured the port, >>>> -     * and the effect is the same as a restart operation. >>>> +     * >>>> +     * The PMD already re-configured the port: >>>>        * a) The following operation will be retained: (alphabetically) >>>>        *    - DCB configuration >>>>        *    - FEC configuration >>>> @@ -3989,6 +3992,9 @@ enum rte_eth_event_type { >>>>        *      (@see RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP) >>>>        * c) Any other configuration will not be stored >>>>        *    and will need to be re-configured. >>>> +     * >>>> +     * The application should restore some additional configuration >>>> +     * (see above case b/c), and then enable data path API invocation. >>>>        */ >>>>       RTE_ETH_EVENT_RECOVERY_SUCCESS, >>>>       /** Port recovery failed. >>>> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map >>>> index 357d1a88c0..c273e0bdae 100644 >>>> --- a/lib/ethdev/version.map >>>> +++ b/lib/ethdev/version.map >>>> @@ -320,6 +320,7 @@ INTERNAL { >>>>       rte_eth_devices; >>>>       rte_eth_dma_zone_free; >>>>       rte_eth_dma_zone_reserve; >>>> +    rte_eth_fp_ops_setup; >>>>       rte_eth_hairpin_queue_peer_bind; >>>>       rte_eth_hairpin_queue_peer_unbind; >>>>       rte_eth_hairpin_queue_peer_update; >>>> -- >>>   Acked-by: Konstantin Ananyev >>> >>>> 2.17.1 >>> >> >