From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D449A00C3; Tue, 1 Feb 2022 13:11:21 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C07EE40698; Tue, 1 Feb 2022 13:11:20 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 97CB640691 for ; Tue, 1 Feb 2022 13:11:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643717479; x=1675253479; h=message-id:date:to:cc:references:from:subject: in-reply-to:content-transfer-encoding:mime-version; bh=mlfIoY4owMJuSI/FoIPOVa6o507qdYzoA1/ql28AgGg=; b=g6cwy0+DQL3rpNgIToOJkbjIbdvPbXRVGQALceH5aFgfVoT7eKOADpDu DuwmgFwdyaprxIpKmcfKEVehBEjKxyNv4UYQpKwum5dUUgzxP8fb2OYO8 c7cZgH4OCdJGSN6q87XyWYRCqOgKRULlBtYXTwO9lOWHCIr8nDkktxWXE 4oBbHx9gbeviIs1ysiVeGpFR4YbnDopMjM5t/ngJ8WGo2W3SDVgPZqc6a hWX63X5Ud2dEhF9aPcJSRHOJ5dGRiNnjfY3j7aPOBoTj3AuXZmnxCtKaA HVEjmLqqMaC0Fux+pBtGWmyzGZ7+2ZgRJ/n17MhcGNzJ+kgKJCNe33tig w==; X-IronPort-AV: E=McAfee;i="6200,9189,10244"; a="228336859" X-IronPort-AV: E=Sophos;i="5.88,333,1635231600"; d="scan'208";a="228336859" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2022 04:11:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,333,1635231600"; d="scan'208";a="482347450" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga006.jf.intel.com with ESMTP; 01 Feb 2022 04:11:17 -0800 Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 1 Feb 2022 04:11:17 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Tue, 1 Feb 2022 04:11:17 -0800 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.176) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Tue, 1 Feb 2022 04:11:13 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BSZ+9ThIroH//0OzCOul98MC80am4z02MDKngpq+5+2lpvSZFeXVDKoaW1i4epaw7eJrboG77eo/WiEBuL7aldtYDkm60vIp55qJs+iReKRhDl3XUXKIR3R9AWSJEjRkec3WaR5qYdFgNkwRSZuI0RNe2MzAbDNtg42oLtnzjveyXkOc/f41OW7xgALQXoLjcXPq6dZCrjrph8ED1x4VJBwey6y5Hy+gtrnPwNJPVebw+EHLtF8dKBxK7o4QXohK3sslABNkYc4BZR821f5pcUR2abfbuI4IReJmz5Yn5kfFgSk5uWMJVoXmOgrniXkdMtboX4b/N6SXFQkTY8LBCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5KTSNo3QECtCCxOzCku16HVx6MPeyndXkU9PaoRxgyg=; b=UB5786XWqHaE1hbiMZ/E+7oaqqu3ODFQnriS3c3x+CqIl6JEpbmaVt222RqUoITqDpbH3M4REggUOc3j7Jf7IKmOwxCAADcP00DwIyoWq8HKzA6QUeqrbBt6vQbSZE/zg5QEvKsHR/pWbrSPrqzCEmCKMVp5ybhyd+V3GWopsl9zN/IAuT/t8DgdciYcoZXIJayIqVP2nASP5bLhcq0NV+xZtMWUwFcoe1rZ2VdIqCC/gb9Cimz3ehlG0gIZiiVoHDt6KJ1iGdXH3cLAFoNFVQ5h276dB4m2sTICVgTBn2RuTNFTRa9rvMaVDoPoRi32gej1hDvH1dYjNjiZoKAi9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) by BL0PR11MB3393.namprd11.prod.outlook.com (2603:10b6:208:62::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.22; Tue, 1 Feb 2022 12:11:10 +0000 Received: from PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::5046:8550:928d:850e]) by PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::5046:8550:928d:850e%7]) with mapi id 15.20.4930.022; Tue, 1 Feb 2022 12:11:10 +0000 Message-ID: <1bd15a49-b53c-b1e7-bed4-2f8aef55a0f9@intel.com> Date: Tue, 1 Feb 2022 12:11:03 +0000 Content-Language: en-US To: Kalesh A P , , "Thomas Monjalon" , Andrew Rybchenko , Konstantin Ananyev CC: , , Stephen Hemminger , "jerinj@marvell.com" References: <20201009034832.10302-1-kalesh-anakkur.purayil@broadcom.com> <20220128124830.427-1-kalesh-anakkur.purayil@broadcom.com> <20220128124830.427-2-kalesh-anakkur.purayil@broadcom.com> From: Ferruh Yigit Subject: Re: [dpdk-dev] [PATCH v7 1/4] ethdev: support device reset and recovery events X-User: ferruhy In-Reply-To: <20220128124830.427-2-kalesh-anakkur.purayil@broadcom.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0045.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:152::14) To PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 29b08f39-9b08-4534-99dd-08d9e57bee9b X-MS-TrafficTypeDiagnostic: BL0PR11MB3393:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: olpfm9uiUH4nArCQRCUrfBrRnZrTOMnUOnKxcSu/wJ/6QNY1/BKQ4+t/QZQgwMTkZCr6aEUhfjOG9SDz1S5vbk49uJqv0sb659rsLCkg3NRdTdMHU1DvkZTgq0AnNvq3mQRD10/y/QYtCptjA3Q0738j0boKBWf+C/xDKddpaiG1THzl2CX/7Dj9utBKvuPtktaoKoAuBUdrBbz2rTiGFPldwrK3z8xv/5EUDaFH2I+9T9bH3cl3KYGEGChhCLQ0AAEiXKgCQNL0cI3mNmKvVcFyIlafQUp240Xj8YcBBnbLxSVhBuglfxb3wD3pNy82xEPoPhAcfcSw6v1V9D2WnL7K5ciNiDD2LKv0IR0FMi7N4zN7VaAuFYGnp48Dsx3OgoVTfCHMcD3deAhjgJTRIDYdK76gOV3qu+sTp5Win+02yoVc/hWPH7nB1aqiL+Ob1ZgMmA3w11mCyBiKS+0OzzLFatPZIpj5lGH+GH9cp3RHH6lWPzflnEh8h3slFZQV7ECQ5X74dB5NhX3UvwlAl4I7LxoSpLfjEF8R0IvkBQd5eqC1oJoYqLPjmAbQWOllsDhIaZnkXSAJtZNmloGR8s3jwU95YkVQCK5hOspgvkRuhqs/DuXY1Waplbq4tsJoaIqu+UXIfK92HLrk3OJ/CZMlB+nrZ1/Jted9k5AbNtp4yf1ncORoPwgzveMEBe5wFgR6OP/m6Es4XecntD7q4g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5000.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(5660300002)(66946007)(86362001)(83380400001)(38100700002)(66476007)(4326008)(186003)(2906002)(26005)(44832011)(66556008)(31696002)(6506007)(53546011)(6666004)(2616005)(316002)(54906003)(55236004)(6512007)(110136005)(82960400001)(6486002)(36756003)(508600001)(6636002)(8936002)(8676002)(31686004)(45980500001)(20210929001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b01ZdUVheldwVFhsaXJ1d01oYURENWd6eGZYWHRzYkFpUHBidDlHeGhuSlVC?= =?utf-8?B?SVIyZXk3ZnBBZFZrVm1xOXYvbU9FV3FlaG8xNWJGWGEzRHBMNlhkMFpRYlBY?= =?utf-8?B?dTg1dHF6cjNLQUxaZ21TY3dyUmVya2dYdGQxZ3ZNcmgzNnErQmo2OUlEMysv?= =?utf-8?B?SXZ6TFE4VmZ5TVpKYS93RlU0eEFDN2NZUGpZbzU2R3ZkcnZrODBoODJSS1pY?= =?utf-8?B?dXBkaFR4bkRSMmhzcGw2akR2Y05JNDJEL0J1d1dpQW55UGtFcDVyd0RVQWFl?= =?utf-8?B?eHRLZ2Q3OExpRnFhYnNXZStQMm53Q2c1aGgyRnVnbzJ6aWdGK011NUxaSFFJ?= =?utf-8?B?WitVZHB4ZlFoNFJvcHQ1ZDFsN0dNVEpmV29ZL3IvRzB1c0dUdVM2M2NSRGhp?= =?utf-8?B?Q1dtblVTTTFKeWZhTHJkUDZsTUpvRTZaOWF5U1YxN2EzRFpGWmN5b3RBZmRo?= =?utf-8?B?SW0vTkc4VVdXVFY5N3Z5WGFVV1FUai9JUGRBblc0aEJOZnE5SWZGb3l0UVo1?= =?utf-8?B?cmp0czRVdHVINTJPWVhITHprWnlCbmtkcnVzM2VnR0lhZTNhdnlhMWt1RkRG?= =?utf-8?B?L3BUNWxBYjlJM1ZIVjh0Q2lwbTdhVk9Bc3lWdnBFRldKVDJZK24yYjJ6R0lz?= =?utf-8?B?cE1lWFZwZzYvcTZHTldINjR6Nk9IdzRXRldBT2Vac3BkbnNWQlZtb0FTNzVH?= =?utf-8?B?ZFpSRE81N2I1TlBwY1h5anJLekF4bU1Ud0FmT0RMUEFRYnRZMUZCMW1jVTRu?= =?utf-8?B?c1JGSmYyOTNhcXZDbGZtbzUvTEhhTGF0b1d0VnFldWZ2cldJRXNnMHZCRyt6?= =?utf-8?B?TjJYTy9kR294RjMvaTRtaXIrSnpUVFhMV1hEMEdLSmVLdEoxRzhtWkxQMzcr?= =?utf-8?B?S0w4cG5TN0EwOE1QOURrK1EzTVZ3K2k2T2NFZWZEU3RpY2QyWHM2YW5DV2Er?= =?utf-8?B?WGc5RVcvWlNtK2w5MlpyU1A2Ykg0bGJVRTcyYkFXdWlaYW9HMG9CWUlwdElJ?= =?utf-8?B?VTZIV0Y4QmNZUnF5Nlc4a29wWUhncGhvUjQxOTJpSmlnWkIxSnJNZFAwUkpo?= =?utf-8?B?ckVVQjdJSnkrSXRyclFZS0VTNGcyYWlaL2cyR29GanBmZUUyRUVRaGxtQjJN?= =?utf-8?B?OEVQM2lVREhmTTNFSHJHKzVMdjcwYVM1WnRBRXQzT3FmL1hrQXViVkpQQWZK?= =?utf-8?B?dW5NaXVsNGRwZHlmVW5RZm9tOXNiVW5mZjdqVGZRZFJ1Y3ZrbHo3V21TWURy?= =?utf-8?B?RGxkbjlxdWszRk1qRnhCTVJ3OVVsZW12Q0JMMG81UlAzRzY0dDlDS2dMd080?= =?utf-8?B?Nk1xZGdLcFIwOURRSGNneTIwVHM3ai9FaGRWWldNWFRnc1V5OEFrdzRGMmh2?= =?utf-8?B?am1tSUZ1WHdWUkRnQWJwQlNFbFBCN2hNb2s5N3hJaFFvenh4c3FuUGxZVDJw?= =?utf-8?B?MmZMdDBqU1JXM3JTbmVFRUgxSDNUZDVHeXJWTG8rb3dBbUpETGp1ZEZzK1c1?= =?utf-8?B?Q0ZDUlNVNlZBdVRyeEVUaUExUXlnYnpxeUM1NFhUN0czMVZyeWtReDlzRHE1?= =?utf-8?B?TGRNejBlVEk0RmVyK3I1R0xRajlTcjJqUER1SWxsVDlIZU1DenJ0eStRbU5m?= =?utf-8?B?eWxPWGUxWVZHbU1Pa3V3cjlGUE0yNXFYZlFja28wcG5CK21IMGU2Ym4vS2Fy?= =?utf-8?B?YWNDL2lkSmxzV0NvemlCL2dPYTh5M3V0eFg1alRiNnVUTEtGb2thSXNlL3Bp?= =?utf-8?B?WTkyTTNVYys0cDNjVWh5QVdLRHpqaXFJdEhLYkxWU2xMd3ZEVGlSSjkvK09l?= =?utf-8?B?Q1lMbGx1bUE4MHBSdFdNb00yNXE4bk1xeU94TVI1MlBsVHhFVFNiSE5BYUpk?= =?utf-8?B?ZVZwcC9LbUxmM2xVVW5pZ3JKa21BNVNHRXREYkwwcWE5RUwwU2pNNXlXL2J4?= =?utf-8?B?ZHUySFRNN3RPaG9yMzdka08zTWJvY1RTZzJkWEluSzdHcG9YTnF6cUx3ODdL?= =?utf-8?B?Zy84SmFGdGl4Q3RWblVjVm05Zmx3eDZ3UDg5M2JoL3JQK2l6bUdHQ3d0WVVj?= =?utf-8?B?dlBwVGg4bnJFMDMzMDc2cFUxR2IwNnZMbkdZUmVtZ1R3YUtpaitqeCszbEV5?= =?utf-8?B?cDY5TklsL2tPSUhyeUlYdjRCdFJFU2VsUlFKSzhKbTlmMWxGMG5YKzZGM0E4?= =?utf-8?Q?rLx8lzXXYy2QkdaSpTWXq0w=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 29b08f39-9b08-4534-99dd-08d9e57bee9b X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5000.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2022 12:11:09.9829 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FxGUd0c+qmfpMdJqPs6mkCfo/bKlTdgyCEc+NnSNf1Y2W+8AJo6xUkg3sU5w41g9jWDfWCM1oRzGnUl6BeqCcw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR11MB3393 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 1/28/2022 12:48 PM, Kalesh A P wrote: > From: Kalesh AP > > Adding support for the device reset and recovery events in the > rte_eth_event framework. FW error and FW reset conditions would be > managed internally by the PMD without needing application intervention. > In such cases, PMD would need reset/recovery events to notify application > that PMD is undergoing a reset. > > While most of the recovery process is transparent to the application since > most of the driver ensures recovery from FW reset or FW error conditions, > the application will have to reprogram any flows which were offloaded to > the underlying hardware. > > Signed-off-by: Kalesh AP > Signed-off-by: Somnath Kotur > Reviewed-by: Ajit Khaparde More developer cc'ed. > --- > doc/guides/prog_guide/poll_mode_drv.rst | 24 ++++++++++++++++++++++++ > lib/ethdev/rte_ethdev.h | 18 ++++++++++++++++++ > 2 files changed, 42 insertions(+) > > diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst > index 6831289..9ecc0e4 100644 > --- a/doc/guides/prog_guide/poll_mode_drv.rst > +++ b/doc/guides/prog_guide/poll_mode_drv.rst > @@ -623,3 +623,27 @@ by application. > The PMD itself should not call rte_eth_dev_reset(). The PMD can trigger > the application to handle reset event. It is duty of application to > handle all synchronization before it calls rte_eth_dev_reset(). > + > +Error recovery support > +~~~~~~~~~~~~~~~~~~~~~~ > + > +When the PMD detects a FW reset or error condition, it may try to recover > +from the error without needing the application intervention. In such cases, > +PMD would need events to notify the application that it is undergoing > +an error recovery. > + > +The PMD should trigger RTE_ETH_EVENT_ERR_RECOVERING event to notify the > +application that PMD detected a FW reset or FW error condition. PMD may > +try to recover from the error by itself. Data path may be quiesced and > +control path operations may fail during the recovery period. The application > +should stop polling till it receives RTE_ETH_EVENT_RECOVERED event from the PMD. > + Between the time FW error occurred and the application receive the RECOVERING event, datapath will continue to poll and application may call control APIs, so the event really is not solving the issue and driver somehow should be sure this won't crash the application, in that case not sure about the benefit of this event. > +The PMD should trigger RTE_ETH_EVENT_RECOVERED event to notify the application > +that the it has recovered from the error condition. PMD re-configures the port > +to the state prior to the error condition. Control path and data path are up now. > +Since the device has undergone a reset, flow rules offloaded prior to reset > +may be lost and the application should recreate the rules again. > + I think the most difficult part here is clarify what application should do when this event received consistent for all devices, "flow rules may be lost" looks very vague to me. Unless it is not clear for application what to do when this event is received, it is not that useful or it will be specific to some PMDs. And I can see it is hard to clarify this but perhaps we can define a set of common behavior. Not sure what others are thinking. > +The PMD should trigger RTE_ETH_EVENT_INTR_RMV event to notify the application > +that it has failed to recover from the error condition. The device may not be > +usable anymore. > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h > index 147cc1c..a46819f 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -3818,6 +3818,24 @@ enum rte_eth_event_type { > RTE_ETH_EVENT_DESTROY, /**< port is released */ > RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */ > RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected */ > + RTE_ETH_EVENT_ERR_RECOVERING, > + /**< port recovering from an error > + * > + * PMD detected a FW reset or error condition. > + * PMD will try to recover from the error. > + * Data path may be quiesced and Control path operations > + * may fail at this time. > + */ Please put multi line comments before enum, Andrew did a set of cleanups for these. > + RTE_ETH_EVENT_RECOVERED, > + /**< port recovered from an error > + * > + * PMD has recovered from the error condition. > + * Control path and Data path are up now. > + * PMD re-configures the port to the state prior to the error. > + * Since the device has undergone a reset, flow rules > + * offloaded prior to reset may be lost and > + * the application should recreate the rules again. > + */ > RTE_ETH_EVENT_MAX /**< max value of this enum */ > }; >