From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 397AEA00C3; Tue, 1 Feb 2022 13:52:36 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B9BD740698; Tue, 1 Feb 2022 13:52:35 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 4178240691 for ; Tue, 1 Feb 2022 13:52:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643719953; x=1675255953; h=message-id:date:to:cc:references:from:subject: in-reply-to:content-transfer-encoding:mime-version; bh=fyUAKAvqzFuTSyIbI5nfTBUto3eNWzB2jqTQxf/+y0o=; b=Dv07X2IYr1IzeTJ71OxceW6RQnxPFNiqcc66RurI3vkXXJyOmuuwaeSm FbEoqAXJudlcgOLDxFa+1XcpPSnnfEVYizjFFVrr0fPsDa3XO9hFTyhEA cM+ij0qQg9GWdWhM1aDeoplXa2sPHap4Wo+1rJvi5Yyyup+tjofI1bgGE TWSZ8ZZZ+7rvEAmAUEE93RlgTYKzYxXiSyQ9VCkoeFo5uck+Fk/XY6riB RNyeOWDLz/Z93Gja7Mbv85k40N7TGCDHvZ5fmFRcpcMPRm3HOQEZp/BxT 3l4Yilf34KKX18/cpOIUosWuwDRDv5lFXXpl6mYL23NG16jiEkEXDxurv g==; X-IronPort-AV: E=McAfee;i="6200,9189,10244"; a="308414534" X-IronPort-AV: E=Sophos;i="5.88,333,1635231600"; d="scan'208";a="308414534" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Feb 2022 04:52:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,333,1635231600"; d="scan'208";a="565581212" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by orsmga001.jf.intel.com with ESMTP; 01 Feb 2022 04:52:31 -0800 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 1 Feb 2022 04:52:31 -0800 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Tue, 1 Feb 2022 04:52:30 -0800 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Tue, 1 Feb 2022 04:52:30 -0800 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (104.47.56.45) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Tue, 1 Feb 2022 04:52:30 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B1JPi3Q0//QmTwHe471M1aBg7LZiDIVP1z7s0+DflUSYwjOQPb+V6hjsiBi2OcW2iu/wWsQB8TELPxJzuRH1PzIsaxDbe5qMF9obspMZG1rvuwQcAA7azhUV5gf7QRYjD/PrHAUuyqnu30Gr48YlZUxX/96zrZOwvD9Q4Iq21lZUNME2LaAwfeeNdz7Z+rzrad97PDYJcwh/IDaLIlYmxJ1YFOFHtENqmyoD634a/ZfUM4jTWEuMqZk+uK81/rZQjRL6vILLmjRc6TqMGpDdwtXman1ZAnCctV4VRZsCcn3kycLkqDJxP5vV7BjHYB04sO6/2bW3Mu6GSA4ICZLxKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XXqMYXi0dZxOe00yki2snnZxhJGvw+Nop5KK1L4Sjww=; b=ZZoNSlb0+VRvUcjuyW1ftV4orMfWA5hECXC2m26Y4FlsOiOprcJme+Cp93akdotdIUsNGsh9Skae0NfBFRFnGpNvItljEU+Ie2edbe/En6EKOoirNvWDzYO37yLhcUTVdTk/QtIS4yTo7Lmtz2cYV74IkQwMFvCjZAW48ZO27iKVGJAvid1X43NvY3L1Jm9AgqxvRKBTdFrWlweje72ytGzGnODjiUoBgKyrKy1OOR+7tHLkypR/LwpEj9DHvKvgF5WID7DM1d069Y+CD9irtA7Jwdf872ex+edmuh4uV2aK1qdOWSQvD7a0vG6eO/7cR/1VHDTqy1NelVrLotuhzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) by MN2PR11MB4351.namprd11.prod.outlook.com (2603:10b6:208:193::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4930.15; Tue, 1 Feb 2022 12:52:23 +0000 Received: from PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::5046:8550:928d:850e]) by PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::5046:8550:928d:850e%7]) with mapi id 15.20.4930.022; Tue, 1 Feb 2022 12:52:23 +0000 Message-ID: Date: Tue, 1 Feb 2022 12:52:18 +0000 Content-Language: en-US To: Kalesh A P , CC: , , David Marchand , Ray Kinsella , Thomas Monjalon , Andrew Rybchenko References: <20201009034832.10302-1-kalesh-anakkur.purayil@broadcom.com> <20220128124830.427-1-kalesh-anakkur.purayil@broadcom.com> <20220128124830.427-2-kalesh-anakkur.purayil@broadcom.com> From: Ferruh Yigit Subject: Re: [dpdk-dev] [PATCH v7 1/4] ethdev: support device reset and recovery events X-User: ferruhy In-Reply-To: <20220128124830.427-2-kalesh-anakkur.purayil@broadcom.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0348.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18d::11) To PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a9a32e23-cf6f-42e2-4ff0-08d9e581b0fc X-MS-TrafficTypeDiagnostic: MN2PR11MB4351:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hM/F287hJlNT2UICeFGsLI59LpHhzZZK0jzUM3zx29n8rcK27S7YWG+5TEVpp3PNMCG7bxoLBWC9OcsfBCLTz568zgCT18ZMYD9PdpxymDwFFTC8L8REHVf5bspbQJYFDUE19PbnwdDlolsb8qwfzYt+nu/NSXoQ3tP+R/wYOwnUvzY/GJBzM1BX1tiyOtTRU/NLRXFi1tuOACyIQYxCJlO9iL7B6++Xln3+5F9a87fVQxNZnrH+jNODN6yvWwZbdj1m2G5waQIIrDxhennu+s/jZ1eu+TJkhP6SvRXwEWDx65oK66JKFBoppObVK1g6wFz2D1krkO/PmNztHnw6brjrz9xlf3+/lKBWQmJf6jTkQC9AiJRZwz+WBmKtuAUJk8ssEaVCFa0oq3PIV2dUMtadQKARLnGCimGoF3W4986e3/rhq0X0vHMzzBo1AySPqUa7eYMqAZhxu4Qqau+zNZGxTOSkD6lchZIaNE7/Wx0zOiTjfXMUvlRZkscnWoz0xT6Xp2ynJjLEBNVjXOJ9uIdr+linbmgOd9ZbzD0lx5bD3Z3PBBVfBj9IkPBB0yLkWPyW1za6be72uALzmwepy5wnF9fA0F1OqAL15IaXqQtmTd8E7XvCe+wsuerdA6W+YQsKxEKMkhESMV9V2X5LYY0dkAZMYukLEsfGRmgxVRXtQ05+Q4CIv4tgssQTvzidJQlKQ13cDtuO1qsekyD7Eg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5000.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(38100700002)(8936002)(66476007)(6486002)(31686004)(316002)(31696002)(86362001)(4326008)(6666004)(66556008)(8676002)(66946007)(36756003)(54906003)(82960400001)(508600001)(53546011)(2616005)(55236004)(5660300002)(186003)(26005)(6506007)(2906002)(44832011)(83380400001)(6512007)(45980500001)(20210929001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?K0RsZDVTUktFRGppZWw3cnp0UkxieHhVMnJNa0RURCtCTS9CcWc3bHpZUlM3?= =?utf-8?B?N3BXOXVZUmx1ZHN5SDIxWE5hMW1DVXdqU3R6OTJFNUhESEYyUzU5bExsL21I?= =?utf-8?B?QWZuUTNIQkFybDRUVkQ5TGxSVzZRSXhJOUw5ZllNTWZoWFNDMWMrdnZ5d3Vz?= =?utf-8?B?UGk2RjlwUTNLQWlCNUxBM0FtVmptYUxKMmt2enVFek5wWDJnT1UxZS9NQ3Z0?= =?utf-8?B?VVg3dDNNVHcrNlo0SVhxcElWNFBtajBpODAyK1h4WTBBa2NtTXVqS0ExRUhQ?= =?utf-8?B?ZWtEa1VqRlpQVlliRjNYVGIwdTl3N09kakRYdUh2ajQrTysvRlRQbUxRVnpu?= =?utf-8?B?S3pDK2pONktUNmlBTnBTOUZVQS91TlZMSWFxMjdMSHR5R1JlYUV4TFMveEEv?= =?utf-8?B?S1FVd1lCSTJlYUl5YXQ1N1RFMklmaHpmU2hvMVhTdjVaMGhxZXc0SUZ6TnEz?= =?utf-8?B?K2VTQitzamVnZTZDOFhhZEVOUHd5bFRqSTRPUDZ6akVqUDU1TnhXSWlLcE5u?= =?utf-8?B?R1lYL2V6WU4xVVZCWTBJdzVTckxnSmh4Q0hqdzZ4WnBlZnRnNEF4cWY5T0FJ?= =?utf-8?B?cjVsME5IQlJFc0xGcmxaUkM2VzJMM2tvVGNhV1N6NDEzNFNqbGk5d2NIK1dJ?= =?utf-8?B?Q2tQaXA0eWhIZFdoVEROaXlBa3ZORjA0WUp1ZGVQN2VsQytOSjhJWVE1NTlm?= =?utf-8?B?MnpIbzFXSDlxa2krK2ZKNXByUittVHEvYW9mSmNUakNRRzBiN1p4UFNhSGpT?= =?utf-8?B?QU1SVDJsVjE0dy9oV1cyUTNYUFdzQlRNZUlUSFg3aEQ5Z1BJeWFlWVI2WjBm?= =?utf-8?B?MitOZEg2a2w0aTFwSWpmTUo0L2luQWh5UVByUVdTUnhGOVpXd1ZYS3M2cjZU?= =?utf-8?B?ajFiMVZmTndPTk9KM0Q2VHRCdGo3enBqUGpNaStybW9UN1F0RXJhamxySmQ5?= =?utf-8?B?WVAwY25wclZLUjd1bDBzNU43Z2QvTW5vUXRKR25YWWZpb1ZRSmtpSUdIbnZ6?= =?utf-8?B?Vm8xVzVReGJlbU1JWHErbStZR2pPSG54ZnRRUzMvZTBTbWlmSERmZ1FqNVRn?= =?utf-8?B?czAyTnM5b3ZsdjY3Q3ppNTZjRkhIYmpvTDNIL3gvMitReUgyU2d3cEo4aUcz?= =?utf-8?B?T0NqRHFHeC9kaDc0dStVbG94YXpsSUgzNVhodGxJZmhYQmVwdUZiTFd6MDg3?= =?utf-8?B?Y0pFb210RlZVdkppbXUvcTN2SlpHOVRaTm96dUFpalNPS3dScnkvWm1VWFFC?= =?utf-8?B?VWtWZFJnT0RNaXZGYXR4cXBXRTF2UnZUa01MNitVcWl5Y09wb2Mzc1BSemFF?= =?utf-8?B?bWRTOUFXdVM1ekZyT2JhMXZzMW8vTWdFVWkxNWx5WmxGTzdCLzMyT1VBT0Uz?= =?utf-8?B?R28zbklYN1E0dDAzL1VuY1VyVzhKcHd4NTJyenNyaTYyRk1xMjF2UHN3bFRP?= =?utf-8?B?UkdMaTJvN0FUTEU1a2trWGo3MEkzVXl5bkZmMnN1NTdvYWtyaC9DYktDZDFN?= =?utf-8?B?amtzeU83RDROdU01L0Q0ekY1NE1iVGlNZkZTVFRtZnhEV25CWnNXYmFZclFs?= =?utf-8?B?NDVqVUliL243RHV0MWJ2Q2l2aDdTZVVrTTlQeldjcU5qTURqZm5PMGNDNnJ5?= =?utf-8?B?Q1c4Z2M0aEUxTkpqTnlEcUJKRXRNYjk5TkhvRG5pTWxjMGNWRkJIaFo4WXdx?= =?utf-8?B?Qis3ZHJsaGpaSmh0M0ZrL1hKODMvL3pYb3h5KzMwcWxXdDdhUk04OGxMdk1n?= =?utf-8?B?WlBpZGlVRm5TemlDWERiR2tZV3B0SWRrN3ZxSzAyNFZ6dEZ2RWZIcURyOUZu?= =?utf-8?B?YlgrK1hIMWJFdHZjM2htT3VrNTR6TEJsamtROHd5QXFnalZkdWVQQnNaM25C?= =?utf-8?B?a3NodnAxcUNkTWRFMUFFcDRheXZ3NEEvQjdBZnZQRUZzQnhuQ0NaN2RGMGRk?= =?utf-8?B?RjFtaXJ4NVk0S3Z1MlloTFVERlVzZEFiV1JOR2dXaGtvU0gxZ2kzOTZQc3VE?= =?utf-8?B?V0ZrWDNRNWtiVGNvZUVCVnc4RjY3TEpiYUYyL21SNTRHYkJZWlNFOWxyZDlV?= =?utf-8?B?b0hLcnppdU11VXArOXN2Zzk3QWxncHhMK3Yya294cW1rNHJrNUo2MnN4WHV0?= =?utf-8?B?WDJDcm16K2FqejhQRmIxTUtYdmNYSVZPMmNQSmphWHQzc3NmZ2htZUxyUTE1?= =?utf-8?Q?zHyX/fjxIXPBF0uvM3nNkYc=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: a9a32e23-cf6f-42e2-4ff0-08d9e581b0fc X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5000.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2022 12:52:23.4739 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cZ3bhnj8LElpX7nr1qhmxpijD6awqnfGNvCNDBZJpxf8JPfKjvIJXGEd9+7389hlYIDFzmBWqOpi3R5T0QbBAg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4351 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 1/28/2022 12:48 PM, Kalesh A P wrote: > From: Kalesh AP > > Adding support for the device reset and recovery events in the > rte_eth_event framework. FW error and FW reset conditions would be > managed internally by the PMD without needing application intervention. > In such cases, PMD would need reset/recovery events to notify application > that PMD is undergoing a reset. > > While most of the recovery process is transparent to the application since > most of the driver ensures recovery from FW reset or FW error conditions, > the application will have to reprogram any flows which were offloaded to > the underlying hardware. > > Signed-off-by: Kalesh AP > Signed-off-by: Somnath Kotur > Reviewed-by: Ajit Khaparde > --- > doc/guides/prog_guide/poll_mode_drv.rst | 24 ++++++++++++++++++++++++ > lib/ethdev/rte_ethdev.h | 18 ++++++++++++++++++ > 2 files changed, 42 insertions(+) > > diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst > index 6831289..9ecc0e4 100644 > --- a/doc/guides/prog_guide/poll_mode_drv.rst > +++ b/doc/guides/prog_guide/poll_mode_drv.rst > @@ -623,3 +623,27 @@ by application. > The PMD itself should not call rte_eth_dev_reset(). The PMD can trigger > the application to handle reset event. It is duty of application to > handle all synchronization before it calls rte_eth_dev_reset(). > + > +Error recovery support > +~~~~~~~~~~~~~~~~~~~~~~ > + > +When the PMD detects a FW reset or error condition, it may try to recover > +from the error without needing the application intervention. In such cases, > +PMD would need events to notify the application that it is undergoing > +an error recovery. > + > +The PMD should trigger RTE_ETH_EVENT_ERR_RECOVERING event to notify the > +application that PMD detected a FW reset or FW error condition. PMD may > +try to recover from the error by itself. Data path may be quiesced and > +control path operations may fail during the recovery period. The application > +should stop polling till it receives RTE_ETH_EVENT_RECOVERED event from the PMD. > + > +The PMD should trigger RTE_ETH_EVENT_RECOVERED event to notify the application > +that the it has recovered from the error condition. PMD re-configures the port > +to the state prior to the error condition. Control path and data path are up now. > +Since the device has undergone a reset, flow rules offloaded prior to reset > +may be lost and the application should recreate the rules again. > + > +The PMD should trigger RTE_ETH_EVENT_INTR_RMV event to notify the application > +that it has failed to recover from the error condition. The device may not be > +usable anymore. > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h > index 147cc1c..a46819f 100644 > --- a/lib/ethdev/rte_ethdev.h > +++ b/lib/ethdev/rte_ethdev.h > @@ -3818,6 +3818,24 @@ enum rte_eth_event_type { > RTE_ETH_EVENT_DESTROY, /**< port is released */ > RTE_ETH_EVENT_IPSEC, /**< IPsec offload related event */ > RTE_ETH_EVENT_FLOW_AGED,/**< New aged-out flows is detected */ > + RTE_ETH_EVENT_ERR_RECOVERING, > + /**< port recovering from an error > + * > + * PMD detected a FW reset or error condition. > + * PMD will try to recover from the error. > + * Data path may be quiesced and Control path operations > + * may fail at this time. > + */ > + RTE_ETH_EVENT_RECOVERED, > + /**< port recovered from an error > + * > + * PMD has recovered from the error condition. > + * Control path and Data path are up now. > + * PMD re-configures the port to the state prior to the error. > + * Since the device has undergone a reset, flow rules > + * offloaded prior to reset may be lost and > + * the application should recreate the rules again. > + */ > RTE_ETH_EVENT_MAX /**< max value of this enum */ Also ABI check complains about 'RTE_ETH_EVENT_MAX' value check, cc'ed more people to evaluate if it is a false positive: 1 function with some indirect sub-type change: [C] 'function int rte_eth_dev_callback_register(uint16_t, rte_eth_event_type, rte_eth_dev_cb_fn, void*)' at rte_ethdev.c:4637:1 has some indirect sub-type changes: parameter 3 of type 'typedef rte_eth_dev_cb_fn' has sub-type changes: underlying type 'int (typedef uint16_t, enum rte_eth_event_type, void*, void*)*' changed: in pointed to type 'function type int (typedef uint16_t, enum rte_eth_event_type, void*, void*)': parameter 2 of type 'enum rte_eth_event_type' has sub-type changes: type size hasn't changed 2 enumerator insertions: 'rte_eth_event_type::RTE_ETH_EVENT_ERR_RECOVERING' value '11' 'rte_eth_event_type::RTE_ETH_EVENT_RECOVERED' value '12' 1 enumerator change: 'rte_eth_event_type::RTE_ETH_EVENT_MAX' from value '11' to '13' at rte_ethdev.h:3807:1