From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <Rasesh.Mody@cavium.com>
Received: from NAM01-BN3-obe.outbound.protection.outlook.com
 (mail-bn3nam01on0073.outbound.protection.outlook.com [104.47.33.73])
 by dpdk.org (Postfix) with ESMTP id 440E57CCC
 for <dev@dpdk.org>; Mon, 24 Jul 2017 12:18:32 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=Inwv4y5bbVfwUlcJwHsppQhjTqJsxqkCJYkJbZ9YHu4=;
 b=m8Z6sas7b81ayFZ/Qc92X3ll+yhaC2docBEpc/eJZIMtJtuaXkJgljpJcPHLZzUb8xUqFh9wxjdRAEw8JqLLIXAwy+iOm2uI2Dw3v1fzIF8luz2SqYkKSQq8I1PIb7SDo+l3ekcecLkJp492cIpODHNwBiB2j5kACUiokMBzw5I=
Received: from BLUPR0701MB739.namprd07.prod.outlook.com (10.141.252.17) by
 BLUPR0701MB740.namprd07.prod.outlook.com (10.141.252.18) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id
 15.1.1282.10; Mon, 24 Jul 2017 10:18:29 +0000
Received: from BLUPR0701MB739.namprd07.prod.outlook.com ([10.141.252.17]) by
 BLUPR0701MB739.namprd07.prod.outlook.com ([10.141.252.17]) with mapi id
 15.01.1282.017; Mon, 24 Jul 2017 10:18:29 +0000
From: "Mody, Rasesh" <Rasesh.Mody@cavium.com>
To: "Mody, Rasesh" <Rasesh.Mody@cavium.com>, "dev@dpdk.org" <dev@dpdk.org>
CC: Dept-Eng DPDK Dev <Dept-EngDPDKDev@cavium.com>
Thread-Topic: [INTERNAL REVIEW 1/7] net/qede/base: fix recovery from previous
 ungraceful exit
Thread-Index: AQHTBGURP2dJoOcZFUKvdLVnQDrgKKJiwyzg
Date: Mon, 24 Jul 2017 10:18:29 +0000
Message-ID: <BLUPR0701MB73940913F4D3A1A103381129FBB0@BLUPR0701MB739.namprd07.prod.outlook.com>
References: <1500891015-21625-1-git-send-email-rasesh.mody@cavium.com>
In-Reply-To: <1500891015-21625-1-git-send-email-rasesh.mody@cavium.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: spf=none (sender IP is )
 smtp.mailfrom=Rasesh.Mody@cavium.com; 
x-originating-ip: [182.237.131.6]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BLUPR0701MB740;
 7:rrASMcAyb7A293q5qI57C6uLechgR6PufDP8A92HSakhyzvWrYwW7Ppd/6p+A+oXgjsqYx5CM+hzfUU0dEycHZiQy/U6UjcEJGQlYfFN0H3XipfZy7Aj+/ML0GgVAEQFvUlgnY0qrPQOe+O2IYg9HuDbWuS6u1VpF7QxB7Sg4e5vAEtPJlr9PbDOQ8DJUBX3/b5FGxoqSgHEGd14NXiBDil5yWT/Ije/7SMgwzr3VPzXdw1Jcci6BPv/Y+2Zcr7OGVIx9RKzoLHOECW6cDTMO1aO/bpu7ocgVTcHAUvFShKdr26DT516fqmFlRTumdS2x5VFeKBKE0HWqGFbParIENeAcF0K91sBo+h0fBqad+IzkMYC0lf0SxVadZpDsM/2iLEiJ33QyCQN9Vjh9832CmsCXhJRbkpdoMS6SMYt08tubHAW/8iFB8zBnvMXMvrqKRpHfm7RXgPNxAnloKXRKcx0Ftb/5oWo/2MvaaMyqnUg1zuIa0/CCvR6h94s5LvspuuyQmfr7ZR3XdC4LwhdHy+zPGF6BHbrl2TlPCc7O4sNWpq8JEJHeHT7/zLiS1ERnWBwXO6gZuFPwfjrShwSgkM+F+goiLqthPSYIgFUA/2x+XeJd9Fg8VN1LrEPeZBMXLy6jKKvV1AH+RiTgHZI98oVStGK0IqXyOG6CW66Qbv9O5mLgvz+KbkhySnpnbPgvjYjc735Wy7jerQArJti1+UBvIKDs6A6BS85SzW2tT8IEhlT+ixcfzUbztTA3mx/WTSzdOGWssDqQqRCIux4SOTGewG4OP8IAJDu2wKoegw=
x-ms-office365-filtering-correlation-id: 2b8794b6-31dd-43b9-ec57-08d4d27d5492
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
 RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);
 SRVR:BLUPR0701MB740; 
x-ms-traffictypediagnostic: BLUPR0701MB740:
x-exchange-antispam-report-test: UriScan:(788757137089);
x-microsoft-antispam-prvs: <BLUPR0701MB740DC563B06C964935FC9749FBB0@BLUPR0701MB740.namprd07.prod.outlook.com>
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
 RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6041248)(20161123562025)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);
 SRVR:BLUPR0701MB740; BCL:0; PCL:0;
 RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);
 SRVR:BLUPR0701MB740; 
x-forefront-prvs: 0378F1E47A
x-forefront-antispam-report: SFV:NSPM;
 SFS:(10009020)(6009001)(39450400003)(39410400002)(39400400002)(39850400002)(39840400002)(199003)(13464003)(189002)(377454003)(53946003)(33656002)(6246003)(53936002)(99286003)(107886003)(38730400002)(2950100002)(6436002)(4326008)(72206003)(229853002)(478600001)(9686003)(77096006)(6506006)(2501003)(189998001)(76176999)(54356999)(2900100001)(55016002)(50986999)(105586002)(81156014)(81166006)(97736004)(8676002)(7736002)(74316002)(14454004)(7696004)(305945005)(3660700001)(53546010)(3280700002)(8936002)(25786009)(106356001)(101416001)(68736007)(3846002)(6116002)(102836003)(86362001)(2906002)(5660300001)(66066001)(217873001)(579004);
 DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0701MB740;
 H:BLUPR0701MB739.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords;
 A:1; MX:1; LANG:en; 
received-spf: None (protection.outlook.com: cavium.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: cavium.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jul 2017 10:18:29.5070 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB740
Subject: Re: [dpdk-dev] [INTERNAL REVIEW 1/7] net/qede/base: fix recovery
 from previous ungraceful exit
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 24 Jul 2017 10:18:32 -0000

Please ignore this patch-set with 'INTERNAL REVIEW' subject prefix. Sending=
 a follow up with proper subject prefix.

Thanks!
-Rasesh

> -----Original Message-----
> From: Rasesh Mody [mailto:rasesh.mody@cavium.com]
> Sent: Monday, July 24, 2017 3:10 AM
> To: dev@dpdk.org
> Cc: Mody, Rasesh <Rasesh.Mody@cavium.com>; Dept-Eng DPDK Dev <Dept-
> EngDPDKDev@cavium.com>
> Subject: [INTERNAL REVIEW 1/7] net/qede/base: fix recovery from previous
> ungraceful exit
>=20
> This patch modifies the recovery flow to allow ongoing PCIe transactions =
to
> be completed. To achieve this, the load sequence is changed such that the
> "final_cleanup" notification is sent while the FID_enable is cleared.
> This change ensures that the chip cleanup actions takes place from previo=
us
> driver instance if needed.
>=20
> Fixes: ec94dbc57362 ("qede: add base driver")
>=20
> Signed-off-by: Rasesh Mody <rasesh.mody@cavium.com>
> ---
>  drivers/net/qede/base/ecore_dev.c     |  121 +++++++++++++++------------
>  drivers/net/qede/base/ecore_dev_api.h |   12 +++
>  drivers/net/qede/base/ecore_int.c     |  144 ++++++++++++++++-----------=
---
> ---
>  drivers/net/qede/base/ecore_int.h     |    3 +
>  drivers/net/qede/base/ecore_mcp.c     |   45 +++++++++++
>  drivers/net/qede/base/ecore_mcp.h     |   11 +++
>  drivers/net/qede/base/ecore_mcp_api.h |   11 +++
>  7 files changed, 224 insertions(+), 123 deletions(-)
>=20
> diff --git a/drivers/net/qede/base/ecore_dev.c
> b/drivers/net/qede/base/ecore_dev.c
> index 4cfa668..65b89b8 100644
> --- a/drivers/net/qede/base/ecore_dev.c
> +++ b/drivers/net/qede/base/ecore_dev.c
> @@ -1080,7 +1080,7 @@ enum _ecore_status_t ecore_final_cleanup(struct
> ecore_hwfn *p_hwfn,
>  	}
>=20
>  	DP_VERBOSE(p_hwfn, ECORE_MSG_IOV,
> -		   "Sending final cleanup for PFVF[%d] [Command %08x\n]",
> +		   "Sending final cleanup for PFVF[%d] [Command %08x]\n",
>  		   id, command);
>=20
>  	ecore_wr(p_hwfn, p_ptt, XSDM_REG_OPERATION_GEN,
> command); @@ -1776,13 +1776,6 @@ static enum _ecore_status_t
> ecore_hw_init_port(struct ecore_hwfn *p_hwfn,
>  	/* perform debug configuration when chip is out of reset */
>  	OSAL_BEFORE_PF_START((void *)p_hwfn->p_dev, p_hwfn-
> >my_id);
>=20
> -	/* Cleanup chip from previous driver if such remains exist */
> -	rc =3D ecore_final_cleanup(p_hwfn, p_ptt, rel_pf_id, false);
> -	if (rc !=3D ECORE_SUCCESS) {
> -		ecore_hw_err_notify(p_hwfn,
> ECORE_HW_ERR_RAMROD_FAIL);
> -		return rc;
> -	}
> -
>  	/* PF Init sequence */
>  	rc =3D ecore_init_run(p_hwfn, p_ptt, PHASE_PF, rel_pf_id, hw_mode);
>  	if (rc)
> @@ -1866,17 +1859,17 @@ static enum _ecore_status_t
> ecore_hw_init_port(struct ecore_hwfn *p_hwfn,
>  	return rc;
>  }
>=20
> -static enum _ecore_status_t
> -ecore_change_pci_hwfn(struct ecore_hwfn *p_hwfn,
> -		      struct ecore_ptt *p_ptt, u8 enable)
> +enum _ecore_status_t ecore_pglueb_set_pfid_enable(struct ecore_hwfn
> *p_hwfn,
> +						  struct ecore_ptt *p_ptt,
> +						  bool b_enable)
>  {
> -	u32 delay_idx =3D 0, val, set_val =3D enable ? 1 : 0;
> +	u32 delay_idx =3D 0, val, set_val =3D b_enable ? 1 : 0;
>=20
> -	/* Change PF in PXP */
> +	/* Configure the PF's internal FID_enable for master transactions */
>  	ecore_wr(p_hwfn, p_ptt,
>  		 PGLUE_B_REG_INTERNAL_PFID_ENABLE_MASTER, set_val);
>=20
> -	/* wait until value is set - try for 1 second every 50us */
> +	/* Wait until value is set - try for 1 second every 50us */
>  	for (delay_idx =3D 0; delay_idx < 20000; delay_idx++) {
>  		val =3D ecore_rd(p_hwfn, p_ptt,
>=20
> PGLUE_B_REG_INTERNAL_PFID_ENABLE_MASTER);
> @@ -1918,14 +1911,21 @@ enum _ecore_status_t ecore_vf_start(struct
> ecore_hwfn *p_hwfn,
>  	return ECORE_SUCCESS;
>  }
>=20
> +static void ecore_pglueb_clear_err(struct ecore_hwfn *p_hwfn,
> +				     struct ecore_ptt *p_ptt)
> +{
> +	ecore_wr(p_hwfn, p_ptt,
> PGLUE_B_REG_WAS_ERROR_PF_31_0_CLR,
> +		 1 << p_hwfn->abs_pf_id);
> +}
> +
>  enum _ecore_status_t ecore_hw_init(struct ecore_dev *p_dev,
>  				   struct ecore_hw_init_params *p_params)  {
>  	struct ecore_load_req_params load_req_params;
> -	u32 load_code, param, drv_mb_param;
> +	u32 load_code, resp, param, drv_mb_param;
>  	bool b_default_mtu =3D true;
>  	struct ecore_hwfn *p_hwfn;
> -	enum _ecore_status_t rc =3D ECORE_SUCCESS, mfw_rc;
> +	enum _ecore_status_t rc =3D ECORE_SUCCESS;
>  	int i;
>=20
>  	if ((p_params->int_mode =3D=3D ECORE_INT_MODE_MSI) && @@ -
> 1942,7 +1942,7 @@ enum _ecore_status_t ecore_hw_init(struct ecore_dev
> *p_dev,
>  	}
>=20
>  	for_each_hwfn(p_dev, i) {
> -		struct ecore_hwfn *p_hwfn =3D &p_dev->hwfns[i];
> +		p_hwfn =3D &p_dev->hwfns[i];
>=20
>  		/* If management didn't provide a default, set one of our
> own */
>  		if (!p_hwfn->hw_info.mtu) {
> @@ -1955,11 +1955,6 @@ enum _ecore_status_t ecore_hw_init(struct
> ecore_dev *p_dev,
>  			continue;
>  		}
>=20
> -		/* Enable DMAE in PXP */
> -		rc =3D ecore_change_pci_hwfn(p_hwfn, p_hwfn->p_main_ptt,
> true);
> -		if (rc !=3D ECORE_SUCCESS)
> -			return rc;
> -
>  		rc =3D ecore_calc_hw_mode(p_hwfn);
>  		if (rc !=3D ECORE_SUCCESS)
>  			return rc;
> @@ -2009,6 +2004,30 @@ enum _ecore_status_t ecore_hw_init(struct
> ecore_dev *p_dev,
>  			qm_lock_init =3D true;
>  		}
>=20
> +		/* Clean up chip from previous driver if such remains exist.
> +		 * This is not needed when the PF is the first one on the
> +		 * engine, since afterwards we are going to init the FW.
> +		 */
> +		if (load_code !=3D FW_MSG_CODE_DRV_LOAD_ENGINE) {
> +			rc =3D ecore_final_cleanup(p_hwfn, p_hwfn-
> >p_main_ptt,
> +						 p_hwfn->rel_pf_id, false);
> +			if (rc !=3D ECORE_SUCCESS) {
> +				ecore_hw_err_notify(p_hwfn,
> +
> ECORE_HW_ERR_RAMROD_FAIL);
> +				goto load_err;
> +			}
> +		}
> +
> +		/* Log and clean previous pglue_b errors if such exist */
> +		ecore_pglueb_rbc_attn_handler(p_hwfn, p_hwfn-
> >p_main_ptt);
> +		ecore_pglueb_clear_err(p_hwfn, p_hwfn->p_main_ptt);
> +
> +		/* Enable the PF's internal FID_enable in the PXP */
> +		rc =3D ecore_pglueb_set_pfid_enable(p_hwfn, p_hwfn-
> >p_main_ptt,
> +						  true);
> +		if (rc !=3D ECORE_SUCCESS)
> +			goto load_err;
> +
>  		switch (load_code) {
>  		case FW_MSG_CODE_DRV_LOAD_ENGINE:
>  			rc =3D ecore_hw_init_common(p_hwfn, p_hwfn-
> >p_main_ptt, @@ -2037,35 +2056,28 @@ enum _ecore_status_t
> ecore_hw_init(struct ecore_dev *p_dev,
>  			break;
>  		}
>=20
> -		if (rc !=3D ECORE_SUCCESS)
> +		if (rc !=3D ECORE_SUCCESS) {
>  			DP_NOTICE(p_hwfn, true,
>  				  "init phase failed for loadcode 0x%x (rc
> %d)\n",
>  				  load_code, rc);
> +			goto load_err;
> +		}
>=20
> -		/* ACK mfw regardless of success or failure of initialization */
> -		mfw_rc =3D ecore_mcp_cmd(p_hwfn, p_hwfn->p_main_ptt,
> -				       DRV_MSG_CODE_LOAD_DONE,
> -				       0, &load_code, &param);
> +		rc =3D ecore_mcp_load_done(p_hwfn, p_hwfn->p_main_ptt);
>  		if (rc !=3D ECORE_SUCCESS)
>  			return rc;
>=20
> -		if (mfw_rc !=3D ECORE_SUCCESS) {
> -			DP_NOTICE(p_hwfn, true,
> -				  "Failed sending a LOAD_DONE
> command\n");
> -			return mfw_rc;
> -		}
> -
>  		/* send DCBX attention request command */
>  		DP_VERBOSE(p_hwfn, ECORE_MSG_DCB,
>  			   "sending phony dcbx set command to trigger DCBx
> attention handling\n");
> -		mfw_rc =3D ecore_mcp_cmd(p_hwfn, p_hwfn->p_main_ptt,
> -				       DRV_MSG_CODE_SET_DCBX,
> -				       1 <<
> DRV_MB_PARAM_DCBX_NOTIFY_SHIFT,
> -				       &load_code, &param);
> -		if (mfw_rc !=3D ECORE_SUCCESS) {
> +		rc =3D ecore_mcp_cmd(p_hwfn, p_hwfn->p_main_ptt,
> +				   DRV_MSG_CODE_SET_DCBX,
> +				   1 <<
> DRV_MB_PARAM_DCBX_NOTIFY_SHIFT, &resp,
> +				   &param);
> +		if (rc !=3D ECORE_SUCCESS) {
>  			DP_NOTICE(p_hwfn, true,
>  				  "Failed to send DCBX attention request\n");
> -			return mfw_rc;
> +			return rc;
>  		}
>=20
>  		p_hwfn->hw_init_done =3D true;
> @@ -2076,7 +2088,7 @@ enum _ecore_status_t ecore_hw_init(struct
> ecore_dev *p_dev,
>  		drv_mb_param =3D STORM_FW_VERSION;
>  		rc =3D ecore_mcp_cmd(p_hwfn, p_hwfn->p_main_ptt,
>=20
> DRV_MSG_CODE_OV_UPDATE_STORM_FW_VER,
> -				   drv_mb_param, &load_code, &param);
> +				   drv_mb_param, &resp, &param);
>  		if (rc !=3D ECORE_SUCCESS)
>  			DP_INFO(p_hwfn, "Failed to update firmware
> version\n");
>=20
> @@ -2094,6 +2106,14 @@ enum _ecore_status_t ecore_hw_init(struct
> ecore_dev *p_dev,
>  	}
>=20
>  	return rc;
> +
> +load_err:
> +	/* The MFW load lock should be released regardless of success or
> failure
> +	 * of initialization.
> +	 * TODO: replace this with an attempt to send cancel_load.
> +	 */
> +	ecore_mcp_load_done(p_hwfn, p_hwfn->p_main_ptt);
> +	return rc;
>  }
>=20
>  #define ECORE_HW_STOP_RETRY_LIMIT	(10)
> @@ -2261,18 +2281,20 @@ enum _ecore_status_t ecore_hw_stop(struct
> ecore_dev *p_dev)
>  		}
>  	} /* hwfn loop */
>=20
> -	if (IS_PF(p_dev)) {
> +	if (IS_PF(p_dev) && !p_dev->recov_in_prog) {
>  		p_hwfn =3D ECORE_LEADING_HWFN(p_dev);
>  		p_ptt =3D ECORE_LEADING_HWFN(p_dev)->p_main_ptt;
>=20
> -		/* Disable DMAE in PXP - in CMT, this should only be done for
> -		 * first hw-function, and only after all transactions have
> -		 * stopped for all active hw-functions.
> -		 */
> -		rc =3D ecore_change_pci_hwfn(p_hwfn, p_ptt, false);
> +		 /* Clear the PF's internal FID_enable in the PXP.
> +		  * In CMT this should only be done for first hw-function, and
> +		  * only after all transactions have stopped for all active
> +		  * hw-functions.
> +		  */
> +		rc =3D ecore_pglueb_set_pfid_enable(p_hwfn, p_hwfn-
> >p_main_ptt,
> +						  false);
>  		if (rc !=3D ECORE_SUCCESS) {
>  			DP_NOTICE(p_hwfn, true,
> -				  "ecore_change_pci_hwfn failed. rc =3D
> %d.\n",
> +				  "ecore_pglueb_set_pfid_enable() failed. rc
> =3D %d.\n",
>  				  rc);
>  			rc2 =3D ECORE_UNKNOWN_ERROR;
>  		}
> @@ -2370,9 +2392,8 @@ static void ecore_hw_hwfn_prepare(struct
> ecore_hwfn *p_hwfn)
>  			 PGLUE_B_REG_PGL_ADDR_94_F0_BB, 0);
>  	}
>=20
> -	/* Clean Previous errors if such exist */
> -	ecore_wr(p_hwfn, p_hwfn->p_main_ptt,
> -		 PGLUE_B_REG_WAS_ERROR_PF_31_0_CLR, 1 << p_hwfn-
> >abs_pf_id);
> +	/* Clean previous pglue_b errors if such exist */
> +	ecore_pglueb_clear_err(p_hwfn, p_hwfn->p_main_ptt);
>=20
>  	/* enable internal target-read */
>  	ecore_wr(p_hwfn, p_hwfn->p_main_ptt,
> diff --git a/drivers/net/qede/base/ecore_dev_api.h
> b/drivers/net/qede/base/ecore_dev_api.h
> index 886407b..eea22e0 100644
> --- a/drivers/net/qede/base/ecore_dev_api.h
> +++ b/drivers/net/qede/base/ecore_dev_api.h
> @@ -584,4 +584,16 @@ enum _ecore_status_t
> ecore_set_queue_coalesce(struct ecore_hwfn *p_hwfn, u16 rx_coal,
>  			 u16 tx_coal, void *p_handle);
>=20
> +/**
> + * @brief ecore_pglueb_set_pfid_enable - Enable or disable PCI BUS
> +MASTER
> + *
> + * @param p_hwfn
> + * @param p_ptt
> + * @param b_enable - true/false
> + *
> + * @return enum _ecore_status_t
> + */
> +enum _ecore_status_t ecore_pglueb_set_pfid_enable(struct ecore_hwfn
> *p_hwfn,
> +						  struct ecore_ptt *p_ptt,
> +						  bool b_enable);
>  #endif
> diff --git a/drivers/net/qede/base/ecore_int.c
> b/drivers/net/qede/base/ecore_int.c
> index 2afca29..b57c510 100644
> --- a/drivers/net/qede/base/ecore_int.c
> +++ b/drivers/net/qede/base/ecore_int.c
> @@ -284,122 +284,119 @@ static enum _ecore_status_t
> ecore_grc_attn_cb(struct ecore_hwfn *p_hwfn)  #define
> ECORE_PGLUE_ATTENTION_ICPL_VALID (1 << 23)  #define
> ECORE_PGLUE_ATTENTION_ZLR_VALID (1 << 25)  #define
> ECORE_PGLUE_ATTENTION_ILT_VALID (1 << 23) -static enum
> _ecore_status_t ecore_pglub_rbc_attn_cb(struct ecore_hwfn *p_hwfn)
> +
> +enum _ecore_status_t ecore_pglueb_rbc_attn_handler(struct ecore_hwfn
> *p_hwfn,
> +						   struct ecore_ptt *p_ptt)
>  {
>  	u32 tmp;
>=20
> -	tmp =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> -		       PGLUE_B_REG_TX_ERR_WR_DETAILS2);
> +	tmp =3D ecore_rd(p_hwfn, p_ptt,
> PGLUE_B_REG_TX_ERR_WR_DETAILS2);
>  	if (tmp & ECORE_PGLUE_ATTENTION_VALID) {
>  		u32 addr_lo, addr_hi, details;
>=20
> -		addr_lo =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_lo =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_TX_ERR_WR_ADD_31_0);
> -		addr_hi =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_hi =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_TX_ERR_WR_ADD_63_32);
> -		details =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		details =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_TX_ERR_WR_DETAILS);
>=20
> -		DP_INFO(p_hwfn,
> -			"Illegal write by chip to [%08x:%08x] blocked."
> -			"Details: %08x [PFID %02x, VFID %02x, VF_VALID
> %02x]"
> -			" Details2 %08x [Was_error %02x BME deassert
> %02x"
> -			" FID_enable deassert %02x]\n",
> -			addr_hi, addr_lo, details,
> -			(u8)((details &
> -
> ECORE_PGLUE_ATTENTION_DETAILS_PFID_MASK) >>
> -
> ECORE_PGLUE_ATTENTION_DETAILS_PFID_SHIFT),
> -			(u8)((details &
> -
> ECORE_PGLUE_ATTENTION_DETAILS_VFID_MASK) >>
> -
> ECORE_PGLUE_ATTENTION_DETAILS_VFID_SHIFT),
> -			(u8)((details &
> ECORE_PGLUE_ATTENTION_DETAILS_VF_VALID)
> -			     ? 1 : 0), tmp,
> -			(u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_WAS_ERR) ? 1
> -			     : 0),
> -			(u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_BME) ? 1 :
> -			     0),
> -			(u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_FID_EN) ? 1
> -			     : 0));
> +		DP_NOTICE(p_hwfn, false,
> +			  "Illegal write by chip to [%08x:%08x] blocked.
> Details: %08x [PFID %02x, VFID %02x, VF_VALID %02x] Details2 %08x
> [Was_error %02x BME deassert %02x FID_enable deassert %02x]\n",
> +			  addr_hi, addr_lo, details,
> +			  (u8)((details &
> +
> 	ECORE_PGLUE_ATTENTION_DETAILS_PFID_MASK) >>
> +
> ECORE_PGLUE_ATTENTION_DETAILS_PFID_SHIFT),
> +			  (u8)((details &
> +
> 	ECORE_PGLUE_ATTENTION_DETAILS_VFID_MASK) >>
> +
> ECORE_PGLUE_ATTENTION_DETAILS_VFID_SHIFT),
> +			  (u8)((details &
> +			       ECORE_PGLUE_ATTENTION_DETAILS_VF_VALID)
> ? 1 : 0),
> +			  tmp,
> +			  (u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_WAS_ERR) ?
> +				1 : 0),
> +			  (u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_BME) ?
> +				1 : 0),
> +			  (u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_FID_EN) ?
> +				1 : 0));
>  	}
>=20
> -	tmp =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> -		       PGLUE_B_REG_TX_ERR_RD_DETAILS2);
> +	tmp =3D ecore_rd(p_hwfn, p_ptt,
> PGLUE_B_REG_TX_ERR_RD_DETAILS2);
>  	if (tmp & ECORE_PGLUE_ATTENTION_RD_VALID) {
>  		u32 addr_lo, addr_hi, details;
>=20
> -		addr_lo =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_lo =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_TX_ERR_RD_ADD_31_0);
> -		addr_hi =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_hi =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_TX_ERR_RD_ADD_63_32);
> -		details =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		details =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_TX_ERR_RD_DETAILS);
>=20
> -		DP_INFO(p_hwfn,
> -			"Illegal read by chip from [%08x:%08x] blocked."
> -			" Details: %08x [PFID %02x, VFID %02x, VF_VALID
> %02x]"
> -			" Details2 %08x [Was_error %02x BME deassert
> %02x"
> -			" FID_enable deassert %02x]\n",
> -			addr_hi, addr_lo, details,
> -			(u8)((details &
> -
> ECORE_PGLUE_ATTENTION_DETAILS_PFID_MASK) >>
> -
> ECORE_PGLUE_ATTENTION_DETAILS_PFID_SHIFT),
> -			(u8)((details &
> -
> ECORE_PGLUE_ATTENTION_DETAILS_VFID_MASK) >>
> -
> ECORE_PGLUE_ATTENTION_DETAILS_VFID_SHIFT),
> -			(u8)((details &
> ECORE_PGLUE_ATTENTION_DETAILS_VF_VALID)
> -			     ? 1 : 0), tmp,
> -			(u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_WAS_ERR) ? 1
> -			     : 0),
> -			(u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_BME) ? 1 :
> -			     0),
> -			(u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_FID_EN) ? 1
> -			     : 0));
> +		DP_NOTICE(p_hwfn, false,
> +			  "Illegal read by chip from [%08x:%08x] blocked.
> Details: %08x [PFID %02x, VFID %02x, VF_VALID %02x] Details2 %08x
> [Was_error %02x BME deassert %02x FID_enable deassert %02x]\n",
> +			  addr_hi, addr_lo, details,
> +			  (u8)((details &
> +
> 	ECORE_PGLUE_ATTENTION_DETAILS_PFID_MASK) >>
> +
> ECORE_PGLUE_ATTENTION_DETAILS_PFID_SHIFT),
> +			  (u8)((details &
> +
> 	ECORE_PGLUE_ATTENTION_DETAILS_VFID_MASK) >>
> +
> ECORE_PGLUE_ATTENTION_DETAILS_VFID_SHIFT),
> +			  (u8)((details &
> +			       ECORE_PGLUE_ATTENTION_DETAILS_VF_VALID)
> ? 1 : 0),
> +			  tmp,
> +			  (u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_WAS_ERR) ?
> +				1 : 0),
> +			  (u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_BME) ?
> +				1 : 0),
> +			  (u8)((tmp &
> ECORE_PGLUE_ATTENTION_DETAILS2_FID_EN) ?
> +				1 : 0));
>  	}
>=20
> -	tmp =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> -		       PGLUE_B_REG_TX_ERR_WR_DETAILS_ICPL);
> +	tmp =3D ecore_rd(p_hwfn, p_ptt,
> PGLUE_B_REG_TX_ERR_WR_DETAILS_ICPL);
>  	if (tmp & ECORE_PGLUE_ATTENTION_ICPL_VALID)
> -		DP_INFO(p_hwfn, "ICPL error - %08x\n", tmp);
> +		DP_NOTICE(p_hwfn, false, "ICPL erorr - %08x\n", tmp);
>=20
> -	tmp =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> -		       PGLUE_B_REG_MASTER_ZLR_ERR_DETAILS);
> +	tmp =3D ecore_rd(p_hwfn, p_ptt,
> PGLUE_B_REG_MASTER_ZLR_ERR_DETAILS);
>  	if (tmp & ECORE_PGLUE_ATTENTION_ZLR_VALID) {
>  		u32 addr_hi, addr_lo;
>=20
> -		addr_lo =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_lo =3D ecore_rd(p_hwfn, p_ptt,
>=20
> PGLUE_B_REG_MASTER_ZLR_ERR_ADD_31_0);
> -		addr_hi =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_hi =3D ecore_rd(p_hwfn, p_ptt,
>=20
> PGLUE_B_REG_MASTER_ZLR_ERR_ADD_63_32);
>=20
> -		DP_INFO(p_hwfn, "ICPL error - %08x [Address
> %08x:%08x]\n",
> -			tmp, addr_hi, addr_lo);
> +		DP_NOTICE(p_hwfn, false,
> +			  "ICPL erorr - %08x [Address %08x:%08x]\n",
> +			  tmp, addr_hi, addr_lo);
>  	}
>=20
> -	tmp =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> -		       PGLUE_B_REG_VF_ILT_ERR_DETAILS2);
> +	tmp =3D ecore_rd(p_hwfn, p_ptt,
> PGLUE_B_REG_VF_ILT_ERR_DETAILS2);
>  	if (tmp & ECORE_PGLUE_ATTENTION_ILT_VALID) {
>  		u32 addr_hi, addr_lo, details;
>=20
> -		addr_lo =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_lo =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_VF_ILT_ERR_ADD_31_0);
> -		addr_hi =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		addr_hi =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_VF_ILT_ERR_ADD_63_32);
> -		details =3D ecore_rd(p_hwfn, p_hwfn->p_dpc_ptt,
> +		details =3D ecore_rd(p_hwfn, p_ptt,
>  				   PGLUE_B_REG_VF_ILT_ERR_DETAILS);
>=20
> -		DP_INFO(p_hwfn,
> -			"ILT error - Details %08x Details2 %08x"
> -			" [Address %08x:%08x]\n",
> -			details, tmp, addr_hi, addr_lo);
> +		DP_NOTICE(p_hwfn, false,
> +			  "ILT error - Details %08x Details2 %08x [Address
> %08x:%08x]\n",
> +			  details, tmp, addr_hi, addr_lo);
>  	}
>=20
>  	/* Clear the indications */
> -	ecore_wr(p_hwfn, p_hwfn->p_dpc_ptt,
> -		 PGLUE_B_REG_LATCHED_ERRORS_CLR, (1 << 2));
> +	ecore_wr(p_hwfn, p_ptt, PGLUE_B_REG_LATCHED_ERRORS_CLR, (1
> << 2));
>=20
>  	return ECORE_SUCCESS;
>  }
>=20
> +static enum _ecore_status_t ecore_pglueb_rbc_attn_cb(struct
> ecore_hwfn
> +*p_hwfn) {
> +	return ecore_pglueb_rbc_attn_handler(p_hwfn, p_hwfn-
> >p_dpc_ptt); }
> +
>  static enum _ecore_status_t ecore_fw_assertion(struct ecore_hwfn
> *p_hwfn)  {
>  	DP_NOTICE(p_hwfn, false, "FW assertion!\n"); @@ -505,7 +502,7
> @@ enum aeu_invert_reg_special_type {
>  	 {			/* After Invert 2 */
>  	  {"PGLUE config_space", ATTENTION_SINGLE, OSAL_NULL,
> MAX_BLOCK_ID},
>  	  {"PGLUE misc_flr", ATTENTION_SINGLE, OSAL_NULL,
> MAX_BLOCK_ID},
> -	  {"PGLUE B RBC", ATTENTION_PAR_INT, ecore_pglub_rbc_attn_cb,
> +	  {"PGLUE B RBC", ATTENTION_PAR_INT, ecore_pglueb_rbc_attn_cb,
>  	   BLOCK_PGLUE_B},
>  	  {"PGLUE misc_mctp", ATTENTION_SINGLE, OSAL_NULL,
> MAX_BLOCK_ID},
>  	  {"Flash event", ATTENTION_SINGLE, OSAL_NULL, MAX_BLOCK_ID},
> @@ -827,8 +824,9 @@ static void ecore_int_attn_print(struct ecore_hwfn
> *p_hwfn,
>  				     ATTN_TYPE_INTERRUPT, !b_fatal);  }
>=20
> +	/* @DPDK */
>  	/* Reach assertion if attention is fatal */
> -	if (b_fatal) {
> +	if (b_fatal || (strcmp(p_bit_name, "PGLUE B RBC") =3D=3D 0)) {
>  		DP_NOTICE(p_hwfn, true, "`%s': Fatal attention\n",
>  			  p_bit_name);
>=20
> diff --git a/drivers/net/qede/base/ecore_int.h
> b/drivers/net/qede/base/ecore_int.h
> index 0c8929e..067ed60 100644
> --- a/drivers/net/qede/base/ecore_int.h
> +++ b/drivers/net/qede/base/ecore_int.h
> @@ -208,4 +208,7 @@ enum _ecore_status_t
> ecore_int_set_timer_res(struct ecore_hwfn *p_hwfn,  #define
> ECORE_MAPPING_MEMORY_SIZE(dev) NUM_OF_SBS(dev)  #endif
>=20
> +enum _ecore_status_t ecore_pglueb_rbc_attn_handler(struct ecore_hwfn
> *p_hwfn,
> +						   struct ecore_ptt *p_ptt);
> +
>  #endif /* __ECORE_INT_H__ */
> diff --git a/drivers/net/qede/base/ecore_mcp.c
> b/drivers/net/qede/base/ecore_mcp.c
> index 03cc901..88c5ceb 100644
> --- a/drivers/net/qede/base/ecore_mcp.c
> +++ b/drivers/net/qede/base/ecore_mcp.c
> @@ -893,6 +893,30 @@ enum _ecore_status_t ecore_mcp_load_req(struct
> ecore_hwfn *p_hwfn,
>  	return ECORE_SUCCESS;
>  }
>=20
> +enum _ecore_status_t ecore_mcp_load_done(struct ecore_hwfn
> *p_hwfn,
> +					 struct ecore_ptt *p_ptt)
> +{
> +	u32 resp =3D 0, param =3D 0;
> +	enum _ecore_status_t rc;
> +
> +	rc =3D ecore_mcp_cmd(p_hwfn, p_ptt,
> DRV_MSG_CODE_LOAD_DONE, 0, &resp,
> +			   &param);
> +	if (rc !=3D ECORE_SUCCESS) {
> +		DP_NOTICE(p_hwfn, false,
> +			  "Failed to send a LOAD_DONE command, rc =3D
> %d\n", rc);
> +		return rc;
> +	}
> +
> +#define FW_MB_PARAM_LOAD_DONE_DID_EFUSE_ERROR     (1 << 0)
> +
> +	/* Check if there is a DID mismatch between nvm-cfg/efuse */
> +	if (param & FW_MB_PARAM_LOAD_DONE_DID_EFUSE_ERROR)
> +		DP_NOTICE(p_hwfn, false,
> +			  "warning: device configuration is not supported on
> this board
> +type. The device may not function as expected.\n");
> +
> +	return ECORE_SUCCESS;
> +}
> +
>  enum _ecore_status_t ecore_mcp_unload_req(struct ecore_hwfn
> *p_hwfn,
>  					  struct ecore_ptt *p_ptt)
>  {
> @@ -2893,6 +2917,27 @@ struct ecore_resc_alloc_out_params {
>  	u32 flags;
>  };
>=20
> +#define ECORE_RECOVERY_PROLOG_SLEEP_MS	100
> +
> +enum _ecore_status_t ecore_recovery_prolog(struct ecore_dev *p_dev) {
> +	struct ecore_hwfn *p_hwfn =3D ECORE_LEADING_HWFN(p_dev);
> +	struct ecore_ptt *p_ptt =3D p_hwfn->p_main_ptt;
> +	enum _ecore_status_t rc;
> +
> +	/* Allow ongoing PCIe transactions to complete */
> +	OSAL_MSLEEP(ECORE_RECOVERY_PROLOG_SLEEP_MS);
> +
> +	/* Clear the PF's internal FID_enable in the PXP */
> +	rc =3D ecore_pglueb_set_pfid_enable(p_hwfn, p_ptt, false);
> +	if (rc !=3D ECORE_SUCCESS)
> +		DP_NOTICE(p_hwfn, false,
> +			  "ecore_pglueb_set_pfid_enable() failed. rc =3D
> %d.\n",
> +			  rc);
> +
> +	return rc;
> +}
> +
>  static enum _ecore_status_t
>  ecore_mcp_resc_allocation_msg(struct ecore_hwfn *p_hwfn,
>  			      struct ecore_ptt *p_ptt,
> diff --git a/drivers/net/qede/base/ecore_mcp.h
> b/drivers/net/qede/base/ecore_mcp.h
> index 37d1835..77fb5a3 100644
> --- a/drivers/net/qede/base/ecore_mcp.h
> +++ b/drivers/net/qede/base/ecore_mcp.h
> @@ -171,6 +171,17 @@ enum _ecore_status_t ecore_mcp_load_req(struct
> ecore_hwfn *p_hwfn,
>  					struct ecore_load_req_params
> *p_params);
>=20
>  /**
> + * @brief Sends a LOAD_DONE message to the MFW
> + *
> + * @param p_hwfn
> + * @param p_ptt
> + *
> + * @return enum _ecore_status_t - ECORE_SUCCESS - Operation was
> successful.
> + */
> +enum _ecore_status_t ecore_mcp_load_done(struct ecore_hwfn
> *p_hwfn,
> +					 struct ecore_ptt *p_ptt);
> +
> +/**
>   * @brief Sends a UNLOAD_REQ message to the MFW
>   *
>   * @param p_hwfn
> diff --git a/drivers/net/qede/base/ecore_mcp_api.h
> b/drivers/net/qede/base/ecore_mcp_api.h
> index 190c135..abc190c 100644
> --- a/drivers/net/qede/base/ecore_mcp_api.h
> +++ b/drivers/net/qede/base/ecore_mcp_api.h
> @@ -736,6 +736,17 @@ enum _ecore_status_t
> ecore_start_recovery_process(struct ecore_hwfn *p_hwfn,
>  						  struct ecore_ptt *p_ptt);
>=20
>  /**
> + * @brief A recovery handler must call this function as its first step.
> + *        It is assumed that the handler is not run from an interrupt co=
ntext.
> + *
> + *  @param p_dev
> + *  @param p_ptt
> + *
> + * @return enum _ecore_status_t
> + */
> +enum _ecore_status_t ecore_recovery_prolog(struct ecore_dev *p_dev);
> +
> +/**
>   * @brief Notify MFW about the change in base device properties
>   *
>   *  @param p_hwfn
> --
> 1.7.10.3