From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BE697A055C; Fri, 27 May 2022 05:10:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 551D940E25; Fri, 27 May 2022 05:10:18 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 9C97840DF7; Fri, 27 May 2022 05:10:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1653621016; x=1685157016; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=kMhfwaf3N0TiQxsujysQq+fIW9w8A6sRoTqNh7Sad4w=; b=cH+2FMEhSIgp+ncdzDX/ldWHO2KsvCcaE0ZOorshmY9BOTt7hWcI4COE BFAavcj9eFxcb/ta2ys4azZuLmJ59MLxkJO4HlCEVJhvzT7dZxkDbtXCR HxrMb7YhIVg2rNMMtrrKozKRfZ9QNtERpzOpTDeWeudiGMaTZ+y17/Q7q tuHuyACStpzeYUvzcj97HfgWfdn/u/1Oy0XgnbCTqsKZ+0IboZNdE2ck9 2R6ycM6D0cGmqRu94KsJblFXzc7LAbPHm3iXdVrOmZrdWNzlx6fXXtjVi BABLArJdOU0Oq+gAhmdwEbqWHPYPBcSSOe/46U4tN1l33ylhge6DJhxDD A==; X-IronPort-AV: E=McAfee;i="6400,9594,10359"; a="254238159" X-IronPort-AV: E=Sophos;i="5.91,254,1647327600"; d="scan'208";a="254238159" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 May 2022 20:10:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,254,1647327600"; d="scan'208";a="631260624" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga008.fm.intel.com with ESMTP; 26 May 2022 20:10:13 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Thu, 26 May 2022 20:10:13 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Thu, 26 May 2022 20:10:13 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Thu, 26 May 2022 20:10:12 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fk+Xuc6mT2FwmqVwLnCO3mH6SPCvNMriOIZKetCuGq5GlEmnp/brqtiIYspujzMPQlhrd09nsBHIz0Bjuo633NsmFwtoq5M7L7vWL87uOFqV9dPngWQLIIyOyFkxOfoKRSvvb5koxcvA1mafiL3zXuutZhvU5Gv+I7TrZXbbEbCBmy1sffl40SMSSjHl2PYZUp6I3PV99qHxITvMfHmiySqR5/JmaCjkOf1BEhQrFY/JqfYtoqcFES0D2XquiQt03J6eANVOyBAZT+AAFDUxpypMbWVAVDbq8o+hzMqntZCgUj9FMy5oIBDIj51EgMw6TEgFNDP9aBeGCtcQq3+ffg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O5XhSvL8xH9+zvLzwwXVYKRKoszd8UAnRgqU062NcX4=; b=VBtSAcMXyGtrpF2gR0bFWjvI3Tp5iNiGjCpWOTA53t7vIjJrKUzwvdk1n2N5M6JpnbbQ8/t4oGqPBzjAZxVoc0zPz2vZCo4R8x/7UrnwJiaP5iBzYp+6B+mPo6PBo9Tw2DlpZ2CVwL76TbW+q70E2xCiV9ppzsXFXbZUxVtOydPYrMV8XnpXfg2YA975ZZ4Zi8fUqDoC8Y/xWVwq1PTHo1CG+8YyGeWg1sInPd+491GL7G36GxiDIkbcM5RVtAzV/mAdpW0LSowSK8qFaU7cEJbVPd67j6Ot6KEZwejROhG28d/oopR6BkKJa8AbeQt5k8OlfyGdKg22Rf4/gh/eMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5483.namprd11.prod.outlook.com (2603:10b6:408:104::10) by SJ0PR11MB5583.namprd11.prod.outlook.com (2603:10b6:a03:3bb::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5293.13; Fri, 27 May 2022 03:10:05 +0000 Received: from BN9PR11MB5483.namprd11.prod.outlook.com ([fe80::4d94:5d8b:51d7:d8b4]) by BN9PR11MB5483.namprd11.prod.outlook.com ([fe80::4d94:5d8b:51d7:d8b4%9]) with mapi id 15.20.5293.013; Fri, 27 May 2022 03:10:05 +0000 From: "Zhang, Tianfei" To: "Huang, Wei" , "dev@dpdk.org" , "thomas@monjalon.net" , "nipun.gupta@nxp.com" , "hemant.agrawal@nxp.com" CC: "stable@dpdk.org" , "Xu, Rosen" , "Zhang, Qi Z" Subject: RE: [PATCH v3 4/5] raw/ifpga: support ofs card probe Thread-Topic: [PATCH v3 4/5] raw/ifpga: support ofs card probe Thread-Index: AQHYcLBWyQrMq1tVQEWDrt6to12i+60yDObQ Date: Fri, 27 May 2022 03:10:05 +0000 Message-ID: References: <1652862549-13131-1-git-send-email-wei.huang@intel.com> <1653535974-1379-1-git-send-email-wei.huang@intel.com> <1653535974-1379-5-git-send-email-wei.huang@intel.com> In-Reply-To: <1653535974-1379-5-git-send-email-wei.huang@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.500.17 dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6cb3bbfc-b7b5-411a-1252-08da3f8e65fa x-ms-traffictypediagnostic: SJ0PR11MB5583:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hs3mZ5N5tnJuMDsC7UWoqxFHvUK8PPp4iwxARUYZZxi1Ot69apMHTvmVnUG1ijZsBZ6Grnf0Jj3kGD8m85cwEum3GNrfU25WkgguJ0WlgP0vQ4LtfV7c9/tkjd4Qo1RNPX3Jloy7Hb37jN9HgLzUvRu0WMTfBVedsDQb24/l1za4h0oo0mDe4F6bjiI3mZIraHUU4f2EH/nXQpzB1YvATAvvQWx8vAXm2GcSqwBmljisvOKm5f/EyOoK8vdYpSND0luOEepBseoKBMvaCsRa3caQLYKPe/6PE0/zr2wJQoBl0mFLkY2zTKA+IuDD3KjZZHbTE6KlqeMNC/NjvKJ5JQXmet8Z0iTrpzIF2cS80+ctjSmeJeyvLcAevaRsgOsZNDA62SuxF6FyxDq5HTvM+Ow7BsvVAPM4l82xjqizjmmlhLZXoPaZTqocRiCull0PxtS+ME+6flkl77VGfzq+vFLUccOJRwpyBQqlinuebmkEVaajULRSlpF09bjNlNdLn6ZOp4dVsarrNO/RzlBINhi8pUy6q/JXO8tE4UFkceEMsEpA3OcGH5PBl2PHBbNs2GLAFC+fOcQwTdMdpuHjxxwy/ICFlcDZ3wxlos9rtpyJ3UsXaILBh2886gV5jHwBioSKDjgyuOahitlZrJGJlZXJ/F1DXDMed4UqXZq12+CmJy4hDeixKkKIRNqbbQlr7Ebbs5HoavgbvO8zhwAQvQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR11MB5483.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(54906003)(5660300002)(38100700002)(110136005)(316002)(107886003)(2906002)(52536014)(30864003)(4326008)(66946007)(66446008)(66476007)(8676002)(64756008)(82960400001)(66556008)(9686003)(8936002)(38070700005)(122000001)(53546011)(7696005)(76116006)(6506007)(83380400001)(26005)(186003)(71200400001)(508600001)(55016003)(86362001)(33656002)(579004)(559001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?P8WI2rXngUSknl0OOszSIhw1zsY65FNJYlqJ/Uk+K2t/KlvCqu7RZV4/u+Jf?= =?us-ascii?Q?dyFPHOncFkM9rhkL7KedvO2CDJC8yzNaicv/w0YBOilr/TDF84uiMxrzV3WK?= =?us-ascii?Q?ACA9aS1N79ztTg29tnihICETlxW9pkTSdbR2S2t0o2oB07FcYOyJuylCU+jQ?= =?us-ascii?Q?d1oEkSFEaEqSLpGKDVpP2rGGXdaiyDuIYXEEP2qzulj0zudSYlXdh2jxFqrC?= =?us-ascii?Q?ACyqwjiSfr6uT8JVEze+zeg0uNa/OXzMKK6EKw3GjSv54idvWoQOkOrbDHj9?= =?us-ascii?Q?2fEvG2qdkadPGTAVkJ2DtlRhjreteR7/9+WglZx4Z11dE/Et2/eyNRHBISyO?= =?us-ascii?Q?BopsYL0WP9rauyKHAvYzQ+Jif7OU/aHx0yZIWjwkt173cNI3PXwF5fMdY7c9?= =?us-ascii?Q?6oRFSuI4P7uPoORDs/WFTV7VFoiOdbSFl1Jt0oFa+XMNMX6ckguLuv/yAHvk?= =?us-ascii?Q?gySbAtjRRSrAAZYmA8wvM1EtlZkp81op0hfOVoV5qsFJ/qKVOn+BXOnbmjP2?= =?us-ascii?Q?gOqEJ/pI/umsCEhEEKpUZ5GqAYPccIRRhWWYA24igveLKGEktE696zDQsSD3?= =?us-ascii?Q?Qzd7z2vjsB610dqZ3HYOw3TOh6Kz5sE805olgCLEZUOiAGOTln0lo5HDT257?= =?us-ascii?Q?KSYObR+niRhperuBuf4XzP0wMrJ0HOvrsX5swUCExZ2GRU0JjnRzqFRb1a19?= =?us-ascii?Q?Y1Qgi4mguJSqvWmtKW61d0BQ5hyKgRNtGAkGzVSe5klag55GPySNXIbX5FhU?= =?us-ascii?Q?5Q9noUTN9HZLE9RJ00q0Lbk38FSszqYFhej6KCfnpvoCCNqKvy/tMxxH0GGn?= =?us-ascii?Q?Wt5J39++20odOt6aA/RMTWD6QtMtlqygbc7zV8xR20gDuqEngxnfMvYjYWfd?= =?us-ascii?Q?fitJYZUebJRblxLuEfKx6suoLP1MeJLFvKa9XhNtPZEhgWsheJvB/oPOTbq5?= =?us-ascii?Q?agBYAiweM3wLaNl7vVZkwltD8MfmF8ib2yxcqREC5sUx8pw2ocboCTURZs93?= =?us-ascii?Q?YFKSxoqYgIIC+doZR4LKbf1AlO1WJwt3NC+j+29khGXRxk+E008UV0/r1qk/?= =?us-ascii?Q?FGDPrxwlLBeo+kKvU4BA97NtX5HiRGPtC5+gSmvOzhRMrRvFZiaAyvdr/GQJ?= =?us-ascii?Q?K8GiHSleLoXy8E5teyx2QBBxZf5XQNBG8UxJ4L8EBrndLE4Re1WqvYHKkV41?= =?us-ascii?Q?k31VC7weXhmLRUa60EsVPpjIw3tadghw3C9HtWEZSdOigdxSUe5wxuMKMCaO?= =?us-ascii?Q?FilG0QpMiAjtgWxFk6RAHdVe5JoK4k4HtYxFwyfQe0nrYaNaPAKjDqjn3twX?= =?us-ascii?Q?1ABrOY7EKIWBjITp+gdCf8GwYEUaF7+CijLNZVNY0uM26IFTZR/46KpOKOtY?= =?us-ascii?Q?0SEUp4j1trVk8ejrp+kSppFryQci1sR89wYW7UA0qUIzEXvB2ENpQ3JRYxYX?= =?us-ascii?Q?4Ac2DviLuTnuPH9UvtgPDF9A8GXqw98gzL9Uw7rEkCse0uyvxshA9Ahrct0j?= =?us-ascii?Q?vzSEEa/RtETC5V4hCHAukcxIpvqxTORrKIshCOwmORgAQ9nmHiq2ojCCcE0n?= =?us-ascii?Q?hDw7CHh77eZCgS6qkUY2AUpAwaiT+DeCZ/cqJs+odZN97PuO9Xjw1jOdlOjJ?= =?us-ascii?Q?2PQycFE/g0gALziZ0wCUnhAEFpPh2NlD7G2XJIn5hv5X+IbjMo1j993Tik1L?= =?us-ascii?Q?VMKMfG2LYGG3lUDiJXOce1gHXiIM/JXMOhbXcjh5w2uW+FJDEcSur0yaAzCH?= =?us-ascii?Q?yGiF46P99A=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5483.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6cb3bbfc-b7b5-411a-1252-08da3f8e65fa X-MS-Exchange-CrossTenant-originalarrivaltime: 27 May 2022 03:10:05.4983 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: lOqe0n8ICCH2t7hHHxxyzH8nWP7gryZGpErVa7PRdndliRBC0M6MyO/Z5DmLNuGtyOhGGdTabRjQpA/p3tmAaA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5583 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Huang, Wei > Sent: Thursday, May 26, 2022 11:33 AM > To: dev@dpdk.org; thomas@monjalon.net; nipun.gupta@nxp.com; > hemant.agrawal@nxp.com > Cc: stable@dpdk.org; Xu, Rosen ; Zhang, Tianfei > ; Zhang, Qi Z ; Huang, Wei > > Subject: [PATCH v3 4/5] raw/ifpga: support ofs card probe >=20 > PAC N6000 is the first OFS platform, its device id is added to ifpga devi= ce support > list. >=20 > Previous FPGA platform like Intel PAC N3000 and N5000, FME DFL (Device > Feature List) starts from BAR0 by default, port DFL location is indicated= in > PORTn_OFFSET register in FME. In OFS implementation, FME DFL and port DFL > location can be defined individually in PCIe VSEC (Vendor Specific Extent= ed > Capabilities). In this patch, DFL definition is searched in VSEC, the leg= acy DFL is > used only when DFL VSEC is not present. >=20 > In original DFL enumeration process, AFU is expected to locate in port DF= L, but > this is not the case in OFS implementation. In this patch, enumeration ca= n search > AFU in any PF/VF which has no FME and port. >=20 > Signed-off-by: Wei Huang > --- > v2: fix build error in UB2004-32 > --- > v3: update commit log with Tianfei's comment, treat 7 as special BAR inde= x. > --- > drivers/raw/ifpga/base/ifpga_api.c | 28 +- > drivers/raw/ifpga/base/ifpga_defines.h | 1 + > drivers/raw/ifpga/base/ifpga_enumerate.c | 681 ++++++++++++++++++++++-= - > ----- > drivers/raw/ifpga/base/ifpga_enumerate.h | 2 + > drivers/raw/ifpga/base/ifpga_feature_dev.c | 18 +- > drivers/raw/ifpga/base/ifpga_hw.h | 2 + > drivers/raw/ifpga/base/opae_hw_api.c | 2 +- > drivers/raw/ifpga/base/opae_hw_api.h | 2 + > drivers/raw/ifpga/ifpga_rawdev.c | 48 +- > 9 files changed, 592 insertions(+), 192 deletions(-) >=20 > diff --git a/drivers/raw/ifpga/base/ifpga_api.c > b/drivers/raw/ifpga/base/ifpga_api.c > index 4610ef1..f19cc26 100644 > --- a/drivers/raw/ifpga/base/ifpga_api.c > +++ b/drivers/raw/ifpga/base/ifpga_api.c > @@ -13,15 +13,22 @@ > static int ifpga_acc_get_uuid(struct opae_accelerator *acc, > struct uuid *uuid) > { > - struct opae_bridge *br =3D acc->br; > - struct ifpga_port_hw *port; > + struct ifpga_afu_info *afu_info =3D acc->data; > + struct opae_reg_region *region; > + u64 val =3D 0; >=20 > - if (!br || !br->data) > - return -EINVAL; > + if (!afu_info) > + return -ENODEV; >=20 > - port =3D br->data; > + region =3D &afu_info->region[0]; > + if (uuid) { > + val =3D readq(region->addr + sizeof(struct feature_header)); > + opae_memcpy(uuid->b, &val, sizeof(u64)); > + val =3D readq(region->addr + sizeof(struct feature_header) + 8); > + opae_memcpy(uuid->b + 8, &val, sizeof(u64)); > + } >=20 > - return fpga_get_afu_uuid(port, uuid); > + return 0; > } >=20 > static int ifpga_acc_set_irq(struct opae_accelerator *acc, @@ -32,6 +39,= 9 @@ > static int ifpga_acc_set_irq(struct opae_accelerator *acc, > struct ifpga_port_hw *port; > struct fpga_uafu_irq_set irq_set; >=20 > + if (!afu_info) > + return -ENODEV; > + > if (!br || !br->data) > return -EINVAL; >=20 > @@ -68,7 +78,7 @@ static int ifpga_acc_get_region_info(struct > opae_accelerator *acc, > struct ifpga_afu_info *afu_info =3D acc->data; >=20 > if (!afu_info) > - return -EINVAL; > + return -ENODEV; >=20 > if (info->index >=3D afu_info->num_regions) > return -EINVAL; > @@ -89,7 +99,7 @@ static int ifpga_acc_read(struct opae_accelerator *acc, > unsigned int region_idx, > struct opae_reg_region *region; >=20 > if (!afu_info) > - return -EINVAL; > + return -ENODEV; >=20 > if (offset + byte <=3D offset) > return -EINVAL; > @@ -129,7 +139,7 @@ static int ifpga_acc_write(struct opae_accelerator *a= cc, > struct opae_reg_region *region; >=20 > if (!afu_info) > - return -EINVAL; > + return -ENODEV; >=20 > if (offset + byte <=3D offset) > return -EINVAL; > diff --git a/drivers/raw/ifpga/base/ifpga_defines.h > b/drivers/raw/ifpga/base/ifpga_defines.h > index 8f62033..9a280eb 100644 > --- a/drivers/raw/ifpga/base/ifpga_defines.h > +++ b/drivers/raw/ifpga/base/ifpga_defines.h > @@ -73,6 +73,7 @@ > enum fpga_id_type { > FME_ID, > PORT_ID, > + AFU_ID, > FPGA_ID_MAX, > }; >=20 > diff --git a/drivers/raw/ifpga/base/ifpga_enumerate.c > b/drivers/raw/ifpga/base/ifpga_enumerate.c > index 48b8af4..0e09904 100644 > --- a/drivers/raw/ifpga/base/ifpga_enumerate.c > +++ b/drivers/raw/ifpga/base/ifpga_enumerate.c > @@ -2,6 +2,10 @@ > * Copyright(c) 2010-2018 Intel Corporation > */ >=20 > +#include > +#include > +#include > + > #include "opae_hw_api.h" > #include "ifpga_api.h" >=20 > @@ -9,6 +13,19 @@ > #include "ifpga_enumerate.h" > #include "ifpga_feature_dev.h" >=20 > +struct dfl_fpga_enum_dfl { > + u64 start; > + u64 len; > + void *addr; > + TAILQ_ENTRY(dfl_fpga_enum_dfl) node; > +}; > + > +TAILQ_HEAD(dfl_fpga_enum_dfls, dfl_fpga_enum_dfl); struct > +dfl_fpga_enum_info { > + struct ifpga_hw *hw; > + struct dfl_fpga_enum_dfls dfls; > +}; > + > struct build_feature_devs_info { > struct opae_adapter_data_pci *pci_data; >=20 > @@ -21,7 +38,6 @@ struct build_feature_devs_info { > void *ioaddr; > void *ioend; > uint64_t phys_addr; > - int current_bar; >=20 > void *pfme_hdr; >=20 > @@ -141,8 +157,10 @@ static u64 feature_id(void __iomem *start) > unsigned int size, unsigned int vec_start, > unsigned int vec_cnt) > { > - return build_info_add_sub_feature(binfo, start, fid, size, vec_start, > - vec_cnt); > + if (binfo->current_type !=3D AFU_ID) > + return build_info_add_sub_feature(binfo, start, fid, size, > + vec_start, vec_cnt); > + return 0; > } >=20 > /* > @@ -152,13 +170,14 @@ static u64 feature_id(void __iomem *start) > */ > static bool feature_is_UAFU(struct build_feature_devs_info *binfo) { > - if (binfo->current_type !=3D PORT_ID) > - return false; > + if ((binfo->current_type =3D=3D PORT_ID) || > + (binfo->current_type =3D=3D AFU_ID)) > + return true; >=20 > - return true; > + return false; > } >=20 > -static int parse_feature_port_uafu(struct build_feature_devs_info *binfo= , > +static int parse_feature_uafu(struct build_feature_devs_info *binfo, > struct feature_header *hdr) > { > u64 id =3D PORT_FEATURE_ID_UAFU; > @@ -169,9 +188,19 @@ static int parse_feature_port_uafu(struct > build_feature_devs_info *binfo, > int ret; > int size; >=20 > + if (binfo->acc_info) { > + dev_info(binfo, "Sub AFU found @ %p.\n", start); > + return 0; > + } > + > capability.csr =3D readq(&port_hdr->capability); >=20 > - size =3D capability.mmio_size << 10; > + if (binfo->current_type =3D=3D AFU_ID) { > + size =3D AFU_REGION_SIZE; > + } else { > + capability.csr =3D readq(&port_hdr->capability); > + size =3D capability.mmio_size << 10; > + } >=20 > ret =3D create_feature_instance(binfo, hdr, id, size, 0, 0); > if (ret) > @@ -185,44 +214,13 @@ static int parse_feature_port_uafu(struct > build_feature_devs_info *binfo, > info->region[0].phys_addr =3D binfo->phys_addr + > (uint8_t *)start - (uint8_t *)binfo->ioaddr; > info->region[0].len =3D size; > - info->num_regions =3D 1; > + info->num_regions =3D AFU_MAX_REGION; >=20 > binfo->acc_info =3D info; >=20 > return ret; > } >=20 > -static int parse_feature_afus(struct build_feature_devs_info *binfo, > - struct feature_header *hdr) > -{ > - int ret; > - struct feature_afu_header *afu_hdr, header; > - u8 __iomem *start; > - u8 __iomem *end =3D binfo->ioend; > - > - start =3D (u8 __iomem *)hdr; > - for (; start < end; start +=3D header.next_afu) { > - if ((unsigned int)(end - start) < > - (unsigned int)(sizeof(*afu_hdr) + sizeof(*hdr))) > - return -EINVAL; > - > - hdr =3D (struct feature_header *)start; > - afu_hdr =3D (struct feature_afu_header *)(hdr + 1); > - header.csr =3D readq(&afu_hdr->csr); > - > - if (feature_is_UAFU(binfo)) { > - ret =3D parse_feature_port_uafu(binfo, hdr); > - if (ret) > - return ret; > - } > - > - if (!header.next_afu) > - break; > - } > - > - return 0; > -} > - > /* create and register proper private data */ static int > build_info_commit_dev(struct build_feature_devs_info *binfo) { @@ -235,1= 3 > +233,9 @@ static int build_info_commit_dev(struct build_feature_devs_info > *binfo) > struct ifpga_fme_hw *fme; > struct ifpga_feature *feature; >=20 > - if (!binfo->fiu) > - return 0; > - > if (binfo->current_type =3D=3D PORT_ID) { > - /* return error if no valid acc info data structure */ > - if (!info) > - return -EFAULT; > + if (!binfo->fiu) > + return 0; >=20 > br =3D opae_bridge_alloc(hw->adapter->name, &ifpga_br_ops, > binfo->fiu); > @@ -254,7 +248,7 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > port =3D &hw->port[binfo->current_port_id]; > feature =3D get_feature_by_id(&port->feature_list, > PORT_FEATURE_ID_UINT); > - if (feature) > + if (feature && info) > info->num_irqs =3D feature->vec_cnt; >=20 > acc =3D opae_accelerator_alloc(hw->adapter->name, > @@ -264,17 +258,21 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > return -ENOMEM; > } >=20 > + acc->adapter =3D hw->adapter; > acc->br =3D br; > if (hw->adapter->mgr) > acc->mgr =3D hw->adapter->mgr; > acc->index =3D br->id; >=20 > fme =3D &hw->fme; > - fme->nums_acc_region =3D info->num_regions; > + fme->nums_acc_region =3D info ? info->num_regions : 0; >=20 > opae_adapter_add_acc(hw->adapter, acc); >=20 > } else if (binfo->current_type =3D=3D FME_ID) { > + if (!binfo->fiu) > + return 0; > + > mgr =3D opae_manager_alloc(hw->adapter->name, > &ifpga_mgr_ops, > &ifpga_mgr_network_ops, binfo->fiu); > if (!mgr) > @@ -282,6 +280,22 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) >=20 > mgr->adapter =3D hw->adapter; > hw->adapter->mgr =3D mgr; > + } else if (binfo->current_type =3D=3D AFU_ID) { > + if (!info) > + return -EFAULT; > + > + info->num_irqs =3D 0; > + acc =3D opae_accelerator_alloc(hw->adapter->name, > + &ifpga_acc_ops, info); > + if (!acc) > + return -ENOMEM; > + > + acc->adapter =3D hw->adapter; > + acc->br =3D NULL; > + acc->mgr =3D NULL; > + acc->index =3D hw->num_afus++; > + > + opae_adapter_add_acc(hw->adapter, acc); > } >=20 > binfo->fiu =3D NULL; > @@ -295,11 +309,15 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) { > int ret; >=20 > + if ((type =3D=3D AFU_ID) && (binfo->current_type =3D=3D PORT_ID)) > + return 0; > + > ret =3D build_info_commit_dev(binfo); > if (ret) > return ret; >=20 > binfo->current_type =3D type; > + binfo->acc_info =3D NULL; >=20 > if (type =3D=3D FME_ID) { > binfo->fiu =3D &binfo->hw->fme; > @@ -311,6 +329,41 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > return 0; > } >=20 > +static int parse_feature_afus(struct build_feature_devs_info *binfo, > + struct feature_header *hdr) > +{ > + int ret; > + struct feature_afu_header *afu_hdr, header; > + u8 __iomem *start; > + u8 __iomem *end =3D binfo->ioend; > + > + ret =3D build_info_create_dev(binfo, AFU_ID, 0); > + if (ret) > + return ret; > + > + start =3D (u8 __iomem *)hdr; > + for (; start < end; start +=3D header.next_afu) { > + if ((unsigned int)(end - start) < > + (unsigned int)(sizeof(*afu_hdr) + sizeof(*hdr))) > + return -EINVAL; > + > + hdr =3D (struct feature_header *)start; > + afu_hdr =3D (struct feature_afu_header *)(hdr + 1); > + header.csr =3D readq(&afu_hdr->csr); > + > + if (feature_is_UAFU(binfo)) { > + ret =3D parse_feature_uafu(binfo, hdr); > + if (ret) > + return ret; > + } > + > + if (!header.next_afu) > + break; > + } > + > + return 0; > +} > + > static int parse_feature_fme(struct build_feature_devs_info *binfo, > struct feature_header *start) > { > @@ -405,7 +458,7 @@ static int parse_feature_fiu(struct > build_feature_devs_info *binfo, > if (ret) > return ret; > } else { > - dev_info(binfo, "No AFUs detected on Port\n"); > + dev_info(binfo, "No AFU detected on Port\n"); > } >=20 > break; > @@ -426,7 +479,7 @@ static void parse_feature_irqs(struct > build_feature_devs_info *binfo, >=20 > id =3D feature_id(start); >=20 > - if (id =3D=3D PORT_FEATURE_ID_UINT) { > + if ((binfo->current_type =3D=3D PORT_ID) && (id =3D=3D > PORT_FEATURE_ID_UINT)) > +{ > struct feature_port_uint *port_uint =3D start; > struct feature_port_uint_cap uint_cap; >=20 > @@ -437,7 +490,8 @@ static void parse_feature_irqs(struct > build_feature_devs_info *binfo, > } else { > dev_debug(binfo, "UAFU doesn't support interrupt\n"); > } > - } else if (id =3D=3D PORT_FEATURE_ID_ERROR) { > + } else if ((binfo->current_type =3D=3D PORT_ID) && > + (id =3D=3D PORT_FEATURE_ID_ERROR)) { > struct feature_port_error *port_err =3D start; > struct feature_port_err_capability port_err_cap; >=20 > @@ -449,7 +503,8 @@ static void parse_feature_irqs(struct > build_feature_devs_info *binfo, > dev_debug(&binfo, "Port error doesn't support > interrupt\n"); > } >=20 > - } else if (id =3D=3D FME_FEATURE_ID_GLOBAL_ERR) { > + } else if ((binfo->current_type =3D=3D FME_ID) && > + (id =3D=3D FME_FEATURE_ID_GLOBAL_ERR)) { > struct feature_fme_err *fme_err =3D start; > struct feature_fme_error_capability fme_err_cap; >=20 > @@ -497,9 +552,15 @@ static int parse_feature_private(struct > build_feature_devs_info *binfo, > return parse_feature_fme_private(binfo, hdr); > case PORT_ID: > return parse_feature_port_private(binfo, hdr); > + case AFU_ID: > + dev_err(binfo, "private feature %x belonging to AFU " > + "is not supported yet.\n", header.id); > + break; > default: > - dev_err(binfo, "private feature %x belonging to AFU %d > (unknown_type) is not supported yet.\n", > + dev_err(binfo, "private feature %x belonging to TYPE %d " > + "(unknown_type) is not supported yet.\n", > header.id, binfo->current_type); > + break; > } > return 0; > } > @@ -530,32 +591,57 @@ static int parse_feature(struct > build_feature_devs_info *binfo, > return ret; > } >=20 > -static int > -parse_feature_list(struct build_feature_devs_info *binfo, u8 __iomem *st= art) > +static int build_info_prepare(struct build_feature_devs_info *binfo, > + struct dfl_fpga_enum_dfl *dfl) > { > + if (!binfo || !dfl) > + return -EINVAL; > + > + binfo->ioaddr =3D dfl->addr; > + binfo->ioend =3D (u8 *)dfl->addr + dfl->len; > + binfo->phys_addr =3D dfl->start; > + > + return 0; > +} > + > +static int parse_feature_list(struct build_feature_devs_info *binfo, > + struct dfl_fpga_enum_dfl *dfl) > +{ > + u8 *start, *end; > struct feature_header *hdr, header; > - u8 __iomem *end =3D (u8 __iomem *)binfo->ioend; > int ret =3D 0; >=20 > + ret =3D build_info_prepare(binfo, dfl); > + if (ret) > + return ret; > + > + start =3D (u8 *)binfo->ioaddr; > + end =3D (u8 *)binfo->ioend; > + > + /* walk through the device feature list via DFH's next DFH pointer. */ > for (; start < end; start +=3D header.next_header_offset) { > if ((unsigned int)(end - start) < (unsigned int)sizeof(*hdr)) { > - dev_err(binfo, "The region is too small to contain a > feature.\n"); > - ret =3D -EINVAL; > + dev_err(binfo, "The region is too small to " > + "contain a feature.\n"); > + ret =3D -EINVAL; > break; > } >=20 > hdr =3D (struct feature_header *)start; > - header.csr =3D readq(hdr); > + header.csr =3D opae_readq(hdr); >=20 > - dev_debug(binfo, "%s: address=3D0x%p, val=3D0x%llx, > header.id=3D0x%x, header.next_offset=3D0x%x, header.eol=3D0x%x, > header.type=3D0x%x\n", > - __func__, hdr, (unsigned long long)header.csr, > - header.id, header.next_header_offset, > - header.end_of_list, header.type); > + dev_debug(binfo, "%s: address=3D0x%p, val=3D0x%"PRIx64", " > + "header.id=3D0x%x, header.next_offset=3D0x%x, " > + "header.eol=3D0x%x, header.type=3D0x%x\n", > + __func__, hdr, header.csr, header.id, > + header.next_header_offset, header.end_of_list, > + header.type); >=20 > ret =3D parse_feature(binfo, hdr); > if (ret) > return ret; >=20 > + /* stop parsing if EOL(End of List) is set or offset is 0 */ > if (header.end_of_list || !header.next_header_offset) > break; > } > @@ -563,82 +649,9 @@ static int parse_feature(struct build_feature_devs_i= nfo > *binfo, > return build_info_commit_dev(binfo); > } >=20 > -/* switch the memory mapping to BAR# @bar */ -static int > parse_switch_to(struct build_feature_devs_info *binfo, int bar) -{ > - struct opae_adapter_data_pci *pci_data =3D binfo->pci_data; > - > - if (!pci_data->region[bar].addr) > - return -ENOMEM; > - > - binfo->ioaddr =3D pci_data->region[bar].addr; > - binfo->ioend =3D (u8 __iomem *)binfo->ioaddr + pci_data- > >region[bar].len; > - binfo->phys_addr =3D pci_data->region[bar].phys_addr; > - binfo->current_bar =3D bar; > - > - return 0; > -} > - > -static int parse_ports_from_fme(struct build_feature_devs_info *binfo) -= { > - struct feature_fme_header *fme_hdr; > - struct feature_fme_port port; > - int i =3D 0, ret =3D 0; > - > - if (!binfo->pfme_hdr) { > - dev_info(binfo, "VF is detected.\n"); > - return ret; > - } > - > - fme_hdr =3D binfo->pfme_hdr; > - > - do { > - port.csr =3D readq(&fme_hdr->port[i]); > - if (!port.port_implemented) > - break; > - > - /* skip port which only could be accessed via VF */ > - if (port.afu_access_control =3D=3D FME_AFU_ACCESS_VF) > - continue; > - > - ret =3D parse_switch_to(binfo, port.port_bar); > - if (ret) > - break; > - > - ret =3D parse_feature_list(binfo, > - (u8 __iomem *)binfo->ioaddr + > - port.port_offset); > - if (ret) > - break; > - } while (++i < MAX_FPGA_PORT_NUM); > - > - return ret; > -} > - > -static struct build_feature_devs_info * -build_info_alloc_and_init(struc= t > ifpga_hw *hw) -{ > - struct build_feature_devs_info *binfo; > - > - binfo =3D zmalloc(sizeof(*binfo)); > - if (!binfo) > - return binfo; > - > - binfo->hw =3D hw; > - binfo->pci_data =3D hw->pci_data; > - > - /* fpga feature list starts from BAR 0 */ > - if (parse_switch_to(binfo, 0)) { > - free(binfo); > - return NULL; > - } > - > - return binfo; > -} > - > static void build_info_free(struct build_feature_devs_info *binfo) { > - free(binfo); > + opae_free(binfo); > } >=20 > static void ifpga_print_device_feature_list(struct ifpga_hw *hw) @@ -648= ,6 > +661,11 @@ static void ifpga_print_device_feature_list(struct ifpga_hw *h= w) > struct ifpga_feature *feature; > int i; >=20 > + if (fme->state =3D=3D IFPGA_FME_UNUSED) { > + dev_info(hw, "FME is not present\n"); > + return; > + } > + > dev_info(hw, "found fme_device, is in PF: %s\n", > is_ifpga_hw_pf(hw) ? "yes" : "no"); >=20 > @@ -685,40 +703,411 @@ static void ifpga_print_device_feature_list(struct > ifpga_hw *hw) > } > } >=20 > -int ifpga_bus_enumerate(struct ifpga_hw *hw) > +static struct dfl_fpga_enum_info *dfl_fpga_enum_info_alloc(struct > +ifpga_hw *hw) > { > - struct build_feature_devs_info *binfo; > + struct dfl_fpga_enum_info *info; > + > + info =3D opae_zmalloc(sizeof(*info)); > + if (!info) > + return NULL; > + > + info->hw =3D hw; > + TAILQ_INIT(&info->dfls); > + > + return info; > +} > + > +static void dfl_fpga_enum_info_free(struct dfl_fpga_enum_info *info) { > + struct dfl_fpga_enum_dfl *tmp, *dfl; > + > + if (!info) > + return; > + > + /* remove all device feature lists in the list. */ > + for (dfl =3D TAILQ_FIRST(&info->dfls); > + dfl && (tmp =3D TAILQ_NEXT(dfl, node), 1); > + dfl =3D tmp) { > + TAILQ_REMOVE(&info->dfls, dfl, node); > + opae_free(dfl); > + } > + > + opae_free(info); > +} > + > +static int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, > + u64 start, u64 len, void *addr) > +{ > + struct dfl_fpga_enum_dfl *dfl; > + > + dfl =3D opae_zmalloc(sizeof(*dfl)); > + if (!dfl) > + return -ENOMEM; > + > + dfl->start =3D start; > + dfl->len =3D len; > + dfl->addr =3D addr; > + > + TAILQ_INSERT_TAIL(&info->dfls, dfl, node); > + > + return 0; > +} > + > +#define PCI_CFG_SPACE_SIZE 256 > +#define PCI_CFG_SPACE_EXP_SIZE 4096 > +#define PCI_EXT_CAP_ID(header) (header & 0x0000ffff) > +#define PCI_EXT_CAP_NEXT(header) ((header >> 20) & 0xffc) > + > +static int > +pci_find_next_ecap(int fd, int start, u32 cap) { > + u32 header; > + int ttl =3D (PCI_CFG_SPACE_EXP_SIZE - PCI_CFG_SPACE_SIZE) / 8; > + int pos =3D PCI_CFG_SPACE_SIZE; > int ret; >=20 > - binfo =3D build_info_alloc_and_init(hw); > + if (start > 0) > + pos =3D start; > + > + ret =3D pread(fd, &header, sizeof(header), pos); > + if (ret < 0) > + return ret; > + > + /* > + * If we have no capabilities, this is indicated by cap ID, > + * cap version and next pointer all being 0. > + */ > + if (header =3D=3D 0) > + return 0; > + > + while (ttl-- > 0) { > + if ((PCI_EXT_CAP_ID(header) =3D=3D cap) && (pos !=3D start)) > + return pos; > + > + pos =3D PCI_EXT_CAP_NEXT(header); > + if (pos < PCI_CFG_SPACE_SIZE) > + break; > + ret =3D pread(fd, &header, sizeof(header), pos); > + if (ret < 0) > + return ret; > + } > + > + return 0; > +} > + > +#define PCI_EXT_CAP_ID_VNDR 0x0B > +#define PCI_VNDR_HEADER 4 > +#define PCI_VNDR_HEADER_ID(x) ((x) & 0xffff) > +#define PCI_VENDOR_ID_INTEL 0x8086 > +#define PCI_VSEC_ID_INTEL_DFLS 0x43 > +#define PCI_VNDR_DFLS_CNT 0x8 > +#define PCI_VNDR_DFLS_RES 0xc > +#define PCI_VNDR_DFLS_RES_BAR_MASK GENMASK(2, 0) #define > +PCI_VNDR_DFLS_RES_OFF_MASK GENMASK(31, 3) > + > +static int find_dfls_by_vsec(struct dfl_fpga_enum_info *info) { > + struct ifpga_hw *hw; > + struct opae_adapter_data_pci *pci_data; > + char path[64]; > + u32 bir, offset, vndr_hdr, i, dfl_cnt, dfl_res; > + int fd, ret, dfl_res_off, voff =3D 0; > + u64 start, len; > + void *addr; > + > + if (!info || !info->hw) > + return -EINVAL; > + hw =3D info->hw; > + > + if (!hw->adapter || !hw->pci_data) > + return -EINVAL; > + pci_data =3D hw->pci_data; > + > + ret =3D snprintf(path, sizeof(path), "/sys/bus/pci/devices/%s/config", > + hw->adapter->name); > + if ((unsigned int)ret >=3D sizeof(path)) > + return -EINVAL; > + > + fd =3D open(path, O_RDWR); > + if (fd < 0) > + return -EIO; > + > + while ((voff =3D pci_find_next_ecap(fd, voff, > + PCI_EXT_CAP_ID_VNDR))) { > + vndr_hdr =3D 0; > + ret =3D pread(fd, &vndr_hdr, sizeof(vndr_hdr), > + voff + PCI_VNDR_HEADER); > + if (ret < 0) > + return -EIO; > + if (PCI_VNDR_HEADER_ID(vndr_hdr) =3D=3D > PCI_VSEC_ID_INTEL_DFLS && > + pci_data->vendor_id =3D=3D PCI_VENDOR_ID_INTEL) > + break; > + } > + > + if (!voff) { > + dev_debug(hw, "%s no DFL VSEC found\n", __func__); > + return -ENODEV; > + } > + > + dfl_cnt =3D 0; > + ret =3D pread(fd, &dfl_cnt, sizeof(dfl_cnt), voff + PCI_VNDR_DFLS_CNT); > + if (ret < 0) > + return -EIO; > + > + dfl_res_off =3D voff + PCI_VNDR_DFLS_RES; > + if (dfl_res_off + (dfl_cnt * sizeof(u32)) > PCI_CFG_SPACE_EXP_SIZE) { > + dev_err(hw, "%s DFL VSEC too big for PCIe config space\n", > + __func__); > + return -EINVAL; > + } > + > + for (i =3D 0; i < dfl_cnt; i++, dfl_res_off +=3D sizeof(u32)) { > + dfl_res =3D GENMASK(31, 0); > + ret =3D pread(fd, &dfl_res, sizeof(dfl_res), dfl_res_off); > + bir =3D dfl_res & PCI_VNDR_DFLS_RES_BAR_MASK; > + if (bir >=3D PCI_MAX_RESOURCE) { > + dev_err(hw, "%s bad bir number %d\n", > + __func__, bir); > + return -EINVAL; > + } > + > + len =3D pci_data->region[bir].len; > + offset =3D dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK; > + if (offset >=3D len) { > + dev_err(hw, "%s bad offset %u >=3D %"PRIu64"\n", > + __func__, offset, len); > + return -EINVAL; > + } > + > + dev_debug(hw, "%s BAR %d offset 0x%x\n", __func__, bir, > offset); > + len -=3D offset; > + start =3D pci_data->region[bir].phys_addr + offset; > + addr =3D pci_data->region[bir].addr + offset; > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } > + > + return 0; > +} > + > +/* default method of finding dfls starting at offset 0 of bar 0 */ > +static int find_dfls_by_default(struct dfl_fpga_enum_info *info) { > + struct ifpga_hw *hw; > + struct opae_adapter_data_pci *pci_data; > + int port_num, bar, i, ret =3D 0; > + u64 start, len; > + void *addr; > + u32 offset; > + struct feature_header hdr; > + struct feature_fme_capability cap; > + struct feature_fme_port port; > + struct feature_fme_header *fme_hdr; > + > + if (!info || !info->hw) > + return -EINVAL; > + hw =3D info->hw; > + > + if (!hw->pci_data) > + return -EINVAL; > + pci_data =3D hw->pci_data; > + > + /* start to find Device Feature List from Bar 0 */ > + addr =3D pci_data->region[0].addr; > + if (!addr) > + return -ENOMEM; > + > + /* > + * PF device has FME and Ports/AFUs, and VF device only has one > + * Port/AFU. Check them and add related "Device Feature List" info > + * for the next step enumeration. > + */ > + hdr.csr =3D opae_readq(addr); > + if ((hdr.type =3D=3D FEATURE_TYPE_FIU) && (hdr.id =3D=3D > FEATURE_FIU_ID_FME)) { > + start =3D pci_data->region[0].phys_addr; > + len =3D pci_data->region[0].len; > + addr =3D pci_data->region[0].addr; > + > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + > + /* > + * find more Device Feature Lists (e.g. Ports) per information > + * indicated by FME module. > + */ > + fme_hdr =3D (struct feature_fme_header *)addr; > + cap.csr =3D opae_readq(&fme_hdr->capability); > + port_num =3D (int)cap.num_ports; > + > + dev_info(hw, "port_num =3D %d\n", port_num); > + if (port_num > MAX_FPGA_PORT_NUM) > + port_num =3D MAX_FPGA_PORT_NUM; > + > + for (i =3D 0; i < port_num; i++) { > + port.csr =3D opae_readq(&fme_hdr->port[i]); > + > + /* skip ports which are not implemented. */ > + if (!port.port_implemented) > + continue; > + > + /* skip port which only could be accessed via VF */ > + if (port.afu_access_control =3D=3D FME_AFU_ACCESS_VF) > + continue; > + > + /* > + * add Port's Device Feature List information for next > + * step enumeration. > + */ > + bar =3D (int)port.port_bar; > + offset =3D port.port_offset; > + if (bar =3D=3D FME_PORT_OFST_BAR_SKIP) { > + continue; > + } else if (bar >=3D PCI_MAX_RESOURCE) { > + dev_err(hw, "bad BAR %d for port %d\n", bar, > i); > + ret =3D -EINVAL; > + break; > + } else { > + dev_info(hw, "BAR %d offset %u\n", bar, > offset); > + } > + > + len =3D pci_data->region[bar].len; > + if (offset >=3D len) { > + dev_warn(hw, "bad port offset %u >=3D %pa\n", > + offset, &len); > + continue; > + } > + > + len -=3D offset; > + start =3D pci_data->region[bar].phys_addr + offset; > + addr =3D pci_data->region[bar].addr + offset; > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } > + } else if ((hdr.type =3D=3D FEATURE_TYPE_FIU) && > + (hdr.id =3D=3D FEATURE_FIU_ID_PORT)) { > + start =3D pci_data->region[0].phys_addr; > + len =3D pci_data->region[0].len; > + addr =3D pci_data->region[0].addr; > + > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } else if (hdr.type =3D=3D FEATURE_TYPE_AFU) { > + start =3D pci_data->region[0].phys_addr; > + len =3D pci_data->region[0].len; > + addr =3D pci_data->region[0].addr; > + > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } else { > + dev_info(hw, "Unknown feature type 0x%x id 0x%x\n", > + hdr.type, hdr.id); > + ret =3D -ENODEV; > + } > + > + return ret; > +} > + > +static int dfl_fpga_feature_devs_enumerate(struct dfl_fpga_enum_info > +*info) { > + struct build_feature_devs_info *binfo; > + struct dfl_fpga_enum_dfl *dfl; > + int ret =3D 0; > + > + if (!info || !info->hw) > + return -EINVAL; > + > + /* create and init build info for enumeration */ > + binfo =3D opae_zmalloc(sizeof(*binfo)); > if (!binfo) > return -ENOMEM; >=20 > - ret =3D parse_feature_list(binfo, binfo->ioaddr); > + binfo->hw =3D info->hw; > + binfo->pci_data =3D info->hw->pci_data; > + > + /* > + * start enumeration for all feature devices based on Device Feature > + * Lists. > + */ > + TAILQ_FOREACH(dfl, &info->dfls, node) { > + ret =3D parse_feature_list(binfo, dfl); > + if (ret) > + break; > + } > + > + build_info_free(binfo); > + > + return ret; > +} > + > +int ifpga_bus_enumerate(struct ifpga_hw *hw) { > + struct dfl_fpga_enum_info *info; > + int ret; > + > + /* allocate enumeration info */ > + info =3D dfl_fpga_enum_info_alloc(hw); > + if (!info) > + return -ENOMEM; > + > + ret =3D find_dfls_by_vsec(info); > + if (ret < 0) > + ret =3D find_dfls_by_default(info); > + > if (ret) > goto exit; >=20 > - ret =3D parse_ports_from_fme(binfo); > - if (ret) > + /* start enumeration with prepared enumeration information */ > + ret =3D dfl_fpga_feature_devs_enumerate(info); > + if (ret < 0) { > + dev_err(hw, "Enumeration failure\n"); > goto exit; > + } >=20 > ifpga_print_device_feature_list(hw); >=20 > exit: > - build_info_free(binfo); > + dfl_fpga_enum_info_free(info); > + > return ret; > } >=20 > -int ifpga_bus_init(struct ifpga_hw *hw) > +static void ifpga_print_acc_list(struct opae_adapter *adapter) > { > + struct opae_accelerator *acc; > + struct ifpga_afu_info *info; > + struct uuid guid; > + char buf[48]; > int i; > + > + opae_adapter_for_each_acc(adapter, acc) { > + info =3D acc->data; > + if (!info) > + continue; > + acc->ops->get_uuid(acc, &guid); > + i =3D sprintf(buf, "%02x%02x%02x%02x-", > + guid.b[15], guid.b[14], guid.b[13], guid.b[12]); > + i +=3D sprintf(buf+i, "%02x%02x-", guid.b[11], guid.b[10]); > + i +=3D sprintf(buf+i, "%02x%02x-", guid.b[9], guid.b[8]); > + i +=3D sprintf(buf+i, "%02x%02x-", guid.b[7], guid.b[6]); > + sprintf(buf+i, "%02x%02x%02x%02x%02x%02x", > + guid.b[5], guid.b[4], guid.b[3], > + guid.b[2], guid.b[1], guid.b[0]); > + dev_info(hw, "AFU(%s-%d)@%p: len:0x%"PRIx64", guid:%s\n", > + acc->name, acc->index, info->region[0].addr, > + info->region[0].len, buf); > + } > +} > + > +int ifpga_bus_init(struct ifpga_hw *hw) { > + int i, ret =3D 0; > struct ifpga_port_hw *port; >=20 > - fme_hw_init(&hw->fme); > + ret =3D fme_hw_init(&hw->fme); > + if (ret) > + return ret; > + > for (i =3D 0; i < MAX_FPGA_PORT_NUM; i++) { > port =3D &hw->port[i]; > port_hw_init(port); > } > + ifpga_print_acc_list(hw->adapter); >=20 > return 0; > } > diff --git a/drivers/raw/ifpga/base/ifpga_enumerate.h > b/drivers/raw/ifpga/base/ifpga_enumerate.h > index 95ed594..e6b04f0 100644 > --- a/drivers/raw/ifpga/base/ifpga_enumerate.h > +++ b/drivers/raw/ifpga/base/ifpga_enumerate.h > @@ -5,6 +5,8 @@ > #ifndef _IFPGA_ENUMERATE_H_ > #define _IFPGA_ENUMERATE_H_ >=20 > +#define FME_PORT_OFST_BAR_SKIP 7 > + > int ifpga_bus_init(struct ifpga_hw *hw); int ifpga_bus_uinit(struct ifp= ga_hw > *hw); int ifpga_bus_enumerate(struct ifpga_hw *hw); diff --git > a/drivers/raw/ifpga/base/ifpga_feature_dev.c > b/drivers/raw/ifpga/base/ifpga_feature_dev.c > index 0813513..dbecc7b 100644 > --- a/drivers/raw/ifpga/base/ifpga_feature_dev.c > +++ b/drivers/raw/ifpga/base/ifpga_feature_dev.c > @@ -59,7 +59,7 @@ int __fpga_port_disable(struct ifpga_port_hw *port) > if (fpga_wait_register_field(port_sftrst_ack, control, > &port_hdr->control, RST_POLL_TIMEOUT, > RST_POLL_INVL)) { > - dev_err(port, "timeout, fail to reset device\n"); > + dev_err(port, "timeout, fail to reset FIM port\n"); > return -ETIMEDOUT; > } >=20 > @@ -277,10 +277,11 @@ static void feature_uinit(struct ifpga_feature_list > *list) > struct ifpga_feature *feature; >=20 > TAILQ_FOREACH(feature, list, next) { > - if (feature->state !=3D IFPGA_FEATURE_ATTACHED) > + if (feature->state !=3D IFPGA_FEATURE_INITED) > continue; > if (feature->ops && feature->ops->uinit) > feature->ops->uinit(feature); > + feature->state =3D IFPGA_FEATURE_ATTACHED; > } > } >=20 > @@ -301,6 +302,9 @@ static int feature_init(struct feature_driver *drv, > ret =3D feature->ops->init(feature); > if (ret) > goto error; > + else > + feature->state =3D > + > IFPGA_FEATURE_INITED; > } > } > } > @@ -315,14 +319,8 @@ static int feature_init(struct feature_driver *drv, >=20 > int fme_hw_init(struct ifpga_fme_hw *fme) { > - int ret; > - > - if (fme->state !=3D IFPGA_FME_IMPLEMENTED) > - return -ENODEV; > - > - ret =3D feature_init(fme_feature_drvs, &fme->feature_list); > - if (ret) > - return ret; > + if (fme->state =3D=3D IFPGA_FME_IMPLEMENTED) > + return feature_init(fme_feature_drvs, &fme->feature_list); >=20 > return 0; > } > diff --git a/drivers/raw/ifpga/base/ifpga_hw.h > b/drivers/raw/ifpga/base/ifpga_hw.h > index ed5edc6..4d56deb 100644 > --- a/drivers/raw/ifpga/base/ifpga_hw.h > +++ b/drivers/raw/ifpga/base/ifpga_hw.h > @@ -15,6 +15,7 @@ > enum ifpga_feature_state { > IFPGA_FEATURE_UNUSED =3D 0, > IFPGA_FEATURE_ATTACHED, > + IFPGA_FEATURE_INITED > }; >=20 > enum feature_type { > @@ -134,6 +135,7 @@ struct ifpga_hw { >=20 > struct ifpga_fme_hw fme; > struct ifpga_port_hw port[MAX_FPGA_PORT_NUM]; > + int num_afus; > }; >=20 > static inline bool is_ifpga_hw_pf(struct ifpga_hw *hw) diff --git > a/drivers/raw/ifpga/base/opae_hw_api.c > b/drivers/raw/ifpga/base/opae_hw_api.c > index 11c9887..87256fc 100644 > --- a/drivers/raw/ifpga/base/opae_hw_api.c > +++ b/drivers/raw/ifpga/base/opae_hw_api.c > @@ -177,7 +177,7 @@ int opae_acc_get_region_info(struct opae_accelerator > *acc, int opae_acc_set_irq(struct opae_accelerator *acc, > u32 start, u32 count, s32 evtfds[]) { > - if (!acc || !acc->data) > + if (!acc) > return -EINVAL; >=20 > if (start + count <=3D start) > diff --git a/drivers/raw/ifpga/base/opae_hw_api.h > b/drivers/raw/ifpga/base/opae_hw_api.h > index 7e04b56..fd40e09 100644 > --- a/drivers/raw/ifpga/base/opae_hw_api.h > +++ b/drivers/raw/ifpga/base/opae_hw_api.h > @@ -143,6 +143,7 @@ struct opae_accelerator { > TAILQ_ENTRY(opae_accelerator) node; > const char *name; > int index; > + struct opae_adapter *adapter; > struct opae_bridge *br; > struct opae_manager *mgr; > struct opae_accelerator_ops *ops; > @@ -240,6 +241,7 @@ struct opae_adapter_data { >=20 > struct opae_reg_region { > u64 phys_addr; > +#define AFU_REGION_SIZE 0x8000 > u64 len; > u8 *addr; > }; > diff --git a/drivers/raw/ifpga/ifpga_rawdev.c > b/drivers/raw/ifpga/ifpga_rawdev.c > index 94df56c..ceb18ae 100644 > --- a/drivers/raw/ifpga/ifpga_rawdev.c > +++ b/drivers/raw/ifpga/ifpga_rawdev.c > @@ -47,11 +47,13 @@ > #define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0 > #define PCIE_DEVICE_ID_PF_DSC_1_X 0x09C4 > #define PCIE_DEVICE_ID_PAC_N3000 0x0B30 > +#define PCIE_DEVICE_ID_PAC_N6000 0xBCCE > /* VF Device */ > #define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF > #define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1 > #define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5 > #define PCIE_DEVICE_ID_VF_PAC_N3000 0x0B31 > +#define PCIE_DEVICE_ID_VF_PAC_N6000 0xBCCF > #define RTE_MAX_RAW_DEVICE 10 >=20 > static const struct rte_pci_id pci_ifpga_map[] =3D { @@ -63,6 +65,8 @@ > { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_DSC_1_X) }, > { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PAC_N3000),}, > { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_PAC_N3000),}, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PAC_N6000),}, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_PAC_N6000),}, > { .vendor_id =3D 0, /* sentinel */ }, > }; >=20 > @@ -110,6 +114,7 @@ struct ifpga_rawdev * >=20 > return IFPGA_RAWDEV_NUM; > } > + > static struct ifpga_rawdev * > ifpga_rawdev_allocate(struct rte_rawdev *rawdev) { @@ -365,7 +370,7 @@ > static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev) > return -ENODEV; >=20 > mgr =3D opae_adapter_get_mgr(adapter); > - if (!mgr) > + if (!mgr || !mgr->sensor_list) > return -ENODEV; >=20 > opae_mgr_for_each_sensor(mgr, sensor) { @@ -377,7 +382,7 @@ > static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev) > goto fail; >=20 > if (value =3D=3D 0xdeadbeef) { > - IFPGA_RAWDEV_PMD_ERR("dev_id %d sensor %s value > %x\n", > + IFPGA_RAWDEV_PMD_DEBUG("dev_id %d sensor %s > value %x\n", > raw_dev->dev_id, sensor->name, > value); > continue; > } > @@ -893,7 +898,7 @@ static int set_surprise_link_check_aer( { > struct opae_adapter *adapter; > struct opae_manager *mgr; > - struct opae_board_info *info; > + struct opae_board_info *info =3D NULL; > struct rte_afu_pr_conf *afu_pr_conf; > int ret; > struct uuid uuid; > @@ -921,17 +926,14 @@ static int set_surprise_link_check_aer( > } >=20 > mgr =3D opae_adapter_get_mgr(adapter); > - if (!mgr) { > - IFPGA_RAWDEV_PMD_ERR("opae_manager of opae_adapter is > NULL"); > - return -1; > - } > - > - if (ifpga_mgr_ops.get_board_info(mgr, &info)) { > - IFPGA_RAWDEV_PMD_ERR("ifpga manager get_board_info > fail!"); > - return -1; > + if (mgr) { > + if (ifpga_mgr_ops.get_board_info(mgr, &info)) { > + IFPGA_RAWDEV_PMD_ERR("ifpga manager > get_board_info fail!"); > + return -1; > + } > } >=20 > - if (info->lightweight) { > + if (info && info->lightweight) { > /* set uuid to all 0, when fpga is lightweight image */ > memset(&afu_pr_conf->afu_id.uuid.uuid_low, 0, sizeof(u64)); > memset(&afu_pr_conf->afu_id.uuid.uuid_high, 0, sizeof(u64)); > @@ -953,7 +955,7 @@ static int set_surprise_link_check_aer( > __func__, > (unsigned long)afu_pr_conf->afu_id.uuid.uuid_low, > (unsigned long)afu_pr_conf->afu_id.uuid.uuid_high); > - } > + } > return 0; > } >=20 > @@ -1592,7 +1594,7 @@ static int fme_clean_fme_error(struct opae_manager > *mgr) > ret =3D opae_adapter_init(adapter, pci_dev->device.name, data); > if (ret) { > ret =3D -ENOMEM; > - goto free_adapter_data; > + goto cleanup; > } >=20 > rawdev->dev_ops =3D &ifpga_rawdev_ops; > @@ -1602,29 +1604,23 @@ static int fme_clean_fme_error(struct > opae_manager *mgr) > /* must enumerate the adapter before use it */ > ret =3D opae_adapter_enumerate(adapter); > if (ret) > - goto free_adapter_data; > + goto cleanup; >=20 > /* get opae_manager to rawdev */ > mgr =3D opae_adapter_get_mgr(adapter); > if (mgr) { > - /* PF function */ > - IFPGA_RAWDEV_PMD_INFO("this is a PF function"); > + ret =3D ifpga_register_msix_irq(dev, 0, IFPGA_FME_IRQ, 0, 0, > + fme_interrupt_handler, "fme_irq", mgr); > + if (ret) > + goto cleanup; > } >=20 > - ret =3D ifpga_register_msix_irq(dev, 0, IFPGA_FME_IRQ, 0, 0, > - fme_interrupt_handler, "fme_irq", mgr); > - if (ret) > - goto free_adapter_data; > - > ret =3D ifpga_monitor_start_func(dev); > if (ret) > - goto free_adapter_data; > + goto cleanup; >=20 > return ret; >=20 > -free_adapter_data: > - if (data) > - opae_adapter_data_free(data); > cleanup: > if (rawdev) > rte_rawdev_pmd_release(rawdev); > -- > 1.8.3.1 It looks good for me, you can add: Acked-by: Tianfei Zhang