From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0DC2EA0543 for ; Tue, 7 Jun 2022 08:04:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 037B740156; Tue, 7 Jun 2022 08:04:36 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 7D9A040156; Tue, 7 Jun 2022 08:04:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1654581873; x=1686117873; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=giqmZ4RklfPXUUnN2SlgrXcLHapLwpujagAJg6p4xYs=; b=NFiAAq2Ab0RBW0FOonJ326C/GJ2+XRHlvvIkMj8CSOoZ5CgLryE8Kp5+ 3AAOYzpkH7+Rz31dCatRwxfHdsara2iR4VFjJ/f3YtZ69hN7DCccOxsL+ E570msCmMDfDp4wQE2QByI9eCo1jhNDUK/kMcWcqUCnckqIPlYA9Yd++P JHRaiT+YzocAe4GuhSwSQmZOX0YSVnQ4b0JiO6riDu527JiOXRvCakE4e gCTNE8e8wscmSfWkLC5CzEWgmKZBYmCxT/Gk2VLWdkecqSXNh70Dx0jVO YA0W/EWGNMf7ILCX0scJGlOSPgwkVpBnNs9WgmI2zbRIZYGKncrwpPlOn w==; X-IronPort-AV: E=McAfee;i="6400,9594,10370"; a="277025636" X-IronPort-AV: E=Sophos;i="5.91,282,1647327600"; d="scan'208";a="277025636" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jun 2022 23:04:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,282,1647327600"; d="scan'208";a="647884429" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by fmsmga004.fm.intel.com with ESMTP; 06 Jun 2022 23:04:11 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Mon, 6 Jun 2022 23:04:11 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Mon, 6 Jun 2022 23:04:11 -0700 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.48) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Mon, 6 Jun 2022 23:04:10 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eaqCu7eKbgIx2vn8s6MIm2o5j7vpv2oFdJeVKebg3phWmGru9EsyYfNRmtx1A+bdpq/HRNyiFPqHzhZJ/EGIAOsYSoozDWPJEHB7laW1w1FOP/yJ9tObxI9j+EN9zjzx9coZwPWYUkNvbcSqZtkvR9bmXPbxml5RyG3Qo8BCXTLeZu4KwDN/eQMjBdyv2KLawnzHPhs95e0T5PV2L9/Ubp6AA0ODXB1Jnckf5ED6KTbLQvZ0bHry0eLBfZ3kRnCeL9xF87xOz9MjTe9AZOSa9ux4ZayPPZ2HC/YbifoJQHHseR8lonbTQxBBrU+D/ouB/GNRaS0coQQ9u2oXHEqsBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UnU4ISXxgcrTffFdmi97QCe4oD8H8KCubr2Xne9dvmU=; b=Lz+aujG8vHFhi0E6i03ahXLItqNltIczhqRYBtuhKcX0dPHrOD9gw9+B2TgDrNFKwtT8xDlPaoWB7/6dbD8VpiD8inMUny2yGXBVFs6DFie9nwReU8i0cBphCBA2V9flhZ1jwEydnEg//vFwee9G8OFR+sX2iXErTvlqgg9qbEnKCPifZBEQ1qKJ0YL7PN9+6kgPR+daeX1RwD8r0bDNNcNGFcaGAYEtF5buMNNYIuVjLE1iNklYAuvTvt594XmGgt3sxCYMB3AVSEUfXJGM4QzzqeZa0Ulcb/P1gxrikGl+OZVQfHEpR3zzlIBh1UZL2aMJDqLRs7uyat/zxl7fkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM6PR11MB4252.namprd11.prod.outlook.com (2603:10b6:5:201::26) by BYAPR11MB3685.namprd11.prod.outlook.com (2603:10b6:a03:fa::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.15; Tue, 7 Jun 2022 06:04:08 +0000 Received: from DM6PR11MB4252.namprd11.prod.outlook.com ([fe80::b9d5:600c:9f13:bc96]) by DM6PR11MB4252.namprd11.prod.outlook.com ([fe80::b9d5:600c:9f13:bc96%7]) with mapi id 15.20.5314.019; Tue, 7 Jun 2022 06:04:08 +0000 From: "Xu, Rosen" To: "Huang, Wei" , "dev@dpdk.org" , "thomas@monjalon.net" , "nipun.gupta@nxp.com" , "hemant.agrawal@nxp.com" CC: "stable@dpdk.org" , "Zhang, Tianfei" , "Zhang, Qi Z" Subject: RE: [PATCH v4 4/5] raw/ifpga: support ofs card probe Thread-Topic: [PATCH v4 4/5] raw/ifpga: support ofs card probe Thread-Index: AQHYcaOMJZncHcgMVk2mGB0TfAJmGK1DhWwQ Date: Tue, 7 Jun 2022 06:04:07 +0000 Message-ID: References: <1653535974-1379-1-git-send-email-wei.huang@intel.com> <1653640433-5066-1-git-send-email-wei.huang@intel.com> <1653640433-5066-5-git-send-email-wei.huang@intel.com> In-Reply-To: <1653640433-5066-5-git-send-email-wei.huang@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNzc0ZDZkOTEtOGUzYy00Y2MzLTgwYzgtMzQyMWM4Yzg4YWFhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiTTVjemVHT2FOMzk4dGFyRUVMdytTMVA1bkRudkd0ZWlPeVZiMXhCdlRndmc3aEFuYzZ1VHRDTWt6b092ZU5JeSJ9 x-ctpclassification: CTP_NT authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c1d2828d-a163-4e50-9457-08da484b88b3 x-ms-traffictypediagnostic: BYAPR11MB3685:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3DWYREXq9QUKmxWOGpGcKKzAvCS0KNINvwU0raq9hEKCWgkByYyPPXmLn6nMr3kcKV221om/84bAdgX2+HNtMTqbNniTJTguA4jUrJzPWjBL489klGvZqNDpgzFkk9qDYJMuZV5mZlwl1Bdc1TQuINQeAMRTCX8m4400PCAw5S9Vr/Ay8tPqcxA/WEsaRcwGf+9RwCSIt1m79Jvnkx3EHjlF7ZQrK1jZ/GIZvEeMcHJWMqO1US8HrNNrnlhIiqG5Od9fwdFB89kJ2XbMqz5Nplg5b4+RKSSpRHTxUHjPecLDy581Wj85IlTJGl1S2yVtmS5kvuHzye/MnQKb8ZFc6WeBLZc3iJVho5rGB8F2bK2k+dbRRNGnws1290HXbYqmGjiheBF0jzaGDegcgRYGZZvI9WkFqTXxF64SNBToD7RijiOmkxi4GZTJpJFueAFmxcVJS/rW4u+HXyG1z++x8ESUIqCtHV6wXKoCEgPe2QXLlGU6QVblSua2hwqGMIQM9Rue3sHJrMvF+zUsqtUBMASqFVPiAanQLpWc4SZWtSaIcKATH71sbMKG0AQojetDRP2iBg6rHoEVIPwf6bLqMoSry32UD/3qw9UdOsvE2K80FR/coPxKE07cKmtPlituU3SfddIkH1XNE0MBxQ0i4t7BtILvFYbnfrCNcTmg4q/q3z8jE+1iUSAQl5fpwBJ9651UDTeTg6mgadXLONBx1w== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4252.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(8676002)(4326008)(76116006)(5660300002)(52536014)(53546011)(66946007)(66556008)(64756008)(66446008)(66476007)(71200400001)(30864003)(38070700005)(508600001)(83380400001)(8936002)(7696005)(82960400001)(26005)(86362001)(9686003)(122000001)(6506007)(186003)(107886003)(110136005)(316002)(54906003)(55016003)(38100700002)(33656002)(2906002)(579004)(559001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?lVX2uJHEs6629LE5QhbXNipweMr9F2B0KMlJgMupHrtfB/Ka9oQXNZ9s1/Qt?= =?us-ascii?Q?LQJDGUULX8pGJGR8RXmDa3eMrWWu9Y9JD40pMQkfD1TYPyp60cYj1oK6cFW5?= =?us-ascii?Q?3aL91VvGs6TIzAYLVXzkY5hitqLcWkLfhTEEWIL9S62xpatxVHC9VIxBiUMX?= =?us-ascii?Q?022lR1E8aXC+CmfQR+/z+CzOAuQHrxgoc58TGZV7miSryycSc7C+zxjkd3rv?= =?us-ascii?Q?s1pBlKJjiwnkpfCqEn+pzuAmnIUDBLtMN+dnq0s3OrKKGgz+PK8DDgK3uc8s?= =?us-ascii?Q?hhZFTP5pooYTLtgrb97TEchzgecunVtv/zp9P7xYM4b04ZK2XZccfHBAwYCD?= =?us-ascii?Q?LLSdp0ma/bFzHDNEmEg9inXq3I9GZ3tuuaNePU6n2TDV1btjl3nnpWwggeuJ?= =?us-ascii?Q?++NOsgqLA2lj6ssIis1ygfyZoxQVfwu+MKjbir1s9cL29e2k5Sm9yT/MUrq8?= =?us-ascii?Q?MeQoe3zK8AGu32Y52a116h+JGZjNG6ITpaXdzCixHkql+B14nLCejNJKH592?= =?us-ascii?Q?06YBxcopAFFzcmkZnRxuddwCBMsprP0tE3EqlfZ4xCNg1YdcwCkZ1PZAP2E5?= =?us-ascii?Q?woSyAqopKgwI/FyQ2kS43bnhxPgc3STRGczjaqZewwt78lWhvTt+3KAzAqO+?= =?us-ascii?Q?RfuEzCXv6wYhzkKWNqTjL1dUXuXlX5Nvyf2IpJdnijqPzB2kv2hbIR5g2mYx?= =?us-ascii?Q?/od0/alIO/KKIt60iSiNtQyZYCKTzQWPfs97DKnAr+1GGB+6DEHsWTphhI8U?= =?us-ascii?Q?sQShB4E/mWdQj5Z+kuNRTHTDCGSujrmF5/LF9jBrPcvJIRp0Pn3RVqa8rCz9?= =?us-ascii?Q?9B35qWyMx0ruLAUoI02hZXWYOVnVlWEha1wW3MWkK/0htNX3rI37tTl3il8x?= =?us-ascii?Q?q+nuvke3aemPi3Mu+Ymw6cFYTL83F8Dt5WhDf6YSUEA2AN1N+v4ZGH+keEig?= =?us-ascii?Q?dXuoXNBHKAMvDmPLQR708Ttuga9EQqJNmouw2/Obm0QBXf2r6ohmmj4kVlYX?= =?us-ascii?Q?5fph0lYMZcsE5qKn+qEzWID2kZSncVNS44s2KPwN56uT05yJ15KVJKMvJjdG?= =?us-ascii?Q?IuD/fCK+hxyUSUbiupTIq9A37IaR86uIIq69Hf0AsuzeEq6IQXHrPWr3DUBC?= =?us-ascii?Q?XY1tzrLHDolrzCrtTXMsFWVNVQ0/BCqVbMRlvQmJgViQKT6arDYuYEdmVoz8?= =?us-ascii?Q?ExLllQHdQJ2GAOVH2kgxVq0qnX/V9Gpp8S4gayTmTORp/3vID1tsESdj7zzI?= =?us-ascii?Q?TZCv3tFoo/xSBQJdcPP3tKYndYV4SIvzEmO/uoQvtWZszCVv19RyVO4RTpXZ?= =?us-ascii?Q?pooqrR7G6YU81ZhibOErJLXq09rblY4parDDWqxAhko67pD00cFwSjA+FK0V?= =?us-ascii?Q?GC+NiSTI7Mqc7Y93GEp7AHlm75a9qVTfMY205zloKuwJ0pUOY1bVKMSmWQRA?= =?us-ascii?Q?aSXa06f1kPgB3U5FdG9YzybH/xZ3VU1MwvLbzmwpxLZUgBbgHMbv/dSVCH4+?= =?us-ascii?Q?jfuhRkR/BFzB0o1x7Acjb8jDQLThhdguYN3YoS7JFSk2xjppG8g7f6HGyya/?= =?us-ascii?Q?uOopx7iCfPuibxriUt14eH4DKSYocwbNIjBzZshzcbzpYd/Eap8J4CSOhe9j?= =?us-ascii?Q?arjGL+jocnc+yebg/NZKbEgnU6fagZXsf33t1PrOBFsdkjOn0yrv9bSc01n2?= =?us-ascii?Q?SixsQtWL2ZSwPB5tIfNXJmx4rnKQmvXT3YmVB7yd/WdkwDECkF1IJIbUQH+/?= =?us-ascii?Q?j9LjwA2lMA=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4252.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c1d2828d-a163-4e50-9457-08da484b88b3 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jun 2022 06:04:07.9407 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: N6SisDxmbFhFgxUM1tcDOPxirH7KpRZA71lbRJpqBIJ1R6jOeFOEoo9W7c6fu53G9hYvlxoEvMU1KgeFiBkIvA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3685 X-OriginatorOrg: intel.com X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, > -----Original Message----- > From: Huang, Wei > Sent: Friday, May 27, 2022 16:34 > To: dev@dpdk.org; thomas@monjalon.net; nipun.gupta@nxp.com; > hemant.agrawal@nxp.com > Cc: stable@dpdk.org; Xu, Rosen ; Zhang, Tianfei > ; Zhang, Qi Z ; Huang, Wei > > Subject: [PATCH v4 4/5] raw/ifpga: support ofs card probe >=20 > PAC N6000 is the first OFS platform, its device id is added to ifpga devi= ce > support list. >=20 > Previous FPGA platform like Intel PAC N3000 and N5000, FME DFL (Device > Feature List) starts from BAR0 by default, port DFL location is indicated= in > PORTn_OFFSET register in FME. In OFS implementation, FME DFL and port > DFL location can be defined individually in PCIe VSEC (Vendor Specific > Extended Capabilities). In this patch, DFL definition is searched in VSEC= , the > legacy DFL is used only when DFL VSEC is not present. >=20 > In original DFL enumeration process, AFU is expected to locate in port DF= L, > but this is not the case in OFS implementation. In this patch, enumeratio= n can > search AFU in any PF/VF which has no FME and port. >=20 > Signed-off-by: Wei Huang > Acked-by: Tianfei Zhang > --- > v2: fix build error in UB2004-32 > --- > v3: update commit log with Tianfei's comment, treat 7 as special BAR inde= x > --- > v4: fix typo and coding style issue > --- > drivers/raw/ifpga/base/ifpga_api.c | 28 +- > drivers/raw/ifpga/base/ifpga_defines.h | 1 + > drivers/raw/ifpga/base/ifpga_enumerate.c | 680 > ++++++++++++++++++++++------- > drivers/raw/ifpga/base/ifpga_enumerate.h | 2 + > drivers/raw/ifpga/base/ifpga_feature_dev.c | 18 +- > drivers/raw/ifpga/base/ifpga_hw.h | 2 + > drivers/raw/ifpga/base/opae_hw_api.c | 2 +- > drivers/raw/ifpga/base/opae_hw_api.h | 2 + > drivers/raw/ifpga/ifpga_rawdev.c | 48 +- > 9 files changed, 591 insertions(+), 192 deletions(-) >=20 > diff --git a/drivers/raw/ifpga/base/ifpga_api.c > b/drivers/raw/ifpga/base/ifpga_api.c > index 4610ef1..f19cc26 100644 > --- a/drivers/raw/ifpga/base/ifpga_api.c > +++ b/drivers/raw/ifpga/base/ifpga_api.c > @@ -13,15 +13,22 @@ > static int ifpga_acc_get_uuid(struct opae_accelerator *acc, > struct uuid *uuid) > { > - struct opae_bridge *br =3D acc->br; > - struct ifpga_port_hw *port; > + struct ifpga_afu_info *afu_info =3D acc->data; > + struct opae_reg_region *region; > + u64 val =3D 0; >=20 > - if (!br || !br->data) > - return -EINVAL; > + if (!afu_info) > + return -ENODEV; >=20 > - port =3D br->data; > + region =3D &afu_info->region[0]; > + if (uuid) { > + val =3D readq(region->addr + sizeof(struct feature_header)); > + opae_memcpy(uuid->b, &val, sizeof(u64)); > + val =3D readq(region->addr + sizeof(struct feature_header) + 8); > + opae_memcpy(uuid->b + 8, &val, sizeof(u64)); > + } >=20 > - return fpga_get_afu_uuid(port, uuid); > + return 0; > } >=20 > static int ifpga_acc_set_irq(struct opae_accelerator *acc, @@ -32,6 +39,= 9 > @@ static int ifpga_acc_set_irq(struct opae_accelerator *acc, > struct ifpga_port_hw *port; > struct fpga_uafu_irq_set irq_set; >=20 > + if (!afu_info) > + return -ENODEV; > + > if (!br || !br->data) > return -EINVAL; >=20 > @@ -68,7 +78,7 @@ static int ifpga_acc_get_region_info(struct > opae_accelerator *acc, > struct ifpga_afu_info *afu_info =3D acc->data; >=20 > if (!afu_info) > - return -EINVAL; > + return -ENODEV; >=20 > if (info->index >=3D afu_info->num_regions) > return -EINVAL; > @@ -89,7 +99,7 @@ static int ifpga_acc_read(struct opae_accelerator *acc, > unsigned int region_idx, > struct opae_reg_region *region; >=20 > if (!afu_info) > - return -EINVAL; > + return -ENODEV; >=20 > if (offset + byte <=3D offset) > return -EINVAL; > @@ -129,7 +139,7 @@ static int ifpga_acc_write(struct opae_accelerator > *acc, > struct opae_reg_region *region; >=20 > if (!afu_info) > - return -EINVAL; > + return -ENODEV; >=20 > if (offset + byte <=3D offset) > return -EINVAL; > diff --git a/drivers/raw/ifpga/base/ifpga_defines.h > b/drivers/raw/ifpga/base/ifpga_defines.h > index 8f62033..9a280eb 100644 > --- a/drivers/raw/ifpga/base/ifpga_defines.h > +++ b/drivers/raw/ifpga/base/ifpga_defines.h > @@ -73,6 +73,7 @@ > enum fpga_id_type { > FME_ID, > PORT_ID, > + AFU_ID, > FPGA_ID_MAX, > }; >=20 > diff --git a/drivers/raw/ifpga/base/ifpga_enumerate.c > b/drivers/raw/ifpga/base/ifpga_enumerate.c > index 48b8af4..7a5d264 100644 > --- a/drivers/raw/ifpga/base/ifpga_enumerate.c > +++ b/drivers/raw/ifpga/base/ifpga_enumerate.c > @@ -2,6 +2,10 @@ > * Copyright(c) 2010-2018 Intel Corporation > */ >=20 > +#include > +#include > +#include > + > #include "opae_hw_api.h" > #include "ifpga_api.h" >=20 > @@ -9,6 +13,19 @@ > #include "ifpga_enumerate.h" > #include "ifpga_feature_dev.h" >=20 > +struct dfl_fpga_enum_dfl { > + u64 start; > + u64 len; > + void *addr; > + TAILQ_ENTRY(dfl_fpga_enum_dfl) node; > +}; > + > +TAILQ_HEAD(dfl_fpga_enum_dfls, dfl_fpga_enum_dfl); struct > +dfl_fpga_enum_info { > + struct ifpga_hw *hw; > + struct dfl_fpga_enum_dfls dfls; > +}; > + > struct build_feature_devs_info { > struct opae_adapter_data_pci *pci_data; >=20 > @@ -21,7 +38,6 @@ struct build_feature_devs_info { > void *ioaddr; > void *ioend; > uint64_t phys_addr; > - int current_bar; >=20 > void *pfme_hdr; >=20 > @@ -141,8 +157,10 @@ static u64 feature_id(void __iomem *start) > unsigned int size, unsigned int vec_start, > unsigned int vec_cnt) > { > - return build_info_add_sub_feature(binfo, start, fid, size, vec_start, > - vec_cnt); > + if (binfo->current_type !=3D AFU_ID) > + return build_info_add_sub_feature(binfo, start, fid, size, > + vec_start, vec_cnt); > + return 0; > } >=20 > /* > @@ -152,13 +170,14 @@ static u64 feature_id(void __iomem *start) > */ > static bool feature_is_UAFU(struct build_feature_devs_info *binfo) { > - if (binfo->current_type !=3D PORT_ID) > - return false; > + if ((binfo->current_type =3D=3D PORT_ID) || > + (binfo->current_type =3D=3D AFU_ID)) > + return true; >=20 > - return true; > + return false; > } >=20 > -static int parse_feature_port_uafu(struct build_feature_devs_info *binfo= , > +static int parse_feature_uafu(struct build_feature_devs_info *binfo, > struct feature_header *hdr) > { > u64 id =3D PORT_FEATURE_ID_UAFU; > @@ -169,9 +188,19 @@ static int parse_feature_port_uafu(struct > build_feature_devs_info *binfo, > int ret; > int size; >=20 > + if (binfo->acc_info) { > + dev_info(binfo, "Sub AFU found @ %p.\n", start); > + return 0; > + } > + > capability.csr =3D readq(&port_hdr->capability); >=20 > - size =3D capability.mmio_size << 10; > + if (binfo->current_type =3D=3D AFU_ID) { > + size =3D AFU_REGION_SIZE; > + } else { > + capability.csr =3D readq(&port_hdr->capability); > + size =3D capability.mmio_size << 10; > + } >=20 > ret =3D create_feature_instance(binfo, hdr, id, size, 0, 0); > if (ret) > @@ -185,44 +214,13 @@ static int parse_feature_port_uafu(struct > build_feature_devs_info *binfo, > info->region[0].phys_addr =3D binfo->phys_addr + > (uint8_t *)start - (uint8_t *)binfo->ioaddr; > info->region[0].len =3D size; > - info->num_regions =3D 1; > + info->num_regions =3D AFU_MAX_REGION; >=20 > binfo->acc_info =3D info; >=20 > return ret; > } >=20 > -static int parse_feature_afus(struct build_feature_devs_info *binfo, > - struct feature_header *hdr) > -{ > - int ret; > - struct feature_afu_header *afu_hdr, header; > - u8 __iomem *start; > - u8 __iomem *end =3D binfo->ioend; > - > - start =3D (u8 __iomem *)hdr; > - for (; start < end; start +=3D header.next_afu) { > - if ((unsigned int)(end - start) < > - (unsigned int)(sizeof(*afu_hdr) + sizeof(*hdr))) > - return -EINVAL; > - > - hdr =3D (struct feature_header *)start; > - afu_hdr =3D (struct feature_afu_header *)(hdr + 1); > - header.csr =3D readq(&afu_hdr->csr); > - > - if (feature_is_UAFU(binfo)) { > - ret =3D parse_feature_port_uafu(binfo, hdr); > - if (ret) > - return ret; > - } > - > - if (!header.next_afu) > - break; > - } > - > - return 0; > -} > - > /* create and register proper private data */ static int > build_info_commit_dev(struct build_feature_devs_info *binfo) { @@ - > 235,13 +233,9 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > struct ifpga_fme_hw *fme; > struct ifpga_feature *feature; >=20 > - if (!binfo->fiu) > - return 0; > - > if (binfo->current_type =3D=3D PORT_ID) { > - /* return error if no valid acc info data structure */ > - if (!info) > - return -EFAULT; > + if (!binfo->fiu) > + return 0; >=20 > br =3D opae_bridge_alloc(hw->adapter->name, &ifpga_br_ops, > binfo->fiu); > @@ -254,7 +248,7 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > port =3D &hw->port[binfo->current_port_id]; > feature =3D get_feature_by_id(&port->feature_list, > PORT_FEATURE_ID_UINT); > - if (feature) > + if (feature && info) > info->num_irqs =3D feature->vec_cnt; >=20 > acc =3D opae_accelerator_alloc(hw->adapter->name, > @@ -264,17 +258,21 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > return -ENOMEM; > } >=20 > + acc->adapter =3D hw->adapter; > acc->br =3D br; > if (hw->adapter->mgr) > acc->mgr =3D hw->adapter->mgr; > acc->index =3D br->id; >=20 > fme =3D &hw->fme; > - fme->nums_acc_region =3D info->num_regions; > + fme->nums_acc_region =3D info ? info->num_regions : 0; >=20 > opae_adapter_add_acc(hw->adapter, acc); >=20 > } else if (binfo->current_type =3D=3D FME_ID) { > + if (!binfo->fiu) > + return 0; > + > mgr =3D opae_manager_alloc(hw->adapter->name, > &ifpga_mgr_ops, > &ifpga_mgr_network_ops, binfo->fiu); > if (!mgr) > @@ -282,6 +280,22 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) >=20 > mgr->adapter =3D hw->adapter; > hw->adapter->mgr =3D mgr; > + } else if (binfo->current_type =3D=3D AFU_ID) { > + if (!info) > + return -EFAULT; > + > + info->num_irqs =3D 0; > + acc =3D opae_accelerator_alloc(hw->adapter->name, > + &ifpga_acc_ops, info); > + if (!acc) > + return -ENOMEM; > + > + acc->adapter =3D hw->adapter; > + acc->br =3D NULL; > + acc->mgr =3D NULL; > + acc->index =3D hw->num_afus++; > + > + opae_adapter_add_acc(hw->adapter, acc); > } >=20 > binfo->fiu =3D NULL; > @@ -295,11 +309,15 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) { > int ret; >=20 > + if ((type =3D=3D AFU_ID) && (binfo->current_type =3D=3D PORT_ID)) > + return 0; > + > ret =3D build_info_commit_dev(binfo); > if (ret) > return ret; >=20 > binfo->current_type =3D type; > + binfo->acc_info =3D NULL; >=20 > if (type =3D=3D FME_ID) { > binfo->fiu =3D &binfo->hw->fme; > @@ -311,6 +329,41 @@ static int build_info_commit_dev(struct > build_feature_devs_info *binfo) > return 0; > } >=20 > +static int parse_feature_afus(struct build_feature_devs_info *binfo, > + struct feature_header *hdr) > +{ > + int ret; > + struct feature_afu_header *afu_hdr, header; > + u8 __iomem *start; > + u8 __iomem *end =3D binfo->ioend; > + > + ret =3D build_info_create_dev(binfo, AFU_ID, 0); > + if (ret) > + return ret; > + > + start =3D (u8 __iomem *)hdr; > + for (; start < end; start +=3D header.next_afu) { > + if ((unsigned int)(end - start) < > + (unsigned int)(sizeof(*afu_hdr) + sizeof(*hdr))) > + return -EINVAL; > + > + hdr =3D (struct feature_header *)start; > + afu_hdr =3D (struct feature_afu_header *)(hdr + 1); > + header.csr =3D readq(&afu_hdr->csr); > + > + if (feature_is_UAFU(binfo)) { > + ret =3D parse_feature_uafu(binfo, hdr); > + if (ret) > + return ret; > + } > + > + if (!header.next_afu) > + break; > + } > + > + return 0; > +} > + > static int parse_feature_fme(struct build_feature_devs_info *binfo, > struct feature_header *start) > { > @@ -405,7 +458,7 @@ static int parse_feature_fiu(struct > build_feature_devs_info *binfo, > if (ret) > return ret; > } else { > - dev_info(binfo, "No AFUs detected on Port\n"); > + dev_info(binfo, "No AFU detected on Port\n"); > } >=20 > break; > @@ -426,7 +479,7 @@ static void parse_feature_irqs(struct > build_feature_devs_info *binfo, >=20 > id =3D feature_id(start); >=20 > - if (id =3D=3D PORT_FEATURE_ID_UINT) { > + if ((binfo->current_type =3D=3D PORT_ID) && (id =3D=3D > PORT_FEATURE_ID_UINT)) > +{ > struct feature_port_uint *port_uint =3D start; > struct feature_port_uint_cap uint_cap; >=20 > @@ -437,7 +490,8 @@ static void parse_feature_irqs(struct > build_feature_devs_info *binfo, > } else { > dev_debug(binfo, "UAFU doesn't support > interrupt\n"); > } > - } else if (id =3D=3D PORT_FEATURE_ID_ERROR) { > + } else if ((binfo->current_type =3D=3D PORT_ID) && > + (id =3D=3D PORT_FEATURE_ID_ERROR)) { > struct feature_port_error *port_err =3D start; > struct feature_port_err_capability port_err_cap; >=20 > @@ -449,7 +503,8 @@ static void parse_feature_irqs(struct > build_feature_devs_info *binfo, > dev_debug(&binfo, "Port error doesn't support > interrupt\n"); > } >=20 > - } else if (id =3D=3D FME_FEATURE_ID_GLOBAL_ERR) { > + } else if ((binfo->current_type =3D=3D FME_ID) && > + (id =3D=3D FME_FEATURE_ID_GLOBAL_ERR)) { > struct feature_fme_err *fme_err =3D start; > struct feature_fme_error_capability fme_err_cap; >=20 > @@ -497,9 +552,15 @@ static int parse_feature_private(struct > build_feature_devs_info *binfo, > return parse_feature_fme_private(binfo, hdr); > case PORT_ID: > return parse_feature_port_private(binfo, hdr); > + case AFU_ID: > + dev_err(binfo, "private feature %x belonging to AFU " > + "is not supported yet.\n", header.id); > + break; > default: > - dev_err(binfo, "private feature %x belonging to AFU %d > (unknown_type) is not supported yet.\n", > + dev_err(binfo, "private feature %x belonging to TYPE %d " > + "(unknown_type) is not supported yet.\n", > header.id, binfo->current_type); > + break; > } > return 0; > } > @@ -530,32 +591,57 @@ static int parse_feature(struct > build_feature_devs_info *binfo, > return ret; > } >=20 > -static int > -parse_feature_list(struct build_feature_devs_info *binfo, u8 __iomem > *start) > +static int build_info_prepare(struct build_feature_devs_info *binfo, > + struct dfl_fpga_enum_dfl *dfl) > { > + if (!binfo || !dfl) > + return -EINVAL; > + > + binfo->ioaddr =3D dfl->addr; > + binfo->ioend =3D (u8 *)dfl->addr + dfl->len; > + binfo->phys_addr =3D dfl->start; > + > + return 0; > +} > + > +static int parse_feature_list(struct build_feature_devs_info *binfo, > + struct dfl_fpga_enum_dfl *dfl) > +{ > + u8 *start, *end; > struct feature_header *hdr, header; > - u8 __iomem *end =3D (u8 __iomem *)binfo->ioend; > int ret =3D 0; >=20 > + ret =3D build_info_prepare(binfo, dfl); > + if (ret) > + return ret; > + > + start =3D (u8 *)binfo->ioaddr; > + end =3D (u8 *)binfo->ioend; > + > + /* walk through the device feature list via DFH's next DFH pointer. */ > for (; start < end; start +=3D header.next_header_offset) { > if ((unsigned int)(end - start) < (unsigned int)sizeof(*hdr)) { > - dev_err(binfo, "The region is too small to contain a > feature.\n"); > - ret =3D -EINVAL; > + dev_err(binfo, "The region is too small to " > + "contain a feature.\n"); > + ret =3D -EINVAL; > break; > } >=20 > hdr =3D (struct feature_header *)start; > - header.csr =3D readq(hdr); > + header.csr =3D opae_readq(hdr); >=20 > - dev_debug(binfo, "%s: address=3D0x%p, val=3D0x%llx, > header.id=3D0x%x, header.next_offset=3D0x%x, header.eol=3D0x%x, > header.type=3D0x%x\n", > - __func__, hdr, (unsigned long long)header.csr, > - header.id, header.next_header_offset, > - header.end_of_list, header.type); > + dev_debug(binfo, "%s: address=3D0x%p, val=3D0x%"PRIx64", " > + "header.id=3D0x%x, header.next_offset=3D0x%x, " > + "header.eol=3D0x%x, header.type=3D0x%x\n", > + __func__, hdr, header.csr, header.id, > + header.next_header_offset, header.end_of_list, > + header.type); >=20 > ret =3D parse_feature(binfo, hdr); > if (ret) > return ret; >=20 > + /* stop parsing if EOL(End of List) is set or offset is 0 */ > if (header.end_of_list || !header.next_header_offset) > break; > } > @@ -563,82 +649,9 @@ static int parse_feature(struct > build_feature_devs_info *binfo, > return build_info_commit_dev(binfo); > } >=20 > -/* switch the memory mapping to BAR# @bar */ -static int > parse_switch_to(struct build_feature_devs_info *binfo, int bar) -{ > - struct opae_adapter_data_pci *pci_data =3D binfo->pci_data; > - > - if (!pci_data->region[bar].addr) > - return -ENOMEM; > - > - binfo->ioaddr =3D pci_data->region[bar].addr; > - binfo->ioend =3D (u8 __iomem *)binfo->ioaddr + pci_data- > >region[bar].len; > - binfo->phys_addr =3D pci_data->region[bar].phys_addr; > - binfo->current_bar =3D bar; > - > - return 0; > -} > - > -static int parse_ports_from_fme(struct build_feature_devs_info *binfo) -= { > - struct feature_fme_header *fme_hdr; > - struct feature_fme_port port; > - int i =3D 0, ret =3D 0; > - > - if (!binfo->pfme_hdr) { > - dev_info(binfo, "VF is detected.\n"); > - return ret; > - } > - > - fme_hdr =3D binfo->pfme_hdr; > - > - do { > - port.csr =3D readq(&fme_hdr->port[i]); > - if (!port.port_implemented) > - break; > - > - /* skip port which only could be accessed via VF */ > - if (port.afu_access_control =3D=3D FME_AFU_ACCESS_VF) > - continue; > - > - ret =3D parse_switch_to(binfo, port.port_bar); > - if (ret) > - break; > - > - ret =3D parse_feature_list(binfo, > - (u8 __iomem *)binfo->ioaddr + > - port.port_offset); > - if (ret) > - break; > - } while (++i < MAX_FPGA_PORT_NUM); > - > - return ret; > -} > - > -static struct build_feature_devs_info * -build_info_alloc_and_init(struc= t > ifpga_hw *hw) -{ > - struct build_feature_devs_info *binfo; > - > - binfo =3D zmalloc(sizeof(*binfo)); > - if (!binfo) > - return binfo; > - > - binfo->hw =3D hw; > - binfo->pci_data =3D hw->pci_data; > - > - /* fpga feature list starts from BAR 0 */ > - if (parse_switch_to(binfo, 0)) { > - free(binfo); > - return NULL; > - } > - > - return binfo; > -} > - > static void build_info_free(struct build_feature_devs_info *binfo) { > - free(binfo); > + opae_free(binfo); > } >=20 > static void ifpga_print_device_feature_list(struct ifpga_hw *hw) @@ -648= ,6 > +661,11 @@ static void ifpga_print_device_feature_list(struct ifpga_hw *h= w) > struct ifpga_feature *feature; > int i; >=20 > + if (fme->state =3D=3D IFPGA_FME_UNUSED) { > + dev_info(hw, "FME is not present\n"); > + return; > + } > + > dev_info(hw, "found fme_device, is in PF: %s\n", > is_ifpga_hw_pf(hw) ? "yes" : "no"); >=20 > @@ -685,40 +703,410 @@ static void ifpga_print_device_feature_list(struct > ifpga_hw *hw) > } > } >=20 > -int ifpga_bus_enumerate(struct ifpga_hw *hw) > +static struct dfl_fpga_enum_info *dfl_fpga_enum_info_alloc(struct > +ifpga_hw *hw) > { > - struct build_feature_devs_info *binfo; > + struct dfl_fpga_enum_info *info; > + > + info =3D opae_zmalloc(sizeof(*info)); > + if (!info) > + return NULL; > + > + info->hw =3D hw; > + TAILQ_INIT(&info->dfls); > + > + return info; > +} > + > +static void dfl_fpga_enum_info_free(struct dfl_fpga_enum_info *info) { > + struct dfl_fpga_enum_dfl *tmp, *dfl; > + > + if (!info) > + return; > + > + /* remove all device feature lists in the list. */ > + for (dfl =3D TAILQ_FIRST(&info->dfls); > + dfl && (tmp =3D TAILQ_NEXT(dfl, node), 1); > + dfl =3D tmp) { > + TAILQ_REMOVE(&info->dfls, dfl, node); > + opae_free(dfl); > + } > + > + opae_free(info); > +} > + > +static int dfl_fpga_enum_info_add_dfl(struct dfl_fpga_enum_info *info, > + u64 start, u64 len, void *addr) > +{ > + struct dfl_fpga_enum_dfl *dfl; > + > + dfl =3D opae_zmalloc(sizeof(*dfl)); > + if (!dfl) > + return -ENOMEM; > + > + dfl->start =3D start; > + dfl->len =3D len; > + dfl->addr =3D addr; > + > + TAILQ_INSERT_TAIL(&info->dfls, dfl, node); > + > + return 0; > +} > + > +#define PCI_CFG_SPACE_SIZE 256 > +#define PCI_CFG_SPACE_EXP_SIZE 4096 > +#define PCI_EXT_CAP_ID(header) (header & 0x0000ffff) > +#define PCI_EXT_CAP_NEXT(header) ((header >> 20) & 0xffc) > + > +static int > +pci_find_next_ecap(int fd, int start, u32 cap) { > + u32 header; > + int ttl =3D (PCI_CFG_SPACE_EXP_SIZE - PCI_CFG_SPACE_SIZE) / 8; > + int pos =3D PCI_CFG_SPACE_SIZE; > int ret; >=20 > - binfo =3D build_info_alloc_and_init(hw); > + if (start > 0) > + pos =3D start; > + > + ret =3D pread(fd, &header, sizeof(header), pos); > + if (ret < 0) > + return ret; > + > + /* > + * If we have no capabilities, this is indicated by cap ID, > + * cap version and next pointer all being 0. > + */ > + if (header =3D=3D 0) > + return 0; > + > + while (ttl-- > 0) { > + if ((PCI_EXT_CAP_ID(header) =3D=3D cap) && (pos !=3D start)) > + return pos; > + > + pos =3D PCI_EXT_CAP_NEXT(header); > + if (pos < PCI_CFG_SPACE_SIZE) > + break; > + ret =3D pread(fd, &header, sizeof(header), pos); > + if (ret < 0) > + return ret; > + } > + > + return 0; > +} > + > +#define PCI_EXT_CAP_ID_VNDR 0x0B > +#define PCI_VNDR_HEADER 4 > +#define PCI_VNDR_HEADER_ID(x) ((x) & 0xffff) > +#define PCI_VENDOR_ID_INTEL 0x8086 > +#define PCI_VSEC_ID_INTEL_DFLS 0x43 > +#define PCI_VNDR_DFLS_CNT 0x8 > +#define PCI_VNDR_DFLS_RES 0xc > +#define PCI_VNDR_DFLS_RES_BAR_MASK GENMASK(2, 0) #define > +PCI_VNDR_DFLS_RES_OFF_MASK GENMASK(31, 3) > + > +static int find_dfls_by_vsec(struct dfl_fpga_enum_info *info) { > + struct ifpga_hw *hw; > + struct opae_adapter_data_pci *pci_data; > + char path[64]; > + u32 bir, offset, vndr_hdr, i, dfl_cnt, dfl_res; > + int fd, ret, dfl_res_off, voff =3D 0; > + u64 start, len; > + void *addr; > + > + if (!info || !info->hw) > + return -EINVAL; > + hw =3D info->hw; > + > + if (!hw->adapter || !hw->pci_data) > + return -EINVAL; > + pci_data =3D hw->pci_data; > + > + ret =3D snprintf(path, sizeof(path), "/sys/bus/pci/devices/%s/config", > + hw->adapter->name); > + if ((unsigned int)ret >=3D sizeof(path)) > + return -EINVAL; > + > + fd =3D open(path, O_RDWR); > + if (fd < 0) > + return -EIO; > + > + while ((voff =3D pci_find_next_ecap(fd, voff, > + PCI_EXT_CAP_ID_VNDR))) { > + vndr_hdr =3D 0; > + ret =3D pread(fd, &vndr_hdr, sizeof(vndr_hdr), > + voff + PCI_VNDR_HEADER); > + if (ret < 0) > + return -EIO; > + if (PCI_VNDR_HEADER_ID(vndr_hdr) =3D=3D > PCI_VSEC_ID_INTEL_DFLS && > + pci_data->vendor_id =3D=3D PCI_VENDOR_ID_INTEL) > + break; > + } > + > + if (!voff) { > + dev_debug(hw, "%s no DFL VSEC found\n", __func__); > + return -ENODEV; > + } > + > + dfl_cnt =3D 0; > + ret =3D pread(fd, &dfl_cnt, sizeof(dfl_cnt), voff + > PCI_VNDR_DFLS_CNT); > + if (ret < 0) > + return -EIO; > + > + dfl_res_off =3D voff + PCI_VNDR_DFLS_RES; > + if (dfl_res_off + (dfl_cnt * sizeof(u32)) > PCI_CFG_SPACE_EXP_SIZE) > { > + dev_err(hw, "%s DFL VSEC too big for PCIe config space\n", > + __func__); > + return -EINVAL; > + } > + > + for (i =3D 0; i < dfl_cnt; i++, dfl_res_off +=3D sizeof(u32)) { > + dfl_res =3D GENMASK(31, 0); > + ret =3D pread(fd, &dfl_res, sizeof(dfl_res), dfl_res_off); > + bir =3D dfl_res & PCI_VNDR_DFLS_RES_BAR_MASK; > + if (bir >=3D PCI_MAX_RESOURCE) { > + dev_err(hw, "%s bad bir number %d\n", > + __func__, bir); > + return -EINVAL; > + } > + > + len =3D pci_data->region[bir].len; > + offset =3D dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK; > + if (offset >=3D len) { > + dev_err(hw, "%s bad offset %u >=3D %"PRIu64"\n", > + __func__, offset, len); > + return -EINVAL; > + } > + > + dev_debug(hw, "%s BAR %d offset 0x%x\n", __func__, bir, > offset); > + len -=3D offset; > + start =3D pci_data->region[bir].phys_addr + offset; > + addr =3D pci_data->region[bir].addr + offset; > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } > + > + return 0; > +} > + > +/* default method of finding dfls starting at offset 0 of bar 0 */ > +static int find_dfls_by_default(struct dfl_fpga_enum_info *info) { > + struct ifpga_hw *hw; > + struct opae_adapter_data_pci *pci_data; > + int port_num, bar, i, ret =3D 0; > + u64 start, len; > + void *addr; > + u32 offset; > + struct feature_header hdr; > + struct feature_fme_capability cap; > + struct feature_fme_port port; > + struct feature_fme_header *fme_hdr; > + > + if (!info || !info->hw) > + return -EINVAL; > + hw =3D info->hw; > + > + if (!hw->pci_data) > + return -EINVAL; > + pci_data =3D hw->pci_data; > + > + /* start to find Device Feature List from Bar 0 */ > + addr =3D pci_data->region[0].addr; > + if (!addr) > + return -ENOMEM; > + > + /* > + * PF device has FME and Ports/AFUs, and VF device only has one > + * Port/AFU. Check them and add related "Device Feature List" info > + * for the next step enumeration. > + */ > + hdr.csr =3D opae_readq(addr); > + if ((hdr.type =3D=3D FEATURE_TYPE_FIU) && (hdr.id =3D=3D > FEATURE_FIU_ID_FME)) { > + start =3D pci_data->region[0].phys_addr; > + len =3D pci_data->region[0].len; > + addr =3D pci_data->region[0].addr; > + > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + > + /* > + * find more Device Feature Lists (e.g. Ports) per information > + * indicated by FME module. > + */ > + fme_hdr =3D (struct feature_fme_header *)addr; > + cap.csr =3D opae_readq(&fme_hdr->capability); > + port_num =3D (int)cap.num_ports; > + > + dev_info(hw, "port_num =3D %d\n", port_num); > + if (port_num > MAX_FPGA_PORT_NUM) > + port_num =3D MAX_FPGA_PORT_NUM; > + > + for (i =3D 0; i < port_num; i++) { > + port.csr =3D opae_readq(&fme_hdr->port[i]); > + > + /* skip ports which are not implemented. */ > + if (!port.port_implemented) > + continue; > + > + /* skip port which only could be accessed via VF */ > + if (port.afu_access_control =3D=3D FME_AFU_ACCESS_VF) > + continue; > + > + /* > + * add Port's Device Feature List information for next > + * step enumeration. > + */ > + bar =3D (int)port.port_bar; > + offset =3D port.port_offset; > + if (bar =3D=3D FME_PORT_OFST_BAR_SKIP) { > + continue; > + } else if (bar >=3D PCI_MAX_RESOURCE) { > + dev_err(hw, "bad BAR %d for port %d\n", bar, > i); > + ret =3D -EINVAL; > + break; > + } > + dev_info(hw, "BAR %d offset %u\n", bar, offset); > + > + len =3D pci_data->region[bar].len; > + if (offset >=3D len) { > + dev_warn(hw, "bad port > offset %u >=3D %pa\n", > + offset, &len); > + continue; > + } > + > + len -=3D offset; > + start =3D pci_data->region[bar].phys_addr + offset; > + addr =3D pci_data->region[bar].addr + offset; > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } > + } else if ((hdr.type =3D=3D FEATURE_TYPE_FIU) && > + (hdr.id =3D=3D FEATURE_FIU_ID_PORT)) { > + start =3D pci_data->region[0].phys_addr; > + len =3D pci_data->region[0].len; > + addr =3D pci_data->region[0].addr; > + > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } else if (hdr.type =3D=3D FEATURE_TYPE_AFU) { > + start =3D pci_data->region[0].phys_addr; > + len =3D pci_data->region[0].len; > + addr =3D pci_data->region[0].addr; > + > + dfl_fpga_enum_info_add_dfl(info, start, len, addr); > + } else { > + dev_info(hw, "Unknown feature type 0x%x id 0x%x\n", > + hdr.type, hdr.id); > + ret =3D -ENODEV; > + } > + > + return ret; > +} > + > +static int dfl_fpga_feature_devs_enumerate(struct dfl_fpga_enum_info > +*info) { > + struct build_feature_devs_info *binfo; > + struct dfl_fpga_enum_dfl *dfl; > + int ret =3D 0; > + > + if (!info || !info->hw) > + return -EINVAL; > + > + /* create and init build info for enumeration */ > + binfo =3D opae_zmalloc(sizeof(*binfo)); > if (!binfo) > return -ENOMEM; >=20 > - ret =3D parse_feature_list(binfo, binfo->ioaddr); > + binfo->hw =3D info->hw; > + binfo->pci_data =3D info->hw->pci_data; > + > + /* > + * start enumeration for all feature devices based on Device Feature > + * Lists. > + */ > + TAILQ_FOREACH(dfl, &info->dfls, node) { > + ret =3D parse_feature_list(binfo, dfl); > + if (ret) > + break; > + } > + > + build_info_free(binfo); > + > + return ret; > +} > + > +int ifpga_bus_enumerate(struct ifpga_hw *hw) { > + struct dfl_fpga_enum_info *info; > + int ret; > + > + /* allocate enumeration info */ > + info =3D dfl_fpga_enum_info_alloc(hw); > + if (!info) > + return -ENOMEM; > + > + ret =3D find_dfls_by_vsec(info); > + if (ret < 0) > + ret =3D find_dfls_by_default(info); > + > if (ret) > goto exit; >=20 > - ret =3D parse_ports_from_fme(binfo); > - if (ret) > + /* start enumeration with prepared enumeration information */ > + ret =3D dfl_fpga_feature_devs_enumerate(info); > + if (ret < 0) { > + dev_err(hw, "Enumeration failure\n"); > goto exit; > + } >=20 > ifpga_print_device_feature_list(hw); >=20 > exit: > - build_info_free(binfo); > + dfl_fpga_enum_info_free(info); > + > return ret; > } >=20 > -int ifpga_bus_init(struct ifpga_hw *hw) > +static void ifpga_print_acc_list(struct opae_adapter *adapter) > { > + struct opae_accelerator *acc; > + struct ifpga_afu_info *info; > + struct uuid guid; > + char buf[48]; > int i; > + > + opae_adapter_for_each_acc(adapter, acc) { > + info =3D acc->data; > + if (!info) > + continue; > + acc->ops->get_uuid(acc, &guid); > + i =3D sprintf(buf, "%02x%02x%02x%02x-", > + guid.b[15], guid.b[14], guid.b[13], guid.b[12]); > + i +=3D sprintf(buf+i, "%02x%02x-", guid.b[11], guid.b[10]); > + i +=3D sprintf(buf+i, "%02x%02x-", guid.b[9], guid.b[8]); > + i +=3D sprintf(buf+i, "%02x%02x-", guid.b[7], guid.b[6]); > + sprintf(buf+i, "%02x%02x%02x%02x%02x%02x", > + guid.b[5], guid.b[4], guid.b[3], > + guid.b[2], guid.b[1], guid.b[0]); > + dev_info(hw, "AFU(%s-%d)@%p: len:0x%"PRIx64", > guid:%s\n", > + acc->name, acc->index, info->region[0].addr, > + info->region[0].len, buf); > + } > +} > + > +int ifpga_bus_init(struct ifpga_hw *hw) { > + int i, ret =3D 0; > struct ifpga_port_hw *port; >=20 > - fme_hw_init(&hw->fme); > + ret =3D fme_hw_init(&hw->fme); > + if (ret) > + return ret; > + > for (i =3D 0; i < MAX_FPGA_PORT_NUM; i++) { > port =3D &hw->port[i]; > port_hw_init(port); > } > + ifpga_print_acc_list(hw->adapter); >=20 > return 0; > } > diff --git a/drivers/raw/ifpga/base/ifpga_enumerate.h > b/drivers/raw/ifpga/base/ifpga_enumerate.h > index 95ed594..e6b04f0 100644 > --- a/drivers/raw/ifpga/base/ifpga_enumerate.h > +++ b/drivers/raw/ifpga/base/ifpga_enumerate.h > @@ -5,6 +5,8 @@ > #ifndef _IFPGA_ENUMERATE_H_ > #define _IFPGA_ENUMERATE_H_ >=20 > +#define FME_PORT_OFST_BAR_SKIP 7 > + > int ifpga_bus_init(struct ifpga_hw *hw); int ifpga_bus_uinit(struct ifp= ga_hw > *hw); int ifpga_bus_enumerate(struct ifpga_hw *hw); diff --git > a/drivers/raw/ifpga/base/ifpga_feature_dev.c > b/drivers/raw/ifpga/base/ifpga_feature_dev.c > index 0813513..dbecc7b 100644 > --- a/drivers/raw/ifpga/base/ifpga_feature_dev.c > +++ b/drivers/raw/ifpga/base/ifpga_feature_dev.c > @@ -59,7 +59,7 @@ int __fpga_port_disable(struct ifpga_port_hw *port) > if (fpga_wait_register_field(port_sftrst_ack, control, > &port_hdr->control, RST_POLL_TIMEOUT, > RST_POLL_INVL)) { > - dev_err(port, "timeout, fail to reset device\n"); > + dev_err(port, "timeout, fail to reset FIM port\n"); > return -ETIMEDOUT; > } >=20 > @@ -277,10 +277,11 @@ static void feature_uinit(struct ifpga_feature_list > *list) > struct ifpga_feature *feature; >=20 > TAILQ_FOREACH(feature, list, next) { > - if (feature->state !=3D IFPGA_FEATURE_ATTACHED) > + if (feature->state !=3D IFPGA_FEATURE_INITED) > continue; > if (feature->ops && feature->ops->uinit) > feature->ops->uinit(feature); > + feature->state =3D IFPGA_FEATURE_ATTACHED; > } > } >=20 > @@ -301,6 +302,9 @@ static int feature_init(struct feature_driver *drv, > ret =3D feature->ops->init(feature); > if (ret) > goto error; > + else > + feature->state =3D > + > IFPGA_FEATURE_INITED; > } > } > } > @@ -315,14 +319,8 @@ static int feature_init(struct feature_driver *drv, >=20 > int fme_hw_init(struct ifpga_fme_hw *fme) { > - int ret; > - > - if (fme->state !=3D IFPGA_FME_IMPLEMENTED) > - return -ENODEV; > - > - ret =3D feature_init(fme_feature_drvs, &fme->feature_list); > - if (ret) > - return ret; > + if (fme->state =3D=3D IFPGA_FME_IMPLEMENTED) > + return feature_init(fme_feature_drvs, &fme->feature_list); >=20 > return 0; > } > diff --git a/drivers/raw/ifpga/base/ifpga_hw.h > b/drivers/raw/ifpga/base/ifpga_hw.h > index ed5edc6..4d56deb 100644 > --- a/drivers/raw/ifpga/base/ifpga_hw.h > +++ b/drivers/raw/ifpga/base/ifpga_hw.h > @@ -15,6 +15,7 @@ > enum ifpga_feature_state { > IFPGA_FEATURE_UNUSED =3D 0, > IFPGA_FEATURE_ATTACHED, > + IFPGA_FEATURE_INITED > }; >=20 > enum feature_type { > @@ -134,6 +135,7 @@ struct ifpga_hw { >=20 > struct ifpga_fme_hw fme; > struct ifpga_port_hw port[MAX_FPGA_PORT_NUM]; > + int num_afus; > }; >=20 > static inline bool is_ifpga_hw_pf(struct ifpga_hw *hw) diff --git > a/drivers/raw/ifpga/base/opae_hw_api.c > b/drivers/raw/ifpga/base/opae_hw_api.c > index 11c9887..87256fc 100644 > --- a/drivers/raw/ifpga/base/opae_hw_api.c > +++ b/drivers/raw/ifpga/base/opae_hw_api.c > @@ -177,7 +177,7 @@ int opae_acc_get_region_info(struct > opae_accelerator *acc, int opae_acc_set_irq(struct opae_accelerator *acc= , > u32 start, u32 count, s32 evtfds[]) { > - if (!acc || !acc->data) > + if (!acc) > return -EINVAL; >=20 > if (start + count <=3D start) > diff --git a/drivers/raw/ifpga/base/opae_hw_api.h > b/drivers/raw/ifpga/base/opae_hw_api.h > index 7e04b56..fd40e09 100644 > --- a/drivers/raw/ifpga/base/opae_hw_api.h > +++ b/drivers/raw/ifpga/base/opae_hw_api.h > @@ -143,6 +143,7 @@ struct opae_accelerator { > TAILQ_ENTRY(opae_accelerator) node; > const char *name; > int index; > + struct opae_adapter *adapter; > struct opae_bridge *br; > struct opae_manager *mgr; > struct opae_accelerator_ops *ops; > @@ -240,6 +241,7 @@ struct opae_adapter_data { >=20 > struct opae_reg_region { > u64 phys_addr; > +#define AFU_REGION_SIZE 0x8000 > u64 len; > u8 *addr; > }; > diff --git a/drivers/raw/ifpga/ifpga_rawdev.c > b/drivers/raw/ifpga/ifpga_rawdev.c > index 94df56c..ceb18ae 100644 > --- a/drivers/raw/ifpga/ifpga_rawdev.c > +++ b/drivers/raw/ifpga/ifpga_rawdev.c > @@ -47,11 +47,13 @@ > #define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0 > #define PCIE_DEVICE_ID_PF_DSC_1_X 0x09C4 > #define PCIE_DEVICE_ID_PAC_N3000 0x0B30 > +#define PCIE_DEVICE_ID_PAC_N6000 0xBCCE > /* VF Device */ > #define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF > #define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1 > #define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5 > #define PCIE_DEVICE_ID_VF_PAC_N3000 0x0B31 > +#define PCIE_DEVICE_ID_VF_PAC_N6000 0xBCCF > #define RTE_MAX_RAW_DEVICE 10 >=20 > static const struct rte_pci_id pci_ifpga_map[] =3D { @@ -63,6 +65,8 @@ > { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_DSC_1_X) }, > { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PAC_N3000),}, > { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_PAC_N3000),}, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_PAC_N6000),}, > + { RTE_PCI_DEVICE(PCI_VENDOR_ID_INTEL, > PCIE_DEVICE_ID_VF_PAC_N6000),}, > { .vendor_id =3D 0, /* sentinel */ }, > }; >=20 > @@ -110,6 +114,7 @@ struct ifpga_rawdev * >=20 > return IFPGA_RAWDEV_NUM; > } > + > static struct ifpga_rawdev * > ifpga_rawdev_allocate(struct rte_rawdev *rawdev) { @@ -365,7 +370,7 > @@ static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev) > return -ENODEV; >=20 > mgr =3D opae_adapter_get_mgr(adapter); > - if (!mgr) > + if (!mgr || !mgr->sensor_list) > return -ENODEV; >=20 > opae_mgr_for_each_sensor(mgr, sensor) { @@ -377,7 +382,7 @@ > static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev) > goto fail; >=20 > if (value =3D=3D 0xdeadbeef) { > - IFPGA_RAWDEV_PMD_ERR("dev_id %d sensor %s > value %x\n", > + IFPGA_RAWDEV_PMD_DEBUG("dev_id %d > sensor %s value %x\n", > raw_dev->dev_id, sensor->name, > value); > continue; > } > @@ -893,7 +898,7 @@ static int set_surprise_link_check_aer( { > struct opae_adapter *adapter; > struct opae_manager *mgr; > - struct opae_board_info *info; > + struct opae_board_info *info =3D NULL; > struct rte_afu_pr_conf *afu_pr_conf; > int ret; > struct uuid uuid; > @@ -921,17 +926,14 @@ static int set_surprise_link_check_aer( > } >=20 > mgr =3D opae_adapter_get_mgr(adapter); > - if (!mgr) { > - IFPGA_RAWDEV_PMD_ERR("opae_manager of > opae_adapter is NULL"); > - return -1; > - } > - > - if (ifpga_mgr_ops.get_board_info(mgr, &info)) { > - IFPGA_RAWDEV_PMD_ERR("ifpga manager get_board_info > fail!"); > - return -1; > + if (mgr) { > + if (ifpga_mgr_ops.get_board_info(mgr, &info)) { > + IFPGA_RAWDEV_PMD_ERR("ifpga manager > get_board_info fail!"); > + return -1; > + } > } >=20 > - if (info->lightweight) { > + if (info && info->lightweight) { > /* set uuid to all 0, when fpga is lightweight image */ > memset(&afu_pr_conf->afu_id.uuid.uuid_low, 0, > sizeof(u64)); > memset(&afu_pr_conf->afu_id.uuid.uuid_high, 0, > sizeof(u64)); @@ -953,7 +955,7 @@ static int set_surprise_link_check_aer( > __func__, > (unsigned long)afu_pr_conf->afu_id.uuid.uuid_low, > (unsigned long)afu_pr_conf->afu_id.uuid.uuid_high); > - } > + } > return 0; > } >=20 > @@ -1592,7 +1594,7 @@ static int fme_clean_fme_error(struct > opae_manager *mgr) > ret =3D opae_adapter_init(adapter, pci_dev->device.name, data); > if (ret) { > ret =3D -ENOMEM; > - goto free_adapter_data; > + goto cleanup; > } >=20 > rawdev->dev_ops =3D &ifpga_rawdev_ops; > @@ -1602,29 +1604,23 @@ static int fme_clean_fme_error(struct > opae_manager *mgr) > /* must enumerate the adapter before use it */ > ret =3D opae_adapter_enumerate(adapter); > if (ret) > - goto free_adapter_data; > + goto cleanup; >=20 > /* get opae_manager to rawdev */ > mgr =3D opae_adapter_get_mgr(adapter); > if (mgr) { > - /* PF function */ > - IFPGA_RAWDEV_PMD_INFO("this is a PF function"); > + ret =3D ifpga_register_msix_irq(dev, 0, IFPGA_FME_IRQ, 0, 0, > + fme_interrupt_handler, "fme_irq", mgr); > + if (ret) > + goto cleanup; > } >=20 > - ret =3D ifpga_register_msix_irq(dev, 0, IFPGA_FME_IRQ, 0, 0, > - fme_interrupt_handler, "fme_irq", mgr); > - if (ret) > - goto free_adapter_data; > - > ret =3D ifpga_monitor_start_func(dev); > if (ret) > - goto free_adapter_data; > + goto cleanup; >=20 > return ret; >=20 > -free_adapter_data: > - if (data) > - opae_adapter_data_free(data); > cleanup: > if (rawdev) > rte_rawdev_pmd_release(rawdev); > -- > 1.8.3.1 Reviewed-by: Rosen Xu