From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3903442BD1; Mon, 29 May 2023 11:25:11 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 28699410DD; Mon, 29 May 2023 11:25:11 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id AA8B8410D7 for ; Mon, 29 May 2023 11:25:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685352310; x=1716888310; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=pzLZdPZDcds2UBMuS4yb39/+zKyun1O9ZbWWdtuL+Uk=; b=Iy7Tpk4MkMVsZcluLMN61yLCXO8nOIc4vA4IqL2PNk1vL3e0iGkQoRxU tFsDsMs1De/kPocyyav91c5cdiO9f4R4xPZ9MYwfwaDH4FB/VG+D3T1VK PA/R1QKBctiWsVZ347eWclqYGnC9DNkbmvcTE5m0gyjFKnKcrWlQZJYEt hfEvm27VN70p5fdRqZwCLDaPcw0uVVc0Ptgjk8uFX01rqWw1Z5XhO39aK 97MTswHtK0vGxOPFnfDUBK2qikzo8XcwZOtsIi5lN7sblP9I8LizLnbRu RooyIiX1U3gZpSi5zd20HPJrWT+LsQFQkYCvGTKYv4hWxsxHxlPt1vOF2 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10724"; a="352169633" X-IronPort-AV: E=Sophos;i="6.00,201,1681196400"; d="scan'208";a="352169633" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2023 02:25:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10724"; a="952677599" X-IronPort-AV: E=Sophos;i="6.00,201,1681196400"; d="scan'208";a="952677599" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by fmsmga006.fm.intel.com with ESMTP; 29 May 2023 02:25:07 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Mon, 29 May 2023 02:25:06 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Mon, 29 May 2023 02:25:06 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.174) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Mon, 29 May 2023 02:25:06 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N6hXeiaitx54eC5fPshLFKv2KlWAixnhRIeMzcsuqe/CP8kjcvjhnh4imAAOz4sWFCHgSOg5wtRgVtc1h74xQ4XU0Vl+18n+2t4rT1qg5lL0x6fl2aT66qxOclYCJr2uvgy2Pr4i5miKULPp2eNFg0tPOSUikZydqq8diTUfEP4SIYZEidQu4nqHX/QOqFQ5AotydV7CmTR7DoIPtOCdiGzjY4XP1pXdKE3U+h+Ot+OSwx0tRLg2+G970jRe1i7uQY7xiXcv0AQbATm/Znh3r9V7nr4fyg26zCy/UdfuarHyl82EIERT7awOiUX+3m3wIxZgJ30BNEF6hADYK3SCMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HDHqbeoJanlY3AntGaPmSSqeUPT1R0x65DO1BZW/6f4=; b=VOQ2NQbPiKrHquxNEuQn48pfs1uC5v406pKgVC3QS66vxVkVgj/8UDq61pZ0EpOGH/A8+J0rHxXM44WaN8slMPFrSAejAlTACNKqXv4H8jjBDDKG4b/INEDmxlLMK1w2gIumxeludtyJ8jJsrxtrDBG/LHTzLqjFCjfKx8HRJ6mt/fK7vWig0Gtn33WKPbzlE2mM8kH5lRioa7LVSev7v292r7CJuPEUEQogO49ZD4sznZ15eHqb0nj5Dr3ZwY8NQwBpdwtFEC76QuaZvj46J3ddvgLuqQ5AQ7dL8hM/cckLpDFdfwmE/2Xxn427DihmcZw5/hFWTmmoEx9d+Q6HOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by CH0PR11MB8141.namprd11.prod.outlook.com (2603:10b6:610:18a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.23; Mon, 29 May 2023 09:25:04 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e427:f0a6:8472:d0f6]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e427:f0a6:8472:d0f6%7]) with mapi id 15.20.6433.020; Mon, 29 May 2023 09:25:04 +0000 From: "Xia, Chenbo" To: "Li, Miao" , "dev@dpdk.org" CC: "skori@marvell.com" , "thomas@monjalon.net" , "david.marchand@redhat.com" , "ferruh.yigit@amd.com" , "Cao, Yahui" , "Burakov, Anatoly" Subject: RE: [PATCH v3 4/4] bus/pci: add VFIO sparse mmap support Thread-Topic: [PATCH v3 4/4] bus/pci: add VFIO sparse mmap support Thread-Index: AQHZjt8Uodl23F/shkKQfbL/ewr2tq9w/0BA Date: Mon, 29 May 2023 09:25:03 +0000 Message-ID: References: <20230515094124.722431-1-miao.li@intel.com> <20230525163116.682000-1-miao.li@intel.com> <20230525163116.682000-5-miao.li@intel.com> In-Reply-To: <20230525163116.682000-5-miao.li@intel.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|CH0PR11MB8141:EE_ x-ms-office365-filtering-correlation-id: 0220cc6e-171a-4e31-ccbd-08db602695b7 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WfgN/845uNdSDBEuWTzhSKpnWWDMZVgp0PBoGS6eJSUQ+FyFk1Ok/rKumz4IQmbLHit4D2su+qffF1SM/hlZ/s6/YJ+GY+wt3H4bfnwBb1TWfsJRrwMRurArt8eFrXISSxGnAKrAyUyC4uMNym1EJ/c7lmtpvjtzFJX8fFczKVK5Y0bvBcUe7l0z50DSvy/DLO0WfB+KGhfLNEFZg6EonVkCrB+TxfS2xcKmaeFb+WbVUqlL/taEKcprF6WhxvyiV2grcavX+ntvyiAhVLjMFEuzODCrOQwJDZPDRjswUhwdbc+CMwWEuA9KMMYEzoJiQ33WuNe+BYw2UBLZFbno07/UHmxqNN3DDiNoWHGudCtmrbtnKl3ghXP84pIsb/yRgywbPyH1aIGzW3ft3HDBgCxzXgCjjTHCWloZV+iuX/8h52FmOJ7gojEx6ZiI5/i9kjjzT0MgLOdAx1Hm6phAovG7MC/tvwkWsdxJUxvnr0veQIZ/JA7rOxFqzfQD92cLmFfVSnIKBkyHxqjG0PKXtWgl+k3FKRFtVpYX3iWkhEPoLdAtYic5CMx17qa862m7ox77j4KzaAsItx38xp/CEJHblJFY2APq1xHi527sMyyRw/DFirbEojXB31FdgHhT x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(136003)(346002)(366004)(396003)(376002)(39860400002)(451199021)(26005)(7696005)(53546011)(9686003)(71200400001)(6506007)(2906002)(33656002)(316002)(186003)(55016003)(52536014)(5660300002)(107886003)(41300700001)(8676002)(8936002)(478600001)(82960400001)(122000001)(38100700002)(54906003)(110136005)(4326008)(86362001)(76116006)(66946007)(38070700005)(66476007)(66556008)(83380400001)(64756008)(66446008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Uckqrj3mboSpmNk9Uz1FHAapOwMQA2cC5raUYblOv63wOhD6s/cJsg2Nd00s?= =?us-ascii?Q?GoghobdpteSqwpj1JvHQpqXKcm24VLBgMvVbKKFd6xg0oFcfoGoelwgJ3BZ5?= =?us-ascii?Q?Fl+oTqiq9B/J8aD/c8e7ILD/y6m628/yXCHn9rGW7uRkxQcn4suVjNZKHihB?= =?us-ascii?Q?rlofSoMdMTYygIDWCycgyejB3rhqfXF8rNoVm/CwLG6tiNs+3nmc3d0VMSzl?= =?us-ascii?Q?itpqIm40LIOY7R2YyEL0U/9l/iCpPbGO9sfGcjp+sI1AyRZpOfOsSbdBj7yY?= =?us-ascii?Q?p+XutFa2jqXg2boMPXv1f4fYaSH+frqwbsBltAXMTyHlq9/NpU+8xkuwOFon?= =?us-ascii?Q?0pXcEhPirOvck2d76tcBHFiFzOoVOtXNAW+BxqcrsxIdRNkzVwkQ/+xa72zI?= =?us-ascii?Q?XYMh7Dst84WQLLyjD61xAK3/QQk+6usWDSyeIxNeNq1oo6XImk4dn9W3nbWM?= =?us-ascii?Q?6+oCuBnkuxnGBAW1hr6jMbC0k/5bczZeutKRpCDsMscMKoYP9MFe9pSFJVAp?= =?us-ascii?Q?yh/zBW5xkyhVtq0+wnnUHkQ1jaX20vh7JhGfSn7mcmgrp4YkYai/TWjetflR?= =?us-ascii?Q?DmICuz7qZjNfKaE6eArm6b9+WUGrcdjUeinxxudYx0ONoDkxIYE9oJ24aipa?= =?us-ascii?Q?E6q/3wI1wAaKWYAhBv/qJRP+Z9HX2/ETycw1+Gpun/CmBM4mCr/l7o8FegNB?= =?us-ascii?Q?E82bU3xS6fUdDL7sHLz5HqVN76JIQbsuxmqqgwqxCBd/mierm8QeCX/vZfOv?= =?us-ascii?Q?wpSoSstFHLu2bJtE4RWzaL05mFOlJ+EIPlkX9JRwK+fE5QVEyU3ULtqBDPNz?= =?us-ascii?Q?7F/wT0mCNfCqIZdDn0aBQwZ7nMxTwE77PRRbgr9zUYY+60SW474ag2HowQ4u?= =?us-ascii?Q?1uU/mBjyFhWXMBIJivexq0mT1Mi22LDY1NhBG/uukqG/M68myVA7R3ciD9q4?= =?us-ascii?Q?oBwIfI8DMKDipLHYK0sAwIJvdihGJQ4s619BvGgjULWTygZbcA00jASQFt0U?= =?us-ascii?Q?C7l+SAnTNL44gFgpdfl025pLtATl/FDZkjIUqlExUmw4tXbIPgvb1vYPJfvi?= =?us-ascii?Q?L/VBZk7YYIcTL5Xe3NcSiin5/M3T/xFeWH6pzTqJ540jAW/hJpElzB+csNo0?= =?us-ascii?Q?7R5re/MSVSUHdTM27RYXEjtHcCxYIG83beruTAZnuCpp1fc0RlTFchqXHox+?= =?us-ascii?Q?OciQGok4B73DemEvVGRwcfWJOfbf7/0qUI9lhEaUGXUBOqLbtPM1RXvYe6qs?= =?us-ascii?Q?aVh+ZR7ga4zDJ8kt5NANelgHjoVykYBK2s+iLAEnIWcO/iP6G25fjLY6CeDP?= =?us-ascii?Q?chGLH1YSK5mzB6iEge3ue5CxNxGXcCarUGblB0XJJ/yaqF6JzvzIzAySqPpZ?= =?us-ascii?Q?9xbtCsPw2VK0+abTb01SQ6NW3Ojzdfcsv5PrtYsHTYXDfQ/XZSUoSoFMiRLM?= =?us-ascii?Q?hAGMIb43uYcQu4CVJwV6D8aKwYoOCCpy6sVRFpyNq1S0MOBK8dYJF1J7me+2?= =?us-ascii?Q?NUXxopyy5KwT3tZtlPLgO/qz3geq3ZjCPqUBKbPFDzs9YPe8RM6AT+wPajNd?= =?us-ascii?Q?MW8XjOGvcHMneS0QpFvWuvFG0jruceOB68oEOAyb?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0220cc6e-171a-4e31-ccbd-08db602695b7 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2023 09:25:03.9855 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: qudiT7nQTUR/49eA0XKoF+JXHp9rzNT6wimSdJ3MEhYQtTeCiRFrB64HS2Tv0zHFlvONsAfEBQhP0pz+PhHHcQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB8141 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Li, Miao > Sent: Friday, May 26, 2023 12:31 AM > To: dev@dpdk.org > Cc: skori@marvell.com; thomas@monjalon.net; david.marchand@redhat.com; > ferruh.yigit@amd.com; Xia, Chenbo ; Cao, Yahui > ; Burakov, Anatoly > Subject: [PATCH v3 4/4] bus/pci: add VFIO sparse mmap support >=20 > This patch adds sparse mmap support in PCI bus. Sparse mmap is a > capability defined in VFIO which allows multiple mmap areas in one > VFIO region. >=20 > In this patch, the sparse mmap regions are mapped to one continuous > virtual address region that follows device-specific BAR layout. So, > driver can still access all mapped sparse mmap regions by using > 'bar_base_address + bar_offset'. >=20 > Signed-off-by: Miao Li > Signed-off-by: Chenbo Xia > --- > drivers/bus/pci/linux/pci_vfio.c | 104 +++++++++++++++++++++++++++---- > drivers/bus/pci/private.h | 2 + > 2 files changed, 94 insertions(+), 12 deletions(-) >=20 > diff --git a/drivers/bus/pci/linux/pci_vfio.c > b/drivers/bus/pci/linux/pci_vfio.c > index 24b0795fbd..c411909976 100644 > --- a/drivers/bus/pci/linux/pci_vfio.c > +++ b/drivers/bus/pci/linux/pci_vfio.c > @@ -673,6 +673,54 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct > mapped_pci_resource *vfio_res, > return 0; > } >=20 > +static int > +pci_vfio_sparse_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource > *vfio_res, > + int bar_index, int additional_flags) > +{ > + struct pci_map *bar =3D &vfio_res->maps[bar_index]; > + struct vfio_region_sparse_mmap_area *sparse; > + void *bar_addr; > + uint32_t i; > + > + if (bar->size =3D=3D 0) { > + RTE_LOG(DEBUG, EAL, "Bar size is 0, skip BAR%d\n", bar_index); > + return 0; > + } > + > + /* reserve the address using an inaccessible mapping */ > + bar_addr =3D mmap(bar->addr, bar->size, 0, MAP_PRIVATE | > + MAP_ANONYMOUS | additional_flags, -1, 0); > + if (bar_addr !=3D MAP_FAILED) { > + void *map_addr =3D NULL; > + for (i =3D 0; i < bar->nr_areas; i++) { > + sparse =3D &bar->areas[i]; > + if (sparse->size) { > + void *addr =3D RTE_PTR_ADD(bar_addr, > (uintptr_t)sparse->offset); > + map_addr =3D pci_map_resource(addr, vfio_dev_fd, > + bar->offset + sparse->offset, sparse->size, > + RTE_MAP_FORCE_ADDRESS); > + if (map_addr =3D=3D NULL) { > + munmap(bar_addr, bar->size); > + RTE_LOG(ERR, EAL, "Failed to map pci > BAR%d\n", > + bar_index); > + goto err_map; > + } > + } > + } > + } else { > + RTE_LOG(ERR, EAL, "Failed to create inaccessible mapping for > BAR%d\n", > + bar_index); > + goto err_map; > + } > + > + bar->addr =3D bar_addr; > + return 0; > + > +err_map: > + bar->nr_areas =3D 0; > + return -1; > +} > + > /* > * region info may contain capability headers, so we need to keep > reallocating > * the memory until we match allocated memory size with argsz. > @@ -875,6 +923,8 @@ pci_vfio_map_resource_primary(struct rte_pci_device > *dev) >=20 > for (i =3D 0; i < vfio_res->nb_maps; i++) { > void *bar_addr; > + struct vfio_info_cap_header *hdr; > + struct vfio_region_info_cap_sparse_mmap *sparse; >=20 > ret =3D pci_vfio_get_region_info(vfio_dev_fd, ®, i); > if (ret < 0) { > @@ -920,12 +970,33 @@ pci_vfio_map_resource_primary(struct rte_pci_device > *dev) > maps[i].size =3D reg->size; > maps[i].path =3D NULL; /* vfio doesn't have per-resource paths > */ >=20 > - ret =3D pci_vfio_mmap_bar(vfio_dev_fd, vfio_res, i, 0); > - if (ret < 0) { > - RTE_LOG(ERR, EAL, "%s mapping BAR%i failed: %s\n", > - pci_addr, i, strerror(errno)); > - free(reg); > - goto err_vfio_res; > + hdr =3D pci_vfio_info_cap(reg, VFIO_REGION_INFO_CAP_SPARSE_MMAP); > + > + if (hdr !=3D NULL) { > + sparse =3D container_of(hdr, > + struct vfio_region_info_cap_sparse_mmap, header); > + if (sparse->nr_areas > 0) { > + maps[i].nr_areas =3D sparse->nr_areas; > + maps[i].areas =3D sparse->areas; I just notice that this is wrong as the memory that pointer 'sparse' points= to will be freed at the end. map[i].areas needs to be allocated by rte_zmalloc and freed correctly. Otherwise it could leads to secondary process segfault when it tries to access maps[i].areas. Will fix this in v4. Thanks, Chenbo > + } > + } > + > + if (maps[i].nr_areas > 0) { > + ret =3D pci_vfio_sparse_mmap_bar(vfio_dev_fd, vfio_res, i, > 0); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "%s sparse mapping BAR%i > failed: %s\n", > + pci_addr, i, strerror(errno)); > + free(reg); > + goto err_vfio_res; > + } > + } else { > + ret =3D pci_vfio_mmap_bar(vfio_dev_fd, vfio_res, i, 0); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "%s mapping BAR%i failed: %s\n", > + pci_addr, i, strerror(errno)); > + free(reg); > + goto err_vfio_res; > + } > } >=20 > dev->mem_resource[i].addr =3D maps[i].addr; > @@ -1008,11 +1079,20 @@ pci_vfio_map_resource_secondary(struct > rte_pci_device *dev) > maps =3D vfio_res->maps; >=20 > for (i =3D 0; i < vfio_res->nb_maps; i++) { > - ret =3D pci_vfio_mmap_bar(vfio_dev_fd, vfio_res, i, MAP_FIXED); > - if (ret < 0) { > - RTE_LOG(ERR, EAL, "%s mapping BAR%i failed: %s\n", > - pci_addr, i, strerror(errno)); > - goto err_vfio_dev_fd; > + if (maps[i].nr_areas > 0) { > + ret =3D pci_vfio_sparse_mmap_bar(vfio_dev_fd, vfio_res, i, > 0); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "%s sparse mapping BAR%i > failed: %s\n", > + pci_addr, i, strerror(errno)); > + goto err_vfio_dev_fd; > + } > + } else { > + ret =3D pci_vfio_mmap_bar(vfio_dev_fd, vfio_res, i, 0); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "%s mapping BAR%i failed: %s\n", > + pci_addr, i, strerror(errno)); > + goto err_vfio_dev_fd; > + } > } >=20 > dev->mem_resource[i].addr =3D maps[i].addr; > @@ -1062,7 +1142,7 @@ find_and_unmap_vfio_resource(struct > mapped_pci_res_list *vfio_res_list, > break; > } >=20 > - if (vfio_res =3D=3D NULL) > + if (vfio_res =3D=3D NULL) > return vfio_res; >=20 > RTE_LOG(INFO, EAL, "Releasing PCI mapped resource for %s\n", > diff --git a/drivers/bus/pci/private.h b/drivers/bus/pci/private.h > index 2d6991ccb7..8b0ce73533 100644 > --- a/drivers/bus/pci/private.h > +++ b/drivers/bus/pci/private.h > @@ -121,6 +121,8 @@ struct pci_map { > uint64_t offset; > uint64_t size; > uint64_t phaddr; > + uint32_t nr_areas; > + struct vfio_region_sparse_mmap_area *areas; > }; >=20 > struct pci_msix_table { > -- > 2.25.1