From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2DBAA059B; Fri, 10 Apr 2020 09:28:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6BDBB1C2E9; Fri, 10 Apr 2020 09:28:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 5C0BA1C2E7 for ; Fri, 10 Apr 2020 09:28:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03A7P5Nh015559; Fri, 10 Apr 2020 00:28:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=i/KOB4nKjL/zJAuv925ct7cZrRtscV5zrjr9cwVEE4Q=; b=BFJsKp4XNQ6S0YNkIhQNCfA10pCL1dVHrVR4jY4+t5Q3ZCtN8yDhy/sHsA7yBgh+Ljek sXAgH8lsfh5sr4d7fNunpZpc3b2zOAdJ4E3zza0KeMTKVa/i6OZf5F4QUjiSaf1TJb5M e8ddG0wlu/04cWtS+3+8yhAQ2IWsXReeS2+u3/DTxMmTtw4g5b2P7gl2jcjgKev67yOg xOvyxqZmwl6u6MwGSItGRaU+ApTOU4W96g3A0cRPZgODw+rB2i0vNwghX12Tzh3tCz/3 lwDbZUKkwxweuKG4SokYSfS7l9i1KDk0bqTYLsUAkq5rXj5on3khyZeNBkHtSn9kEmW6 Mg== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 3091mec6aj-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 10 Apr 2020 00:28:33 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 10 Apr 2020 00:28:31 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.40) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Fri, 10 Apr 2020 00:28:30 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OhkvClttNZp3ogvtzGV4qopRhTiKnH4eAeGLrASf0mZ/z0eQMqXawcbdgAoulxHPzjsyrxLJ9flcbq4RFwUzz/INZXd0jmUMmDUir84NuelcDtl4v4WnptPS0p5L+L+x8++MSoDgPmVk/Ci9ei2StiD53hpZme0vTNmrQ84CKtw8WGGHfGThsazGNRejjF9et+hT4AJaQBHSva02KDL+fK1uY0Jpgwhd2k/iCabFOqSuqx6qCIR6dX5FobdKV/OIF7GDp4QO2zysL4JZ9IzAZf8eSz6saaHgHD8WwinoVDUN8ntHn0z/3orJRyuDVKepRj5oVm5ztUpWgkgBLLCFHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i/KOB4nKjL/zJAuv925ct7cZrRtscV5zrjr9cwVEE4Q=; b=nfUCNKqO6mOpLFe1YuJmklxToqFzVIcLc9cnJRCGO7O7O9ne7BnFkGrjrVM4fnHd6uW05d9AOJI+7+KDfw2pKuwt4Vv0B+7P+SwjSzOIeqej0Zhr8KtxbPbPKwB3awKUmd8b2IZBu/oQUtjyCfqBszdsZxFQyJPxBB+83/WDPncAX0b4kKQBtyZaiRswrLA9sT7qb3FGZK5YaFqXG5nMF7Dmi1kc6BzihXcOkVJ1KPZfYru/zvCFkkdVjM8XbMkNlXdBrPzK933ivcPuWt0/Z7/j6mngDjDOoDfBNnqNMqvC+dAD7xAsIkkVNTMZEcka2o6AAPlPzUk8bbCwjECNHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i/KOB4nKjL/zJAuv925ct7cZrRtscV5zrjr9cwVEE4Q=; b=iXhezKMjLlwN6Ae7YPVltAs2oGRK4H7DpqMA/u3/73YvtrnK1nvHsKXsuTxchxn1U6lRBVkfmgIi5gbNV7vgWPBk6WH6BPmIZYdRxxBhWZGhwfbgUgMvIXVL2UJeQzU9wDl5vxIIWKrRrf9gBH2UZXWR3xNjlpEfxC0qZp6gITE= Received: from MWHPR18MB1645.namprd18.prod.outlook.com (2603:10b6:300:ca::9) by MWHPR18MB1552.namprd18.prod.outlook.com (2603:10b6:300:d2::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.16; Fri, 10 Apr 2020 07:28:29 +0000 Received: from MWHPR18MB1645.namprd18.prod.outlook.com ([fe80::34df:bd9:ffe5:ff77]) by MWHPR18MB1645.namprd18.prod.outlook.com ([fe80::34df:bd9:ffe5:ff77%8]) with mapi id 15.20.2878.023; Fri, 10 Apr 2020 07:28:29 +0000 From: Vamsi Krishna Attunuru To: "Wang, Haiyue" , "dev@dpdk.org" CC: Jerin Jacob Kollanukkaran , "alex.williamson@redhat.com" , "thomas@monjalon.net" , "david.marchand@redhat.com" Thread-Topic: [dpdk-dev] [RFC v1 1/1] vfio: set vf token and gain vf device access Thread-Index: AQHV8qc6xT0zyxqXgE6lUpUnglKd+ahl8zcAgAw5pfA= Date: Fri, 10 Apr 2020 07:28:29 +0000 Message-ID: References: <20200305043311.17065-1-vattunuru@marvell.com> <28a7b43ac5704c9781a035ab4a14d352@intel.com> In-Reply-To: <28a7b43ac5704c9781a035ab4a14d352@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [157.47.119.179] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dd6a97c0-53dd-482e-9148-08d7dd20c3ef x-ms-traffictypediagnostic: MWHPR18MB1552: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:494; x-forefront-prvs: 0369E8196C x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR18MB1645.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(136003)(396003)(346002)(39860400002)(366004)(5660300002)(2906002)(81156014)(30864003)(4326008)(316002)(55016002)(19627235002)(71200400001)(86362001)(66946007)(76116006)(7696005)(8676002)(966005)(478600001)(6506007)(26005)(110136005)(66476007)(66446008)(8936002)(53546011)(64756008)(33656002)(66556008)(52536014)(54906003)(9686003)(186003); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4G/Vi/KK3r7WLTxg9fID3xdXoYO/WWATJIMzcDyfWR+qVh/Eg7cCc7NT8W++Y0eTNiw5fD9yDUjSIDB2k5hrYLP4iKL7voj86wRBlkDZ4YBjNYZ5D+chDTvAjv/spfchGbjQOYpKzCGvDWuCDFj24gNIzbinPJfAE5HYv6ryfCwssCsATzxf9X6IQ/NvTzg5fExcAPBe+I1ggIplGuZ91wfD7hsjJRkdo2nCAeAvkH67WdEqiLku2tXbmHki1RyioB/jBDJW8V3hrG4gRMDlbMj+psd5TNuz55dKArNg3Qd3QI27J6d+67lz3dP/GZlkjshiy45foBGaP07TUEiWjErXcM2+8YzhXJSHZ19iAX2jA1pGT2fm1CE9dxg9P3sQ133qNAm1zDqjzBO89q2BUHOb3teVLz3iJZckRN4C+vEd6Q2wF9pleof1Wlbr6HrHT7Z6yjL9TdHVf8IS6tf5TovpMqbcE7otRZwPVNQj3U2tXdp6udkF1cloGnKz7R6CaI80sp231wI/Xp9RkLmpfg== x-ms-exchange-antispam-messagedata: k+SI7/PRuBHbJkvPy5tHg76PKSiaYLnNAKdQhU4OfIakUhv5y9ppf2wT0KOMLCTjoqKk3GqbfDrky16Gj7/gGvi2FEwVizo4N1MLrJLdFP/WJyKkIq+gKkG+C+FET5Kx9pR1mPMnd8yJWoIRmC68vA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: dd6a97c0-53dd-482e-9148-08d7dd20c3ef X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2020 07:28:29.0904 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8gQ3p3MDh9fnm9DMiCOYw1QH7iBd8silmVVMTc0bbEiaqxUf/f+NaLAwChXtilaQT9E2Vs6GnxI/3M61CAzRTg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1552 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-10_02:2020-04-07, 2020-04-10 signatures=0 Subject: Re: [dpdk-dev] [RFC v1 1/1] vfio: set vf token and gain vf device access X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Wang, > -----Original Message----- > From: dev On Behalf Of Wang, Haiyue > Sent: Thursday, April 2, 2020 6:13 PM > To: Vamsi Krishna Attunuru ; dev@dpdk.org > Cc: Jerin Jacob Kollanukkaran ; > alex.williamson@redhat.com; thomas@monjalon.net; > david.marchand@redhat.com > Subject: Re: [dpdk-dev] [RFC v1 1/1] vfio: set vf token and gain vf devic= e > access >=20 > Hi A Vamsi, >=20 > > -----Original Message----- > > From: dev On Behalf Of vattunuru@marvell.com > > Sent: Thursday, March 5, 2020 12:33 > > To: dev@dpdk.org > > Cc: jerinj@marvell.com; alex.williamson@redhat.com; > > thomas@monjalon.net; david.marchand@redhat.com; Vamsi Attunuru > > > > Subject: [dpdk-dev] [RFC v1 1/1] vfio: set vf token and gain vf device > > access > > > > From: Vamsi Attunuru > > > > vfio-pci driver enables virtual function access from the DPDK > > applications when those vf device's physical function is also bound to > > vfio driver. > > > > Patch adds the required configuration and checks to enable DPDK > > applications to access both pf and it's vf devices through vfio-pci > > driver. > > > > See background on vf token scheme in linux vfio driver. > > https://urldefense.proofpoint.com/v2/url?u=3Dhttp-3A__patches.dpdk.org_= c > > > over_65915_&d=3DDwIFAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3D2rpxxNF2qeP02g > VZIWTVrW > > -6zNZz5- > uKt9pRqpR_M3U&m=3D1z1lwkWphoVghrYUNLI304mER44Vei152GYrXBwn4kk& > s=3D > > tvJ4Li8zpLMbp_p3vKQL7QV09DttNcl70YAr6BVHZm8&e=3D > > > > When a physical function is enabled with non-zero virtual functions, > > patch sets the UUID using VFIO_DEVICE_FEATURE ioctl from physical > > function's file descriptor. Same UUID is used to gain the access for > > the virtual functions on those physical function. > > > > Following changes required on top of this DPDK patch > > * Kernel version check for VFIO_DEVICE_FEATURE ioctl > > * Use uuid gen API to generate UUID. > > > > Signed-off-by: Vamsi Attunuru > > > > diff --git a/lib/librte_eal/linux/eal/eal_vfio.c > > b/lib/librte_eal/linux/eal/eal_vfio.c > > index 01b5ef3..e2fdd35 100644 > > --- a/lib/librte_eal/linux/eal/eal_vfio.c > > +++ b/lib/librte_eal/linux/eal/eal_vfio.c > > @@ -12,6 +12,7 @@ > > #include > > #include > > #include > > +#include > > #include > > > > #include "eal_filesystem.h" > > @@ -50,6 +51,9 @@ struct vfio_config { > > struct user_mem_maps mem_maps; > > }; > > > > +rte_uuid_t uuid_token =3D RTE_UUID_INIT(0xf8615163, 0xdf3e, 0x46c5, > > + 0x913f, 0xf2d2f965ed0eULL); > > + > > /* per-process VFIO config */ > > static struct vfio_config vfio_cfgs[VFIO_MAX_CONTAINERS]; static > > struct vfio_config *default_vfio_cfg =3D &vfio_cfgs[0]; @@ -657,6 > > +661,102 @@ rte_vfio_clear_group(int vfio_group_fd) > > return 0; > > } > > > > +static bool > > +rte_vfio_dev_is_physfn(const char *sysfs_base, const char *dev_addr) > > +{ > > + char linkname[PATH_MAX]; > > + char filename[PATH_MAX]; > > + int ret; > > + > > + memset(linkname, 0, sizeof(linkname)); > > + memset(filename, 0, sizeof(filename)); > > + > > + /* check if physfn directory exist for this device */ > > + snprintf(linkname, sizeof(linkname), > > + "%s/%s/physfn", sysfs_base, dev_addr); > > + > > + ret =3D readlink(linkname, filename, sizeof(filename)); > > + > > + /* For PFs, physfn directory does not exist */ > > + if (ret < 0) > > + return true; > > + > > + return false; > > +} > > + > > +static int > > +is_vf_token_required(const char *sysfs_base, const char *dev_addr) { > > + char *tok[16], *physfn, *physfn_drv; > > + char linkname[PATH_MAX]; > > + char filename[PATH_MAX]; > > + int ret; > > + > > + memset(linkname, 0, sizeof(linkname)); > > + memset(filename, 0, sizeof(filename)); > > + > > + snprintf(linkname, sizeof(linkname), > > + "%s/%s/physfn", sysfs_base, dev_addr); > > + > > + ret =3D readlink(linkname, filename, sizeof(filename)); > > + if (ret < 0) > > + return -1; > > + > > + ret =3D rte_strsplit(filename, sizeof(filename), > > + tok, RTE_DIM(tok), '/'); > > + if (ret <=3D 0) { > > + RTE_LOG(ERR, EAL, " %s cannot get it's physfn\n", dev_addr); > > + return -1; > > + } > > + > > + physfn =3D tok[ret - 1]; > > + > > + snprintf(linkname, sizeof(linkname), > > + "/sys/bus/pci/devices/%s/driver", physfn); > > + ret =3D readlink(linkname, filename, sizeof(filename)); > > + if (ret < 0) > > + return -1; > > + > > + ret =3D rte_strsplit(filename, sizeof(filename), > > + tok, RTE_DIM(tok), '/'); > > + if (ret <=3D 0) { > > + RTE_LOG(ERR, EAL, " %s cannot get it's physfn driver info\n", > > + dev_addr); > > + return -1; > > + } > > + > > + physfn_drv =3D tok[ret - 1]; > > + > > + if (strncmp(physfn_drv, "vfio-pci", sizeof("vfio-pci"))) > > + return 1; > > + > > + /* physfn is bound to vfio-pci */ > > + return 0; > > +} > > + >=20 > Based on Alex's vfio for qemu patch > (https://urldefense.proofpoint.com/v2/url?u=3Dhttps- > 3A__lore.kernel.org_lkml_20200204161737.34696b91- > 40w520.home_&d=3DDwIFAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3D2rpxxNF2qeP0 > 2gVZIWTVrW-6zNZz5- > uKt9pRqpR_M3U&m=3D1z1lwkWphoVghrYUNLI304mER44Vei152GYrXBwn4kk& > s=3D0pgRda84NrL2omZa0mn-ufSEzKksSi440O2AYImZLaI&e=3D ), and understand > the SR-IOV design by reading your RFC, I scratched a simple design. Thanks, looks good to me, I guess now it serves the QEMU based use cases as= well. >=20 > 1. ./usertools/dpdk-devbind.py -b vfio-pci 0000:87:00.0 2. echo 2 > > /sys/bus/pci/devices/0000:87:00.0/sriov_numvfs > 3. ./x86_64-native-linux-gcc/app/testpmd -l 22-25 -n 4 -w > 87:00.0,vf_token=3D2ab74924-c335-45f4-9b16-8569e5b08258 --file-prefix=3Dp= f -- > -i >=20 > If no token in VF: > ./x86_64-native-linux-gcc/app/testpmd -l 26-29 -n 4 -w 87:02.0 --file- > prefix=3Dvf1 -- -i > Kernel error: vfio-pci 0000:87:02.0: VF token required to access device >=20 > ./x86_64-native-linux-gcc/app/testpmd -l 26-29 -n 4 -w > 87:02.0,vf_token=3D2ab74924-c335-45f4-9b16-8569e5b08258 --file-prefix=3Dv= f1 - > - -i >=20 >=20 > static int > vfio_pci_vf_token_arg_handler(__rte_unused const char *key, > const char *value, void *opaque) { > if (rte_uuid_parse(value, opaque)) { > RTE_LOG(ERR, EAL, > "The VF token is not a valid uuid : %s\n", value); > return -1; > } >=20 > return 0; > } >=20 > static int > vfio_pci_vf_token_arg(struct rte_devargs *devargs, rte_uuid_t uu) { > const char *key =3D "vf_token"; > struct rte_kvargs *kvlist; > int ret =3D 0; >=20 > if (devargs =3D=3D NULL) > return 0; >=20 > kvlist =3D rte_kvargs_parse(devargs->args, NULL); > if (kvlist =3D=3D NULL) > return 0; >=20 > if (!rte_kvargs_count(kvlist, key)) > goto exit; >=20 > if (rte_kvargs_process(kvlist, key, > vfio_pci_vf_token_arg_handler, uu) < 0) > goto exit; >=20 > ret =3D 1; >=20 > exit: > rte_kvargs_free(kvlist); > return ret; > } >=20 >=20 >=20 > static int > pci_vfio_map_resource_primary(struct rte_pci_device *dev) { > struct vfio_device_info device_info =3D { .argsz =3D sizeof(device_info= ) }; > + rte_uuid_t vf_token =3D RTE_UUID_INIT(0, 0, 0, 0, 0ULL); > char pci_addr[PATH_MAX] =3D {0}; > int vfio_dev_fd; > struct rte_pci_addr *loc =3D &dev->addr; @@ -668,8 +712,9 @@ > pci_vfio_map_resource_primary(struct rte_pci_device *dev) > snprintf(pci_addr, sizeof(pci_addr), PCI_PRI_FMT, > loc->domain, loc->bus, loc->devid, loc->function); >=20 > + vfio_pci_vf_token_arg(dev->device.devargs, vf_token); > ret =3D rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr, > - &vfio_dev_fd, &device_info); > + &vfio_dev_fd, &device_info, > vf_token); > if (ret) > return ret; >=20 >=20 > int > rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, > - int *vfio_dev_fd, struct vfio_device_info *device_info) > + int *vfio_dev_fd, struct vfio_device_info *device_info, > + rte_uuid_t vf_token) > { > struct vfio_group_status group_status =3D { > .argsz =3D sizeof(group_status) > @@ -712,6 +713,7 @@ rte_vfio_setup_device(const char *sysfs_base, const > char *dev_addr, > int vfio_container_fd; > int vfio_group_fd; > int iommu_group_num; > + char dev[PATH_MAX]; > int i, ret; >=20 > /* get group number */ > @@ -895,8 +897,19 @@ rte_vfio_setup_device(const char *sysfs_base, > const char *dev_addr, > t->type_id, t->name); > } >=20 > + if (!rte_uuid_is_null(vf_token)) { > + char vf_token_str[PATH_MAX]; > + > + rte_uuid_unparse(vf_token, vf_token_str, > sizeof(vf_token_str)); > + snprintf(dev, sizeof(dev), > + "%s vf_token=3D%s", dev_addr, vf_token_str); > + } else { > + snprintf(dev, sizeof(dev), > + "%s", dev_addr); > + } > + > /* get a file descriptor for the device */ > - *vfio_dev_fd =3D ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, > dev_addr); > + *vfio_dev_fd =3D ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, > dev); >=20 >=20 >=20 > > +static bool > > +rte_vfio_dev_has_nonzero_numvfs(const char *sysfs_base, const char > > +*dev_addr) { > > + char linkname[PATH_MAX]; > > + unsigned long num_vfs; > > + int ret; > > + > > + if (!rte_vfio_dev_is_physfn(sysfs_base, dev_addr)) > > + return false; > > + > > + memset(linkname, 0, sizeof(linkname)); > > + > > + snprintf(linkname, sizeof(linkname), > > + "%s/%s/sriov_numvfs", sysfs_base, dev_addr); > > + > > + ret =3D eal_parse_sysfs_value(linkname, &num_vfs); > > + > > + if ((ret < 0) || (num_vfs =3D=3D 0)) > > + return false; > > + > > + return true; > > +} > > + > > int > > rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr, > > int *vfio_dev_fd, struct vfio_device_info *device_info) @@ - > 669,6 > > +769,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char > *dev_addr, > > int vfio_container_fd; > > int vfio_group_fd; > > int iommu_group_num; > > + char dev[PATH_MAX]; > > int i, ret; > > > > /* get group number */ > > @@ -683,6 +784,29 @@ rte_vfio_setup_device(const char *sysfs_base, > const char *dev_addr, > > if (ret < 0) > > return -1; > > > > + snprintf(dev, sizeof(dev), "%s", dev_addr); > > + > > + if (!rte_vfio_dev_is_physfn(sysfs_base, dev_addr)) { > > + char vf_token[PATH_MAX]; > > + /* > > + * Check if vf_token is required or not, > > + * vf_token is required when the VF's physfn is > > + * binded with vfio-pci driver > > + */ > > + ret =3D is_vf_token_required(sysfs_base, dev_addr); > > + /* if negative, something failed */ > > + if (ret < 0) > > + return -1; > > + > > + if (ret =3D=3D 0) { > > + /* vf_token required to open device file descriptor */ > > + rte_uuid_unparse(uuid_token, > > + vf_token, sizeof(vf_token)); > > + snprintf(dev, sizeof(dev), > > + "%s vf_token=3D%s", dev_addr, vf_token); > > + } > > + } > > + > > /* get the actual group fd */ > > vfio_group_fd =3D rte_vfio_get_group_fd(iommu_group_num); > > if (vfio_group_fd < 0) > > @@ -853,7 +977,7 @@ rte_vfio_setup_device(const char *sysfs_base, > const char *dev_addr, > > } > > > > /* get a file descriptor for the device */ > > - *vfio_dev_fd =3D ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, > dev_addr); > > + *vfio_dev_fd =3D ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, > dev); > > if (*vfio_dev_fd < 0) { > > /* if we cannot get a device fd, this implies a problem with > > * the VFIO group or the container not having IOMMU > configured. > > @@ -877,6 +1001,31 @@ rte_vfio_setup_device(const char *sysfs_base, > const char *dev_addr, > > rte_vfio_clear_group(vfio_group_fd); > > return -1; > > } > > + > > + if (rte_vfio_dev_has_nonzero_numvfs(sysfs_base, dev_addr)) { > #define > > +VF_TOKEN (sizeof(struct vfio_device_feature) + sizeof(rte_uuid_t)) > > + > > + struct vfio_device_feature *vf_token; > > + uint8_t local[VF_TOKEN]; > > + > > + memset(local, 0, VF_TOKEN); > > + vf_token =3D (struct vfio_device_feature *)local; > > + vf_token->argsz =3D VF_TOKEN; > > + vf_token->flags =3D VFIO_DEVICE_FEATURE_SET | > > + > VFIO_DEVICE_FEATURE_PCI_VF_TOKEN; > > + > > + memcpy(local + sizeof(struct vfio_device_feature), > > + &uuid_token, sizeof(uuid_token)); > > + > > + ret =3D ioctl(*vfio_dev_fd, VFIO_DEVICE_FEATURE, vf_token); > > + if (ret) { > > + RTE_LOG(ERR, EAL, " Failed to set UUID on %s " > > + "error %i (%s)\n", dev_addr, errno, > > + strerror(errno)); > > + return -1; > > + } > > + } > > + > > vfio_group_device_get(vfio_group_fd); > > > > return 0; > > -- > > 2.8.4