From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 57D503B5 for ; Fri, 28 Apr 2017 15:26:00 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Apr 2017 06:25:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.37,388,1488873600"; d="scan'208";a="80098817" Received: from irsmsx153.ger.corp.intel.com ([163.33.192.75]) by orsmga002.jf.intel.com with ESMTP; 28 Apr 2017 06:25:58 -0700 Received: from irsmsx109.ger.corp.intel.com ([169.254.13.12]) by IRSMSX153.ger.corp.intel.com ([163.33.192.75]) with mapi id 14.03.0319.002; Fri, 28 Apr 2017 14:25:57 +0100 From: "Burakov, Anatoly" To: Alejandro Lucero , "dev@dpdk.org" Thread-Topic: [PATCH] vfio: fix device unplug when several devices per vfio group Thread-Index: AQHSvnrWz3bhAYyYYEW1xiVW6Yh0waHayKOA Date: Fri, 28 Apr 2017 13:25:57 +0000 Message-ID: References: <1493203787-25555-1-git-send-email-alejandro.lucero@netronome.com> In-Reply-To: <1493203787-25555-1-git-send-email-alejandro.lucero@netronome.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_IC x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOTgwYmQyMDAtMGY4OC00N2MwLWI0MGYtOWU0NmMxOWM3N2QwIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6ImdDSU9JdW1hdFMyQ1VLcFFaUlAwUVRhOUttT3FzSkdFVDhYUFR0aEI2aGM9In0= dlp-product: dlpe-windows dlp-version: 10.0.102.7 dlp-reaction: no-action x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] vfio: fix device unplug when several devices per vfio group X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Apr 2017 13:26:00 -0000 > From: Alejandro Lucero [mailto:alejandro.lucero@netronome.com] > Sent: Wednesday, April 26, 2017 11:50 AM > To: dev@dpdk.org > Cc: Burakov, Anatoly > Subject: [PATCH] vfio: fix device unplug when several devices per vfio gr= oup >=20 > VFIO allows a secure way of assigning devices to user space and those > devices which can not be isolated from other ones are set in same VFIO > group. Releasing or unplugging a device should be aware of remaining > devices is the same group for avoiding to close such a group. >=20 > Fixes: 94c0776b1bad ("vfio: support hotplug") >=20 > Signed-off-by: Alejandro Lucero > --- > lib/librte_eal/linuxapp/eal/eal_vfio.c | 32 ++++++++++++++++++++++++---- > ---- lib/librte_eal/linuxapp/eal/eal_vfio.h | 1 + > 2 files changed, 25 insertions(+), 8 deletions(-) >=20 > diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c > b/lib/librte_eal/linuxapp/eal/eal_vfio.c > index 6e2e84c..6e24273 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c > +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c > @@ -184,6 +184,7 @@ > if (vfio_cfg.vfio_groups[i].fd =3D=3D vfio_group_fd) { > vfio_cfg.vfio_groups[i].group_no =3D -1; > vfio_cfg.vfio_groups[i].fd =3D -1; > + vfio_cfg.vfio_groups[i].devices =3D 0; > vfio_cfg.vfio_active_groups--; > return 0; > } > @@ -353,6 +354,7 @@ > clear_group(vfio_group_fd); > return -1; > } > + vfio_cfg.vfio_groups[vfio_group_fd].devices++; >=20 > return 0; > } > @@ -390,17 +392,30 @@ > * code will unset the container and the IOMMU mappings. > */ >=20 > - if (close(vfio_group_fd) < 0) > - RTE_LOG(INFO, EAL, "Error when closing vfio_group_fd for > %s\n", > - dev_addr); > - > - if (close(vfio_dev_fd) < 0) > + /* Closing a device */ > + if (close(vfio_dev_fd) < 0) { > RTE_LOG(INFO, EAL, "Error when closing vfio_dev_fd for > %s\n", > dev_addr); > + return -1; > + } >=20 > - if (clear_group(vfio_group_fd) < 0) > - RTE_LOG(INFO, EAL, "Error when clearing group for %s\n", > - dev_addr); > + /* An VFIO group can have several devices attached. Just when there > is > + * no devices remaining should the group be closed. > + */ > + if (--vfio_cfg.vfio_groups[vfio_group_fd].devices =3D=3D 0) { > + > + if (close(vfio_group_fd) < 0) { > + RTE_LOG(INFO, EAL, "Error when closing > vfio_group_fd for %s\n", > + dev_addr); > + return -1; > + } > + > + if (clear_group(vfio_group_fd) < 0) { > + RTE_LOG(INFO, EAL, "Error when clearing group for > %s\n", > + dev_addr); > + return -1; > + } > + } >=20 > return 0; > } > @@ -415,6 +430,7 @@ > for (i =3D 0; i < VFIO_MAX_GROUPS; i++) { > vfio_cfg.vfio_groups[i].fd =3D -1; > vfio_cfg.vfio_groups[i].group_no =3D -1; > + vfio_cfg.vfio_groups[i].devices =3D 0; > } >=20 > /* inform the user that we are probing for VFIO */ diff --git > a/lib/librte_eal/linuxapp/eal/eal_vfio.h > b/lib/librte_eal/linuxapp/eal/eal_vfio.h > index 7fcec2c..2c7cb94 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_vfio.h > +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.h > @@ -103,6 +103,7 @@ struct vfio_iommu_spapr_tce_remove { struct > vfio_group { > int group_no; > int fd; > + int devices; > }; >=20 > struct vfio_config { > -- > 1.9.1 I have tested this on my setup on an old kernel with multiple attach/detach= es, and it works (whereas it fails without this patch). Acked-by: Anatoly Burakov