[dpdk-dev] [PATCH] eal/vfio: fix sPAPR IOMMU mapping

[dpdk-dev] [PATCH] eal/vfio: fix sPAPR IOMMU mapping

[Takeshi Yoshimura]

Commit 73a639085938 ("vfio: allow to map other memory regions")
introduced a bug in sPAPR IOMMU mapping. The commit removed necessary
ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also, vfio_spapr_map_walk
should call vfio_spapr_dma_do_map instead of vfio_spapr_dma_mem_map.

Fixes: 73a639085938 ("vfio: allow to map other memory regions")

Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
---
 lib/librte_eal/linuxapp/eal/eal_vfio.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c
index c68dc38e0..68e862946 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c
@@ -1145,8 +1145,22 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
 	struct vfio_iommu_type1_dma_map dma_map;
 	struct vfio_iommu_type1_dma_unmap dma_unmap;
 	int ret;
+	struct vfio_iommu_spapr_register_memory reg = {
+		.argsz = sizeof(reg),
+		.flags = 0
+	};
+	reg.vaddr = (uintptr_t) vaddr;
+	reg.size = len;
 
 	if (do_map != 0) {
+		ret = ioctl(vfio_container_fd,
+				VFIO_IOMMU_SPAPR_REGISTER_MEMORY, &reg);
+		if (ret) {
+			RTE_LOG(ERR, EAL, "  cannot register vaddr for IOMMU, "
+				"error %i (%s)\n", errno, strerror(errno));
+			return -1;
+		}
+
 		memset(&dma_map, 0, sizeof(dma_map));
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = vaddr;
@@ -1163,13 +1177,6 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
 		}
 
 	} else {
-		struct vfio_iommu_spapr_register_memory reg = {
-			.argsz = sizeof(reg),
-			.flags = 0
-		};
-		reg.vaddr = (uintptr_t) vaddr;
-		reg.size = len;
-
 		ret = ioctl(vfio_container_fd,
 				VFIO_IOMMU_SPAPR_UNREGISTER_MEMORY, &reg);
 		if (ret) {
@@ -1201,7 +1208,7 @@ vfio_spapr_map_walk(const struct rte_memseg_list *msl __rte_unused,
 {
 	int *vfio_container_fd = arg;
 
-	return vfio_spapr_dma_mem_map(*vfio_container_fd, ms->addr_64, ms->iova,
+	return vfio_spapr_dma_do_map(*vfio_container_fd, ms->addr_64, ms->iova,
 			ms->len, 1);
 }
 
-- 
2.15.1

Re: [dpdk-dev] [PATCH] eal/vfio: fix sPAPR IOMMU mapping

[Thomas Monjalon]

06/08/2018 10:43, Takeshi Yoshimura:
> Commit 73a639085938 ("vfio: allow to map other memory regions")
> introduced a bug in sPAPR IOMMU mapping. The commit removed necessary
> ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also, vfio_spapr_map_walk
> should call vfio_spapr_dma_do_map instead of vfio_spapr_dma_mem_map.

This is fixing an old patch:
	http://git.dpdk.org/dpdk/commit/?id=73a639085938

> Fixes: 73a639085938 ("vfio: allow to map other memory regions")

You probably want it to be backported in previous release,
so you need to add Cc: stable@dpdk.org

> Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>

It is common to have bugs in sPAPR implementation.
How can we be sure this one is OK?
Should it be added in last minute of 18.08?
Or can it wait 18.11?

Adding Anatoly and Chao who are maintainers to decide.

Re: [dpdk-dev] [PATCH] eal/vfio: fix sPAPR IOMMU mapping

[Takeshi T Yoshimura]

>06/08/2018 10:43, Takeshi Yoshimura:
>> Commit 73a639085938 ("vfio: allow to map other memory regions")
>> introduced a bug in sPAPR IOMMU mapping. The commit removed
>necessary
>> ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also,
>vfio_spapr_map_walk
>> should call vfio_spapr_dma_do_map instead of
>vfio_spapr_dma_mem_map.
>
>This is fixing an old patch:
>	INVALID URI REMOVED
>k_commit_-3Fid-3D73a639085938&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=EZR
>6Jx10q0q3dTopeH3WIQ&m=JqmiDyEAYS0sG-jH6aSYikeyXYLr12jnaWDUpBN1mgI&s=7
>bvq2dyXbquauFqqhuOTPVOpoqEPKg8MoncacU4OdJU&e=
>
>> Fixes: 73a639085938 ("vfio: allow to map other memory regions")
>
>You probably want it to be backported in previous release,
>so you need to add Cc: stable@dpdk.org
>
>> Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
>
>It is common to have bugs in sPAPR implementation.
>How can we be sure this one is OK?
>Should it be added in last minute of 18.08?
>Or can it wait 18.11?
>
>Adding Anatoly and Chao who are maintainers to decide.
>

OK. I will resend the patch with Cc: stable@dpdk.org.

[dpdk-dev] [PATCH] eal/vfio: fix sPAPR IOMMU mapping

[Takeshi Yoshimura]

Commit 73a639085938 ("vfio: allow to map other memory regions")
introduced a bug in sPAPR IOMMU mapping. The commit removed necessary
ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also, vfio_spapr_map_walk
should call vfio_spapr_dma_do_map instead of vfio_spapr_dma_mem_map.

Fixes: 73a639085938 ("vfio: allow to map other memory regions")

Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
---
 lib/librte_eal/linuxapp/eal/eal_vfio.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c
index c68dc38e0..68e862946 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c
@@ -1145,8 +1145,22 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
 	struct vfio_iommu_type1_dma_map dma_map;
 	struct vfio_iommu_type1_dma_unmap dma_unmap;
 	int ret;
+	struct vfio_iommu_spapr_register_memory reg = {
+		.argsz = sizeof(reg),
+		.flags = 0
+	};
+	reg.vaddr = (uintptr_t) vaddr;
+	reg.size = len;
 
 	if (do_map != 0) {
+		ret = ioctl(vfio_container_fd,
+				VFIO_IOMMU_SPAPR_REGISTER_MEMORY, &reg);
+		if (ret) {
+			RTE_LOG(ERR, EAL, "  cannot register vaddr for IOMMU, "
+				"error %i (%s)\n", errno, strerror(errno));
+			return -1;
+		}
+
 		memset(&dma_map, 0, sizeof(dma_map));
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = vaddr;
@@ -1163,13 +1177,6 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
 		}
 
 	} else {
-		struct vfio_iommu_spapr_register_memory reg = {
-			.argsz = sizeof(reg),
-			.flags = 0
-		};
-		reg.vaddr = (uintptr_t) vaddr;
-		reg.size = len;
-
 		ret = ioctl(vfio_container_fd,
 				VFIO_IOMMU_SPAPR_UNREGISTER_MEMORY, &reg);
 		if (ret) {
@@ -1201,7 +1208,7 @@ vfio_spapr_map_walk(const struct rte_memseg_list *msl __rte_unused,
 {
 	int *vfio_container_fd = arg;
 
-	return vfio_spapr_dma_mem_map(*vfio_container_fd, ms->addr_64, ms->iova,
+	return vfio_spapr_dma_do_map(*vfio_container_fd, ms->addr_64, ms->iova,
 			ms->len, 1);
 }
 
-- 
2.15.1

[dpdk-dev] [PATCH v2] eal/vfio: fix sPAPR IOMMU mapping

[Takeshi Yoshimura]

Commit 73a639085938 ("vfio: allow to map other memory regions")
introduced a bug in sPAPR IOMMU mapping. The commit removed necessary
ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also, vfio_spapr_map_walk
should call vfio_spapr_dma_do_map instead of vfio_spapr_dma_mem_map.

Fixes: 73a639085938 ("vfio: allow to map other memory regions")

Cc: stable@dpdk.org

Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
---

v2: Added Cc in message

 lib/librte_eal/linuxapp/eal/eal_vfio.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c
index c68dc38e0..68e862946 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c
@@ -1145,8 +1145,22 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
 	struct vfio_iommu_type1_dma_map dma_map;
 	struct vfio_iommu_type1_dma_unmap dma_unmap;
 	int ret;
+	struct vfio_iommu_spapr_register_memory reg = {
+		.argsz = sizeof(reg),
+		.flags = 0
+	};
+	reg.vaddr = (uintptr_t) vaddr;
+	reg.size = len;
 
 	if (do_map != 0) {
+		ret = ioctl(vfio_container_fd,
+				VFIO_IOMMU_SPAPR_REGISTER_MEMORY, &reg);
+		if (ret) {
+			RTE_LOG(ERR, EAL, "  cannot register vaddr for IOMMU, "
+				"error %i (%s)\n", errno, strerror(errno));
+			return -1;
+		}
+
 		memset(&dma_map, 0, sizeof(dma_map));
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = vaddr;
@@ -1163,13 +1177,6 @@ vfio_spapr_dma_do_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova,
 		}
 
 	} else {
-		struct vfio_iommu_spapr_register_memory reg = {
-			.argsz = sizeof(reg),
-			.flags = 0
-		};
-		reg.vaddr = (uintptr_t) vaddr;
-		reg.size = len;
-
 		ret = ioctl(vfio_container_fd,
 				VFIO_IOMMU_SPAPR_UNREGISTER_MEMORY, &reg);
 		if (ret) {
@@ -1201,7 +1208,7 @@ vfio_spapr_map_walk(const struct rte_memseg_list *msl __rte_unused,
 {
 	int *vfio_container_fd = arg;
 
-	return vfio_spapr_dma_mem_map(*vfio_container_fd, ms->addr_64, ms->iova,
+	return vfio_spapr_dma_do_map(*vfio_container_fd, ms->addr_64, ms->iova,
 			ms->len, 1);
 }
 
-- 
2.15.1

Re: [dpdk-dev] [PATCH] eal/vfio: fix sPAPR IOMMU mapping

[Burakov, Anatoly]

On 06-Aug-18 10:50 AM, Thomas Monjalon wrote:
> 06/08/2018 10:43, Takeshi Yoshimura:
>> Commit 73a639085938 ("vfio: allow to map other memory regions")
>> introduced a bug in sPAPR IOMMU mapping. The commit removed necessary
>> ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also, vfio_spapr_map_walk
>> should call vfio_spapr_dma_do_map instead of vfio_spapr_dma_mem_map.
> 
> This is fixing an old patch:
> 	http://git.dpdk.org/dpdk/commit/?id=73a639085938
> 
>> Fixes: 73a639085938 ("vfio: allow to map other memory regions")
> 
> You probably want it to be backported in previous release,
> so you need to add Cc: stable@dpdk.org
> 
>> Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
> 
> It is common to have bugs in sPAPR implementation.
> How can we be sure this one is OK?
> Should it be added in last minute of 18.08?
> Or can it wait 18.11?
> 
> Adding Anatoly and Chao who are maintainers to decide.
> 

The patch appears to be correct - we did have a 
VFIO_IOMMU_SPAPR_REGISTER_MEMORY ioctl in the old map() function, but 
not in the new one.

However, i agree with Thomas - without more testing from other sPAPR 
users/IBM, i would be hesitant to allow it in at the last minute. In any 
case, this code has been there for a while and no one has reported any 
issues - so this can probably wait until 18.11, seeing how this codepath 
is so popular :)

-- 
Thanks,
Anatoly

Re: [dpdk-dev] [PATCH v2] eal/vfio: fix sPAPR IOMMU mapping

[Thomas Monjalon]

07/08/2018 04:35, Takeshi Yoshimura:
> Commit 73a639085938 ("vfio: allow to map other memory regions")
> introduced a bug in sPAPR IOMMU mapping. The commit removed necessary
> ioctl with VFIO_IOMMU_SPAPR_REGISTER_MEMORY. Also, vfio_spapr_map_walk
> should call vfio_spapr_dma_do_map instead of vfio_spapr_dma_mem_map.
> 
> Fixes: 73a639085938 ("vfio: allow to map other memory regions")
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>

We don't have required ack from the PPC maintainer.
But after more than 2 months, we must consider it is OK.

Applied, thanks