From: "Hu, Jiayu" <jiayu.hu@intel.com>
To: "Ding, Xuan" <xuan.ding@intel.com>, "dev@dpdk.org" <dev@dpdk.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>,
"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
"Xia, Chenbo" <chenbo.xia@intel.com>
Cc: "Jiang, Cheng1" <cheng1.jiang@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"Pai G, Sunil" <sunil.pai.g@intel.com>,
"Wang, Yinan" <yinan.wang@intel.com>,
"Yang, YvonneX" <yvonnex.yang@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 2/2] vhost: enable IOMMU for async vhost
Date: Thu, 23 Sep 2021 14:39:03 +0000 [thread overview]
Message-ID: <917d6068d3cc484b95e9e884cc9a4f3b@intel.com> (raw)
In-Reply-To: <20210917052546.23883-3-xuan.ding@intel.com>
Hi Xuan,
> -----Original Message-----
> From: Ding, Xuan <xuan.ding@intel.com>
> Sent: Friday, September 17, 2021 1:26 PM
> To: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>;
> maxime.coquelin@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
> Cc: Hu, Jiayu <jiayu.hu@intel.com>; Jiang, Cheng1 <cheng1.jiang@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Pai G, Sunil
> <sunil.pai.g@intel.com>; Wang, Yinan <yinan.wang@intel.com>; Yang,
> YvonneX <yvonnex.yang@intel.com>; Ding, Xuan <xuan.ding@intel.com>
> Subject: [PATCH v2 2/2] vhost: enable IOMMU for async vhost
>
> The use of IOMMU has many advantages, such as isolation and address
> translation. This patch extends the capbility of DMA engine to use IOMMU if
> the DMA engine is bound to vfio.
>
> When set memory table, the guest memory will be mapped into the default
> container of DPDK.
>
> Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> ---
> lib/vhost/rte_vhost.h | 1 +
> lib/vhost/vhost_user.c | 57
> +++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 57 insertions(+), 1 deletion(-)
>
> diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h index
> 8d875e9322..e0537249f3 100644
> --- a/lib/vhost/rte_vhost.h
> +++ b/lib/vhost/rte_vhost.h
> @@ -127,6 +127,7 @@ struct rte_vhost_mem_region {
> void *mmap_addr;
> uint64_t mmap_size;
> int fd;
> + uint64_t dma_map_success;
How about using bool for dma_map_success?
> };
>
> /**
> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index
> 29a4c9af60..7d1d592b86 100644
> --- a/lib/vhost/vhost_user.c
> +++ b/lib/vhost/vhost_user.c
> @@ -45,6 +45,8 @@
> #include <rte_common.h>
> #include <rte_malloc.h>
> #include <rte_log.h>
> +#include <rte_vfio.h>
> +#include <rte_errno.h>
>
> #include "iotlb.h"
> #include "vhost.h"
> @@ -141,6 +143,46 @@ get_blk_size(int fd)
> return ret == -1 ? (uint64_t)-1 : (uint64_t)stat.st_blksize; }
>
> +static int
> +async_dma_map(struct rte_vhost_mem_region *region, bool do_map) {
> + int ret = 0;
> + uint64_t host_iova;
> + host_iova = rte_mem_virt2iova((void *)(uintptr_t)region-
> >host_user_addr);
> + if (do_map) {
> + /* Add mapped region into the default container of DPDK. */
> + ret =
> rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
> + region->host_user_addr,
> + host_iova,
> + region->size);
> + region->dma_map_success = ret == 0;
> + if (ret) {
> + if (rte_errno != ENODEV && rte_errno != ENOTSUP) {
> + VHOST_LOG_CONFIG(ERR, "DMA engine map
> failed\n");
> + return ret;
> + }
> + return 0;
Why return 0, if ret is -1 here?
Thanks,
Jiayu
> + }
> + return ret;
> + } else {
> + /* No need to do vfio unmap if the map failed. */
> + if (!region->dma_map_success)
> + return 0;
> +
> + /* Remove mapped region from the default container of
> DPDK. */
> + ret =
> rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> + region->host_user_addr,
> + host_iova,
> + region->size);
> + if (ret) {
> + VHOST_LOG_CONFIG(ERR, "DMA engine unmap
> failed\n");
> + return ret;
> + }
> + region->dma_map_success = 0;
> + }
> + return ret;
> +}
> +
> static void
> free_mem_region(struct virtio_net *dev) { @@ -153,6 +195,9 @@
> free_mem_region(struct virtio_net *dev)
> for (i = 0; i < dev->mem->nregions; i++) {
> reg = &dev->mem->regions[i];
> if (reg->host_user_addr) {
> + if (dev->async_copy && rte_vfio_is_enabled("vfio"))
> + async_dma_map(reg, false);
> +
> munmap(reg->mmap_addr, reg->mmap_size);
> close(reg->fd);
> }
> @@ -1157,6 +1202,7 @@ vhost_user_mmap_region(struct virtio_net *dev,
> uint64_t mmap_size;
> uint64_t alignment;
> int populate;
> + int ret;
>
> /* Check for memory_size + mmap_offset overflow */
> if (mmap_offset >= -region->size) {
> @@ -1210,13 +1256,22 @@ vhost_user_mmap_region(struct virtio_net *dev,
> region->mmap_size = mmap_size;
> region->host_user_addr = (uint64_t)(uintptr_t)mmap_addr +
> mmap_offset;
>
> - if (dev->async_copy)
> + if (dev->async_copy) {
> if (add_guest_pages(dev, region, alignment) < 0) {
> VHOST_LOG_CONFIG(ERR,
> "adding guest pages to region
> failed.\n");
> return -1;
> }
>
> + if (rte_vfio_is_enabled("vfio")) {
> + ret = async_dma_map(region, true);
> + if (ret < 0) {
> + VHOST_LOG_CONFIG(ERR, "Configure
> IOMMU for DMA engine failed\n");
> + return -1;
> + }
> + }
> + }
> +
> VHOST_LOG_CONFIG(INFO,
> "guest memory region size: 0x%" PRIx64 "\n"
> "\t guest physical addr: 0x%" PRIx64 "\n"
> --
> 2.17.1
next prev parent reply other threads:[~2021-09-23 14:39 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-01 5:30 [dpdk-dev] [PATCH 0/2] *** support IOMMU for DMA device *** Xuan Ding
2021-09-01 5:30 ` [dpdk-dev] [PATCH 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-01 5:30 ` [dpdk-dev] [PATCH 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-17 5:25 ` [dpdk-dev] [PATCH v2 0/2] support IOMMU for DMA device Xuan Ding
2021-09-17 5:25 ` [dpdk-dev] [PATCH v2 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-17 5:25 ` [dpdk-dev] [PATCH v2 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-23 14:39 ` Hu, Jiayu [this message]
2021-09-23 14:56 ` Maxime Coquelin
2021-09-24 1:53 ` Xia, Chenbo
2021-09-24 7:13 ` Maxime Coquelin
2021-09-24 7:35 ` Xia, Chenbo
2021-09-24 8:18 ` Ding, Xuan
2021-09-25 10:03 ` [dpdk-dev] [PATCH v3 0/2] support IOMMU for DMA device Xuan Ding
2021-09-25 10:03 ` [dpdk-dev] [PATCH v3 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-25 10:03 ` [dpdk-dev] [PATCH v3 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-27 4:17 ` Hu, Jiayu
2021-09-27 4:55 ` Ding, Xuan
2021-09-25 10:33 ` [dpdk-dev] [PATCH v4 0/2] support IOMMU for DMA device Xuan Ding
2021-09-25 10:33 ` [dpdk-dev] [PATCH v4 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-25 10:33 ` [dpdk-dev] [PATCH v4 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-27 7:48 ` [dpdk-dev] [PATCH v5 0/2] support IOMMU for DMA device Xuan Ding
2021-09-27 7:48 ` [dpdk-dev] [PATCH v5 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-27 7:48 ` [dpdk-dev] [PATCH v5 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-27 12:13 ` Burakov, Anatoly
2021-09-28 9:03 ` Ding, Xuan
2021-09-29 2:41 ` [dpdk-dev] [PATCH v6 0/2] support IOMMU for DMA device Xuan Ding
2021-09-29 2:41 ` [dpdk-dev] [PATCH v6 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-29 2:41 ` [dpdk-dev] [PATCH v6 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-29 6:12 ` Hu, Jiayu
2021-09-29 9:39 ` Burakov, Anatoly
2021-09-30 5:17 ` Hu, Jiayu
2021-09-30 5:19 ` Hu, Jiayu
2021-10-11 7:59 ` [dpdk-dev] [PATCH v7 0/2] Support IOMMU for DMA device Xuan Ding
2021-10-11 7:59 ` [dpdk-dev] [PATCH v7 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-10-13 6:57 ` Yang, YvonneX
2021-10-21 9:50 ` Maxime Coquelin
2021-10-11 7:59 ` [dpdk-dev] [PATCH v7 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-10-13 6:57 ` Yang, YvonneX
2021-10-21 10:00 ` Maxime Coquelin
2021-10-21 12:33 ` [dpdk-dev] [PATCH v7 0/2] Support IOMMU for DMA device Maxime Coquelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=917d6068d3cc484b95e9e884cc9a4f3b@intel.com \
--to=jiayu.hu@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=chenbo.xia@intel.com \
--cc=cheng1.jiang@intel.com \
--cc=dev@dpdk.org \
--cc=maxime.coquelin@redhat.com \
--cc=sunil.pai.g@intel.com \
--cc=xuan.ding@intel.com \
--cc=yinan.wang@intel.com \
--cc=yvonnex.yang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).