From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: "Hu, Jiayu" <jiayu.hu@intel.com>,
"Ding, Xuan" <xuan.ding@intel.com>, "dev@dpdk.org" <dev@dpdk.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>,
"Xia, Chenbo" <chenbo.xia@intel.com>
Cc: "Jiang, Cheng1" <cheng1.jiang@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"Pai G, Sunil" <sunil.pai.g@intel.com>,
"Wang, Yinan" <yinan.wang@intel.com>,
"Yang, YvonneX" <yvonnex.yang@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 2/2] vhost: enable IOMMU for async vhost
Date: Thu, 23 Sep 2021 16:56:27 +0200 [thread overview]
Message-ID: <e61db398-02a8-aac7-f1ed-1359fc64c7cd@redhat.com> (raw)
In-Reply-To: <917d6068d3cc484b95e9e884cc9a4f3b@intel.com>
On 9/23/21 16:39, Hu, Jiayu wrote:
> Hi Xuan,
>
>> -----Original Message-----
>> From: Ding, Xuan <xuan.ding@intel.com>
>> Sent: Friday, September 17, 2021 1:26 PM
>> To: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>;
>> maxime.coquelin@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
>> Cc: Hu, Jiayu <jiayu.hu@intel.com>; Jiang, Cheng1 <cheng1.jiang@intel.com>;
>> Richardson, Bruce <bruce.richardson@intel.com>; Pai G, Sunil
>> <sunil.pai.g@intel.com>; Wang, Yinan <yinan.wang@intel.com>; Yang,
>> YvonneX <yvonnex.yang@intel.com>; Ding, Xuan <xuan.ding@intel.com>
>> Subject: [PATCH v2 2/2] vhost: enable IOMMU for async vhost
>>
>> The use of IOMMU has many advantages, such as isolation and address
>> translation. This patch extends the capbility of DMA engine to use IOMMU if
>> the DMA engine is bound to vfio.
>>
>> When set memory table, the guest memory will be mapped into the default
>> container of DPDK.
>>
>> Signed-off-by: Xuan Ding <xuan.ding@intel.com>
>> ---
>> lib/vhost/rte_vhost.h | 1 +
>> lib/vhost/vhost_user.c | 57
>> +++++++++++++++++++++++++++++++++++++++++-
>> 2 files changed, 57 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h index
>> 8d875e9322..e0537249f3 100644
>> --- a/lib/vhost/rte_vhost.h
>> +++ b/lib/vhost/rte_vhost.h
>> @@ -127,6 +127,7 @@ struct rte_vhost_mem_region {
>> void *mmap_addr;
>> uint64_t mmap_size;
>> int fd;
>> + uint64_t dma_map_success;
>
> How about using bool for dma_map_success?
The bigger problem here is that you are breaking the ABI.
>> };
>>
>> /**
>> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index
>> 29a4c9af60..7d1d592b86 100644
>> --- a/lib/vhost/vhost_user.c
>> +++ b/lib/vhost/vhost_user.c
>> @@ -45,6 +45,8 @@
>> #include <rte_common.h>
>> #include <rte_malloc.h>
>> #include <rte_log.h>
>> +#include <rte_vfio.h>
>> +#include <rte_errno.h>
>>
>> #include "iotlb.h"
>> #include "vhost.h"
>> @@ -141,6 +143,46 @@ get_blk_size(int fd)
>> return ret == -1 ? (uint64_t)-1 : (uint64_t)stat.st_blksize; }
>>
>> +static int
>> +async_dma_map(struct rte_vhost_mem_region *region, bool do_map) {
>> + int ret = 0;
>> + uint64_t host_iova;
>> + host_iova = rte_mem_virt2iova((void *)(uintptr_t)region-
>>> host_user_addr);
>> + if (do_map) {
>> + /* Add mapped region into the default container of DPDK. */
>> + ret =
>> rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
>> + region->host_user_addr,
>> + host_iova,
>> + region->size);
>> + region->dma_map_success = ret == 0;
>> + if (ret) {
>> + if (rte_errno != ENODEV && rte_errno != ENOTSUP) {
>> + VHOST_LOG_CONFIG(ERR, "DMA engine map
>> failed\n");
>> + return ret;
>> + }
>> + return 0;
>
> Why return 0, if ret is -1 here?
>
> Thanks,
> Jiayu
>
>> + }
>> + return ret;
>> + } else {
>> + /* No need to do vfio unmap if the map failed. */
>> + if (!region->dma_map_success)
>> + return 0;
>> +
>> + /* Remove mapped region from the default container of
>> DPDK. */
>> + ret =
>> rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
>> + region->host_user_addr,
>> + host_iova,
>> + region->size);
>> + if (ret) {
>> + VHOST_LOG_CONFIG(ERR, "DMA engine unmap
>> failed\n");
>> + return ret;
>> + }
>> + region->dma_map_success = 0;
>> + }
>> + return ret;
>> +}
>> +
>> static void
>> free_mem_region(struct virtio_net *dev) { @@ -153,6 +195,9 @@
>> free_mem_region(struct virtio_net *dev)
>> for (i = 0; i < dev->mem->nregions; i++) {
>> reg = &dev->mem->regions[i];
>> if (reg->host_user_addr) {
>> + if (dev->async_copy && rte_vfio_is_enabled("vfio"))
>> + async_dma_map(reg, false);
>> +
>> munmap(reg->mmap_addr, reg->mmap_size);
>> close(reg->fd);
>> }
>> @@ -1157,6 +1202,7 @@ vhost_user_mmap_region(struct virtio_net *dev,
>> uint64_t mmap_size;
>> uint64_t alignment;
>> int populate;
>> + int ret;
>>
>> /* Check for memory_size + mmap_offset overflow */
>> if (mmap_offset >= -region->size) {
>> @@ -1210,13 +1256,22 @@ vhost_user_mmap_region(struct virtio_net *dev,
>> region->mmap_size = mmap_size;
>> region->host_user_addr = (uint64_t)(uintptr_t)mmap_addr +
>> mmap_offset;
>>
>> - if (dev->async_copy)
>> + if (dev->async_copy) {
>> if (add_guest_pages(dev, region, alignment) < 0) {
>> VHOST_LOG_CONFIG(ERR,
>> "adding guest pages to region
>> failed.\n");
>> return -1;
>> }
>>
>> + if (rte_vfio_is_enabled("vfio")) {
>> + ret = async_dma_map(region, true);
>> + if (ret < 0) {
>> + VHOST_LOG_CONFIG(ERR, "Configure
>> IOMMU for DMA engine failed\n");
>> + return -1;
>> + }
>> + }
>> + }
>> +
>> VHOST_LOG_CONFIG(INFO,
>> "guest memory region size: 0x%" PRIx64 "\n"
>> "\t guest physical addr: 0x%" PRIx64 "\n"
>> --
>> 2.17.1
>
next prev parent reply other threads:[~2021-09-23 14:56 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-01 5:30 [dpdk-dev] [PATCH 0/2] *** support IOMMU for DMA device *** Xuan Ding
2021-09-01 5:30 ` [dpdk-dev] [PATCH 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-01 5:30 ` [dpdk-dev] [PATCH 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-17 5:25 ` [dpdk-dev] [PATCH v2 0/2] support IOMMU for DMA device Xuan Ding
2021-09-17 5:25 ` [dpdk-dev] [PATCH v2 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-17 5:25 ` [dpdk-dev] [PATCH v2 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-23 14:39 ` Hu, Jiayu
2021-09-23 14:56 ` Maxime Coquelin [this message]
2021-09-24 1:53 ` Xia, Chenbo
2021-09-24 7:13 ` Maxime Coquelin
2021-09-24 7:35 ` Xia, Chenbo
2021-09-24 8:18 ` Ding, Xuan
2021-09-25 10:03 ` [dpdk-dev] [PATCH v3 0/2] support IOMMU for DMA device Xuan Ding
2021-09-25 10:03 ` [dpdk-dev] [PATCH v3 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-25 10:03 ` [dpdk-dev] [PATCH v3 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-27 4:17 ` Hu, Jiayu
2021-09-27 4:55 ` Ding, Xuan
2021-09-25 10:33 ` [dpdk-dev] [PATCH v4 0/2] support IOMMU for DMA device Xuan Ding
2021-09-25 10:33 ` [dpdk-dev] [PATCH v4 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-25 10:33 ` [dpdk-dev] [PATCH v4 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-27 7:48 ` [dpdk-dev] [PATCH v5 0/2] support IOMMU for DMA device Xuan Ding
2021-09-27 7:48 ` [dpdk-dev] [PATCH v5 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-27 7:48 ` [dpdk-dev] [PATCH v5 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-27 12:13 ` Burakov, Anatoly
2021-09-28 9:03 ` Ding, Xuan
2021-09-29 2:41 ` [dpdk-dev] [PATCH v6 0/2] support IOMMU for DMA device Xuan Ding
2021-09-29 2:41 ` [dpdk-dev] [PATCH v6 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-09-29 2:41 ` [dpdk-dev] [PATCH v6 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-09-29 6:12 ` Hu, Jiayu
2021-09-29 9:39 ` Burakov, Anatoly
2021-09-30 5:17 ` Hu, Jiayu
2021-09-30 5:19 ` Hu, Jiayu
2021-10-11 7:59 ` [dpdk-dev] [PATCH v7 0/2] Support IOMMU for DMA device Xuan Ding
2021-10-11 7:59 ` [dpdk-dev] [PATCH v7 1/2] vfio: allow partially unmapping adjacent memory Xuan Ding
2021-10-13 6:57 ` Yang, YvonneX
2021-10-21 9:50 ` Maxime Coquelin
2021-10-11 7:59 ` [dpdk-dev] [PATCH v7 2/2] vhost: enable IOMMU for async vhost Xuan Ding
2021-10-13 6:57 ` Yang, YvonneX
2021-10-21 10:00 ` Maxime Coquelin
2021-10-21 12:33 ` [dpdk-dev] [PATCH v7 0/2] Support IOMMU for DMA device Maxime Coquelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e61db398-02a8-aac7-f1ed-1359fc64c7cd@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=chenbo.xia@intel.com \
--cc=cheng1.jiang@intel.com \
--cc=dev@dpdk.org \
--cc=jiayu.hu@intel.com \
--cc=sunil.pai.g@intel.com \
--cc=xuan.ding@intel.com \
--cc=yinan.wang@intel.com \
--cc=yvonnex.yang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).