From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0A59FA0C43; Thu, 23 Sep 2021 16:39:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C94E741260; Thu, 23 Sep 2021 16:39:11 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 0861A41257 for ; Thu, 23 Sep 2021 16:39:09 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10115"; a="223887480" X-IronPort-AV: E=Sophos;i="5.85,316,1624345200"; d="scan'208";a="223887480" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Sep 2021 07:39:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,316,1624345200"; d="scan'208";a="513994182" Received: from irsmsx601.ger.corp.intel.com ([163.33.146.7]) by fmsmga008.fm.intel.com with ESMTP; 23 Sep 2021 07:39:08 -0700 Received: from shsmsx606.ccr.corp.intel.com (10.109.6.216) by irsmsx601.ger.corp.intel.com (163.33.146.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Thu, 23 Sep 2021 15:39:06 +0100 Received: from shsmsx606.ccr.corp.intel.com ([10.109.6.216]) by SHSMSX606.ccr.corp.intel.com ([10.109.6.216]) with mapi id 15.01.2242.012; Thu, 23 Sep 2021 22:39:03 +0800 From: "Hu, Jiayu" To: "Ding, Xuan" , "dev@dpdk.org" , "Burakov, Anatoly" , "maxime.coquelin@redhat.com" , "Xia, Chenbo" CC: "Jiang, Cheng1" , "Richardson, Bruce" , "Pai G, Sunil" , "Wang, Yinan" , "Yang, YvonneX" Thread-Topic: [PATCH v2 2/2] vhost: enable IOMMU for async vhost Thread-Index: AQHXq4V+sKikRLZEPkSE0QtjqoXhmauxq18w Date: Thu, 23 Sep 2021 14:39:03 +0000 Message-ID: <917d6068d3cc484b95e9e884cc9a4f3b@intel.com> References: <20210901053044.109901-1-xuan.ding@intel.com> <20210917052546.23883-1-xuan.ding@intel.com> <20210917052546.23883-3-xuan.ding@intel.com> In-Reply-To: <20210917052546.23883-3-xuan.ding@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.6.200.16 dlp-product: dlpe-windows x-originating-ip: [10.239.127.36] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 2/2] vhost: enable IOMMU for async vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Xuan, > -----Original Message----- > From: Ding, Xuan > Sent: Friday, September 17, 2021 1:26 PM > To: dev@dpdk.org; Burakov, Anatoly ; > maxime.coquelin@redhat.com; Xia, Chenbo > Cc: Hu, Jiayu ; Jiang, Cheng1 ; > Richardson, Bruce ; Pai G, Sunil > ; Wang, Yinan ; Yang, > YvonneX ; Ding, Xuan > Subject: [PATCH v2 2/2] vhost: enable IOMMU for async vhost >=20 > The use of IOMMU has many advantages, such as isolation and address > translation. This patch extends the capbility of DMA engine to use IOMMU = if > the DMA engine is bound to vfio. >=20 > When set memory table, the guest memory will be mapped into the default > container of DPDK. >=20 > Signed-off-by: Xuan Ding > --- > lib/vhost/rte_vhost.h | 1 + > lib/vhost/vhost_user.c | 57 > +++++++++++++++++++++++++++++++++++++++++- > 2 files changed, 57 insertions(+), 1 deletion(-) >=20 > diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h index > 8d875e9322..e0537249f3 100644 > --- a/lib/vhost/rte_vhost.h > +++ b/lib/vhost/rte_vhost.h > @@ -127,6 +127,7 @@ struct rte_vhost_mem_region { > void *mmap_addr; > uint64_t mmap_size; > int fd; > + uint64_t dma_map_success; How about using bool for dma_map_success? > }; >=20 > /** > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index > 29a4c9af60..7d1d592b86 100644 > --- a/lib/vhost/vhost_user.c > +++ b/lib/vhost/vhost_user.c > @@ -45,6 +45,8 @@ > #include > #include > #include > +#include > +#include >=20 > #include "iotlb.h" > #include "vhost.h" > @@ -141,6 +143,46 @@ get_blk_size(int fd) > return ret =3D=3D -1 ? (uint64_t)-1 : (uint64_t)stat.st_blksize; } >=20 > +static int > +async_dma_map(struct rte_vhost_mem_region *region, bool do_map) { > + int ret =3D 0; > + uint64_t host_iova; > + host_iova =3D rte_mem_virt2iova((void *)(uintptr_t)region- > >host_user_addr); > + if (do_map) { > + /* Add mapped region into the default container of DPDK. */ > + ret =3D > rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD, > + region->host_user_addr, > + host_iova, > + region->size); > + region->dma_map_success =3D ret =3D=3D 0; > + if (ret) { > + if (rte_errno !=3D ENODEV && rte_errno !=3D ENOTSUP) { > + VHOST_LOG_CONFIG(ERR, "DMA engine map > failed\n"); > + return ret; > + } > + return 0; Why return 0, if ret is -1 here? Thanks, Jiayu > + } > + return ret; > + } else { > + /* No need to do vfio unmap if the map failed. */ > + if (!region->dma_map_success) > + return 0; > + > + /* Remove mapped region from the default container of > DPDK. */ > + ret =3D > rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD, > + region->host_user_addr, > + host_iova, > + region->size); > + if (ret) { > + VHOST_LOG_CONFIG(ERR, "DMA engine unmap > failed\n"); > + return ret; > + } > + region->dma_map_success =3D 0; > + } > + return ret; > +} > + > static void > free_mem_region(struct virtio_net *dev) { @@ -153,6 +195,9 @@ > free_mem_region(struct virtio_net *dev) > for (i =3D 0; i < dev->mem->nregions; i++) { > reg =3D &dev->mem->regions[i]; > if (reg->host_user_addr) { > + if (dev->async_copy && rte_vfio_is_enabled("vfio")) > + async_dma_map(reg, false); > + > munmap(reg->mmap_addr, reg->mmap_size); > close(reg->fd); > } > @@ -1157,6 +1202,7 @@ vhost_user_mmap_region(struct virtio_net *dev, > uint64_t mmap_size; > uint64_t alignment; > int populate; > + int ret; >=20 > /* Check for memory_size + mmap_offset overflow */ > if (mmap_offset >=3D -region->size) { > @@ -1210,13 +1256,22 @@ vhost_user_mmap_region(struct virtio_net *dev, > region->mmap_size =3D mmap_size; > region->host_user_addr =3D (uint64_t)(uintptr_t)mmap_addr + > mmap_offset; >=20 > - if (dev->async_copy) > + if (dev->async_copy) { > if (add_guest_pages(dev, region, alignment) < 0) { > VHOST_LOG_CONFIG(ERR, > "adding guest pages to region > failed.\n"); > return -1; > } >=20 > + if (rte_vfio_is_enabled("vfio")) { > + ret =3D async_dma_map(region, true); > + if (ret < 0) { > + VHOST_LOG_CONFIG(ERR, "Configure > IOMMU for DMA engine failed\n"); > + return -1; > + } > + } > + } > + > VHOST_LOG_CONFIG(INFO, > "guest memory region size: 0x%" PRIx64 "\n" > "\t guest physical addr: 0x%" PRIx64 "\n" > -- > 2.17.1