From: Nithin Dabilpuram <ndabilpuram@marvell.com> To: "Burakov, Anatoly" <anatoly.burakov@intel.com> Cc: <jerinj@marvell.com>, <dev@dpdk.org>, <stable@dpdk.org> Subject: Re: [dpdk-stable] [EXT] Re: [PATCH 2/2] vfio: fix partial DMA unmapping for VFIO type1 Date: Thu, 15 Oct 2020 11:39:14 +0530 Message-ID: <20201015060914.GA32207@outlook.office365.com> (raw) In-Reply-To: <05afb7f5-96bf-dffd-15dd-2024586f7290@intel.com> On Wed, Oct 14, 2020 at 04:07:10PM +0100, Burakov, Anatoly wrote: > External Email > > ---------------------------------------------------------------------- > On 12-Oct-20 9:11 AM, Nithin Dabilpuram wrote: > > Partial unmapping is not supported for VFIO IOMMU type1 > > by kernel. Though kernel gives return as zero, the unmapped size > > returned will not be same as expected. So check for > > returned unmap size and return error. > > > > For case of DMA map/unmap triggered by heap allocations, > > maintain granularity of memseg page size so that heap > > expansion and contraction does not have this issue. > > This is quite unfortunate, because there was a different bug that had to do > with kernel having a very limited number of mappings available [1], as a > result of which the page concatenation code was added. > > It should therefore be documented that the dma_entry_limit parameter should > be adjusted should the user run out of the DMA entries. > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_lkml_155414977872.12780.13728555131525362206.stgit-40gimli.home_T_&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=3GMg-634_cdUCY4WpQPwjzZ_S4ckuMHOnt2FxyyjXMk&s=TJLzppkaDS95VGyRHX2hzflQfb9XLK0OiOszSXoeXKk&e= Ack, I'll document it in guides/linux_gsg/linux_drivers.rst in vfio section. > > > > > For user requested DMA map/unmap disallow partial unmapping > > for VFIO type1. > > > > Fixes: 73a639085938 ("vfio: allow to map other memory regions") > > Cc: anatoly.burakov@intel.com > > Cc: stable@dpdk.org > > > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > > --- > > lib/librte_eal/linux/eal_vfio.c | 34 ++++++++++++++++++++++++++++------ > > lib/librte_eal/linux/eal_vfio.h | 1 + > > 2 files changed, 29 insertions(+), 6 deletions(-) > > > > diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c > > index d26e164..ef95259 100644 > > --- a/lib/librte_eal/linux/eal_vfio.c > > +++ b/lib/librte_eal/linux/eal_vfio.c > > @@ -69,6 +69,7 @@ static const struct vfio_iommu_type iommu_types[] = { > > { > > .type_id = RTE_VFIO_TYPE1, > > .name = "Type 1", > > + .partial_unmap = false, > > .dma_map_func = &vfio_type1_dma_map, > > .dma_user_map_func = &vfio_type1_dma_mem_map > > }, > > @@ -76,6 +77,7 @@ static const struct vfio_iommu_type iommu_types[] = { > > { > > .type_id = RTE_VFIO_SPAPR, > > .name = "sPAPR", > > + .partial_unmap = true, > > .dma_map_func = &vfio_spapr_dma_map, > > .dma_user_map_func = &vfio_spapr_dma_mem_map > > }, > > @@ -83,6 +85,7 @@ static const struct vfio_iommu_type iommu_types[] = { > > { > > .type_id = RTE_VFIO_NOIOMMU, > > .name = "No-IOMMU", > > + .partial_unmap = true, > > .dma_map_func = &vfio_noiommu_dma_map, > > .dma_user_map_func = &vfio_noiommu_dma_mem_map > > }, > > @@ -525,12 +528,19 @@ vfio_mem_event_callback(enum rte_mem_event type, const void *addr, size_t len, > > /* for IOVA as VA mode, no need to care for IOVA addresses */ > > if (rte_eal_iova_mode() == RTE_IOVA_VA && msl->external == 0) { > > uint64_t vfio_va = (uint64_t)(uintptr_t)addr; > > - if (type == RTE_MEM_EVENT_ALLOC) > > - vfio_dma_mem_map(default_vfio_cfg, vfio_va, vfio_va, > > - len, 1); > > - else > > - vfio_dma_mem_map(default_vfio_cfg, vfio_va, vfio_va, > > - len, 0); > > + uint64_t page_sz = msl->page_sz; > > + > > + /* Maintain granularity of DMA map/unmap to memseg size */ > > + for (; cur_len < len; cur_len += page_sz) { > > + if (type == RTE_MEM_EVENT_ALLOC) > > + vfio_dma_mem_map(default_vfio_cfg, vfio_va, > > + vfio_va, page_sz, 1); > > + else > > + vfio_dma_mem_map(default_vfio_cfg, vfio_va, > > + vfio_va, page_sz, 0); > > + vfio_va += page_sz; > > + } > > + > > You'd also have to revert d1c7c0cdf7bac5eb40d3a2a690453aefeee5887b because > currently the PA path will opportunistically concantenate contiguous > segments into single mapping too. Ack, I'll change it even for IOVA as PA mode. I missed that. > > > return; > > } > > @@ -1383,6 +1393,12 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, > > RTE_LOG(ERR, EAL, " cannot clear DMA remapping, error %i (%s)\n", > > errno, strerror(errno)); > > return -1; > > + } else if (dma_unmap.size != len) { > > + RTE_LOG(ERR, EAL, " unexpected size %"PRIu64" of DMA " > > + "remapping cleared instead of %"PRIu64"\n", > > + (uint64_t)dma_unmap.size, len); > > + rte_errno = EIO; > > + return -1; > > } > > } > > @@ -1853,6 +1869,12 @@ container_dma_unmap(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova, > > /* we're partially unmapping a previously mapped region, so we > > * need to split entry into two. > > */ > > + if (!vfio_cfg->vfio_iommu_type->partial_unmap) { > > + RTE_LOG(DEBUG, EAL, "DMA partial unmap unsupported\n"); > > + rte_errno = ENOTSUP; > > + ret = -1; > > + goto out; > > + } > > How would we ever arrive here if we never do more than 1 page worth of > memory anyway? I don't think this is needed. container_dma_unmap() is called by user via rte_vfio_container_dma_unmap() and when he maps we don't split it as we don't about his memory. So if he maps multiple pages and tries to unmap partially, then we should fail. > > > if (user_mem_maps->n_maps == VFIO_MAX_USER_MEM_MAPS) { > > RTE_LOG(ERR, EAL, "Not enough space to store partial mapping\n"); > > rte_errno = ENOMEM; > > diff --git a/lib/librte_eal/linux/eal_vfio.h b/lib/librte_eal/linux/eal_vfio.h > > index cb2d35f..6ebaca6 100644 > > --- a/lib/librte_eal/linux/eal_vfio.h > > +++ b/lib/librte_eal/linux/eal_vfio.h > > @@ -113,6 +113,7 @@ typedef int (*vfio_dma_user_func_t)(int fd, uint64_t vaddr, uint64_t iova, > > struct vfio_iommu_type { > > int type_id; > > const char *name; > > + bool partial_unmap; > > vfio_dma_user_func_t dma_user_map_func; > > vfio_dma_func_t dma_map_func; > > }; > > > > > -- > Thanks, > Anatoly
next prev parent reply other threads:[~2020-10-15 6:09 UTC|newest] Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top [not found] <20201012081106.10610-1-ndabilpuram@marvell.com> 2020-10-12 8:11 ` [dpdk-stable] " Nithin Dabilpuram 2020-10-14 15:07 ` Burakov, Anatoly 2020-10-15 6:09 ` Nithin Dabilpuram [this message] 2020-10-15 10:00 ` [dpdk-stable] [EXT] " Burakov, Anatoly 2020-10-15 11:38 ` Nithin Dabilpuram 2020-10-15 11:50 ` Nithin Dabilpuram 2020-10-15 11:57 ` [dpdk-stable] [dpdk-dev] " Nithin Dabilpuram 2020-10-15 15:10 ` Burakov, Anatoly 2020-10-16 7:10 ` Nithin Dabilpuram 2020-10-17 16:14 ` Burakov, Anatoly 2020-10-19 9:43 ` Nithin Dabilpuram 2020-10-22 12:13 ` Nithin Dabilpuram 2020-10-28 13:04 ` Burakov, Anatoly 2020-10-28 14:17 ` Nithin Dabilpuram 2020-10-28 16:07 ` Burakov, Anatoly 2020-10-28 16:31 ` Nithin Dabilpuram [not found] ` <20201105090423.11954-1-ndabilpuram@marvell.com> 2020-11-05 9:04 ` [dpdk-stable] [PATCH v2 1/3] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2020-11-05 9:04 ` [dpdk-stable] [PATCH v2 2/3] vfio: fix DMA mapping granularity for type1 iova as va Nithin Dabilpuram 2020-11-10 14:04 ` Burakov, Anatoly 2020-11-10 14:22 ` [dpdk-stable] [dpdk-dev] " Burakov, Anatoly 2020-11-10 14:17 ` [dpdk-stable] " Burakov, Anatoly 2020-11-11 5:08 ` [dpdk-stable] [dpdk-dev] " Nithin Dabilpuram 2020-11-11 10:00 ` Burakov, Anatoly [not found] ` <20201201193302.28131-1-ndabilpuram@marvell.com> 2020-12-01 19:32 ` [dpdk-stable] [PATCH v3 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2020-12-01 19:33 ` [dpdk-stable] [PATCH v3 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram [not found] ` <20201202054647.3449-1-ndabilpuram@marvell.com> 2020-12-02 5:46 ` [dpdk-stable] [PATCH v4 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2020-12-02 18:36 ` David Christensen 2020-12-02 5:46 ` [dpdk-stable] [PATCH v4 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram 2020-12-02 18:38 ` David Christensen [not found] ` <20201214081935.23577-1-ndabilpuram@marvell.com> 2020-12-14 8:19 ` [dpdk-stable] [PATCH v5 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2020-12-14 8:19 ` [dpdk-stable] [PATCH v5 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram [not found] ` <20201217190604.29803-1-ndabilpuram@marvell.com> 2020-12-17 19:06 ` [dpdk-stable] [PATCH v6 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2020-12-17 19:06 ` [dpdk-stable] [PATCH v6 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram [not found] ` <20210112173923.30320-1-ndabilpuram@marvell.com> 2021-01-12 17:39 ` [dpdk-stable] [PATCH v7 1/3] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2021-01-12 17:39 ` [dpdk-stable] [PATCH v7 2/3] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram [not found] ` <20210115073243.7025-1-ndabilpuram@marvell.com> 2021-01-15 7:32 ` [dpdk-stable] [PATCH v8 1/3] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram 2021-03-05 7:50 ` David Marchand 2021-03-05 13:54 ` Burakov, Anatoly 2021-03-05 15:50 ` [dpdk-stable] [dpdk-dev] " Nithin Dabilpuram 2021-01-15 7:32 ` [dpdk-stable] [PATCH v8 2/3] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram 2021-01-15 7:32 ` [dpdk-stable] [PATCH v8 3/3] test: change external memory test to use system page sz Nithin Dabilpuram 2021-02-11 11:21 ` Burakov, Anatoly
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201015060914.GA32207@outlook.office365.com \ --to=ndabilpuram@marvell.com \ --cc=anatoly.burakov@intel.com \ --cc=dev@dpdk.org \ --cc=jerinj@marvell.com \ --cc=stable@dpdk.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
patches for DPDK stable branches This inbox may be cloned and mirrored by anyone: git clone --mirror http://inbox.dpdk.org/stable/0 stable/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 stable stable/ http://inbox.dpdk.org/stable \ stable@dpdk.org public-inbox-index stable Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.stable AGPL code for this site: git clone https://public-inbox.org/public-inbox.git