From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C193EA09DF for ; Wed, 2 Dec 2020 19:38:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CED4AC9C6; Wed, 2 Dec 2020 19:38:14 +0100 (CET) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by dpdk.org (Postfix) with ESMTP id 235A7C9A8; Wed, 2 Dec 2020 19:38:10 +0100 (CET) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0B2IX9Bo050118; Wed, 2 Dec 2020 13:38:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=iHwS6mc/NoXf4G/npGDVceTeTHsMWiSXgtG6jNjxQ6c=; b=H0ikftRMjntU//eQf/hl11lI8t2EJz70TAJFlO0Whg5YblX9Mh/kZetDay8ns6+xXEE/ gw8z3JIr5sItRu9kVvwQYbs6esWq1+DEnnJOXLJWJ1qZCAhHYVXMpX9B5qfzUlMBBF+Q WgpswEP91E75uhFmYyr7o9Db+hHfGhRCx1zKRACJX55uN/gBH26yLbP0+1J9UVzpmiGq lzFOuK7HF7rNMKOFohRox1DCWAX3rD9eruqgpsO33I1Y0q2jtmG13BMfy4gGi+KwDwoc e8X4YbAU0MjPgDM6mkBaUSwavelWljtffVbG2jt5sMiWcLg2nMGimUerlBRhgUIzi43T 1g== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 355vcraywx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 13:38:09 -0500 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 0B2IXSQq054953; Wed, 2 Dec 2020 13:38:08 -0500 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 355vcraywg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 13:38:08 -0500 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0B2IRTVN023075; Wed, 2 Dec 2020 18:38:07 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma02dal.us.ibm.com with ESMTP id 3569xubej6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 02 Dec 2020 18:38:07 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0B2Ic6cU22282642 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 2 Dec 2020 18:38:06 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7A6E2BE04F; Wed, 2 Dec 2020 18:38:06 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0379CBE051; Wed, 2 Dec 2020 18:38:06 +0000 (GMT) Received: from Davids-MBP.randomparity.org (unknown [9.163.49.161]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Wed, 2 Dec 2020 18:38:05 +0000 (GMT) To: Nithin Dabilpuram , anatoly.burakov@intel.com, david.marchand@redhat.com Cc: jerinj@marvell.com, dev@dpdk.org, stable@dpdk.org References: <20201012081106.10610-1-ndabilpuram@marvell.com> <20201202054647.3449-1-ndabilpuram@marvell.com> <20201202054647.3449-3-ndabilpuram@marvell.com> From: David Christensen Message-ID: <66b5c897-6682-bdc0-f715-0f032f1cd474@linux.vnet.ibm.com> Date: Wed, 2 Dec 2020 10:38:05 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <20201202054647.3449-3-ndabilpuram@marvell.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-12-02_10:2020-11-30, 2020-12-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 mlxscore=0 lowpriorityscore=0 adultscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 suspectscore=0 mlxlogscore=999 phishscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012020107 Subject: Re: [dpdk-stable] [PATCH v4 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" On 12/1/20 9:46 PM, Nithin Dabilpuram wrote: > Partial unmapping is not supported for VFIO IOMMU type1 > by kernel. Though kernel gives return as zero, the unmapped size > returned will not be same as expected. So check for > returned unmap size and return error. > > For IOVA as PA, DMA mapping is already at memseg size > granularity. Do the same even for IOVA as VA mode as > DMA map/unmap triggered by heap allocations, > maintain granularity of memseg page size so that heap > expansion and contraction does not have this issue. > > For user requested DMA map/unmap disallow partial unmapping > for VFIO type1. > > Fixes: 73a639085938 ("vfio: allow to map other memory regions") > Cc: anatoly.burakov@intel.com > Cc: stable@dpdk.org > > Signed-off-by: Nithin Dabilpuram > Acked-by: Anatoly Burakov > --- > lib/librte_eal/linux/eal_vfio.c | 34 ++++++++++++++++++++++++++++------ > lib/librte_eal/linux/eal_vfio.h | 1 + > 2 files changed, 29 insertions(+), 6 deletions(-) > > diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c > index 64b134d..b15b758 100644 > --- a/lib/librte_eal/linux/eal_vfio.c > +++ b/lib/librte_eal/linux/eal_vfio.c > @@ -70,6 +70,7 @@ static const struct vfio_iommu_type iommu_types[] = { > { > .type_id = RTE_VFIO_TYPE1, > .name = "Type 1", > + .partial_unmap = false, > .dma_map_func = &vfio_type1_dma_map, > .dma_user_map_func = &vfio_type1_dma_mem_map > }, > @@ -77,6 +78,7 @@ static const struct vfio_iommu_type iommu_types[] = { > { > .type_id = RTE_VFIO_SPAPR, > .name = "sPAPR", > + .partial_unmap = true, > .dma_map_func = &vfio_spapr_dma_map, > .dma_user_map_func = &vfio_spapr_dma_mem_map > }, > @@ -84,6 +86,7 @@ static const struct vfio_iommu_type iommu_types[] = { > { > .type_id = RTE_VFIO_NOIOMMU, > .name = "No-IOMMU", > + .partial_unmap = true, > .dma_map_func = &vfio_noiommu_dma_map, > .dma_user_map_func = &vfio_noiommu_dma_mem_map > }, > @@ -526,12 +529,19 @@ vfio_mem_event_callback(enum rte_mem_event type, const void *addr, size_t len, > /* for IOVA as VA mode, no need to care for IOVA addresses */ > if (rte_eal_iova_mode() == RTE_IOVA_VA && msl->external == 0) { > uint64_t vfio_va = (uint64_t)(uintptr_t)addr; > - if (type == RTE_MEM_EVENT_ALLOC) > - vfio_dma_mem_map(default_vfio_cfg, vfio_va, vfio_va, > - len, 1); > - else > - vfio_dma_mem_map(default_vfio_cfg, vfio_va, vfio_va, > - len, 0); > + uint64_t page_sz = msl->page_sz; > + > + /* Maintain granularity of DMA map/unmap to memseg size */ > + for (; cur_len < len; cur_len += page_sz) { > + if (type == RTE_MEM_EVENT_ALLOC) > + vfio_dma_mem_map(default_vfio_cfg, vfio_va, > + vfio_va, page_sz, 1); > + else > + vfio_dma_mem_map(default_vfio_cfg, vfio_va, > + vfio_va, page_sz, 0); > + vfio_va += page_sz; > + } > + > return; > } > > @@ -1348,6 +1358,12 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, > RTE_LOG(ERR, EAL, " cannot clear DMA remapping, error %i (%s)\n", > errno, strerror(errno)); > return -1; > + } else if (dma_unmap.size != len) { > + RTE_LOG(ERR, EAL, " unexpected size %"PRIu64" of DMA " > + "remapping cleared instead of %"PRIu64"\n", > + (uint64_t)dma_unmap.size, len); > + rte_errno = EIO; > + return -1; > } > } > > @@ -1823,6 +1839,12 @@ container_dma_unmap(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova, > /* we're partially unmapping a previously mapped region, so we > * need to split entry into two. > */ > + if (!vfio_cfg->vfio_iommu_type->partial_unmap) { > + RTE_LOG(DEBUG, EAL, "DMA partial unmap unsupported\n"); > + rte_errno = ENOTSUP; > + ret = -1; > + goto out; > + } > if (user_mem_maps->n_maps == VFIO_MAX_USER_MEM_MAPS) { > RTE_LOG(ERR, EAL, "Not enough space to store partial mapping\n"); > rte_errno = ENOMEM; > diff --git a/lib/librte_eal/linux/eal_vfio.h b/lib/librte_eal/linux/eal_vfio.h > index cb2d35f..6ebaca6 100644 > --- a/lib/librte_eal/linux/eal_vfio.h > +++ b/lib/librte_eal/linux/eal_vfio.h > @@ -113,6 +113,7 @@ typedef int (*vfio_dma_user_func_t)(int fd, uint64_t vaddr, uint64_t iova, > struct vfio_iommu_type { > int type_id; > const char *name; > + bool partial_unmap; > vfio_dma_user_func_t dma_user_map_func; > vfio_dma_func_t dma_map_func; > }; > Acked-by: David Christensen