From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5DAFAA0A0E for ; Mon, 10 May 2021 18:03:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 338D14003E; Mon, 10 May 2021 18:03:33 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2051.outbound.protection.outlook.com [40.107.94.51]) by mails.dpdk.org (Postfix) with ESMTP id E3CD24003E for ; Mon, 10 May 2021 18:03:31 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jGeZIWJXySWb2cWDanH0268g9pE3pZmaHu15uEoN//Y6TkniMIlapzXhbtA5nuQJI/tLaZQNo9A8MFtjJHpIMDykY/n2m8q6Fuo9QSVoN8jYLL0MoVzVHAI5efe89+VKzZgf7hzRBUdeOwu5Y9KSWtMMGTlLk1HpBU+Q7q2x8AeIqkAsE9OJKlf/DGjj41N1Qm5oea5j8yQ4WC56GkwglcT55SXkMUY7zwOVRMNznjhCnGCjhcADGcvGCGtKdk5IyA++7pHcV/6lUZdtNbVb8dx6txycwc7nSE441WL+8nr7aBddIPeL6mucyOum1TPyhwXFeMn+Qqnc07XOZqfbOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7pN5kCOJjq+C7alRJgtN1O0hbaGBNvZ8tUOLwgZ4YU0=; b=LDNcryo82rL7Q81wkm17RYCMuV9Qkbk1hT9u3xf3alB/tX32ZNLuwmpfXn00GbMM2pyzYuBf8B9/DLBIvOm8Brg3shmMFVU95LLh13aWp6x4FNHlvJZswRhn/S6oUvfeUDBy226X/6omCuvTVCgFRpatSLc2GBPG1L4YbLNi8fJN0P/76PljKvmf7LAVuIAu8NzRUlmgCZCSU9MKXbfwVgK1v8ktnUTLbOTBEqxLelSyh3oWQHf2jxVq5VE6yY5kqD7/KpC2/1jpGZlLLsReHmf/5el+FQjgQPQxj6uXWHtlOY6h7Eb83QD78SxvmTieYFJDVtJRHk095APrY+kZBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=linux.vnet.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7pN5kCOJjq+C7alRJgtN1O0hbaGBNvZ8tUOLwgZ4YU0=; b=A+HCPqJBUtfPQIow9cyOehbApcJTE8nzDRXhfo9v0IR0NFF7hQjt21SiBafaPvtPAhTTRQRe7pKQP+XwLtBXNDP4SIravNwaRIK5jDal6dAL2mhcgtLlkIsYEMwxbdLJztH1r1jGlxSu/XaruEQjjPDc82kV2LTY7HMzVmdc+5x5dvT8JYnmGkGNu0KVEQTHg3VpX+pSfVUsv5pFZ8GxjhL6A645CbzCRlt/TtR7B+YyxnTJqnHvfxOwttthDhdrb56Y7Bpt0KnnuLUWB7cAi1ZkluSa4uDcfJpT6I25oODssdaAmr7FNda8JFIF8xNwCJKzmx2zZl4VJMt/jSdqQw== Received: from BN8PR15CA0060.namprd15.prod.outlook.com (2603:10b6:408:80::37) by DM6PR12MB3369.namprd12.prod.outlook.com (2603:10b6:5:117::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:03:30 +0000 Received: from BN8NAM11FT030.eop-nam11.prod.protection.outlook.com (2603:10b6:408:80:cafe::c5) by BN8PR15CA0060.outlook.office365.com (2603:10b6:408:80::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25 via Frontend Transport; Mon, 10 May 2021 16:03:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; linux.vnet.ibm.com; dkim=none (message not signed) header.d=none;linux.vnet.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT030.mail.protection.outlook.com (10.13.177.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4108.25 via Frontend Transport; Mon, 10 May 2021 16:03:30 +0000 Received: from nvidia.com (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 10 May 2021 16:03:27 +0000 From: Xueming Li To: Nithin Dabilpuram CC: Luca Boccassi , Anatoly Burakov , David Christensen , dpdk stable Date: Mon, 10 May 2021 23:59:10 +0800 Message-ID: <20210510160258.30982-1-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9aaeca84-c880-44a5-3a9c-08d913cd278b X-MS-TrafficTypeDiagnostic: DM6PR12MB3369: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0bYbDrthe48lCX2p9+8Q6R9Oe2mWR6SD9roa/lkuVWLlQok/MPqAXG1UJ63L5tlWRlOE7hzYd2V5tWQVsjOn8fEMWv7ds4Kt3MEICS4abmT/+MV2GokvcPIOk/SnaeG+JCIvsN9ZiuptXG2mVUvAeUOpCNXWLpMJttzM30KzYuhtm9k+u1NA1K8JBWY8M0EsjFFmep1q7gZ/JyK3m8LUNkqco6YQjZ5NLKK7BjfdWAnJfbAqawHc70qOI7gew3QHv3ti67x8i/RQv9p6A9hRugjucVeJqtOWDswoOpYg+ChZbZTIaVbFkZKQeb6l3g+JTlS+CEQZqkvSFmvtq5BKhklW2STm81y/WFFTys0JW/oJZYyYUkR4eQq1MGwZ8nKOWdPTt1qr4uaG/wAcnneeLrvVh4560xnT3DGas41VnEoioq4f63aM9d4241/aZB2IsgtymQU9zNEZBImGBXheeUZdSxBkLBs40iLcQxu16AB12zzV4BMo+7JUotUidB6fFtKrR5SoDecH9AaYTjJCEAeV1sHoPP5Nd+5BXOjiq429/hLmZQ/Q5wcDrtYNEfg56v2RmoxcvGgWjPy6aVT7sJcPwIRTGwZn1lC5cW5pT2c6MQUS4zo+I44Oa/FUoVbeEzy+0KiToDC91FEEsvckfIwJhz5kuE26MqnGsItid5v0edinBgsWEXN15oz+b5GLn+KHTjZO5iwtEawisM0gg7DId7Ia9I55dMRm87E8rUauKXSswhLGijcwoQuDmpjJ8tfYHl9WfElBUSMEtj5C1Q== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(346002)(396003)(136003)(376002)(46966006)(36840700001)(7696005)(36756003)(2616005)(26005)(55016002)(36906005)(6286002)(8676002)(316002)(36860700001)(8936002)(336012)(47076005)(6666004)(6916009)(82310400003)(2906002)(16526019)(426003)(54906003)(5660300002)(86362001)(478600001)(356005)(966005)(82740400003)(70586007)(7636003)(186003)(53546011)(70206006)(4326008)(1076003)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:03:30.0506 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9aaeca84-c880-44a5-3a9c-08d913cd278b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT030.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3369 Subject: [dpdk-stable] patch 'vfio: fix DMA mapping granularity for IOVA as VA' has been queued to stable release 20.11.2 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 20.11.2 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 05/12/21. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/steevenlee/dpdk This queued commit can be viewed at: https://github.com/steevenlee/dpdk/commit/7f904ea0c6725fabfe16812f3d36d250ec3981c2 Thanks. Xueming Li --- >From 7f904ea0c6725fabfe16812f3d36d250ec3981c2 Mon Sep 17 00:00:00 2001 From: Nithin Dabilpuram Date: Fri, 15 Jan 2021 13:02:42 +0530 Subject: [PATCH] vfio: fix DMA mapping granularity for IOVA as VA Cc: Luca Boccassi [ upstream commit c13ca4e81cac591904c893a65ffbbf446af0268a ] Partial unmapping is not supported for VFIO IOMMU type1 by kernel. Though kernel gives return as zero, the unmapped size returned will not be same as expected. So check for returned unmap size and return error. For IOVA as PA, DMA mapping is already at memseg size granularity. Do the same even for IOVA as VA mode as DMA map/unmap triggered by heap allocations, maintain granularity of memseg page size so that heap expansion and contraction does not have this issue. For user requested DMA map/unmap disallow partial unmapping for VFIO type1. Fixes: 73a639085938 ("vfio: allow to map other memory regions") Signed-off-by: Nithin Dabilpuram Acked-by: Anatoly Burakov Acked-by: David Christensen --- lib/librte_eal/linux/eal_vfio.c | 34 +++++++++++++++++++++++++++------ lib/librte_eal/linux/eal_vfio.h | 1 + 2 files changed, 29 insertions(+), 6 deletions(-) diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c index 64b134d530..b15b75882b 100644 --- a/lib/librte_eal/linux/eal_vfio.c +++ b/lib/librte_eal/linux/eal_vfio.c @@ -70,6 +70,7 @@ static const struct vfio_iommu_type iommu_types[] = { { .type_id = RTE_VFIO_TYPE1, .name = "Type 1", + .partial_unmap = false, .dma_map_func = &vfio_type1_dma_map, .dma_user_map_func = &vfio_type1_dma_mem_map }, @@ -77,6 +78,7 @@ static const struct vfio_iommu_type iommu_types[] = { { .type_id = RTE_VFIO_SPAPR, .name = "sPAPR", + .partial_unmap = true, .dma_map_func = &vfio_spapr_dma_map, .dma_user_map_func = &vfio_spapr_dma_mem_map }, @@ -84,6 +86,7 @@ static const struct vfio_iommu_type iommu_types[] = { { .type_id = RTE_VFIO_NOIOMMU, .name = "No-IOMMU", + .partial_unmap = true, .dma_map_func = &vfio_noiommu_dma_map, .dma_user_map_func = &vfio_noiommu_dma_mem_map }, @@ -526,12 +529,19 @@ vfio_mem_event_callback(enum rte_mem_event type, const void *addr, size_t len, /* for IOVA as VA mode, no need to care for IOVA addresses */ if (rte_eal_iova_mode() == RTE_IOVA_VA && msl->external == 0) { uint64_t vfio_va = (uint64_t)(uintptr_t)addr; - if (type == RTE_MEM_EVENT_ALLOC) - vfio_dma_mem_map(default_vfio_cfg, vfio_va, vfio_va, - len, 1); - else - vfio_dma_mem_map(default_vfio_cfg, vfio_va, vfio_va, - len, 0); + uint64_t page_sz = msl->page_sz; + + /* Maintain granularity of DMA map/unmap to memseg size */ + for (; cur_len < len; cur_len += page_sz) { + if (type == RTE_MEM_EVENT_ALLOC) + vfio_dma_mem_map(default_vfio_cfg, vfio_va, + vfio_va, page_sz, 1); + else + vfio_dma_mem_map(default_vfio_cfg, vfio_va, + vfio_va, page_sz, 0); + vfio_va += page_sz; + } + return; } @@ -1348,6 +1358,12 @@ vfio_type1_dma_mem_map(int vfio_container_fd, uint64_t vaddr, uint64_t iova, RTE_LOG(ERR, EAL, " cannot clear DMA remapping, error %i (%s)\n", errno, strerror(errno)); return -1; + } else if (dma_unmap.size != len) { + RTE_LOG(ERR, EAL, " unexpected size %"PRIu64" of DMA " + "remapping cleared instead of %"PRIu64"\n", + (uint64_t)dma_unmap.size, len); + rte_errno = EIO; + return -1; } } @@ -1823,6 +1839,12 @@ container_dma_unmap(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova, /* we're partially unmapping a previously mapped region, so we * need to split entry into two. */ + if (!vfio_cfg->vfio_iommu_type->partial_unmap) { + RTE_LOG(DEBUG, EAL, "DMA partial unmap unsupported\n"); + rte_errno = ENOTSUP; + ret = -1; + goto out; + } if (user_mem_maps->n_maps == VFIO_MAX_USER_MEM_MAPS) { RTE_LOG(ERR, EAL, "Not enough space to store partial mapping\n"); rte_errno = ENOMEM; diff --git a/lib/librte_eal/linux/eal_vfio.h b/lib/librte_eal/linux/eal_vfio.h index cb2d35fb12..6ebaca6a0c 100644 --- a/lib/librte_eal/linux/eal_vfio.h +++ b/lib/librte_eal/linux/eal_vfio.h @@ -113,6 +113,7 @@ typedef int (*vfio_dma_user_func_t)(int fd, uint64_t vaddr, uint64_t iova, struct vfio_iommu_type { int type_id; const char *name; + bool partial_unmap; vfio_dma_user_func_t dma_user_map_func; vfio_dma_func_t dma_map_func; }; -- 2.25.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2021-05-10 23:59:26.774717800 +0800 +++ 0002-vfio-fix-DMA-mapping-granularity-for-IOVA-as-VA.patch 2021-05-10 23:59:26.310000000 +0800 @@ -1 +1 @@ -From c13ca4e81cac591904c893a65ffbbf446af0268a Mon Sep 17 00:00:00 2001 +From 7f904ea0c6725fabfe16812f3d36d250ec3981c2 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Luca Boccassi + +[ upstream commit c13ca4e81cac591904c893a65ffbbf446af0268a ] @@ -21 +23,0 @@ -Cc: stable@dpdk.org