From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0070.outbound.protection.outlook.com [104.47.42.70]) by dpdk.org (Postfix) with ESMTP id 673FC1B03C for ; Fri, 8 Dec 2017 06:22:38 +0100 (CET) Received: from BN6PR03CA0074.namprd03.prod.outlook.com (10.164.122.140) by CY1PR0301MB0731.namprd03.prod.outlook.com (10.160.159.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.302.9; Fri, 8 Dec 2017 05:22:32 +0000 Received: from BY2FFO11FD040.protection.gbl (2a01:111:f400:7c0c::147) by BN6PR03CA0074.outlook.office365.com (2603:10b6:405:6f::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.302.9 via Frontend Transport; Fri, 8 Dec 2017 05:22:31 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; NXP1.onmicrosoft.com; dkim=none (message not signed) header.d=none;NXP1.onmicrosoft.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11FD040.mail.protection.outlook.com (10.1.14.225) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.20.282.5 via Frontend Transport; Fri, 8 Dec 2017 05:22:28 +0000 Received: from bf-netperf1.ap.freescale.net (bf-netperf1.ap.freescale.net [10.232.134.28]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id vB85MKGa013022; Thu, 7 Dec 2017 22:22:28 -0700 From: Hemant Agrawal To: CC: , Date: Fri, 8 Dec 2017 10:51:16 +0530 Message-ID: <1512710487-32388-4-git-send-email-hemant.agrawal@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1512710487-32388-1-git-send-email-hemant.agrawal@nxp.com> References: <1512710487-32388-1-git-send-email-hemant.agrawal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131571841487336321; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(7966004)(336005)(39860400002)(346002)(39380400002)(376002)(2980300002)(1110001)(1109001)(339900001)(189003)(199004)(8676002)(33646002)(2351001)(106466001)(47776003)(53936002)(316002)(2906002)(16586007)(5660300001)(105606002)(48376002)(50466002)(498600001)(6666003)(356003)(77096006)(2950100002)(296002)(4326008)(6916009)(54906003)(81156014)(305945005)(8656006)(81166006)(36756003)(51416003)(50226002)(76176011)(86362001)(8936002)(68736007)(97736004)(85426001)(104016004); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0301MB0731; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD040; 1:d+ufjDjFx+55/y25wOpLPjAzrlQq3oBKI1bWsYhs1xMrDb5KQHYxYQv5pl1IxhVjx9BGvHS7F+axRM7ZEAy50qdcvKiwr5vGv9d7ez4Y463BEw/dl7/gUhFvgVU81eRw MIME-Version: 1.0 Content-Type: text/plain X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 863abc81-1616-4baa-f381-08d53dfbacc2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4628075)(201703131517081)(2017052603307); SRVR:CY1PR0301MB0731; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0731; 3:7HXMBTNlEFttmuriU8VeWt8NneaLXAXxvUoq2Rg21g05npBkHfmJ7UgZ5oAbs9NEdfuj5rK+AT0hC9AnYg0gZQh8bq5U25jan16AlTKVTLDz2UKYDlHbOQx9eyRyE/uDhTV1C5ZC6XKN7dfyXOR+xhm8IojYrdARbfhaooQN6zt8WjnL3XUycHJPGke73Km1uVE0ma/Mg/gc2xWt/hX0XTo2lT0K6+Xfr9NE6T5xDjLYVd8RaLUnFF4f5PXGW9s6SHCQDwIBPLV2OccEHLWN2qo4wwkxdPS7aZHT8+rZAM3in+kwrz9eXoE2ieIaSVDNMM8gU/7xFUYevDX2q3b4p7EeLvb1g/VT35ecdeWfUto=; 25:mqQ8UtuhjgkjMYpoq2p9mXaFngJrLHXFZUd0+DMZ3MBfWMmsCRJXs5Sl4QXu/7YrEmB7d6+q3xEoUODf+v2QJzD8FI1PeODQuEYNb9I3jifjpvsWJnmAaiebx0gMpsJ89qfY+1W6KuB6OnI3n+Aw+jK6rOS23Y+G8NV1n8h6j+tDMHmlcapz5/mA8QF+XtbEX6U4qNB9/QBJA3m4DupXQzXZx7mAFC9wYBE4U2h+LwGIcPB0wRIwfQUBvvW1RX/r6Gc3KXnOZMkzP9wCpbdKZDuP0AAMyyjRtK1gu1J7RWcbbep9wNCn7vi4naTK/e3u7hxEgXl3p8G7pysSjzIYoA== X-MS-TrafficTypeDiagnostic: CY1PR0301MB0731: X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0731; 31:Rw8P5gQjskY2rjGwdlj0gbEcQy6Du7MxdmBEFWefsp5CoVj1Ge8JT4nFGdHVsB/5qkbPZt9HHxfKd+CWTuIvvkOaJjBVMC6Rr96zPH2BsrxfKWErf98oRJNqbyZ8U0xOLfO6lmD/A1bfDJsYubeYTCBW6Cbq6Mv/Vn7MIGnCZbFTwfHrQ+CtH9qHbOu7If0A9lbaxPaaYLYzwFAjjKD19lUdEQ183t2+t6PeRiLUdrE=; 4:bfesYFsklkmIcVG+4ANjJA/DOdjm2NgOzDmfVV2IKabsE0KOw/swKs6vSwce43b4flnFVs6QiASmVLFxJo+f6hp0K9jSjl/jU1CeNcPUUL+oOiaETwvBtbFPM8u8CA0DPM/ubi2jHwdODYZEOXlBkD0WHL3bRQxTInWekWcUsmXbgiMwrvbbkgLqMtkSISWtD72gRTt6Qk+SKb2mhvBBDTJjkVyCmyAhJLKC2ksTyAD9RQG4wx7OLVF2QZahOgegDrrNpWXQ6KEHmNYX5CQeT/rj/suzmK5BMecbxRYrH9yiY5fX2RTBNoxBeebBreb8q34FF0DcbjSaiIsV8MwBSJGOF4xbCvbDU+vdpNDwCP8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(3231022)(6055026)(6096035)(20161123563025)(20161123561025)(201703131430075)(201703131520075)(201703131433075)(201703131441075)(201703131448075)(201703161259150)(20161123556025)(20161123559100)(20161123565025)(201708071742011); SRVR:CY1PR0301MB0731; BCL:0; PCL:0; RULEID:(100000803101)(100110400095)(400006); SRVR:CY1PR0301MB0731; X-Forefront-PRVS: 0515208626 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB0731; 23:Mh+tKoc5i6JWJHxVIr4ppUU/MPT1dg9eF31O+Xt?= =?us-ascii?Q?God6jaSZgDDMD4odOOcHzSu6rsI48cN00e5l9jXVQayRO5lTgg8S5T6QPL4c?= =?us-ascii?Q?QVor8AhkGEHXrCzmCggzcrHu565QtrvB5uo8d1YKzRwNMN9nXN+UbeWaPvjo?= =?us-ascii?Q?CM7K3Bdn4ckyK3npBNWnGUeGilX28lXLFQswlAXSLtDHDzyPVrc6qVihG71S?= =?us-ascii?Q?j+V7zV3LZvWvgiw/XY37q/5XFc4OupfeeN4iRRUzN74XkH46HmWyjnj3qdk+?= =?us-ascii?Q?2MIJwiAPuhvRXmhB3dfsh8oh3zbRZeOZhn/fTHhd0YO13e9l4VQzdRVNEhRU?= =?us-ascii?Q?r2g0mNQ488PF+bsmLwihpER+bAHMKwbMkI1BrpSCZmoXBfeAtxKhVekg/mcp?= =?us-ascii?Q?/qm0YPXIL0cIvl5W5EX25lrFA5OHyJ1wYc4DfLwuGoSPAdZ2KjOR6SFS3fPN?= =?us-ascii?Q?Z14ec8ZTftXHVthC7d69EKUGtF8YtERYiBWOfhEnK7zUF5FqTVpc/Ce4lU5c?= =?us-ascii?Q?i6S+t6Yr/NfFJSMMxHlg7HJiZlvmJVD8fIzoBrLJxGAeqZ+KzcnOK4JMqrS0?= =?us-ascii?Q?eK6Pb5mVvj8AjdNI/PWTNwSdrX9/hOJG++5g2V4vQzVzd2BJpq6KuEG5Ygb9?= =?us-ascii?Q?zfak8C5Nd69JW5pLDQQ5AfqVpnGUy31bjsTOoaxMkyRN0QQgg+IvTIDfOYMD?= =?us-ascii?Q?pRRiTHSAPh5eUFvrP6uGVK4F6GWxJhJyokUw4t0k1cCABMW56omPA21+HHWk?= =?us-ascii?Q?z2ut+EO1eidWsa2bxk9Dl0vforryYwUNz/l54kqoJLFQXJtHz3mBUbKOVGTB?= =?us-ascii?Q?4rL5kDjxUoBgyOZsj5q9U0x/ynkqEz+LurGZWglMeroe1Zii9NfihwOErRU9?= =?us-ascii?Q?cvH5kjN2sSEZIfhNUsiYN3P4HqE1vSDRFfAs6OJWwfRqwYdamOFsiE9o3pZZ?= =?us-ascii?Q?VoA8nY20lPNbcw3TWMK7Iy8970h91sakbDBeVQmlVWk690kjJc8d30opgbl/?= =?us-ascii?Q?INmqNWBEdtwuZjh8tlFd35FHCiTh5yc8PvXokU16tshg0PLJwnDLzKVUBzw5?= =?us-ascii?Q?r+naRABCtbI2m2f5KWBr95a9Nn2Ajzdb7UX7eYFd67tnwGycl7l+YQHjMNDP?= =?us-ascii?Q?H3yk2QIkKiAs=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0731; 6:jvFqopt5EzaIe6v07TY8uRMO9BAnIBQr4k00S9KPicqYkZP46gdHxR6UuyGkd91bsG+zrKmTvQWf20KFuXo4CFAZ8+QKrSXNLQhu3tFW7/scMYnBSY0mMOvsx7iNF7xJGKe1NL7kbL51jWJAt8osJTFnjbKemnov0Jm9aI6mlkMGGyXEYAoS8Y6UPGsbYty/vz1voZFpPZApKbd2+wVk9H3th3PlCn3xcwY6ou6cyKW4453ywjCW+j9koItfcFYNyYLBHEHzaw2SCmHq9nSarOkx8b3xtwvtoWCrgQVh5YRF0cChCQX7CPX1xIWYW7w5pPwp9FQ8xx8F2lzY8DB3QGqLA1F6JG5hle4yWuPN/gE=; 5:9SghUB8yn3nc7eD5UFpLwpFOD3U1D7Qa1UPy+JLqqc7tuiP1n4IpW92ZVgGUsKbRyhG+z8GTzkQJI4PgceSNtlwhhqB7OayK2Wjcb0IN7AdE+gsbq6eajh0i12sIpJU94DfKPF45nc8yWxqM7IdGpLMA/Pcn43xyClNDa56eXsU=; 24:TesKjZLi0LwKSVfY4/+FVRaqmyUwKUysJd4LBnJEDeMVijKc8+7CxBGmjX4MuvkgrNDtK0fg1n/nENmK+dFkIbXYErN3hKE789XaLel8cAE=; 7:QVro0gVka7wW/fcaXGP5YcX0VtxtCXcR39rcUqaIs1xB5Gn4gBK2k7Q1R0qRokiHXw7WP9cWaTGXtv2+TfAQYoyKxKkri3BtGRToI9gHaqxazwQpdw2D9gylb4R3vOMO/9JBIj4KWGkQ49khqxjH9BXsYPIFSztqTbt740yclaCJdhmNJ+1h7aLdCX4kQ+HtYD1TbauZJnqCdmItpMoh9bxOkNNHyEKqxn1Vq4eertk8PxKEU0f1uy6YJelOcr+2 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2017 05:22:28.5464 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 863abc81-1616-4baa-f381-08d53dfbacc2 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0301MB0731 Subject: [dpdk-dev] [PATCH 03/14] bus/fslmc: add support for dynamic iova for DPAA2 devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Dec 2017 05:22:39 -0000 This patch add support for dynamic iova detection for DPAA2 devices and use of virtual address in such cases. Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/fslmc_bus.c | 44 +++++++++++++++++++++++++++++ drivers/bus/fslmc/fslmc_vfio.c | 5 +++- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 15 ++++++++-- drivers/bus/fslmc/rte_bus_fslmc_version.map | 7 +++++ drivers/bus/fslmc/rte_fslmc.h | 3 ++ drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 1 + drivers/net/dpaa2/dpaa2_ethdev.c | 1 + 7 files changed, 73 insertions(+), 3 deletions(-) diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 480857e..63c333a 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -51,6 +51,7 @@ #define VFIO_IOMMU_GROUP_PATH "/sys/kernel/iommu_groups" struct rte_fslmc_bus rte_fslmc_bus; +uint8_t dpaa2_virt_mode; static void cleanup_fslmc_device_list(void) @@ -300,6 +301,9 @@ rte_fslmc_probe(void) } } + if (rte_eal_iova_mode() == RTE_IOVA_VA) + dpaa2_virt_mode = 1; + return 0; } @@ -347,11 +351,51 @@ rte_fslmc_driver_unregister(struct rte_dpaa2_driver *driver) } /* + * All device has iova as va + */ +static inline int +fslmc_all_device_support_iova(void) +{ + int ret = 0; + struct rte_dpaa2_device *dev; + struct rte_dpaa2_driver *drv; + + TAILQ_FOREACH(dev, &rte_fslmc_bus.device_list, next) { + TAILQ_FOREACH(drv, &rte_fslmc_bus.driver_list, next) { + ret = rte_fslmc_match(drv, dev); + if (ret) + continue; + /* if the driver is not supporting IOVA */ + if (!(drv->drv_flags & RTE_DPAA2_DRV_IOVA_AS_VA)) + return 0; + } + } + return 1; +} + +/* * Get iommu class of DPAA2 devices on the bus. */ static enum rte_iova_mode rte_dpaa2_get_iommu_class(void) { + bool is_vfio_noiommu_enabled = 1; + bool has_iova_va; + + if (TAILQ_EMPTY(&rte_fslmc_bus.device_list)) + return RTE_IOVA_DC; + + /* check if all devices on the bus support Virtual addressing or not */ + has_iova_va = fslmc_all_device_support_iova(); + +#ifdef VFIO_PRESENT + is_vfio_noiommu_enabled = rte_vfio_noiommu_is_enabled() == true ? + true : false; +#endif + + if (has_iova_va && !is_vfio_noiommu_enabled) + return RTE_IOVA_VA; + return RTE_IOVA_PA; } diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index e47215c..c0709de 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -249,7 +249,10 @@ int rte_fslmc_vfio_dmamap(void) dma_map.size = memseg[i].len; dma_map.vaddr = memseg[i].addr_64; #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA - dma_map.iova = memseg[i].iova; + if (rte_eal_iova_mode() == RTE_IOVA_VA) + dma_map.iova = dma_map.vaddr; + else + dma_map.iova = memseg[i].iova; #else dma_map.iova = dma_map.vaddr; #endif diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h index ece1a7d..7937293 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h +++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h @@ -276,13 +276,19 @@ enum qbman_fd_format { #define DPAA2_EQ_RESP_ALWAYS 1 #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA +extern uint8_t dpaa2_virt_mode; static void *dpaa2_mem_ptov(phys_addr_t paddr) __attribute__((unused)); /* todo - this is costly, need to write a fast coversion routine */ static void *dpaa2_mem_ptov(phys_addr_t paddr) { - const struct rte_memseg *memseg = rte_eal_get_physmem_layout(); + const struct rte_memseg *memseg; int i; + if (dpaa2_virt_mode) + return (void *)paddr; + + memseg = rte_eal_get_physmem_layout(); + for (i = 0; i < RTE_MAX_MEMSEG && memseg[i].addr_64 != 0; i++) { if (paddr >= memseg[i].iova && (char *)paddr < (char *)memseg[i].iova + memseg[i].len) @@ -295,9 +301,14 @@ static void *dpaa2_mem_ptov(phys_addr_t paddr) static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) __attribute__((unused)); static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) { - const struct rte_memseg *memseg = rte_eal_get_physmem_layout(); + const struct rte_memseg *memseg; int i; + if (dpaa2_virt_mode) + return vaddr; + + memseg = rte_eal_get_physmem_layout(); + for (i = 0; i < RTE_MAX_MEMSEG && memseg[i].addr_64 != 0; i++) { if (vaddr >= memseg[i].addr_64 && vaddr < memseg[i].addr_64 + memseg[i].len) diff --git a/drivers/bus/fslmc/rte_bus_fslmc_version.map b/drivers/bus/fslmc/rte_bus_fslmc_version.map index 51a2ac6..a1e30d6 100644 --- a/drivers/bus/fslmc/rte_bus_fslmc_version.map +++ b/drivers/bus/fslmc/rte_bus_fslmc_version.map @@ -89,3 +89,10 @@ DPDK_17.11 { rte_dpaa2_intr_enable; } DPDK_17.08; + +DPDK_18.02 { + global: + + dpaa2_virt_mode; + +} DPDK_17.11; diff --git a/drivers/bus/fslmc/rte_fslmc.h b/drivers/bus/fslmc/rte_fslmc.h index 0814e69..0c7872d 100644 --- a/drivers/bus/fslmc/rte_fslmc.h +++ b/drivers/bus/fslmc/rte_fslmc.h @@ -62,6 +62,9 @@ extern "C" { #define FSLMC_OBJECT_MAX_LEN 32 /**< Length of each device on bus */ +/** Device driver supports IOVA as VA */ +#define RTE_DPAA2_DRV_IOVA_AS_VA 0X0040 + struct rte_dpaa2_driver; /* DPAA2 Device and Driver lists for FSLMC bus */ diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 9c64c5d..5e52390 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -2420,6 +2420,7 @@ cryptodev_dpaa2_sec_remove(struct rte_dpaa2_device *dpaa2_dev) } static struct rte_dpaa2_driver rte_dpaa2_sec_driver = { + .drv_flags = RTE_DPAA2_DRV_IOVA_AS_VA, .drv_type = DPAA2_CRYPTO, .driver = { .name = "DPAA2 SEC PMD" diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c index 202f84f..38de3d9 100644 --- a/drivers/net/dpaa2/dpaa2_ethdev.c +++ b/drivers/net/dpaa2/dpaa2_ethdev.c @@ -2003,6 +2003,7 @@ rte_dpaa2_remove(struct rte_dpaa2_device *dpaa2_dev) } static struct rte_dpaa2_driver rte_dpaa2_pmd = { + .drv_flags = RTE_DPAA2_DRV_IOVA_AS_VA, .drv_type = DPAA2_ETH, .probe = rte_dpaa2_probe, .remove = rte_dpaa2_remove, -- 2.7.4