From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0057.outbound.protection.outlook.com [104.47.32.57]) by dpdk.org (Postfix) with ESMTP id E6D2C199AE for ; Mon, 18 Sep 2017 12:43:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=4vVfd0TWf5WsZubLh1Qvf6xTsg4Ey1nHNGDCrQ+xeaY=; b=OXzTdxbtdKvEARFlrPibZnuExCnXEuN4HiS/PP7Y8Wj7f2kAH1NFTnPh2qnTohNL+esMcC+S362SIGe8J9bBnkoaiLvCX1ir8gxFWhs+giVlf10u85Q1HiDZO8cERuXKHrPAVR9g0nsR+GHb+b83y/dF99kpFxwB6uaObSbfG0w= Received: from localhost.localdomain (14.140.2.178) by DM5PR07MB3097.namprd07.prod.outlook.com (10.172.85.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.9; Mon, 18 Sep 2017 10:43:05 +0000 From: Santosh Shukla To: dev@dpdk.org Cc: olivier.matz@6wind.com, thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, aconole@redhat.com, stephen@networkplumber.org, anatoly.burakov@intel.com, gaetan.rivet@6wind.com, shreyansh.jain@nxp.com, bruce.richardson@intel.com, sergio.gonzalez.monroy@intel.com, maxime.coquelin@redhat.com, Santosh Shukla Date: Mon, 18 Sep 2017 16:12:25 +0530 Message-Id: <20170918104234.9149-1-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170831032618.7120-1-santosh.shukla@caviumnetworks.com> References: <20170831032618.7120-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: PN1PR01CA0093.INDPRD01.PROD.OUTLOOK.COM (10.174.144.161) To DM5PR07MB3097.namprd07.prod.outlook.com (10.172.85.7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c99275e3-bf50-4ab5-2475-08d4fe820f29 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DM5PR07MB3097; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3097; 3:wuc0h3a0egbSu+Rxj/AqEJod7T1X8NZ5vuRus19OhpFQJ4UeohnPqYM9UcOrofoY39uyKHupMIFennLebsrVvLxqwJavHgjrq8y4fhu2uTl23B08e7KiWRlNNo9EnmmRFFAlAgpYiNJIY+w8L7X/YDAQBYUFnuWefWoHKyxnTZuLs/0tk8Q5XvRYpmDWbwnvo8yojLdOwsACFnQIuUGBirmrwUlNKum7jBMRLoq/ayEQ6wfTBdqOw8fbQ9u9Fjg6; 25:Xmhg0CD+Rx8QrqreEr7kqy1HfeaqzD6tlfEDS4aNuXEe/yFYJApbPtEpMRB5WxgHJVCr3Vm9fiZDEdWS+AxOE02oUHQc4Amng1F47HUuI+V1tSYAd6gvBfO7RoKm+MVvOn3k/DOEVe/Cyd1wjx0sIocBsWItCr+7He6OP3/1vmv2gKTwDG79TrSsep8Kc8cqkb8YXJXpvsNqju3gXRRAUnfrjfm6q9TICfufr77h/GLn9eskmH+k1qMl2MgEFORdGdyY3aHGxacRzUzQ8gysVrX+y+1gj16n/gaVwhJPM9MPgXFOWRc9Bo3A7CmVP/EktLE2WUxqp64ZYXeMbESLOA==; 31:4Ui1OjJgH4D3yL/0ksa7cZt2SQM5XOpzIj1LHXldD91RL43i2dVgaX2XHNUwCTsgYUSFIvXN3FU6uy8ahKNRe1I2xrKnlpakVOO26GrT6Q/Ga30/dzeI/ovRCDqhwxthE5eRMik37MB4RyYuk2740dlLcEbKdyMHBNKUKEuXlj+gi8BH5E91FB/pi6+nG8xl7uKi/g+SeTbU7p9cAwr52BX0cGKLtXyxGV8jCE5HtkM= X-MS-TrafficTypeDiagnostic: DM5PR07MB3097: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3097; 20:9Cu6IuvH4rl1kJfUo8Mjsu+IpxT6z+P9ojWJeqcYYF2+IHtwo8H7f7s7dQd0CgcwEeDMoIwEmunZRMy7OYKG1Lze2k1n57R9KCUOG1ow9Igrg3+d2xjJfTmSA3uz95guukM328DvVvlvXEidh9Z4e9fHqAeCr0B9JWbTYj7DyL59WKRKqwMklgEkjdeZ0bx81IGOqDZzHm42LIHaVg3zU8tVjLuiDEvYYAz/DcleTOrTZV2FN6DjPMf+NOKOOzyTySJwO7ala5Ke8PziRWn8gEyJwmEMzQE6LnEvreJNt4YLGLuUX/3Gh21n7zzFAVmTyUnxtNrVj1WtSu9pd7xm0ujwz5Gzekx1ZKKwXq9IoBXhwCvDP164ZvtOqtGbelC58XuhdawYv0wZ6yJCpwlyAz3EHL3azB3uER0pJkDNHGyFucj2pyjlOXEHG2N2G1/T1tlda5FFBm3xuIakYDKzFKobr4gSA1P1zD/PIFjoUIIrMp1jVhtCsEVN+pKSu615gEd/oK5MdMz5lTfLUzLjnj984FPPj+fJJYjDn8nG1wcEoBpmV5fs9FJFvoSCv7sPnG+uvtdy98+HXadVO9qVU+2A/kP608fKlmUjW7vsGBM= X-Exchange-Antispam-Report-Test: UriScan:(788757137089)(278021516957215); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(93006095)(10201501046)(100000703101)(100105400095)(6041248)(20161123560025)(20161123558100)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR07MB3097; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR07MB3097; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3097; 4:4vKfqQmjdPaSdykzkzoXK5pKe4/5RPAYag7T0kXqV69zqhfzpKqmR7ITbvv0mYxd7cljDBXXaPEooR4YitGwOGFx+jS/j0fNkCUQQywI5ZxpQ+Gi+cEcmWSZ/Jex6Lj2dw9lwgMilCpt+TpU1CWdI7oyz4nvkPiQq5pKOrGBtOWQE1xG9Bx0Tj/zTKXRACCxioLLul++EtuHsrGNoYHhkGfS1FAd1Xo5FmXn8/mLEX6IZ74HKmsqW598fPEuWJLj3G1eUQnkVGIcx7M2zhBDrb2PrqD8Pfd3VfKWnobOEo9zaZU14MjPrXvTiFkPl+T7j8CzAx9qoMoK88CTxsAp4w== X-Forefront-PRVS: 04347F8039 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(6009001)(346002)(376002)(199003)(189002)(6486002)(76176999)(6506006)(48376002)(50466002)(25786009)(5003940100001)(107886003)(110136004)(53376002)(53936002)(4326008)(101416001)(97736004)(6512007)(16586007)(106356001)(105586002)(2351001)(47776003)(305945005)(189998001)(33646002)(478600001)(5660300001)(2361001)(42882006)(6916009)(6666003)(2950100002)(7416002)(5890100001)(36756003)(66066001)(6306002)(2906002)(7736002)(81156014)(68736007)(50986999)(8656003)(81166006)(50226002)(316002)(966005)(8676002)(3846002)(5009440100003)(8936002)(16526017)(6116002)(1076002)(72206003)(110426004); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR07MB3097; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR07MB3097; 23:DzOoQOCowPQSNvdwfjDO2Syg9zovS4DTB0z7qHV4x?= =?us-ascii?Q?jkSjSB3oGPUGHwhkoTDS9QLAvUMs+LwtFkyEWLiA5LDNOw/cDoBOhJmVXYu8?= =?us-ascii?Q?I5Z2mvGXrgQQ7mEBa16R9I3yk0y+mFug3nH5Bc6XhQ6n/bGP12oDZQcq224G?= =?us-ascii?Q?stN/kAE6VeVXJ4Lzz9Gy8EPwUPa1hhmD6YpoeL+Tme/ZYri2I1zRn56EGKip?= =?us-ascii?Q?kbp+h2f5MPbK8lMC+HnoAPPJYOeqVCvIUjMw9yLvhsFPJBzqNr6N00mWHn2u?= =?us-ascii?Q?L/Da+qcthePRRUzs+qzJglgtWXCi52M4549IPEMP/ZSCNBhIrEB/8lk42sBh?= =?us-ascii?Q?jy0HMNeFptpHHzbcXePT+mSbFKn95BPyomaH2pL0uQLfrK/+lfxQVTbTsx4/?= =?us-ascii?Q?wOt99lK2woZP9To/pLKShImWzWk9kQlKl93DlAHd2TnVmHug2iDRJ7resf6g?= =?us-ascii?Q?ZDa1Pe9IuWt/yFqxafRO7cjSW70jydVEinebxDnsWgNN+omXEYRB0ZfeTuyC?= =?us-ascii?Q?xM9tUnzd2/sD+J/n62dz4xOdhHD0oWI2gm3MkLWStCWXs/1fVvbdN3zR2Ns3?= =?us-ascii?Q?m8xjrFi4vPZPyAD6rPZ3Nwh1VM+bv8O5z7zD2r4e9hbNTqYHjt2vhgKXXc+6?= =?us-ascii?Q?SUPRj7DHA2CIDJXkfq//gF6OTyiAqfJUM6CU9AD+xl5cIpn3rthCaIJw0CaG?= =?us-ascii?Q?cV7CoG2sMdwXsNJ7e+rMumhkfXG8btuYDPE748e+QsaXHwS0vlIUAf4q3RZI?= =?us-ascii?Q?+N2JtUrzr4T+xGdon33LPHmbuaCXzzIjSt5j6JlhDBXsJjOio19sYHqWkfRE?= =?us-ascii?Q?W6gpKls5JN7WxXZ2baaPpr6gMM6wrIj4irUmwqRYx2CsUOG1yFd4FyUoTUYz?= =?us-ascii?Q?WJ4ovkU60yo19/ZUxre7v26bO73lrix6iF2xmuS1SJamgJAjkAZmzc84VlL2?= =?us-ascii?Q?4HfMpZRDT4KAvptLeSddzdtOCtowVQnP1OTEg8yz0Br8hvOOj9CfKgu6tx7q?= =?us-ascii?Q?JjED3r6nBM0pnpoRg1EMPn94WyzJT2MMJg0kK1uCazynZEIhjsgGDDCFsCsD?= =?us-ascii?Q?wGGXI4E6P5KWN/olD7BIA3cr626d3aaEHzLnBuj4H5wzRtP//OzltOLYEgz1?= =?us-ascii?Q?QE6FyIua8vxP0g4LfWvAyC/e2ChxfADbwfv6TJMYOr8bg3u8VNbB/hUCl//q?= =?us-ascii?Q?fx6IaWWbFQzoq/Nh9Rx9NUmFuph15Sl5z2ITuVXipOXcFAnIDtlzI4ImgFEN?= =?us-ascii?Q?s7r3fqMvpDQ5LbLOjBCTdvI4keLqTRhw3ToBU2WgC36vECHMVLj2TxyIn7uV?= =?us-ascii?Q?P8IpraWP1KqGTDGLqAiPWP7weSXB6URX5a3MHzCeJnRwRfNKPxj2R4ioizX7?= =?us-ascii?Q?nbaYu3bUcqMb3su+ZcXwVZwCc8=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3097; 6:jw0Utn/06b3Qtk34JWFkAZdCc2qMSp1IyDCGXsYmSOfijhcUERhLLqKUvRHbCaMwO0k2F3k4u87atbTZrVSFyE5lvxYtE+0FMNR4Ed7TYfyGVdH+iGB54E2XcNd8FkXcRB5wJp7AxrT1mz/zT5mupebokafK0X03l6mo27dS2/X+umn+LhLFayfbtOuV1l4aVRdn9vsOLn1umSxIwul50Vc+NcMXlLMicfdk1oMVggEobj6SWdrpZB0CXQGjyafaj/Il++jpywaN0oHnuPg8xe7tUXG+rxghTCkkjD+j+IA9uMS7scq1uDgyZAEUw5f+WYo4KtrcX+0XjmrzLmcuCg==; 5:kkVUNz0Z+SFswY4uzqnYnX2XPZKWqWkA0Aysp7FarcPUhNfBoC/35kRaRipDBIOtXMPva0wNmyNCu2xSsJhNwrlewS6nCuC5FsjqgoUn5Z8MrMq526a0V58OnlYe0LAEyqrapLpdVjfZfq9q6kpbsg==; 24:0Q7+R8DU2A4v5PmV9gU6VTAGOdDUr82un9WHIMWD2D0R0HIZdKAxwg71VvEy2utjwXAjlylEtKvcvyN9MmrPL6XgiKX1p0XpYKR6AnaVFaQ=; 7:rJqyCXY50p+5sn0T5a26TtFc+8DkyXNTVA+awJRoSOTEBK7iT1jxdawYrL6eYAZ3eHix7dG4FZ2LvPjov6daErAbsIa6DgTzcxrax64A9wTc3UoGW/V6qyZtkTfZ1Rf521v7yx7JZ9MifyPmt6snh2kf0KlePtTh8h8JAFwdpviytuL9AyXjc0uiixo0ybyIrJltxkiXJTkE+F4GHwqP6lfJhvOOTUjJMzoboDHJSRI= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Sep 2017 10:43:05.3077 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB3097 Subject: [dpdk-dev] [PATCH v8 0/9] Infrastructure to detect iova mapping on the bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Sep 2017 10:43:14 -0000 v8: Includes minor review changes per v7 review comment from Anatoly. Patches rebased on Tip commit:3d2e0448eb. v7: Includes no major change, minor change detailing: - patch sqashing (Aaron suggestion) - added run_once for device_parse() and bus_scan() in eal init (Aaron suggestion) - Moved rte_eal_device_parse() up in eal initialization order. - Patches rebased on top of version: 17.11-rc0 For v6 info refer [11]. v6: Sending v5 series rebased on top of version: 17.11-rc0. v5: Introducing RTE_PCI_DRV_IOVA_AS_VA flag for autodetection of iova va mapping. If a PCI driver demand for IOVA as VA scheme then the driver can add it in the PCI driver registration function. Algorithm to select IOVA as VA for PCI bus case: 0. If no device bound then return with RTE_IOVA_DC mapping mode, else goto 1). 1. Look for device attached to vfio kdrv and has .drv_flag set to RTE_PCI_DRV_IOVA_AS_VA. 2. Look for any device attached to UIO class of driver. 3. Check for vfio-noiommu mode enabled. If 2) & 3) is false and 1) is true then select mapping scheme as RTE_IOVA_VA. Otherwise use default mapping scheme (RTE_IOVA_PA). That way, Bus can truly autodetect the iova mapping mode for a device Or a set of the device. Change History: v7 --> v8: - Replace 0 / 1 with true/false boolean values (Suggested by Anatoly). v6 --> v7: - Patches squashed per v6. - Added run_once in eal per v6. - Moved rte_eal_device_parse() up in eal init oder. v5 --> v6: - Added api info in eal's versiom.map (release DPDK_v17.11). v4 --> v5: - Change DPDK_17.08 to DPDK_17.11 in _version.map. - Reworded bus api description (suggested by Hemant). - Added reviewed-by from Maxime in v5. - Added acked-by from Hemant for pci and bus patches. v3 --> v4: - Re-introduced RTE_IOVA_DEC mode (Suggested by Hemant [5]). - Renamed flag to RTE_PCI_DRV_IOVA_AS_VA (Suggested by Maxime). - Reworded WARNING message(suggested by Maxime[7]). - Created a separate patch for rte_pci_get_iommu_class (suggested by Maxime[]). - Added VFIO_PRESENT ifdef build fix. v2 --> v3: - Removed rte_mempool_virt2phy (suggested by Olivier [4]) v1 --> v2: - Removed override eal option i.e. (--iova-mode=<>) Because we have means to truly autodetect the iova mode. - Introduced RTE_PCI_DRV_NEED_IOVA_VA drv_flag (Suggested by Maxime [3]). - Using NEED_IOVA_VA drv_flag in autodetection logic. - Removed Linux version check macro in vfio code, As per Maxime feedback. - Moved rte_pci_match API from local to global. Patch Summary: 1) 1nd: declare rte_pci_match api in pci header. Required for autodetection in follow up patches. 2) 2nd - 3rd - 4th : autodetection mapping infrastructure for Linux/bsdapp. 3) 5th: iova mode helper API. 4) 6th: Infra to detect iova mode. 5) 7th: make vfio mapping iova aware. 6) 8th - 9th : Check for IOVA_VA mode in below APIs - rte_mem_virt2phy - rte_malloc_virt2phy Test History: - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). - Tested for arm64/thunderx vNIC Integrated NIC for both modes - Tested for arm64/Octeontx integrated NICs for only Iova_va mode(It supports only one mode.) - Ran standalone tests like mempool_autotest, mbuf_autotest. - Verified for Doxygen. Work History: For v1, Refer [1]. For v2, Refer [2]. For v3, Refer [9]. For v4, refer [10]. for v6, refer [11]. Checkpatch result: * None Thanks., [1] https://www.mail-archive.com/dev@dpdk.org/msg67438.html [2] https://www.mail-archive.com/dev@dpdk.org/msg70674.html [3] https://www.mail-archive.com/dev@dpdk.org/msg70279.html [4] https://www.mail-archive.com/dev@dpdk.org/msg70692.html [5] http://dpdk.org/ml/archives/dev/2017-July/071282.html [6] http://dpdk.org/ml/archives/dev/2017-July/070951.html [7] http://dpdk.org/ml/archives/dev/2017-July/070941.html [8] http://dpdk.org/ml/archives/dev/2017-July/070952.html [9] http://dpdk.org/ml/archives/dev/2017-July/070918.html [10] http://dpdk.org/ml/archives/dev/2017-July/071754.html [11] http://dpdk.org/ml/archives/dev/2017-August/072871.html Santosh Shukla (9): eal/pci: export match function eal/pci: get iommu class linuxapp/eal_pci: get iommu class bus: get iommu class eal: introduce iova mode helper api eal: auto detect iova mode linuxapp/eal_vfio: honor iova mode before mapping linuxapp/eal_memory: honor iova mode in virt2phy eal/rte_malloc: honor iova mode in virt2phy lib/librte_eal/bsdapp/eal/eal.c | 33 ++++++--- lib/librte_eal/bsdapp/eal/eal_pci.c | 10 +++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 10 +++ lib/librte_eal/common/eal_common_bus.c | 23 ++++++ lib/librte_eal/common/eal_common_pci.c | 11 +-- lib/librte_eal/common/include/rte_bus.h | 35 +++++++++ lib/librte_eal/common/include/rte_eal.h | 12 ++++ lib/librte_eal/common/include/rte_pci.h | 28 ++++++++ lib/librte_eal/common/rte_malloc.c | 9 ++- lib/librte_eal/linuxapp/eal/eal.c | 33 ++++++--- lib/librte_eal/linuxapp/eal/eal_memory.c | 3 + lib/librte_eal/linuxapp/eal/eal_pci.c | 96 +++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.c | 29 +++++++- lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 10 +++ 15 files changed, 312 insertions(+), 34 deletions(-) -- 2.14.1