From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0054.outbound.protection.outlook.com [104.47.36.54]) by dpdk.org (Postfix) with ESMTP id 3347B2C00 for ; Thu, 31 Aug 2017 05:26:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=GF6bKeKToC8uFnQ+gaZ0OF0decaVyc4SYbhUvG2cMzk=; b=fBxj1OkA5Y4eW1aa8zDSaw3Cmy3vR6G7rXneVmlNIXxexbp6kpAUvAF/eSabPP0RABf2n/2QSrv4Pq995qHUV476qLh0OSpn8T12fROaO0kzpqQQjHGk5ql3ZyVfTMjpgB1sYagz7z9dfsqCxltnKV8q1cvF9yvzDzAnVGmphXo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (111.93.218.67) by BN6PR07MB3089.namprd07.prod.outlook.com (10.172.105.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1385.9; Thu, 31 Aug 2017 03:26:54 +0000 From: Santosh Shukla To: dev@dpdk.org Cc: thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, olivier.matz@6wind.com, maxime.coquelin@redhat.com, sergio.gonzalez.monroy@intel.com, bruce.richardson@intel.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, stephen@networkplumber.org, aconole@redhat.com, Santosh Shukla Date: Thu, 31 Aug 2017 03:26:09 +0000 Message-Id: <20170831032618.7120-1-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.13.0 In-Reply-To: <20170814161059.6684-1-santosh.shukla@caviumnetworks.com> References: <20170814161059.6684-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: PN1PR01CA0118.INDPRD01.PROD.OUTLOOK.COM (10.174.144.34) To BN6PR07MB3089.namprd07.prod.outlook.com (10.172.105.7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 49708762-42dc-4b3f-5375-08d4f02022b2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:BN6PR07MB3089; X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3089; 3:Wl5SXHkClWJVxasyavLeJcpMEf3AkCK/Z2yHfQeBJBwzoB8A57BL/1x+j/nEnQeTRpCikTSpkNj8xjKGiLWskY4I9dV2AcVVEvKyxLGszkajAvCW2ZKlF8y5hj5YknvzbivQOi7E9mnJ514XFAhAWC/PrvLe3atSvVUON4GIqd8CNfNyRG9uWnMh/F4rWxSLiyiRtOc0ThRxlYbEqX8lv1w2AFwuukJb//NusMZPZYY/FH/HELaZJzvewVhPLhe6; 25:Lfn0tlle7blJg0qxfdAuEsnUFsucK2hby3hW+OAzDF5uVwvJv4kNtyrQV74LUpNh2BuUTjhw4JLcjfNwF3tfyNPm+Q2GLw+OLmj33HNFJ4Zx9gyCwQdlLQkMWcs0V5o7Jl9b1uKGlOJFD06YrvZ+prfN3Kyk0iqFz8lzlXJMynNBgPD/NEZTkcGcdKCGucOokup1mpymACONquU01wPyIHQjAEwyM9i17OqeoMr2CAv6A3L1y5FMZdGKBDOpFL/24RYMc/c1lnWOqEeCAmz4dBRoN86arJKB49byh6G6zqsVbUUm1yYZZ1S6+XJ6smCyQZMWg5ot8Hk/S3cFuOEH8g==; 31:fX3TBEPeOPnlDFwFOSRJXuZ7UxUHO4bMsxsn0tZmDG06X9H8Xkll4nRuoSPL5qm4mjd+UlvyI8mP8QpSavz/KWnxekakG8cNCJ4gYQ5ds6oMHFZoug5QL17d9Jkg/qTry3pE1ECOW5RjHza1zWCe7LrzkheuP2KIfRx/2b+MbLNTOIXitdsRj3QCjZ0+wBYdZsvRqtl97NhnvxQOeJYV8W3giQxl7I78zZf7tKlTknk= X-MS-TrafficTypeDiagnostic: BN6PR07MB3089: X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3089; 20:b9y/Nb/UhnHlakXcRq7GHzvc0bouruImS+Fw0Lfqh/ZZdkSDFlkOGLcOGLUuqSn6AF/UVP0Q2EIF9Zo+ogZXP+dNTtOuMZB6P/dmFVrKtJl8qts8gCvpEHjfh11H2i2pejkFWRy/ZoilLQy/VLhbgFMzrjxTIKgbd5J4nxMLwtyxZd3yNhoKW26cGNVIXhOTfUexNxFI8TdiYVUN96BYEUmBLRtKvwWSwyHJ9AM1fHoDOEQfexO/xV12srkznRn4sF/cQuPgDKlumAkjlXKqlQQMEVQxSeDff+VYplh8j103f2FvzcLkILBdxpNJzZlWU2PIudEMkRvnGIRS5HCOvU4UhYJRPBRd3jHfM5wBpffa+Lo8aeEXPHhPIUClRlrSb/AyyAR+RJm/NxeZLIiu2lupHYHy4pcCQYl9yP7R5U9yps0MVdiYDb5sgsks1czn29H4oD3Tb14r402wvH+ho6XA5Zedzt6XKPyJpwT0GuQAId3C/wY+UaK3G6ScOnJzBOG7YMeqAayG4WLtf6WZ6nIZ0IfxSuWqqJLNVB4oexWkum4/Wvm+W0wyIZDVtb0eD7/FTGDk6+trHZ38rmqx9rw3LsOySopy3jPo5hk6l80=; 4:k7piDl2gk+5siYHRt/utsOX0QrGtxYW3lKO8agV/w2YWR75urKylrZA7aNnoF1wX3Voje9sy4tjMdSy3ByZBBM+3UVM1gkPAqiKoMantwDRI/mbpoz3kF/Uii4ABbctbSchTeohRRmfI3FBKOGM2q2n4xLuw0a4BMTRtmNNmdwl3mUk7cYNtaBpsyuloV7r21knyS9Ogl1cTYEgI5UY0VxQDDBEu2giX62V7utPDQSV9P6SO74o9Z9l+L+TkieRTL3q7WdemntfbZqcVTiWynk6hpx62iWOQOiADANVkxow= X-Exchange-Antispam-Report-Test: UriScan:(278021516957215); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(6041248)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123562025)(20161123555025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BN6PR07MB3089; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BN6PR07MB3089; X-Forefront-PRVS: 04163EF38A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(6009001)(199003)(189002)(81156014)(8936002)(81166006)(8676002)(7736002)(6666003)(5003940100001)(305945005)(2950100002)(7416002)(2906002)(6916009)(42882006)(76176999)(50986999)(101416001)(50226002)(5660300001)(6486002)(6506006)(48376002)(50466002)(72206003)(5890100001)(36756003)(97736004)(478600001)(189998001)(42186005)(2361001)(2351001)(33646002)(106356001)(105586002)(110136004)(107886003)(5009440100003)(8656003)(6512007)(6306002)(53376002)(53936002)(4326008)(966005)(68736007)(1076002)(6116002)(3846002)(66066001)(47776003)(25786009); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR07MB3089; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN6PR07MB3089; 23:zTrKQxsvozyJfG2JKNeBItUF8gRQeD13UN6R4boLk?= =?us-ascii?Q?SbzS8OQuOstN+cZOC0TLxWT4GmAnwI6NDUeRAlPyHabDoJanH2fug9seGHeq?= =?us-ascii?Q?2Q+Q/DJK8lR/mEsTYaPPLGDd5zfIocY4TZswoRUQpO0so7fXMqI2WxSs7ZSG?= =?us-ascii?Q?tazcbWrZQWLXa9yQnO5GZwk4ZySskvNiho9xfKTNEDukqEPGfhg3pUuGEHIC?= =?us-ascii?Q?8qfa7KKrMd0TGFlhli63Y1jPi7Ko2HTyrOxE4V7vuqrMNogmmXdRMThT52EQ?= =?us-ascii?Q?RUGtH8+j1GBLpHL4YiuckHkJAbRGNjwSb85W+8GBTgCHFOtafT6Ev7AnH4gf?= =?us-ascii?Q?ErTPhhGMSap0jekG7W8jW2FzG3dTVcaW5x0GFLPhPBVrEkKry7n8IUddRIrJ?= =?us-ascii?Q?97oQWlM6jo3pKx8NM+kNAKak6+rHRzvTJWpX9rnwJsZDSc3+nT9YKzutREBq?= =?us-ascii?Q?FDZ4sNs74Qya2Knb+An5G/k/gJphZPkDlpUVuLlQ+jBj45lELMYQHKSw5wOL?= =?us-ascii?Q?AR1NGfLQelRoyoQJSkUAMh5BaaSWvxLzFYjatXAOxhnTtn5F9A+ijMwvBqSs?= =?us-ascii?Q?X4JHNdEdqbuwe0rMd4UOprnAce3uCTa2CVjRmN2ZBGzIRhhAeRsSig80tWds?= =?us-ascii?Q?YDpwSMoNmq0aP1MxqphN2t2zOOjvYRa52iI+6Z7cvXiXHBv7nqUtygOORfcT?= =?us-ascii?Q?X67v4mzUv1oQptlNR6RFQQ1o1pKt8+aVFPEcAweK4MPXfACbz02ekv3Wh8O2?= =?us-ascii?Q?1qKAN+ZZ1j6IaIBabSzc7T6CVQsE0wxDJXfC/ltB5hyFJZEYXDQTN7+x1lbt?= =?us-ascii?Q?VlEa/lZk6fN8Vm/SkYogqTa+ug6cr+JZG/kmhY8zxDCsNUBHtI7b2wrnBGd3?= =?us-ascii?Q?9BQ19RLTN8sXtl3vKLiYtwQHuSz2n7RcR0Tananj2/YC4qFdx6j/IViHQLa1?= =?us-ascii?Q?6oWxKGsEjK1+vMKYFXZutw2x5ptYSZdBfGuWSN4i5/VbfKSG+vK0DYKlIGxD?= =?us-ascii?Q?TO0PzrcZ3aH5zrCV7EoRX9DIol9r4Y5nREZziBcetCAqSW4S347a7+gFCnv8?= =?us-ascii?Q?tGjDSQn6Sh6hH4yBfUWJ4WRIOOcLijANQpIyoQ2KY9R5F8HBz8plJH9qmnW+?= =?us-ascii?Q?2Vk2h3OYMVEmxjfp+4b2rAN5dpL1cqMj2CSqtnCNrqZxVYQQNRfHjvnlF42s?= =?us-ascii?Q?WUIroIToKawmJ/aUxTM4MT9ekDRd5NFG7zPIuMsQ9Hf2m+FK7cHY8kqA7fsi?= =?us-ascii?Q?VHJ2oWzJmx965itxUP1RnZUpTuugmhGVXfJLhsS6VBDMwe3d1jTfyaQeNqiN?= =?us-ascii?B?QT09?= X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3089; 6:gyGKigWMf+4JudJGxF4QBSleIQaRiiSqDkt3fL7p+/+NrqMINOmlw6vwAPnvIzpJjhberU5tdb9z1zOmOcYnefxVplZPgSXCboN6nFL9xx61y0YnkDSPA0WSbYKqQ3HI0WiZKzlVAzMi3IC9Ra29hZpFqHaXU3yEDGcJ7n/nCFy+icVV8SlNBKnOGpeCM73br93Jua/xmFffYeT2OOKcyo7qK99CT5y/DakzH5t+/c9aCT5KUYawYjsh1iVlbRoQrx5VvadHo6eOgWJjAvpf2UZBA4DPJ4oB4dSOC0DyCONoI5C6rDqb4mLXkuADwNe0kd7ny6AOUjN6qv0+4llSZA==; 5:+Q03L03VPrZlwIQ7z2iQVlhXxIc6kIT/CZtTtYF0EAW7lmYcpptfjB5zQu3ceTeX2rEPoyY2pxwm/yLF5+d1nTaXH5UNi/Mi0FFdNoQpxxU5Es74gjvDZGs/7iv3MJ4J9UDoaj/7CGxwDpCxqhJgUQ==; 24:qAdhiLU5fQY845BE63gzeq94zhu4bvrhxHBWK9+d/ZjT7iC+lUIuO9p8QkzN2gdx50znbARbhsd5dQ6cqxD9HLDNPS2D3F6dQZSJgmj1Y28=; 7:+OGBeyM9BIMHWbWLGX6WOS6YDpgxlTIwUlEnamytz4FeGSsnSQnPWwzCnsF1aNFUMqlW4sEaqfaTae2B5OOfLRts2a1NR5S/FlwdIlNw0a2iUedEA/zBGDQKCSkTQQnltpgn3qcXQoz0mJHAz9ieSQ1ft8NxUtKwQwz/w4AadEhH9fwJKW/VFZGx8l1wJCxSOvXPFbBnUOp2XPVlvTf7ylgiICgTsbmwEp7nOoWb+jU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2017 03:26:54.0703 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR07MB3089 Subject: [dpdk-dev] [PATCH v7 0/9] Infrastructure to detect iova mapping on the bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Aug 2017 03:26:59 -0000 v7: Includes no major change, minor change detailing: - patch sqashing (Aaron suggestion) - added run_once for device_parse() and bus_scan() in eal init (Aaron suggestion) - Moved rte_eal_device_parse() up in eal initialization order. - Patches rebased on top of version: 17.11-rc0 For v6 info refer [11]. v6: Sending v5 series rebased on top of version: 17.11-rc0. v5: Introducing RTE_PCI_DRV_IOVA_AS_VA flag for autodetection of iova va mapping. If a PCI driver demand for IOVA as VA scheme then the driver can add it in the PCI driver registration function. Algorithm to select IOVA as VA for PCI bus case: 0. If no device bound then return with RTE_IOVA_DC mapping mode, else goto 1). 1. Look for device attached to vfio kdrv and has .drv_flag set to RTE_PCI_DRV_IOVA_AS_VA. 2. Look for any device attached to UIO class of driver. 3. Check for vfio-noiommu mode enabled. If 2) & 3) is false and 1) is true then select mapping scheme as RTE_IOVA_VA. Otherwise use default mapping scheme (RTE_IOVA_PA). That way, Bus can truly autodetect the iova mapping mode for a device Or a set of the device. v6 --> v7: - Patches squashed per v6. - Added run_once in eal per v6. - Moved rte_eal_device_parse() up in eal init oder. v5 --> v6: - Added api info in eal's versiom.map (release DPDK_v17.11). v4 --> v5: - Change DPDK_17.08 to DPDK_17.11 in _version.map. - Reworded bus api description (suggested by Hemant). - Added reviewed-by from Maxime in v5. - Added acked-by from Hemant for pci and bus patches. v3 --> v4: - Re-introduced RTE_IOVA_DEC mode (Suggested by Hemant [5]). - Renamed flag to RTE_PCI_DRV_IOVA_AS_VA (Suggested by Maxime). - Reworded WARNING message(suggested by Maxime[7]). - Created a separate patch for rte_pci_get_iommu_class (suggested by Maxime[]). - Added VFIO_PRESENT ifdef build fix. v2 --> v3: - Removed rte_mempool_virt2phy (suggested by Olivier [4]) v1 --> v2: - Removed override eal option i.e. (--iova-mode=<>) Because we have means to truly autodetect the iova mode. - Introduced RTE_PCI_DRV_NEED_IOVA_VA drv_flag (Suggested by Maxime [3]). - Using NEED_IOVA_VA drv_flag in autodetection logic. - Removed Linux version check macro in vfio code, As per Maxime feedback. - Moved rte_pci_match API from local to global. Patch Summary: 1) 1nd: declare rte_pci_match api in pci header. Required for autodetection in follow up patches. 2) 2nd - 3rd - 4th : autodetection mapping infrastructure for Linux/bsdapp. 3) 5th: iova mode helper API. 4) 6th: Infra to detect iova mode. 5) 7th: make vfio mapping iova aware. 6) 8th - 9th : Check for IOVA_VA mode in below APIs - rte_mem_virt2phy - rte_malloc_virt2phy Test History: - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). - Tested for arm64/thunderx vNIC Integrated NIC for both modes - Tested for arm64/Octeontx integrated NICs for only Iova_va mode(It supports only one mode.) - Ran standalone tests like mempool_autotest, mbuf_autotest. - Verified for Doxygen. Work History: For v1, Refer [1]. For v2, Refer [2]. For v3, Refer [9]. For v4, refer [10]. for v6, refer [11]. Checkpatch result: * Debug message - WARNING: line over 80 characters Thanks., [1] https://www.mail-archive.com/dev@dpdk.org/msg67438.html [2] https://www.mail-archive.com/dev@dpdk.org/msg70674.html [3] https://www.mail-archive.com/dev@dpdk.org/msg70279.html [4] https://www.mail-archive.com/dev@dpdk.org/msg70692.html [5] http://dpdk.org/ml/archives/dev/2017-July/071282.html [6] http://dpdk.org/ml/archives/dev/2017-July/070951.html [7] http://dpdk.org/ml/archives/dev/2017-July/070941.html [8] http://dpdk.org/ml/archives/dev/2017-July/070952.html [9] http://dpdk.org/ml/archives/dev/2017-July/070918.html [10] http://dpdk.org/ml/archives/dev/2017-July/071754.html [11] http://dpdk.org/ml/archives/dev/2017-August/072871.html Santosh Shukla (9): eal/pci: export match function eal/pci: get iommu class linuxapp/eal_pci: get iommu class bus: get iommu class eal: introduce iova mode helper api eal: auto detect iova mode linuxapp/eal_vfio: honor iova mode before mapping linuxapp/eal_memory: honor iova mode in virt2phy eal/rte_malloc: honor iova mode in virt2phy lib/librte_eal/bsdapp/eal/eal.c | 33 ++++++--- lib/librte_eal/bsdapp/eal/eal_pci.c | 10 +++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 10 +++ lib/librte_eal/common/eal_common_bus.c | 23 ++++++ lib/librte_eal/common/eal_common_pci.c | 11 +-- lib/librte_eal/common/include/rte_bus.h | 35 +++++++++ lib/librte_eal/common/include/rte_eal.h | 12 ++++ lib/librte_eal/common/include/rte_pci.h | 28 ++++++++ lib/librte_eal/common/rte_malloc.c | 9 ++- lib/librte_eal/linuxapp/eal/eal.c | 33 ++++++--- lib/librte_eal/linuxapp/eal/eal_memory.c | 3 + lib/librte_eal/linuxapp/eal/eal_pci.c | 95 +++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.c | 29 +++++++- lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 10 +++ 15 files changed, 311 insertions(+), 34 deletions(-) -- 2.13.0