From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0055.outbound.protection.outlook.com [104.47.41.55]) by dpdk.org (Postfix) with ESMTP id EAB9A7D7E for ; Mon, 14 Aug 2017 18:12:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=8HvE/MHY0JWH95cXglLAe3f43HnveDclkN3JOqNvpjE=; b=nX41KWoU+nhZxDYwX+PQa5b2yrbCemru0RLmmptz97UGMXJLESfzUoZMDLhu9WNeG3r9gPHTcDd5SA1AwGNfKbgPbhTm+xjS4T7qA32XL/cURZYeZO48qqaLXKI1yIkMT6UopCj9w0sDWSxaYFXGJF4MBPpPokcMrVnzZ+1HibY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (14.140.2.178) by CY4PR07MB3093.namprd07.prod.outlook.com (10.172.115.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1341.21; Mon, 14 Aug 2017 16:12:05 +0000 From: Santosh Shukla To: dev@dpdk.org Cc: olivier.matz@6wind.com, thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, maxime.coquelin@redhat.com, sergio.gonzalez.monroy@intel.com, bruce.richardson@intel.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com, anatoly.burakov@intel.com, stephen@networkplumber.org, Santosh Shukla Date: Mon, 14 Aug 2017 21:40:47 +0530 Message-Id: <20170814161059.6684-1-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170724084004.25542-1-santosh.shukla@caviumnetworks.com> References: <20170724084004.25542-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: PN1PR01CA0087.INDPRD01.PROD.OUTLOOK.COM (10.174.144.155) To CY4PR07MB3093.namprd07.prod.outlook.com (10.172.115.7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 48d2ee28-e9c6-4374-d628-08d4e32f37f2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:CY4PR07MB3093; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3093; 3:Fi5eywYMHFsMWlAdsJFgJc7MLvKYGulXSPMcO0pQ/wek0GOBRoohKdzk+gnDRTD86rkW2NU+tA1LTAgfsMOQ8Qno2gADHV8mhQqjhmrTHdp6uLV+/3kLWRBf1CNXa3PKsimuSAZKiZIOhLP4mytRV3LCaWqwinyyJ/6XE3vhKlJ43yJB/4HVUTIccOSjhQXUNS417iOnTH4Yk8bWT+tSrfMeD5XReVHaybxt/tSggfiOG4moKhuZlrpXmyNINsks; 25:CrjfMjPIbvlNTJoFY3MOzM6znDIf12MPy8MJmS2tHZr1t5R6KEEbFfgYG4RXeVKOk2YL0DKGxNxr02QkNOXQI8sfSfup4LjTdwkE+dEd1q49Xqqjpz2WQQyMIA0u5aAHlslAMdNEaxwaqUNv7181jcHb9eq4xzY4F+KGHAr6zcre+9CoCqvl2xIDc4temlqg2BHhivW5+XKCseX4lDlRXEJW9X00X99I41aDJLrgW1Y6hGXZZE2cRxeoeMEl8ssY1muKmRRqyZQMQ8AjVyKwbnAoAruF2uf6Xt8IyHYXpglN4D1Q0YbDRiguCyBa7qf348iLL9jUN8H6zHFzjvzoXg==; 31:9UMtqpq8UZT3bz3I53bMSeq4DFvIVg7yXBNY3OIJop6RrXRJmR/ay/Pg/EUlwyrGVtGOVUNXrRORF5Y89VxlUZyBdFlgi0N5BDFcalQG6mrJYYGm8gH4iKTL7O2EJOYr4u+6q4EOPElakCJYJMuzylyz6jYpTCVXZAzvLHDPkgxR9rKHvIffESBTi+NcCOADbW7u9Kc8mEf8AORgUIce+x22L0KpZ1QxuH1j9qwU318= X-MS-TrafficTypeDiagnostic: CY4PR07MB3093: X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3093; 20:T+yduII7hYpc93yEVK/hev4wX8fzNYTFfS0vQON/r36T6N4C01T8Valp2ktf6LlctOUKwsV5PKSq33uGn9+Bt5JA2EyyjZLkD0eOFfz14o0t+xtGGdGDqCTdYv7IbpT0q21gExjmboEnx7mc9Gl6pEdYtpCeeyESs4aFj2ncNKrA/xoQq17CH1y8+I4163tFBDE/vVjfJiYLo0fmEzCwS27Krw3dZXgLF1210DNG35O/BDx5wxxA2g6OV/byzUFjvtxHigfUgojDNgilrsTm1tC3Mq6VLeE1tKLXE0j0bpqCYugNzBz580o3vJNMfICoMrmKrR2FX7vXHXTxGuWiqOo+xcwmkyKIYyQeD9sD/mDBcYYntOJIzuu+PtZgg+6kcoVKikXXfpKd4ee/72wAuqJqRCYKeLVPzEamMrQ7maEHyccEpfp2IE5IaAyOyUjJiHGCJo91yqKvsmIcOrLC3rQO6vcBsYhnNfWcigHG15Dee99U+eiWxglAaJnhA2k4a0PVnt/eF6XaqWiwn4MeH2PyccgVExCbbL+SYETis0ecbjP64xdSJDIgPKlzqQl9PFiYA6bnoT/D9i3p3SJWSVhSGM2TkG6hUxS3Jpp/IbU=; 4:9D9voq7O89vD+XiaKFmyUoej49f10BI3F0pq/rt9m+1k2/4qPW/w/ynjsjJUXcv7XGFgh6kjejnF2p/X0YcKegfQYNNHgYEHlbCmqryYqhBWIdb6h3GutTNpb2xl2R/4TXkoaMKvnCkVwZGiYDQMTzgWWP/y1FWRvoINRYAkim26EIqlUYwayagDhHqmNlR6PZK164pjlHL4xO78QkbPUgjSUtTH8r5mLv7dr416F1zAEfKKdirmjqdA+WMTBxBG9LKMjZJbSgyAMXKtxkg3uhhxOIk8zZX8rrF4zGuH/sk= X-Exchange-Antispam-Report-Test: UriScan:(278021516957215); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(100000703101)(100105400095)(3002001)(10201501046)(6041248)(20161123562025)(20161123560025)(20161123558100)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR07MB3093; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR07MB3093; X-Forefront-PRVS: 039975700A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7370300001)(4630300001)(6069001)(6009001)(189002)(199003)(76176999)(101416001)(1076002)(50986999)(8656003)(6116002)(66066001)(72206003)(7350300001)(5009440100003)(3846002)(107886003)(53376002)(47776003)(110136004)(6306002)(2361001)(8676002)(42186005)(6486002)(50466002)(48376002)(105586002)(106356001)(189998001)(4326008)(6506006)(5660300001)(2351001)(25786009)(6512007)(81156014)(478600001)(36756003)(33646002)(7416002)(5003940100001)(7736002)(305945005)(68736007)(97736004)(5890100001)(6666003)(50226002)(6916009)(42882006)(2906002)(2950100002)(81166006)(966005)(53936002)(110426004); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3093; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3093; 23:y2eNcao65CqukDj7ev10DwK6It3A1JktQp/Qh5v4g?= =?us-ascii?Q?26P7ZvW0XQ85OGAYrXZrmUXImi+u1BUFyJaTt6O0J2DxdA1BMaBuP9G94HL6?= =?us-ascii?Q?7uI9wWNttuXAoAEYhHCFAf/rMrArTtvqbNnMzq6KtJ3KYW18GrbLkdmMqq8y?= =?us-ascii?Q?Z87ebsKqlKfMVerUz5ehJ6Xb8V+M0+t5fJCVTNToE1IOqI52meAcqeuUDzIS?= =?us-ascii?Q?hKPphktPqv3EOphFHt+dCC2EPxVEXusnEtemwvVw40nKNVXGguohHNdIikCR?= =?us-ascii?Q?lkax85ycP6m4tsIxMtioUj5r2kCDyEajnD3Rqmuni/jge5M5PVXD2v2m6zjg?= =?us-ascii?Q?HHCpyv1PniQBGB1fpc2bUTzjlwdHlSidpznn0BVF7LM4nFwwmqBQ3wtPABUZ?= =?us-ascii?Q?EaFWUeHZHATErWx69qTGzp0nnFeKWHQL72p9sSfgOhBhossSJD3LT7Nu8/gF?= =?us-ascii?Q?SIkV9bvR53UtfXS3RKboSudNKimPxJBanSzhlkby66J8FXxdZE9sVMGNI7aS?= =?us-ascii?Q?3N9aBtRANqRrkwego2wDRFsct7T1dAK8YGPmZp2CZynCrNX/ELhq3SGEKBsI?= =?us-ascii?Q?/+qfzmjrDP94RQ6rnKyIoR6jlrvKmZCL2zB6NUmk0Ft4p3aHggSQ1TH2MHYZ?= =?us-ascii?Q?f4qiNt5KoUq7mClmb0mth6n/FUcG8UT5HcZMvh3tbaUc6rh+PiSwEifSuPff?= =?us-ascii?Q?0NWo1xAedl8l6r5S/S8s+tLCE/Kw7hbidurYwN6hut/xKV4iUrYMFVKg5lyA?= =?us-ascii?Q?cfJYyF0b9JrYKQtjEViBogkCkQd/wERWx/++rKIRcl9VzFkIMLF9zSBi3FER?= =?us-ascii?Q?AnJv5diW8xlnkG6YtQWI24iW4MRwx3mWJa7vx3MlzEH7Uc4DlXPN3qUnc3o+?= =?us-ascii?Q?hjXUDpiqhdN9z88IG25Naa/cuvritQ1Q4N07aF7fUYKCZ8fQ9vqInLBsO0/2?= =?us-ascii?Q?rKP0vBsAz2sniln90oo5fufpHIKCb8AW5Y9wr4zso3/85cP5Yw3OrZlvRAJz?= =?us-ascii?Q?SnKhVR1PqbZ3wNsX/qZOf14pJxCxsfcel561jOfrl2yRI7jeH9RnfdOILAB/?= =?us-ascii?Q?xeHbWamv235iAPbebvQb3yL4wQzdwa2fYm+1I6Is6ypwoJkNsZ2LHLY8CR+A?= =?us-ascii?Q?mvtVZboawQpZcNo/sxu2D0Q8TaY/SDJPezw63Mw0B10/zAMLHiWqY7zdfKim?= =?us-ascii?Q?Vq1KuLkthDkQRBtQxXQjCVs6/CBF4P6BZhizcwR4o2SBSLpZz2ZGIsfctYOM?= =?us-ascii?Q?X2IAqS8ZVkc4e8jiVmbqrHurTKZ2RHUgXTMhFgebP06PDYdtps3SpBvDkY9E?= =?us-ascii?Q?IN6xxSOtSeDbdpXWfVJellxo7EZOdtOH+LtOET63Mg2M18j7aLmuiZzerlIe?= =?us-ascii?Q?cOo5Swnj32mRgWNKM2iTkM0GXY=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3093; 6:PCWOYq8p3h83wBwM1bdxXB8Nj8oPQyg54ZFbtuGvVO5rkQ3QkXgVjT+fD/D1NAsUGVp1XcldXz/W4OEkWoRm+ZxbLxHad4Our3048+R2L6fofgsI9YBPqqRgebQN12rtPqzNnKvQSVz1/u+ZYpX9qVi98ltqMegeHJsKbUrtC0WNXFxtfqFTBg83wcCH1O/mn0ZAiDzX4J5Yc7NiKODK47aMq9bM9Z43nhGFFnUe8uA7jg7D8I7o+HD3ZHPNFhPAcAYwSveQmAG4gmRPTLAvSt+Tnlwb8n3YZjgVNrC2BNUw7aUHsjneKnf8AetJUMDUcSltJtoM/gtji9eseq9ALA==; 5:JmcSD3Igwx5r6CBb3KjzTrWWKjtOLksPe91zXaBflJjXInAtLtEN4uXiFV57tWv+7uGhbbXU1Q3WEL13AdovnuIWLs2MLbxsoK7dn96w77lZsCYzHNqj973n2qiWDRer208jNHUZO9w06bgCzEyNFg==; 24:Pd5LsWQARoA1hSXtpat64xkT/p+vHr/nxhCMDCv2qF2+fMCD45wfKpTJa5bFc/OYskJMWGWzCRrFE+JIP57vSAxdp51RZD6Ids3DYLGIE7U=; 7:4aPOYwfpbpSTj/R9PJcDbqSE/qnBMvDHaNRbdt3V03Z6qBrxJUEl8pOBLgGxx4NboE8HFwe0++RSLE8irjbnbYiyiyfitIgCkFo2iu1niqqIiVh0YcYWHtNwSkYT3tCR2vn6wZVt11nkugLjoWdszh00dMywK8Rn6kPGMBJdXk+vovvBKBXZkYK8BlegP+8FXoEY+8XLXjwEjotyWP9L1O6a9RQHwp5EzF6ms0xB/Xw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Aug 2017 16:12:05.8336 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3093 Subject: [dpdk-dev] [PATCH v6 00/12] Infrastructure to detect iova mapping on the bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Aug 2017 16:12:13 -0000 v6: Sending v5 series rebased on top of version: 17.11-rc0. v5: Introducing RTE_PCI_DRV_IOVA_AS_VA flag for autodetection of iova va mapping. If a PCI driver demand for IOVA as VA scheme then the driver can add it in the PCI driver registration function. Algorithm to select IOVA as VA for PCI bus case: 0. If no device bound then return with RTE_IOVA_DC mapping mode, else goto 1). 1. Look for device attached to vfio kdrv and has .drv_flag set to RTE_PCI_DRV_IOVA_AS_VA. 2. Look for any device attached to UIO class of driver. 3. Check for vfio-noiommu mode enabled. If 2) & 3) is false and 1) is true then select mapping scheme as RTE_IOVA_VA. Otherwise use default mapping scheme (RTE_IOVA_PA). That way, Bus can truly autodetect the iova mapping mode for a device Or a set of the device. Patch series rebased on version-17.08-rc2: '67c4b6db68e199247b5dbd63f560582640b180bf'. v5 --> v6: - Added api info in eal's versiom.map (release DPDK_v17.11). v4 --> v5: - Change DPDK_17.08 to DPDK_17.11 in _version.map. - Reworded bus api description (suggested by Hemant). - Added reviewed-by from Maxime in v5. - Added acked-by from Hemant for pci and bus patches. v3 --> v4: - Re-introduced RTE_IOVA_DEC mode (Suggested by Hemant [5]). - Renamed flag to RTE_PCI_DRV_IOVA_AS_VA (Suggested by Maxime). - Reworded WARNING message(suggested by Maxime[7]). - Created a separate patch for rte_pci_get_iommu_class (suggested by Maxime[]). - Added VFIO_PRESENT ifdef build fix. v2 --> v3: - Removed rte_mempool_virt2phy (suggested by Olivier [4]) v1 --> v2: - Removed override eal option i.e. (--iova-mode=<>) Because we have means to truly autodetect the iova mode. - Introduced RTE_PCI_DRV_NEED_IOVA_VA drv_flag (Suggested by Maxime [3]). - Using NEED_IOVA_VA drv_flag in autodetection logic. - Removed Linux version check macro in vfio code, As per Maxime feedback. - Moved rte_pci_match API from local to global. Patch Summary: 0) 1st: Introducing a new flag in rte_pci_drv 1) 2nd: declare rte_pci_match api in pci header. Required for autodetection in follow up patches. 2) 3rd: declare rte_pci_get_iommu_class. 3) 4nd - 5th: autodetection mapping infrastructure for Linux/bsdapp. 4) 6th: Introduces global bus API named rte_bus_get_iommu_class. 5) 7th: iova mode helper API. 6) 8th - 9th: Calls rte_bus_get_iommu_class API for Linux/bsdapp and returns their iova mode. 7) 10th: Check iova mode and accordingly map vfio.dma_map to _pa or _va. 8) 11th - 12th: Check for IOVA_VA mode in below APIs - rte_mem_virt2phy - rte_malloc_virt2phy Test History: - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). - Tested for arm64/thunderx vNIC Integrated NIC for both modes - Tested for arm64/Octeontx integrated NICs for only Iova_va mode(It supports only one mode.) - Ran standalone tests like mempool_autotest, mbuf_autotest. - Verified for Doxygen. Work History: For v1, Refer [1]. For v2, Refer [2]. For v3, Refer [9]. For v4, refer [10]. Checkpatch result: * Debug message - WARNING: line over 80 characters Thanks., [1] https://www.mail-archive.com/dev@dpdk.org/msg67438.html [2] https://www.mail-archive.com/dev@dpdk.org/msg70674.html [3] https://www.mail-archive.com/dev@dpdk.org/msg70279.html [4] https://www.mail-archive.com/dev@dpdk.org/msg70692.html [5] http://dpdk.org/ml/archives/dev/2017-July/071282.html [6] http://dpdk.org/ml/archives/dev/2017-July/070951.html [7] http://dpdk.org/ml/archives/dev/2017-July/070941.html [8] http://dpdk.org/ml/archives/dev/2017-July/070952.html [9] http://dpdk.org/ml/archives/dev/2017-July/070918.html [10] http://dpdk.org/ml/archives/dev/2017-July/071754.html Santosh Shukla (12): eal/pci: introduce PCI driver iova as va flag eal/pci: export match function eal/pci: get iommu class bsdapp/eal_pci: get iommu class linuxapp/eal_pci: get iommu class bus: get iommu class eal: introduce iova mode helper api linuxapp/eal: auto detect iova mode bsdapp/eal: auto detect iova mapping mode linuxapp/eal_vfio: honor iova mode before mapping linuxapp/eal_memory: honor iova mode in virt2phy eal/rte_malloc: honor iova mode in virt2phy lib/librte_eal/bsdapp/eal/eal.c | 20 ++++-- lib/librte_eal/bsdapp/eal/eal_pci.c | 10 +++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 10 +++ lib/librte_eal/common/eal_common_bus.c | 23 ++++++ lib/librte_eal/common/eal_common_pci.c | 11 +-- lib/librte_eal/common/include/rte_bus.h | 35 +++++++++ lib/librte_eal/common/include/rte_eal.h | 12 ++++ lib/librte_eal/common/include/rte_pci.h | 28 ++++++++ lib/librte_eal/common/rte_malloc.c | 9 ++- lib/librte_eal/linuxapp/eal/eal.c | 21 ++++-- lib/librte_eal/linuxapp/eal/eal_memory.c | 3 + lib/librte_eal/linuxapp/eal/eal_pci.c | 95 +++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.c | 29 +++++++- lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 10 +++ 15 files changed, 296 insertions(+), 24 deletions(-) -- 2.11.0