From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0088.outbound.protection.outlook.com [104.47.32.88]) by dpdk.org (Postfix) with ESMTP id EE061235 for ; Wed, 20 Sep 2017 13:24:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=yBlLSueFx7shB5Yx6ssFyRql0ooYosPxYZ2Kc0dHxlU=; b=BaL5TIV5p2mPwQ8NmomtmcOg4e3RKNk0ETEShOhDiwgd4BIkPH9MuhUPtvIaSFMDLvMSdarkJexr+wx76WIKIKjmgZrhebM2/rjcu1Lnit+RtrrIfLtUgAHLZJBrJjOuZ3EHS8sIgBLWUfbueGEhW7OEQhEulPlo800OFMuAghg= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (111.93.218.67) by DM5PR07MB3098.namprd07.prod.outlook.com (10.172.85.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Wed, 20 Sep 2017 11:24:38 +0000 From: Santosh Shukla To: dev@dpdk.org Cc: olivier.matz@6wind.com, thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, aconole@redhat.com, stephen@networkplumber.org, anatoly.burakov@intel.com, gaetan.rivet@6wind.com, shreyansh.jain@nxp.com, bruce.richardson@intel.com, sergio.gonzalez.monroy@intel.com, maxime.coquelin@redhat.com, Santosh Shukla Date: Wed, 20 Sep 2017 16:53:47 +0530 Message-Id: <20170920112356.17629-1-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170918104234.9149-1-santosh.shukla@caviumnetworks.com> References: <20170918104234.9149-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: BM1PR01CA0076.INDPRD01.PROD.OUTLOOK.COM (10.174.208.144) To DM5PR07MB3098.namprd07.prod.outlook.com (10.172.85.8) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 888b2005-3f83-455e-a994-08d5001a30e2 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DM5PR07MB3098; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3098; 3:L9r8yZRQLKPQc+IFkhhjOq6AkUVSNJvl/iylSujTPsslzwxAq/J5hIDaQhCKn86nfVOi8Ewl65xQeu4Lphz44krk8oZVDzt4+Q3nEAfBjxccApRJEE2EVlxh0tKZHjjqdsobmzHiW19PP3/2MdEnJvtLtlw5Lh/HSVL1NEApztlWJLq0E/8XfEvDW/loqft8sjaDqh47yndOx9jpQNmdjca/HLk6hk6U2boDN1i/ZXNtkudFbl5UBgYZGq3e6aZO; 25:S9aWjx+mwf9IdzYXhcEPC7mvWWGCiZo1OW53QM133tat7nXuJ/62yvnmIR2MOz2Z7CD7Jb4C1vvsoqPov0nLOViCbIAqd2aPNlGNqVAVDq/076Io3rPl5JP85IPsmiYyNKXsuP+PfPjK4cg33asiflZNDabLfnOfm0abOCBIMhg2/QOBfLyXplmFEWkDPRulxdbJOSVLatpyjcgC3iGiPjjxr+6Bg3pUgDWGFoW9bbAgj8EAKShTK9KPE/bcxnyaxHe9ga7sfy+jxxkyUDjnYj/gWQDvPmwa758oR/LT1CWyfavBnzNFGT4eYfOSeslfNe95ofHN8h7iKLvNoacL3w==; 31:2B+oyK2B1gM3aSMKYznuTGC9uEb6SNv5LLxbiqH9FoBP7EmO3gqooD/dkO9Eudyx5jv0EI0driBbfG9d+3qfnWogixbXco7xoVZ6lqUWabgGa5Xdmz4dlydI6LpA8ieaQcpbzTYyiRY4tDlR5ipwtfDBzXHzMo6UFhhZ4jGfUgB/1Jp8zQyNLmv0dhQAcQm95JGLdNqC7fxDlHBs4Fu52lFQowoQaVSHB3FLB5PGX84= X-MS-TrafficTypeDiagnostic: DM5PR07MB3098: X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3098; 20:6MXdvXbvuxQQdiHY/1B8nf9KyqqDJyTzGlh4OF/aPpH4hUIxJEg7JKmKjO2EWlKn9KiQuv6eWdURc5NC4adFhBr7DOgc4fyIqUVw3KRd8JZqy4f12/qQ9N0I634tjADw00+PaOT1EZ96Ku0q2oIPvIDWAUOyWjfWxeRc0ANIB4u4CPtpVS1NFHVBGgJv4R8+pVwipoi4EV63HFDAqNlRBrTuogmes/qhsV9W+lDJndqkkTJD3307qapXul5TPzv3o9qAN494ypHZM9sAnIx+kXhrC897mStKWF/JnmifhaTWSE8sv5uyTn4z6BNRlsoA3T5pFuYY0fRgLu/aWoFDW4R9X35SqZBh8O/HaBL0DgpTtSQR84JE42kbGpkHrpNjqQ/jUy1wkyVqzU/d9cvf8lkEyVhS/taaTORRgqKf82ptFd6VQjOJUJAh5Is0njKTemCsYNhUh+YPhIpEy0APA42WMvNpNbg+GCB5P40btcaJIPvAwSGBs9YG9jyl2hnXY2WCOnxIlJ1RT3GzubcQcRBORF9PadtgLR8wwBA044nxgRiHfBVD0GfelNGv3CbyZM1lpdZ9H1P/9+x4TNBsv79HZrtU029lY6KIxmeqkr8= X-Exchange-Antispam-Report-Test: UriScan:(788757137089)(278021516957215); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(100000703101)(100105400095)(93006095)(3002001)(6041248)(20161123564025)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR07MB3098; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR07MB3098; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3098; 4:fxsGGKCn4/OHA6nOglraDfUe+ylvA87vqXwtP5ZQo7H400GaOoRsRc/gt+82Ah6tjPNytLJYxKBqOSWSuW7fTisxuZoPeA26Aa5CPhlpkyssCZh1BoWZqmbNdGi4qWeK1sRdfYuzifuKXldugfxA/gtAAcHtv9CgUGEryfJVK5O0HIqSule3QHpjQ2ZB7Rfg3mO9jqvX3XPmpNVQyqxDUx4BlfXYdmH15d1yq5DzazYbMol9lbrHJcYSamfEUAC5yvikLHmxxhIB8GYVBUPjsfuYKj3zakyV6qqIdDQLA60HBrQv6Ya9mgu4hLVrYQW+6ELE0SUTezbPdjw3dh094g== X-Forefront-PRVS: 04362AC73B X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(346002)(376002)(189002)(199003)(66066001)(3846002)(106356001)(8936002)(7736002)(2361001)(33646002)(6506006)(2351001)(47776003)(25786009)(105586002)(53936002)(6116002)(316002)(36756003)(6486002)(107886003)(68736007)(5009440100003)(6666003)(101416001)(6512007)(81156014)(6306002)(81166006)(97736004)(53376002)(4326008)(50986999)(76176999)(16526017)(8676002)(2906002)(16586007)(5890100001)(189998001)(305945005)(478600001)(72206003)(2950100002)(5003940100001)(966005)(1076002)(48376002)(50226002)(6916009)(42882006)(50466002)(5660300001)(8656003)(7416002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR07MB3098; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR07MB3098; 23:29J0A19T/VZ5UNHCwHSFJOZIl3XzB3NkEIwxIZuvL?= =?us-ascii?Q?viXnaA0im+3DrZJbXp880p81VH3hyA6VeDEtfv7uHTsrpbBgHJbH1vEAzh1i?= =?us-ascii?Q?b1+Mt33tMzABsJ6OfK4QUr4SlsypAtZQiXjicLWDp+Ho16pmo2m58OLMut8E?= =?us-ascii?Q?hu+BObvIecFCLcFeuqRWrMNpQW7BQQe5Y3lkxvoyqCz+UabjtygHQXV/rORW?= =?us-ascii?Q?XTLh2QxzsbkDq9bYhY90NNbZ6LfdHqawQhsj6r2S+6m353z7Dnvn2pPPv+QI?= =?us-ascii?Q?QnaOuDtUfoRzmGsa8f77KH1NhYgWyeT6mAmM0IVzbAj18LPLBufPV2dHluO5?= =?us-ascii?Q?G8Fn8/X6jaFEnKFRPeK/OdWoHUWaafspM9H/GqqQB7wx8BoxzrpavNV2ofUv?= =?us-ascii?Q?zJT6CYYzQqJwBsja33JZy2n3E1QTZjUzWVHZSc2f0vOTpFkJ0aNYBZ405X8I?= =?us-ascii?Q?q5SPZMhe4C/NKgDmDzTVepT8yw2Cw2xM6FPBxzlb9Iljef6ZgW+m7BCk2J/u?= =?us-ascii?Q?RCwdEX+Hyssy3XLSznby0x4V2uoHvsAUL2cHqNqCRnqsf3350YPUfSjIQrIH?= =?us-ascii?Q?l21Mx3s+Rb5zp031EOSIEpuqfCiMnFdjOw1PzUlg3WnMs4ZGqFRzpi0BnH1/?= =?us-ascii?Q?cpRwTXS5a3q8pskOaf3HS3EcSn/XjRVBMTCyAtb3iu2iT6BvbRsJTN1Z4AVQ?= =?us-ascii?Q?I0kPEkYBBbvHsrwKTfE3q9QLWp8cLlHgdr0DEcG3r+qqybUzMXKTayParpk2?= =?us-ascii?Q?mla3BYG7e1wmyvdAtAYjcWWuq6M6TCYs9E/EZ9kyS4H/8g6vJxC+xiM8cX5Z?= =?us-ascii?Q?Al2qgJHqNOgZwY7hBmtMEGmduM03Co96wN1VnI32Oc7N+W+f4A9HqcfdoL7X?= =?us-ascii?Q?mtEX51OLqcmWhmuafDej2Ua6yumguzvQvx5WnLK7lrhZWYmZTkQPYp/Tr2wD?= =?us-ascii?Q?xiTQu0Uf64j2ORVOZt+do+LNhwSFP5PVVdQQSRH4l5d3I9VYDrrL316DgnAO?= =?us-ascii?Q?opuw3IQwqQ7z3hQMzBfjcry2vkNehgGZ7eqXGyxwCqhsnL4+gKS8bb5PKbTL?= =?us-ascii?Q?8PqCL5PUpUFdgpEZKHcCeGKv0ghU3dpcazb4gUYyQEbVobRkEbTvAeITKXNS?= =?us-ascii?Q?3wvg7vOfLdosVGJ2UWMTDXjWtDbHVBGSSHxE73x3FhNNYE6vwwzwQw79Eyz1?= =?us-ascii?Q?ROTIMy6q0n7KyBAMyJpvOGVWvx3MSFxUiLAxpFeZa8N594zO7R1g+zjYdVKk?= =?us-ascii?Q?oAJ0giFr4EDV4Jby32m/VnpnMLUKO4ac6FQ1YdEW8uAlBFPFtfMzsfYcNRTY?= =?us-ascii?Q?ZZPbq3isYRuFNK0q+vsmHMtLLqmbOYm2Gc+ArnIBhkQ?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3098; 6:gkCEe/2QDpRdUOKNAkjdrlCJl6mrh8V+r0pmgYCZUrLwnVPLffJ2sd0XNmhdSS8wdeMQGRSl2evNLTnvLa8GRGzLJ6vKrR7WMXOerO3/PXeisRlBK80AlANt6sCHVnCEB+cP+66fhl5VFnldb1P8gQ8HRw/4/jtY61FV966QEtsJ0gj4p7PbZw2KYCRQ2WGcL8VldRVCAKwP5hSGQYxsy4i0+wQMb9cM+PDM21OvM1DFku8Tl8u+RVtRx/jdbGiaO3JRbqEEQUA+47sIRkqON6/4SB93rVUL3pBgLH4W2V/LoXCf6afn+7HN6lbiFNBSXurUdYW+7XR0H5VPU/GN+w==; 5:MHJUrfcMwGYSqs+bfeM/96sE+T5EVJtOBLUbZKLZIQWxUWLqXw0f+jTY77qHEowF8qo41RZ7YLbZjNg3w2nQBfZlrXVn0cKPSCP/hMj2v+iqK0/5PsFKkcoo8bVNSr5MK+gP8Fj2W8h6Gn/CfpoEhA==; 24:8zrvF1ae7jbSqRUh8+FMTSe5WtLjdVXZ3+ajforlWx9YDS3KRleCtstX5GfA/Yw6sqLWlgAlok/VEYR/fRn8sp+h0Oc1Xv2AGMXCB40Lz7k=; 7:t2n4jmaiY0wqc75NUQAOXZY9bLGecRFhix8GjEk9KjiCX4uQsytyIuttv1x85tQwqp4PH0qR4dRtPVuKRJN53xkAXEKRF1fgGS2Bi+F9L419ISknxYFCIeIPGkp9yCIXuqFjIcYdB+tPi6AcXNhnUJELnaUGRQ2dfy4DI3dWpQ+8hiVtj6LSr4mTKHAjrDfr9uz6AXhebVptpRM1Ku1ljLuugULcufryNW43sb2H01w= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2017 11:24:38.0384 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB3098 Subject: [dpdk-dev] [PATCH v9 0/9] Infrastructure to detect iova mapping on the bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2017 11:24:45 -0000 v9: - Added Tested-By: to series. - Includes minor changes related to linuxapp api stub in [02/09] (Suggested by Anatoly) - Series rebased on tip commit : aee62e90 v8: Includes minor review changes per v7 review comment from Anatoly. Patches rebased on Tip commit:3d2e0448eb. v7: Includes no major change, minor change detailing: - patch sqashing (Aaron suggestion) - added run_once for device_parse() and bus_scan() in eal init (Aaron suggestion) - Moved rte_eal_device_parse() up in eal initialization order. - Patches rebased on top of version: 17.11-rc0 For v6 info refer [11]. v6: Sending v5 series rebased on top of version: 17.11-rc0. v5: Introducing RTE_PCI_DRV_IOVA_AS_VA flag for autodetection of iova va mapping. If a PCI driver demand for IOVA as VA scheme then the driver can add it in the PCI driver registration function. Algorithm to select IOVA as VA for PCI bus case: 0. If no device bound then return with RTE_IOVA_DC mapping mode, else goto 1). 1. Look for device attached to vfio kdrv and has .drv_flag set to RTE_PCI_DRV_IOVA_AS_VA. 2. Look for any device attached to UIO class of driver. 3. Check for vfio-noiommu mode enabled. If 2) & 3) is false and 1) is true then select mapping scheme as RTE_IOVA_VA. Otherwise use default mapping scheme (RTE_IOVA_PA). That way, Bus can truly autodetect the iova mapping mode for a device Or a set of the device. Change History: v8 --> v9: - Added Tested-by: signature of Hemant. - Added linuxapp stub api definition in [02/09] (Suggested by Amatoly) v7 --> v8: - Replace 0 / 1 with true/false boolean values (Suggested by Anatoly). v6 --> v7: - Patches squashed per v6. - Added run_once in eal per v6. - Moved rte_eal_device_parse() up in eal init oder. v5 --> v6: - Added api info in eal's versiom.map (release DPDK_v17.11). v4 --> v5: - Change DPDK_17.08 to DPDK_17.11 in _version.map. - Reworded bus api description (suggested by Hemant). - Added reviewed-by from Maxime in v5. - Added acked-by from Hemant for pci and bus patches. v3 --> v4: - Re-introduced RTE_IOVA_DEC mode (Suggested by Hemant [5]). - Renamed flag to RTE_PCI_DRV_IOVA_AS_VA (Suggested by Maxime). - Reworded WARNING message(suggested by Maxime[7]). - Created a separate patch for rte_pci_get_iommu_class (suggested by Maxime[]). - Added VFIO_PRESENT ifdef build fix. v2 --> v3: - Removed rte_mempool_virt2phy (suggested by Olivier [4]) v1 --> v2: - Removed override eal option i.e. (--iova-mode=<>) Because we have means to truly autodetect the iova mode. - Introduced RTE_PCI_DRV_NEED_IOVA_VA drv_flag (Suggested by Maxime [3]). - Using NEED_IOVA_VA drv_flag in autodetection logic. - Removed Linux version check macro in vfio code, As per Maxime feedback. - Moved rte_pci_match API from local to global. Patch Summary: 1) 1nd: declare rte_pci_match api in pci header. Required for autodetection in follow up patches. 2) 2nd - 3rd - 4th : autodetection mapping infrastructure for Linux/bsdapp. 3) 5th: iova mode helper API. 4) 6th: Infra to detect iova mode. 5) 7th: make vfio mapping iova aware. 6) 8th - 9th : Check for IOVA_VA mode in below APIs - rte_mem_virt2phy - rte_malloc_virt2phy Test History: - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). - Tested for arm64/thunderx vNIC Integrated NIC for both modes - Tested for arm64/Octeontx integrated NICs for only Iova_va mode(It supports only one mode.) - Ran standalone tests like mempool_autotest, mbuf_autotest. - Verified for Doxygen. Work History: For v1, Refer [1]. For v2, Refer [2]. For v3, Refer [9]. For v4, refer [10]. for v6, refer [11]. Checkpatch result: * None Thanks., [1] https://www.mail-archive.com/dev@dpdk.org/msg67438.html [2] https://www.mail-archive.com/dev@dpdk.org/msg70674.html [3] https://www.mail-archive.com/dev@dpdk.org/msg70279.html [4] https://www.mail-archive.com/dev@dpdk.org/msg70692.html [5] http://dpdk.org/ml/archives/dev/2017-July/071282.html [6] http://dpdk.org/ml/archives/dev/2017-July/070951.html [7] http://dpdk.org/ml/archives/dev/2017-July/070941.html [8] http://dpdk.org/ml/archives/dev/2017-July/070952.html [9] http://dpdk.org/ml/archives/dev/2017-July/070918.html [10] http://dpdk.org/ml/archives/dev/2017-July/071754.html [11] http://dpdk.org/ml/archives/dev/2017-August/072871.html Santosh Shukla (9): eal/pci: export match function eal/pci: get iommu class linuxapp/eal_pci: get iommu class bus: get iommu class eal: introduce helper API for iova mode eal: auto detect iova mode linuxapp/eal_vfio: honor iova mode before mapping linuxapp/eal_memory: honor iova mode in virt2phy eal/rte_malloc: honor iova mode in virt2phy lib/librte_eal/bsdapp/eal/eal.c | 33 ++++++--- lib/librte_eal/bsdapp/eal/eal_pci.c | 10 +++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 10 +++ lib/librte_eal/common/eal_common_bus.c | 23 ++++++ lib/librte_eal/common/eal_common_pci.c | 11 +-- lib/librte_eal/common/include/rte_bus.h | 35 +++++++++ lib/librte_eal/common/include/rte_eal.h | 12 ++++ lib/librte_eal/common/include/rte_pci.h | 28 ++++++++ lib/librte_eal/common/rte_malloc.c | 9 ++- lib/librte_eal/linuxapp/eal/eal.c | 33 ++++++--- lib/librte_eal/linuxapp/eal/eal_memory.c | 3 + lib/librte_eal/linuxapp/eal/eal_pci.c | 96 +++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.c | 29 +++++++- lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 10 +++ 15 files changed, 312 insertions(+), 34 deletions(-) -- 2.14.1