From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0058.outbound.protection.outlook.com [104.47.38.58]) by dpdk.org (Postfix) with ESMTP id 090BD1B2A5 for ; Fri, 6 Oct 2017 13:04:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=kAuwtQKIfNbOkIwjX8JkDLObwC4/B8P/LIGQyMdDDrQ=; b=JW8Bmy1nAMxS+GpiQtv2q1xqz1yfk2M1EQthHCO9Lrn0kqS3hs++Kvna6yNHtPa1proXAPdY1gYRlXeec1jOyGMj1tloAV5K73vVWkKgVXPVID9TZl7ACCFPR1LHvN0h0hAd7yEbE6+76lvnUZ4DDTmm0RL6SmPMpb2khvfl1is= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Santosh.Shukla@cavium.com; Received: from localhost.localdomain (111.93.218.67) by CY4PR07MB3094.namprd07.prod.outlook.com (10.172.115.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Fri, 6 Oct 2017 11:04:07 +0000 From: Santosh Shukla To: olivier.matz@6wind.com, dev@dpdk.org Cc: thomas@monjalon.net, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, aconole@redhat.com, stephen@networkplumber.org, anatoly.burakov@intel.com, gaetan.rivet@6wind.com, shreyansh.jain@nxp.com, bruce.richardson@intel.com, sergio.gonzalez.monroy@intel.com, maxime.coquelin@redhat.com, Santosh Shukla Date: Fri, 6 Oct 2017 16:33:37 +0530 Message-Id: <20171006110346.13247-1-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170920112356.17629-1-santosh.shukla@caviumnetworks.com> References: <20170920112356.17629-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: CY4PR1201CA0022.namprd12.prod.outlook.com (10.172.75.32) To CY4PR07MB3094.namprd07.prod.outlook.com (10.172.115.8) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4bb0e266-92c4-45cf-b35e-08d50ca9f984 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:CY4PR07MB3094; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3094; 3:BhAJefVvVGUFnXY1AYB5IaulIWCtVozdP12fskxkfZXYww1RuCZDPXxqjWTOiW1ajm/Ctf8ciuCeql+/3FLfo+EiZGWVkD39DIJKZjDOhVb1zdutSRQZSXarktpUtQ0mNTHmQocBeYQQFo+z66zC9BKPUs3PSDcPDZbLzuUgYaSIt+7uylYjeQ0iqjwvsD1FdQeKCUQpdJe1NXU8hV5nztjJO0navEFx6drJmsvsei+2lys7Z+hJHu2EGYrnKQiN; 25:cXpDkq+q4BNqLHZloYUtt5NLbdNlwuHlRkzed7ED/erm30NDGvZxEKiI8CeVubHzYv1w7SON5qnMSLKPPCLw7oaeBPy0XbD7xxC0GUCMp01248d9OZnf5VeaRWMtc0yuMWMK09wf+ordx92HEgac7fL1Y2gBIO6qEAInX/NkuIKw7hCiK+9mtKihJFRnRCuV7/Qi9TIQ+AT8z877ypi425xorO0eVlKzikteapmK3HOB/JqBqxIn+6P0uDw/glzvW32hXTKPH0aAeOOTxNpabfACOxLMDSl9L+z6/oXEdzRe/se3b1POI8wKdXSR9ELn2zT4u0pqMA6yjdeuMDPCdQ==; 31:R8ysdrXHIjJ0gnf4ofIYrA4Pv97wy1I4uUQQ75A+xmrdyu92c2FCS+rlRqBMfNwvTqozwKSFcWudWcxiYDX68xm3+wi5NROknCUrABp+19eu+1bn4m6kEHoTXKJ5E2/17ywZKM2K7g6ujt3fPRUfirXygpiKDAsHLuEYtABnf7fSrR7qLnnvbzcq4emiqZSyCGw4apbj6AiabEzU3z8OxkQsuoe/YjHkMJl0KoubYFI= X-MS-TrafficTypeDiagnostic: CY4PR07MB3094: X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3094; 20:XDR4NlwWCF4c97zs43bAjyTkQFf8ZYrTyTYmPT3IVlpyhJetPgWEIpB+qPX8icvznkxl/1azh67NlxUBVQ1Y1MAwta6Y6YdzeJafa72+o9EU1j3ATJjyRSVcxaCM3je7GvHSunC+ZtTr7vmovplJo5v7irwhknV55sS/t5/XACKQNfXjVNNIICGXsCwoAW7fiy9f3XAQtJ3SaNik2DcYMxE4gt5nKu+yTiq9xkvX32pWJtp/OlfUbQnf+eqiKlv9kxA/aJkHPMLyTlC3L22El5B++dVGIHSPgP5ymUAZaEa12i7UGy4MNYrtFM18viRsaFr9/8E0aKvAlOFX6h52MUKhYuLPEQ3bEMmB0suUBWnWReLOwbnMOA6NHinJJKB3atPyn+Bm7KAmiuNERDu4s67IdCAsUsWYHQCsQoOX3w2t28cjPCWnztLJ9FHNr1ezVFG+Uieo1sJBS7ar9XNf5+4db3q524cJERxviQw1S1+OzbjXnsqT8NvxzvCICTPbLrOKqtpq6sXH5RN53w4ZnrWnFZUrxiqB5TWoqS2eSsGZDXo3X4tzBF2oxYBGKyoETskPSPFTfaYLV69gC8dvCBnw/1IGvtU2+EEAww+J8Gw= X-Exchange-Antispam-Report-Test: UriScan:(788757137089)(278021516957215); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(6041248)(20161123555025)(20161123564025)(20161123560025)(20161123558100)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR07MB3094; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR07MB3094; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3094; 4:uK5FJtsgGzrdlD47Sm9IUbhkE338BCnFEnb14ITzPLhXDsfXyuSzCKivxZ62Nzv38sYPMtC4leZpSav7OVgofXLZTfu5TztiPSVKgORoGN9/ZJve28Sgvp8OkE8sb6H+vPP8hi8gJxJp01QR7YONE9bjjaiHj8T2lBvt/U2aUudTLQd9wvEIziVMc/57lcv7hnNlCMJ0CWHTiSQ+na+jChBU2XTAbvPoCTdmImt2i8c5zOAN/rD1EGtSWdbZ+/FyxZJ6IruHZHO4/rr160hRjQQkUsBaBYAlt3EWq2TtIqrPSL7WEPQ99hrqXhfs81rXQAZZ+aZu+coLOLLKP2x0Sg== X-Forefront-PRVS: 0452022BE1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(346002)(376002)(189002)(199003)(97736004)(2950100002)(5660300001)(7416002)(105586002)(6306002)(6512007)(53936002)(106356001)(478600001)(6666003)(50226002)(189998001)(8936002)(25786009)(81166006)(81156014)(8676002)(107886003)(2906002)(5003940100001)(6486002)(76176999)(305945005)(8656003)(50466002)(36756003)(4326008)(53376002)(966005)(1076002)(68736007)(48376002)(7736002)(72206003)(3846002)(6506006)(316002)(33646002)(16526018)(5009440100003)(101416001)(66066001)(6116002)(47776003)(5890100001)(16586007)(50986999)(42882006); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3094; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3094; 23:G7LrXypHH4KiiljBovpT0DYUqDfcVG+hMH8kXsO+g?= =?us-ascii?Q?YkHnL1MRuyS5br8aQlKT16BEQQEcBf2agaexigTO5og5agFm0frjshrB4mNO?= =?us-ascii?Q?Bw9TGEnwwxgui4qK50m0lijCzpNn1MogB5FbBAho6HUt3P6QPT+MMcikKbV9?= =?us-ascii?Q?VFb5DtXyMLzROI58Zfq1FuWoI8TmwGWkPOJJGm68VdTVQvyO2wmb7EXfQvw+?= =?us-ascii?Q?OhIckKXqbFkdw1MurtXuY206/z5SjQB+xR1T5YSwfctgGu5xVGcon+xAZa59?= =?us-ascii?Q?Z0GTFX3ZShIj+MFzKi9Sv3uR82oSpMwKTfI3Zaf2picY+CmoU1iATsg0+pA/?= =?us-ascii?Q?2md4kmR2ejn+3v0/Gd7S2qlQCMMOjOXXNk/2udzmmWfQyD+NG5+h2s/w0O/3?= =?us-ascii?Q?x88gEQNaTYSxg47qPai3B2Qd6dKwi3dyzqFZiU8omdWWiou8byemcc4vKCLZ?= =?us-ascii?Q?OaD1xBFkT76YSkCCnP9BavJLcbkyJoMm67xB5JMGu/ebKs1WkhZh3khY6MGR?= =?us-ascii?Q?vHOghYRAFLfX0ZkwV/q38yKj64WycBYePpnyTvPdCmVi5JwRxTH1MMOjG1aW?= =?us-ascii?Q?IvKdUvo79E0+uNYtF8QIVQ/1pXJXKSrxPB4ImTSH705OVzbL2UsK4medXKCQ?= =?us-ascii?Q?S4sjAzZU/gZw6hK8rP58jp2vMj4GN9SkgpOZksiUsSdWKQCcmv7dAoXeLdMv?= =?us-ascii?Q?8Unzyrwzyp/A2T01I8qIT9dmTblprHiHtxIcobsYBGHB6yLIX8tErvi+bcsg?= =?us-ascii?Q?/TR8B3esEzdgM8LzyTEZuzn/FBF2alQBdmxSllbkojUI6nu5wB1PXGOdOSXU?= =?us-ascii?Q?USn5Z+mef2y0jWmkxQ8IWFKHpjVJI1nm9pXcBT84554RgshYyNkTcBv7rLKw?= =?us-ascii?Q?bJUjYPwkMSVF2PzhkC6Z2tT4XyiCJfrsS5shgIpBlb+NWSrxmQsyQBz18SHy?= =?us-ascii?Q?eGf+zLCgdX3M8rPVsM3JqBdUubDD6+d88TfJVNjxYyPJFWLhgcfHZPgkR7jG?= =?us-ascii?Q?2pgjjndG0Bgl2JtEZq9PakjiC3rb/Rw44RlTl/w++zJtkUVWjAGrksWSV+bA?= =?us-ascii?Q?0k7COPyZzUvnXizJjV1sOa1fh/OdcWkjCvgYO12eyDID7BZVb2FheOKJIk8Y?= =?us-ascii?Q?Mgye2aWe1c6xZXtUKbSgqASdncjrya8lxI1ql1YMLP88IdocsfaY27kjA8aa?= =?us-ascii?Q?+q4cAkkzS8mgseZJIDLo90aU34OZpzzmJio09P3XuDORZevN57LsP0fT7FlY?= =?us-ascii?Q?Wb6mc2sFXw/GNrsva+AwVoRN6ENsBWe8uLcvaWz?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3094; 6:oGaygmXRl0uJHGgf4cxU/KhfP/eD613fLtj6zPVn2PC1Jmgns5VKHQIxGepUhYHwxVAXermvpJ+2mbhXWyQXFK7qtp4JQDuM6uqUy1MIYVXm3UO9D2Ge8BbOsN5i+j0bqe1zgXntnK7z2eeOvYSkib1VQ3bLjS8fd0O9WVNd/PzgdZ1yGGtZlgJBqUmOYGGSLbHIQshf/BuxFVJ/peL7G+XpKw5pvzn5+dYhHL9BVRtFoyY6e3kPcf0QtAwyqbNS33xiWnACymU1JGQorvFdXUwDxdxhuAlCoObeAYOb0OOc2u3LoJpwHq+wrYuoqTaAPZRcq4pYiS5kmLQhVvCZsw==; 5:wlE7Bjsea+ERzd3FFx1lJS61OEwC/x8/nPZSLaIiQPavPIl1z86kYSCfSa1OA28jJjhz8MJN1K8v4qUniAXZmN2jWAlmalryCVcwT9Nt5MVjJtpwAa0a4ipgS/aZKvt0mQB+Y6DBY9v96ikxMOMubw==; 24:Y4K12CCGZpNhJfbpwvjjEPF7n/Ih7V0DRTuNwvCC53C8Qk8Bwh/67RLYhOAv+npRBwApH4L2+vumPbWavD+k4EJKv5SNBl2U63j1TeuWTfQ=; 7:f2k/AX/5R5A7IzRF1O9GT8xvuPDgGP0gAZr+t//7MyXRWyS2TXzGvPYf0fjUCjP5XyXqXR01CWY7bfmlhkUPiol9oyOXxAPBZcNgs6ubSpkmCveKo2T8Bz0KLl0NWncOdn7Xlog5D3cpYbX0XwR36+DDclJjbn39+c8O/HYa0TSLZ5Tmch4uYK4qWSmuK9/0/csfWCBLdxDgoa1KU7H1Zjiu0WwpS7Q5sziLJVJQb/M= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2017 11:04:07.0522 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3094 Subject: [dpdk-dev] [PATCH v10 0/9] Infrastructure to detect iova mapping on the bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2017 11:04:15 -0000 v10: - Added doxygen specific comment for iova mapping mode in patch [2/09] (Suggested by Olivier) - Added pci_one_ for pci_device_has_iova_va and other api for patch [3/9] (Suggested by Olivier) - Added More verbose description in patch summary for patch [6/09] (Suggested by Olivier) v9: - Added Tested-By: to series. - Includes minor changes related to linuxapp api stub in [02/09] (Suggested by Anatoly) - Series rebased on tip commit : aee62e90 v8: Includes minor review changes per v7 review comment from Anatoly. Patches rebased on Tip commit:3d2e0448eb. v7: Includes no major change, minor change detailing: - patch sqashing (Aaron suggestion) - added run_once for device_parse() and bus_scan() in eal init (Aaron suggestion) - Moved rte_eal_device_parse() up in eal initialization order. - Patches rebased on top of version: 17.11-rc0 For v6 info refer [11]. v6: Sending v5 series rebased on top of version: 17.11-rc0. v5: Introducing RTE_PCI_DRV_IOVA_AS_VA flag for autodetection of iova va mapping. If a PCI driver demand for IOVA as VA scheme then the driver can add it in the PCI driver registration function. Algorithm to select IOVA as VA for PCI bus case: 0. If no device bound then return with RTE_IOVA_DC mapping mode, else goto 1). 1. Look for device attached to vfio kdrv and has .drv_flag set to RTE_PCI_DRV_IOVA_AS_VA. 2. Look for any device attached to UIO class of driver. 3. Check for vfio-noiommu mode enabled. If 2) & 3) is false and 1) is true then select mapping scheme as RTE_IOVA_VA. Otherwise use default mapping scheme (RTE_IOVA_PA). That way, Bus can truly autodetect the iova mapping mode for a device Or a set of the device. Change History: v9 --> v10: - Refer top description. v8 --> v9: - Added Tested-by: signature of Hemant. - Added linuxapp stub api definition in [02/09] (Suggested by Amatoly) v7 --> v8: - Replace 0 / 1 with true/false boolean values (Suggested by Anatoly). v6 --> v7: - Patches squashed per v6. - Added run_once in eal per v6. - Moved rte_eal_device_parse() up in eal init oder. v5 --> v6: - Added api info in eal's versiom.map (release DPDK_v17.11). v4 --> v5: - Change DPDK_17.08 to DPDK_17.11 in _version.map. - Reworded bus api description (suggested by Hemant). - Added reviewed-by from Maxime in v5. - Added acked-by from Hemant for pci and bus patches. v3 --> v4: - Re-introduced RTE_IOVA_DEC mode (Suggested by Hemant [5]). - Renamed flag to RTE_PCI_DRV_IOVA_AS_VA (Suggested by Maxime). - Reworded WARNING message(suggested by Maxime[7]). - Created a separate patch for rte_pci_get_iommu_class (suggested by Maxime[]). - Added VFIO_PRESENT ifdef build fix. v2 --> v3: - Removed rte_mempool_virt2phy (suggested by Olivier [4]) v1 --> v2: - Removed override eal option i.e. (--iova-mode=<>) Because we have means to truly autodetect the iova mode. - Introduced RTE_PCI_DRV_NEED_IOVA_VA drv_flag (Suggested by Maxime [3]). - Using NEED_IOVA_VA drv_flag in autodetection logic. - Removed Linux version check macro in vfio code, As per Maxime feedback. - Moved rte_pci_match API from local to global. Patch Summary: 1) 1nd: declare rte_pci_match api in pci header. Required for autodetection in follow up patches. 2) 2nd - 3rd - 4th : autodetection mapping infrastructure for Linux/bsdapp. 3) 5th: iova mode helper API. 4) 6th: Infra to detect iova mode. 5) 7th: make vfio mapping iova aware. 6) 8th - 9th : Check for IOVA_VA mode in below APIs - rte_mem_virt2phy - rte_malloc_virt2phy Test History: - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). - Tested for arm64/thunderx vNIC Integrated NIC for both modes - Tested for arm64/Octeontx integrated NICs for only Iova_va mode(It supports only one mode.) - Ran standalone tests like mempool_autotest, mbuf_autotest. - Verified for Doxygen. Work History: For v1, Refer [1]. For v2, Refer [2]. For v3, Refer [9]. For v4, refer [10]. for v6, refer [11]. Checkpatch result: * None Thanks., [1] https://www.mail-archive.com/dev@dpdk.org/msg67438.html [2] https://www.mail-archive.com/dev@dpdk.org/msg70674.html [3] https://www.mail-archive.com/dev@dpdk.org/msg70279.html [4] https://www.mail-archive.com/dev@dpdk.org/msg70692.html [5] http://dpdk.org/ml/archives/dev/2017-July/071282.html [6] http://dpdk.org/ml/archives/dev/2017-July/070951.html [7] http://dpdk.org/ml/archives/dev/2017-July/070941.html [8] http://dpdk.org/ml/archives/dev/2017-July/070952.html [9] http://dpdk.org/ml/archives/dev/2017-July/070918.html [10] http://dpdk.org/ml/archives/dev/2017-July/071754.html [11] http://dpdk.org/ml/archives/dev/2017-August/072871.html Santosh Shukla (9): eal/pci: export match function eal/pci: get iommu class linuxapp/eal_pci: get iommu class bus: get iommu class eal: introduce helper API for iova mode eal: auto detect iova mode linuxapp/eal_vfio: honor iova mode before mapping linuxapp/eal_memory: honor iova mode in virt2phy eal/rte_malloc: honor iova mode in virt2phy lib/librte_eal/bsdapp/eal/eal.c | 33 ++++++--- lib/librte_eal/bsdapp/eal/eal_pci.c | 10 +++ lib/librte_eal/bsdapp/eal/rte_eal_version.map | 10 +++ lib/librte_eal/common/eal_common_bus.c | 23 ++++++ lib/librte_eal/common/eal_common_pci.c | 11 +-- lib/librte_eal/common/include/rte_bus.h | 40 +++++++++++ lib/librte_eal/common/include/rte_eal.h | 12 ++++ lib/librte_eal/common/include/rte_pci.h | 28 ++++++++ lib/librte_eal/common/rte_malloc.c | 9 ++- lib/librte_eal/linuxapp/eal/eal.c | 33 ++++++--- lib/librte_eal/linuxapp/eal/eal_memory.c | 3 + lib/librte_eal/linuxapp/eal/eal_pci.c | 96 +++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.c | 29 +++++++- lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 10 +++ 15 files changed, 317 insertions(+), 34 deletions(-) -- 2.14.1