From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0053.outbound.protection.outlook.com [104.47.40.53]) by dpdk.org (Postfix) with ESMTP id 4CDD7BD34 for ; Tue, 11 Apr 2017 15:38:08 +0200 (CEST) Received: from BN3PR0301CA0042.namprd03.prod.outlook.com (10.160.180.180) by DM2PR0301MB1296.namprd03.prod.outlook.com (10.160.222.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.17; Tue, 11 Apr 2017 13:38:06 +0000 Received: from BL2FFO11FD016.protection.gbl (2a01:111:f400:7c09::167) by BN3PR0301CA0042.outlook.office365.com (2a01:111:e400:4000::52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.17 via Frontend Transport; Tue, 11 Apr 2017 13:38:06 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; caviumnetworks.com; dkim=none (message not signed) header.d=none; caviumnetworks.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BL2FFO11FD016.mail.protection.outlook.com (10.173.160.224) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.1019.14 via Frontend Transport; Tue, 11 Apr 2017 13:38:05 +0000 Received: from bf-netperf1.ap.freescale.net (bf-netperf1.ap.freescale.net [10.232.134.28]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v3BDbY35023442; Tue, 11 Apr 2017 06:38:02 -0700 From: Hemant Agrawal To: CC: , , , , , Date: Tue, 11 Apr 2017 19:07:14 +0530 Message-ID: <1491917847-3935-9-git-send-email-hemant.agrawal@nxp.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1491917847-3935-1-git-send-email-hemant.agrawal@nxp.com> References: <1491917847-3935-1-git-send-email-hemant.agrawal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131363914859806594; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(39840400002)(39450400003)(39860400002)(39400400002)(39410400002)(39850400002)(39380400002)(2980300002)(1109001)(1110001)(339900001)(199003)(189002)(9170700003)(8936002)(8656002)(356003)(77096006)(53936002)(85426001)(33646002)(48376002)(38730400002)(8676002)(97736004)(81166006)(54906002)(104016004)(110136004)(36756003)(2906002)(4326008)(575784001)(2351001)(76176999)(47776003)(50986999)(189998001)(5003940100001)(50466002)(105606002)(86362001)(6666003)(305945005)(6916009)(2950100002)(50226002)(5660300001)(106466001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0301MB1296; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; MLV:ovrnspm; A:1; MX:1; PTR:InfoDomainNonexistent; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD016; 1:mTaAcUSqGyxCjUPai8lNIXH4s9yKjeue9wFBjve42oYssvnZPQ08y4dD2z5TJCPgOBNoQure4Ql5Nhp7FvEEQoRAS7LSVjwEdq0QqQ2iqsqi6PU2EnaisL9RjjPoHOvi4yC+8CY/ibhUadvROQqegsO1zCBEMbIxW/vWrBZAv1c6+h93ObGI2b+S/H+148sgMg+xV6yMhp76vZYUcWG26xJGGsxVnHd3OfPQ/kAaKnp7key8TQuIIyAZJCU9bPMSdixiWfedFaCrVIbK5c/RmLzDosdzzMEi1N36LYOZYdk4JpZo+4A0PmfE1kOmf7Rl07slDFC9iqMqpzGQYjXInYQSnzuTRlLPFLyOUTN22iyFl8E57mY1kspRVM8FVLvqElfzrrv+wJ820K4F2wbGyMgdLy42Jg+zW9+slbT+NBIl7yKaNBWk05uHYchpNiv3nhUEXEzngpARNJ3owLUtCZoB3zZx/pzabufr/tlvB0Kj0AjiNH3AoM9WdJXieIThbxwmzIjW+dUG8ahdOqK4WtNoMApC3ThIjE8ww20x7guAdG7GT4E/qkNHtS+IbSxurvkadSOzw3+BAkcmC5a+hULVc9cinTJJdIbvQ/QSWnHR9xxx2d4m+uobArkz8pNcvMBER1Ad1IN6QnD8Bgx9vpLPpkysIPJGcBO1uYHcZQw= MIME-Version: 1.0 Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: c0980427-fc93-4be9-9e21-08d480dffbfa X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030255120)(201703131430075)(201703131517081); SRVR:DM2PR0301MB1296; X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1296; 3:AuZilf5Clj7QgI9EHAl0G2Fe7MjVaflKYhlbvBUrbdfruNjtxqPq2SSI9tbHIfXA6Btpw8MmVCTczyjBWi5mw4OTnVUJqL4CHH4/HmNaYgdc4HSFkvE8fN+AGWW/RN7k6kx+6QTjJmZL7S0aiTDhlj8sZwOyiBoZshv8dzse+mMg4ZGvTp+Yu+UP1UUwOgoUoWIUx66cf51Ov46oFX/x4ZgXZCK1mpLk4e7bvAnRZy5zl7JUgsJvvxQSAerMmUHBgJcnnJknNuXcWMNkz5FltNttM20CBHNh3BJ/51A0F+HSzlYKJDdZcAmhbebI3gzWGQiPWYNQrfxVNqixmBJGgDtVr5gMMsgIHMVSpHSjgjDQ8I94OZT8PKJaiMKUHA8RZqeJEwXLRTsW2KLkwxOAKYlcXXYtAAyQ4CXdzng0D+tIX7sqdPAY9nfPxJXGp0dKeLW9ClOLKD496HqeSn4NsA== X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1296; 25:hlDOevC/c/2odz62Ak1VLdWxNu1SzeZenWfNV0U+xhMDrQXGIiAfZlLWx+E16U/fAcf4rj8TBXQYcoFYapVlv6S297ByF1C/O5CBOS0zgKtlsEKHOvmqOrphTQrmB5Hwv+nn04tmgZgz2NeSXYryzAPSE9bBbmDBjAAf6org4EdTS0PSlFlze6BeefzxukEiblFR//UkKRF1nvJKl1mcrrxUDe86+lq9sCiBg+vUOoIIdywfaITLwwx1XAlJQpM9ySMiCxyFShtMx7zbnY8xygc/fp4q1TdzTOsGVc+ETmaRLTAsELRK+E9YwrgnLlvfPgQIMb0iAFZivuCo28pm6d9kT0xP0IFoxYvR86ShsWWL8sMIaw6s04U//Ybu30kpmylWL3bX+m1PCO3MxyjwOErE5NdufKapIYL6WGHnioX9G4uq2nGuMN+U2EjtuY5mYwSeRSWTd3uvYqeyUuVX+Q==; 31:QlHxZD0GhkQNQ3Itlv3pkgxe6alKo7IxMKkYxa+9WbryXLBqUFCKrzpgTTtjD9UD5rJtFoAzWSbRNeB6lAL6duXCTRTIOmNNUbh+/6pYnIfSzHjzffcjqo+HSBYwD4PQwbnz80k9XL3xJGs8/FOzm4TRlrTunXWSiCxg8w8VMsoTtZp40YaBg9vR/6UbhfQh/QfchixRO2dfXxJ2Z1IiB2mM+OGjStpxn6tTatz2OzxiJ6+mMTywdA1ie/NuHEd68Rp4dGfFh+PkxbS7hupSvIEHUZ2gsDVhUJkXu2UZTfA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684)(21532816269658); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(601004)(2401047)(13015025)(8121501046)(5005006)(13018025)(13017025)(13024025)(13023025)(3002001)(10201501046)(93006095)(93001095)(6055026)(6096035)(201703131430075)(201703131448075)(201703131433075)(201703161259075)(20161123563025)(20161123561025)(20161123565025)(20161123556025); SRVR:DM2PR0301MB1296; BCL:0; PCL:0; RULEID:(400006); SRVR:DM2PR0301MB1296; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM2PR0301MB1296; 4:b1pxOeJL3YJb06Y5zhNKpHPWE7ko4qWX+4cB0DVU?= =?us-ascii?Q?51oTI0f+Ni+XhoenI8j0JYeg8YrJVRVlkpwpyHsxd8zmQTSt1WG/6zJaEXBR?= =?us-ascii?Q?xGwURIS+DHAh1wPIfrpJbqrmDPkAunBIABmZlL38vFryn+yaWXsSjvFBsqlv?= =?us-ascii?Q?q2hvwx9MiFBHPkFAk8TdQKEqQowO2C5igdHozusqDsiNnPf8P35EXN7+cAk0?= =?us-ascii?Q?9G+Ab+lLsjro85wAM9Kn485QHoTCaj+eS5vl/sf7kCWdcEp9yQ2pNZ6Jx8LA?= =?us-ascii?Q?6z+xskKExtT5CgEkMlSaOJ/X6YXSN84SQ4DcosoN0yyJyRkvQ/ZGGc16UPut?= =?us-ascii?Q?3dHOb2APDZzKgvczpHaTC3P9Yx+9FXXHLG1hqhNSFv20sL/wOp3MSrxJ9iOj?= =?us-ascii?Q?TYzm/Eg4fMsR7D7mzlXqdQEKTvDjCC9bdSzcOtTAQvxwYxWrBnlmGbMjWSXW?= =?us-ascii?Q?MHIAIt3Sz3hExaFnIGOo017fLuilvzkMoCGOyZg4WL4J5Faad376Y7Vuo+vF?= =?us-ascii?Q?5oe4hQTh/UljTxVZsWPL3xxutzCmWHAZN4kzrUCzD7dRWmif5TfAjP9viF4b?= =?us-ascii?Q?tgwW2YPNkhG02SxqyE0aNWJWEjxMyIMFyOCLDsE24fK9yUtZU8HQNd7DgHp5?= =?us-ascii?Q?FW63+fj9DV1RngM0ZTCZAmNnbdIH5UzaTGGOqTbQcZGf9U2YBEzCGgkBRc4O?= =?us-ascii?Q?qnlnCqh7XTwHgHHU9pADOuPYDLhKbGAIRDWMLvK7WEu3GoAiIRP3FTMUVJX+?= =?us-ascii?Q?w93XeaoayjPG+VdupWQrefiRhWAajvLBz9mgEOpjq5nCjgYaKrDBWszEUal7?= =?us-ascii?Q?sfQFluD+BdzgMsAQsKSf4K26+PvW5XfNTr7hXEjIj1aWDbkEwCeK58rGbrHR?= =?us-ascii?Q?QpEIH05UvJvskF2R06RrO1D5sddDsCzoSHsNzyu7Gi1tOvg8cl+kMzUxKBIK?= =?us-ascii?Q?Bm1ZPG/AU78UETrj/Pa5bAeeaRRxbvMa9vZo0/jRQw=3D=3D?= X-Forefront-PRVS: 0274272F87 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM2PR0301MB1296; 23:rhiEK06yX7Rj0UQG1PdShaIfd8wVYKPgyaqIsyU?= =?us-ascii?Q?G4xhwmrtyy5SUoDtf2ikwTv2Tu36j85lhusG80O0igxjIRJQoLPbUhMUnXOV?= =?us-ascii?Q?HgnMt6q0gQw8gmCgkcjqo48OSRkcpvIX81gOKSmFweywPZbuYK9Z3t14OFv3?= =?us-ascii?Q?0Klt+07EIG3Uqhnew8Zp5f6+yCvSSeclPBBFrNmFMhPTJ2co/DJ4lOTOi35q?= =?us-ascii?Q?gyVi3KFsNwFLFXVGjxw3RR1FFBGid4qzW0pFchAZyeXZGWe7IooxTMSwujcU?= =?us-ascii?Q?HoUJVS/enwtqwkXGEb4DBa4s3CVdgt3v22hKkM3eJKocpjDwGPbm3BAiytIW?= =?us-ascii?Q?Wj54rAPeo1Ojjds7QW+RiW6DlBmigulwwKeUN2sVNbvZ8QvJ+P2YOFukbo3M?= =?us-ascii?Q?iPm6BLkDSEnOTLn3habKIBDSyrGyroZ9KUggIGA6zapG6DtGJO32oFoFijyb?= =?us-ascii?Q?kwGmwoxe1lvNBmnn7g0MhJ2syE4d5GExEECeCpa7imIPJLWvnUMBSSmM8s/r?= =?us-ascii?Q?+oXnuJI+WHy2ppyPsyDqJU3cM60JtnyvIWkB8eDcs5yTnYBbdUWfrgj0KkyX?= =?us-ascii?Q?JtsS3FyVYF3bT4XRLNlfQPAWM/RHmox0aKZTWcQA9LYqxiQzNAxYc173whEg?= =?us-ascii?Q?5OTYCIqrT10Itc/zCa8Aky5726Yns/3/KD4y4uWJCEMhNUOYBUCIKDAQGrUF?= =?us-ascii?Q?dbYYrA46QCUMI5rjLGTV6VnTO2dSeM2+GZuL+ADdTVGsRaUmuflrMZ9BNM8/?= =?us-ascii?Q?5p6x3HUmTXbRrHiBBlDRqVx9k0uKwwRFpaI/5yjOGJ20Iah83GFQXOXtifB+?= =?us-ascii?Q?+ozTtZSbSJ3onSYUZICUTpQpntm8RiCtINaP4vovAYRUA4l1Kqs2G9LeDFA9?= =?us-ascii?Q?3yytXUm/YScyIVCJrfBmhd5jY+8gI3yQYQdBiNMvscY33QRuZ+NGOFFBHNhG?= =?us-ascii?Q?gv554LOYFQ/U0GQL4J6K2x0Ep1X4VWVuv9lLoig0jCEkeE+n/Qj88ZujuHK+?= =?us-ascii?Q?H2s04hW6AQU1Sy7BMVtG8Em9jluucIaexLrHLt4Lxe7qmycZd/gUgx0CQ/Qd?= =?us-ascii?Q?rxxDoPaho7HLxW8v9L7cenuI8vqaFtgXI7mUby2uAoEZG6dlXQBvBUcLcGUd?= =?us-ascii?Q?PbClHP8pGWPMGub8aEUjPHB/5MvGalTKBvEy8E0MXQYOEpr1W4B6Qcbw6Rpg?= =?us-ascii?Q?ybfvrUoUnQkyrcmK5KKOruKzJAvjWqy/zg3oeGexZlyoR9OjB2ZlhgmRIKhW?= =?us-ascii?Q?9MWGIlKAUlPkfkzTMbgTNVxquO6Jn14u6pLujlLkO?= X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1296; 6:IiQ8/X3zH1soxhMEzoAzUJaLYlcIRTrCQA6YoCd6Lxl2bd9j4SYnLDa2l03mf4Db7SmBQN7EemovJRkM2WE4mL7VHt82YfJ/s3IyMF23dk1NjP0VhUUoXJ7IyInbx//72HlrFGhtiKNY5sAycN8h416STi6YfpeBf9ZZDw/tg/2mxHVXisDQv2ZCS1BsJRH80PX5V+STT9dVATD0mq4eobC6lEZTKwI/M/984sdYfVHXjcLdjcF3iMPPWnsIRnRt/+cLJiVZrquuhY6Nt8npoRyywjeE3L2RjfrFf6LMfoVxj/1Cp2WZ9i0O78KTiwbUEXXEy3EGmJ024S6TOLp8XZjSO+wfB9j2hsmKgtR41FTDavm2KbDd2aDlwCrgTxWdq5hQUZ9VmkwvXLomPRAcnqekUxkt7tFbeMIMSg9rpevMRdmHVbYOOLQVntvhdUO2MDjaiAnyNCmpUHjTv8fAgA==; 5:aQltWXRrSMkotpnqhL1d+yrpuT3BPR5tpH7w1jEHTYMsmIHIuadZNSjgOn+viKSX6AVKsHfT+5S8Reo6Mv1Y66+eySPoSppo6Jch+WjEK5ILzRz0hgPKz2l06VrJquyQxIX71Vr8vPGjqZjJS5+i3wQ2gCDngLiF8DuVD9TypyJ8aAveuVbTM7pIUHIRHhIp; 24:iG0z4vWIXG6SUuoYr5V5EXSxjnbzAEjEY341mC3vOF+XjKeM3KdaP6ZQoA5NPtr+MdD2A36DCBCfePnplmeD7zvAzf4zgBe6ImXi09hJhFY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM2PR0301MB1296; 7:0sgJ0vQAI6sRIBeFZgJ2rNaMVysCzkAs+GsptaZhnYBBWGE2TjpOrovac0CXt4hgWqETA+PVCaXVMxD529Qn2CwX4yuXeeu5BQvBy3qgJQXpsDWDstf0kJXgSXxE27crDpHi7lUWmbXx/dsMBKPRULw+85s5cDCao30x9IfXSL4oKx5600WA/WlhwFbMhKUYLnVWguLIBcolQkcppBXpztot1gtIkMh/wLl6pPKigU4LHBZPuJUnfLW/Kmir8y8A3cBCFj0KE0HgwwyEotnlzQXisc1Myow/k4/aZSMLT0t9bbouA7NqymUOlcjMSsNeEFVegGaa1HBdTn/J6fqXHg== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2017 13:38:05.7934 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0301MB1296 Subject: [dpdk-dev] [PATCH v4 08/21] bus/fslmc: add vfio support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2017 13:38:09 -0000 Add support for using VFIO for dpaa2 based fsl-mc bus. There are some differences in the way vfio used for fsl-mc bus from the eal vfio. - The scanning of bus for individual objects on the basis of the DPRC container. - The use and mapping of MC portal for object access With the evolution of bus model, they canbe further aligned with eal vfio code. Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/Makefile | 1 + drivers/bus/fslmc/fslmc_bus.c | 16 + drivers/bus/fslmc/fslmc_vfio.c | 463 ++++++++++++++++++++++++++++ drivers/bus/fslmc/fslmc_vfio.h | 74 +++++ drivers/bus/fslmc/rte_bus_fslmc_version.map | 1 + 5 files changed, 555 insertions(+) create mode 100644 drivers/bus/fslmc/fslmc_vfio.c create mode 100644 drivers/bus/fslmc/fslmc_vfio.h diff --git a/drivers/bus/fslmc/Makefile b/drivers/bus/fslmc/Makefile index e7eff10..b99a4f0 100644 --- a/drivers/bus/fslmc/Makefile +++ b/drivers/bus/fslmc/Makefile @@ -67,6 +67,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += \ mc/dpio.c \ mc/mc_sys.c +SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc_vfio.c SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc_bus.c include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 7848207..b24642d 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -42,6 +42,7 @@ #include #include "rte_fslmc.h" +#include "fslmc_vfio.h" #define FSLMC_BUS_LOG(level, fmt, args...) \ RTE_LOG(level, EAL, "%s(): " fmt "\n", __func__, ##args) @@ -51,6 +52,21 @@ static int rte_fslmc_scan(void) { + int ret; + + ret = fslmc_vfio_setup_group(); + if (ret) { + FSLMC_BUS_LOG(ERR, "fslmc: Unable to setup VFIO"); + return ret; + } + + ret = fslmc_vfio_process_group(); + if (ret) { + FSLMC_BUS_LOG(ERR, "fslmc: Unable to setup devices"); + return -1; + } + + RTE_LOG(INFO, EAL, "fslmc: Bus scan completed\n"); return 0; } diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c new file mode 100644 index 0000000..8223e48 --- /dev/null +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -0,0 +1,463 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright (c) 2016 NXP. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Freescale Semiconductor, Inc nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rte_fslmc.h" +#include "fslmc_vfio.h" + +#define VFIO_MAX_CONTAINERS 1 + +#define FSLMC_VFIO_LOG(level, fmt, args...) \ + RTE_LOG(level, EAL, "%s(): " fmt "\n", __func__, ##args) + +/** Pathname of FSL-MC devices directory. */ +#define SYSFS_FSL_MC_DEVICES "/sys/bus/fsl-mc/devices" + +/* Number of VFIO containers & groups with in */ +static struct fslmc_vfio_group vfio_groups[VFIO_MAX_GRP]; +static struct fslmc_vfio_container vfio_containers[VFIO_MAX_CONTAINERS]; +static int container_device_fd; +void *(*rte_mcp_ptr_list); +static uint32_t mcp_id; + +static int vfio_connect_container(struct fslmc_vfio_group *vfio_group) +{ + struct fslmc_vfio_container *container; + int i, fd, ret; + + /* Try connecting to vfio container if already created */ + for (i = 0; i < VFIO_MAX_CONTAINERS; i++) { + container = &vfio_containers[i]; + if (!ioctl(vfio_group->fd, VFIO_GROUP_SET_CONTAINER, + &container->fd)) { + FSLMC_VFIO_LOG(INFO, "Container pre-exists with" + " FD[0x%x] for this group", + container->fd); + vfio_group->container = container; + return 0; + } + } + + /* Opens main vfio file descriptor which represents the "container" */ + fd = vfio_get_container_fd(); + if (fd < 0) { + FSLMC_VFIO_LOG(ERR, "Failed to open VFIO container"); + return -errno; + } + + /* Check whether support for SMMU type IOMMU present or not */ + if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU)) { + /* Connect group to container */ + ret = ioctl(vfio_group->fd, VFIO_GROUP_SET_CONTAINER, &fd); + if (ret) { + FSLMC_VFIO_LOG(ERR, "Failed to setup group container"); + close(fd); + return -errno; + } + + ret = ioctl(fd, VFIO_SET_IOMMU, VFIO_TYPE1_IOMMU); + if (ret) { + FSLMC_VFIO_LOG(ERR, "Failed to setup VFIO iommu"); + close(fd); + return -errno; + } + } else { + FSLMC_VFIO_LOG(ERR, "No supported IOMMU available"); + close(fd); + return -EINVAL; + } + + container = NULL; + for (i = 0; i < VFIO_MAX_CONTAINERS; i++) { + if (vfio_containers[i].used) + continue; + FSLMC_VFIO_LOG(DEBUG, "Unused container at index %d", i); + container = &vfio_containers[i]; + } + if (!container) { + FSLMC_VFIO_LOG(ERR, "No free container found"); + close(fd); + return -ENOMEM; + } + + container->used = 1; + container->fd = fd; + container->group_list[container->index] = vfio_group; + vfio_group->container = container; + container->index++; + return 0; +} + +int vfio_dmamap_mem_region(uint64_t vaddr, + uint64_t iova, + uint64_t size) +{ + struct fslmc_vfio_group *group; + struct vfio_iommu_type1_dma_map dma_map = { + .argsz = sizeof(dma_map), + .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE, + }; + + dma_map.vaddr = vaddr; + dma_map.size = size; + dma_map.iova = iova; + + /* SET DMA MAP for IOMMU */ + group = &vfio_groups[0]; + if (ioctl(group->container->fd, VFIO_IOMMU_MAP_DMA, &dma_map)) { + FSLMC_VFIO_LOG(ERR, "VFIO_IOMMU_MAP_DMA (errno = %d)", errno); + return -1; + } + return 0; +} + +static int64_t vfio_map_mcp_obj(struct fslmc_vfio_group *group, char *mcp_obj) +{ + int64_t v_addr = (int64_t)MAP_FAILED; + int32_t ret, mc_fd; + + struct vfio_device_info d_info = { .argsz = sizeof(d_info) }; + struct vfio_region_info reg_info = { .argsz = sizeof(reg_info) }; + + /* getting the mcp object's fd*/ + mc_fd = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, mcp_obj); + if (mc_fd < 0) { + FSLMC_VFIO_LOG(ERR, "error in VFIO get device %s fd from group" + " %d", mcp_obj, group->fd); + return v_addr; + } + + /* getting device info*/ + ret = ioctl(mc_fd, VFIO_DEVICE_GET_INFO, &d_info); + if (ret < 0) { + FSLMC_VFIO_LOG(ERR, "error in VFIO getting DEVICE_INFO"); + goto MC_FAILURE; + } + + /* getting device region info*/ + ret = ioctl(mc_fd, VFIO_DEVICE_GET_REGION_INFO, ®_info); + if (ret < 0) { + FSLMC_VFIO_LOG(ERR, "error in VFIO getting REGION_INFO"); + goto MC_FAILURE; + } + + FSLMC_VFIO_LOG(DEBUG, "region offset = %llx , region size = %llx", + reg_info.offset, reg_info.size); + + v_addr = (uint64_t)mmap(NULL, reg_info.size, + PROT_WRITE | PROT_READ, MAP_SHARED, + mc_fd, reg_info.offset); + +MC_FAILURE: + close(mc_fd); + + return v_addr; +} + +/* Following function shall fetch total available list of MC devices + * from VFIO container & populate private list of devices and other + * data structures + */ +int fslmc_vfio_process_group(void) +{ + struct fslmc_vfio_device *vdev; + struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; + char *temp_obj, *object_type __rte_unused, *mcp_obj, *dev_name; + int32_t object_id, i, dev_fd; + DIR *d; + struct dirent *dir; + char path[PATH_MAX]; + int64_t v_addr; + int ndev_count; + struct fslmc_vfio_group *group = &vfio_groups[0]; + static int process_once; + + /* if already done once */ + if (process_once) { + FSLMC_VFIO_LOG(DEBUG, "Already scanned once - re-scan " + "not supported"); + return 0; + } + process_once = 0; + + sprintf(path, "/sys/kernel/iommu_groups/%d/devices", group->groupid); + + d = opendir(path); + if (!d) { + FSLMC_VFIO_LOG(ERR, "Unable to open directory %s", path); + return -1; + } + + /*Counting the number of devices in a group and getting the mcp ID*/ + ndev_count = 0; + mcp_obj = NULL; + while ((dir = readdir(d)) != NULL) { + if (dir->d_type == DT_LNK) { + ndev_count++; + if (!strncmp("dpmcp", dir->d_name, 5)) { + if (mcp_obj) + free(mcp_obj); + mcp_obj = malloc(sizeof(dir->d_name)); + if (!mcp_obj) { + FSLMC_VFIO_LOG(ERR, "mcp obj:Unable to" + " allocate memory"); + closedir(d); + return -ENOMEM; + } + strcpy(mcp_obj, dir->d_name); + temp_obj = strtok(dir->d_name, "."); + temp_obj = strtok(NULL, "."); + sscanf(temp_obj, "%d", &mcp_id); + } + } + } + closedir(d); + d = NULL; + if (!mcp_obj) { + FSLMC_VFIO_LOG(ERR, "DPAA2 MCP Object not Found"); + return -ENODEV; + } + RTE_LOG(INFO, EAL, "fslmc: DPRC contains = %d devices\n", ndev_count); + + /* Allocate the memory depends upon number of objects in a group*/ + group->vfio_device = (struct fslmc_vfio_device *)malloc(ndev_count * + sizeof(struct fslmc_vfio_device)); + if (!(group->vfio_device)) { + FSLMC_VFIO_LOG(ERR, "vfio device: Unable to allocate memory\n"); + free(mcp_obj); + return -ENOMEM; + } + + /* Allocate memory for MC Portal list */ + rte_mcp_ptr_list = malloc(sizeof(void *) * 1); + if (!rte_mcp_ptr_list) { + FSLMC_VFIO_LOG(ERR, "portal list: Unable to allocate memory!"); + free(mcp_obj); + goto FAILURE; + } + + v_addr = vfio_map_mcp_obj(group, mcp_obj); + free(mcp_obj); + if (v_addr == (int64_t)MAP_FAILED) { + FSLMC_VFIO_LOG(ERR, "Error mapping region (errno = %d)", errno); + goto FAILURE; + } + + FSLMC_VFIO_LOG(DEBUG, "DPAA2 MC has VIR_ADD = %ld", v_addr); + + rte_mcp_ptr_list[0] = (void *)v_addr; + + d = opendir(path); + if (!d) { + FSLMC_VFIO_LOG(ERR, "Unable to open %s Directory", path); + goto FAILURE; + } + + i = 0; + FSLMC_VFIO_LOG(DEBUG, "DPAA2 - Parsing devices:"); + /* Parsing each object and initiating them*/ + while ((dir = readdir(d)) != NULL) { + if (dir->d_type != DT_LNK) + continue; + if (!strncmp("dprc", dir->d_name, 4) || + !strncmp("dpmcp", dir->d_name, 5)) + continue; + dev_name = malloc(sizeof(dir->d_name)); + if (!dev_name) { + FSLMC_VFIO_LOG(ERR, "name: Unable to allocate memory"); + goto FAILURE; + } + strcpy(dev_name, dir->d_name); + object_type = strtok(dir->d_name, "."); + temp_obj = strtok(NULL, "."); + sscanf(temp_obj, "%d", &object_id); + FSLMC_VFIO_LOG(DEBUG, " - %s ", dev_name); + + /* getting the device fd*/ + dev_fd = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, dev_name); + if (dev_fd < 0) { + FSLMC_VFIO_LOG(ERR, "VFIO_GROUP_GET_DEVICE_FD error" + " Device fd: %s, Group: %d", + dev_name, group->fd); + free(dev_name); + goto FAILURE; + } + + free(dev_name); + vdev = &group->vfio_device[group->object_index++]; + vdev->fd = dev_fd; + vdev->index = i; + i++; + /* Get Device inofrmation */ + if (ioctl(vdev->fd, VFIO_DEVICE_GET_INFO, &device_info)) { + FSLMC_VFIO_LOG(ERR, "DPAA2 VFIO_DEVICE_GET_INFO fail"); + goto FAILURE; + } + } + closedir(d); + + return 0; + +FAILURE: + if (d) + closedir(d); + if (rte_mcp_ptr_list) { + free(rte_mcp_ptr_list); + rte_mcp_ptr_list = NULL; + } + + free(group->vfio_device); + group->vfio_device = NULL; + return -1; +} + +int fslmc_vfio_setup_group(void) +{ + struct fslmc_vfio_group *group = NULL; + int groupid; + int ret, i; + char *container; + struct vfio_group_status status = { .argsz = sizeof(status) }; + + /* if already done once */ + if (container_device_fd) + return 0; + + container = getenv("DPRC"); + + if (container == NULL) { + FSLMC_VFIO_LOG(ERR, "VFIO container not set in env DPRC"); + return -EOPNOTSUPP; + } + + /* get group number */ + ret = vfio_get_group_no(SYSFS_FSL_MC_DEVICES, container, &groupid); + if (ret == 0) { + RTE_LOG(WARNING, EAL, "%s not managed by VFIO, skipping\n", + container); + return -EOPNOTSUPP; + } + + /* if negative, something failed */ + if (ret < 0) + return ret; + + FSLMC_VFIO_LOG(DEBUG, "VFIO iommu group id = %d", groupid); + + /* Check if group already exists */ + for (i = 0; i < VFIO_MAX_GRP; i++) { + group = &vfio_groups[i]; + if (group->groupid == groupid) { + FSLMC_VFIO_LOG(ERR, "groupid already exists %d", + groupid); + return 0; + } + } + + /* get the actual group fd */ + ret = vfio_get_group_fd(groupid); + if (ret < 0) + return ret; + group->fd = ret; + + /* + * at this point, we know that this group is viable (meaning, + * all devices are either bound to VFIO or not bound to anything) + */ + + ret = ioctl(group->fd, VFIO_GROUP_GET_STATUS, &status); + if (ret) { + FSLMC_VFIO_LOG(ERR, " VFIO error getting group status"); + close(group->fd); + return ret; + } + + if (!(status.flags & VFIO_GROUP_FLAGS_VIABLE)) { + FSLMC_VFIO_LOG(ERR, "VFIO group not viable"); + close(group->fd); + return -EPERM; + } + /* Since Group is VIABLE, Store the groupid */ + group->groupid = groupid; + + /* check if group does not have a container yet */ + if (!(status.flags & VFIO_GROUP_FLAGS_CONTAINER_SET)) { + /* Now connect this IOMMU group to given container */ + ret = vfio_connect_container(group); + if (ret) { + FSLMC_VFIO_LOG(ERR, "VFIO error connecting container" + " with groupid %d", groupid); + close(group->fd); + return ret; + } + } + + /* Get Device information */ + ret = ioctl(group->fd, VFIO_GROUP_GET_DEVICE_FD, container); + if (ret < 0) { + FSLMC_VFIO_LOG(ERR, "VFIO error getting device %s fd from" + " group %d", container, group->groupid); + return ret; + } + container_device_fd = ret; + FSLMC_VFIO_LOG(DEBUG, "VFIO Container FD is [0x%X]", + container_device_fd); + + return 0; +} diff --git a/drivers/bus/fslmc/fslmc_vfio.h b/drivers/bus/fslmc/fslmc_vfio.h new file mode 100644 index 0000000..5e58211 --- /dev/null +++ b/drivers/bus/fslmc/fslmc_vfio.h @@ -0,0 +1,74 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright (c) 2016 NXP. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Freescale Semiconductor, Inc nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _FSLMC_VFIO_H_ +#define _FSLMC_VFIO_H_ + +#include "eal_vfio.h" + +#define DPAA2_VENDOR_ID 0x1957 +#define DPAA2_MC_DPNI_DEVID 7 +#define DPAA2_MC_DPSECI_DEVID 3 + +#define VFIO_MAX_GRP 1 + +typedef struct fslmc_vfio_device { + int fd; /* fslmc root container device ?? */ + int index; /*index of child object */ + struct fslmc_vfio_device *child; /* Child object */ +} fslmc_vfio_device; + +typedef struct fslmc_vfio_group { + int fd; /* /dev/vfio/"groupid" */ + int groupid; + struct fslmc_vfio_container *container; + int object_index; + struct fslmc_vfio_device *vfio_device; +} fslmc_vfio_group; + +typedef struct fslmc_vfio_container { + int fd; /* /dev/vfio/vfio */ + int used; + int index; /* index in group list */ + struct fslmc_vfio_group *group_list[VFIO_MAX_GRP]; +} fslmc_vfio_container; + +int vfio_dmamap_mem_region( + uint64_t vaddr, + uint64_t iova, + uint64_t size); + +int fslmc_vfio_setup_group(void); +int fslmc_vfio_process_group(void); + +#endif /* _FSLMC_VFIO_H_ */ diff --git a/drivers/bus/fslmc/rte_bus_fslmc_version.map b/drivers/bus/fslmc/rte_bus_fslmc_version.map index 67b0dc8..f64572f 100644 --- a/drivers/bus/fslmc/rte_bus_fslmc_version.map +++ b/drivers/bus/fslmc/rte_bus_fslmc_version.map @@ -36,6 +36,7 @@ DPDK_17.05 { qbman_swp_send_multiple; rte_fslmc_driver_register; rte_fslmc_driver_unregister; + rte_mcp_ptr_list; local: *; }; -- 1.9.1