From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0041.outbound.protection.outlook.com [104.47.42.41]) by dpdk.org (Postfix) with ESMTP id 7754FF91B for ; Thu, 29 Dec 2016 06:15:08 +0100 (CET) Received: from BN3PR0301CA0059.namprd03.prod.outlook.com (10.160.152.155) by CY1PR0301MB0745.namprd03.prod.outlook.com (10.160.159.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.803.11; Thu, 29 Dec 2016 05:15:06 +0000 Received: from BL2FFO11FD011.protection.gbl (2a01:111:f400:7c09::178) by BN3PR0301CA0059.outlook.office365.com (2a01:111:e400:401e::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.817.10 via Frontend Transport; Thu, 29 Dec 2016 05:15:06 +0000 Authentication-Results: spf=fail (sender IP is 192.88.158.2) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com;nxp.com; dkim=none (message not signed) header.d=none; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.158.2 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.158.2; helo=az84smr01.freescale.net; Received: from az84smr01.freescale.net (192.88.158.2) by BL2FFO11FD011.mail.protection.outlook.com (10.173.161.17) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.789.10 via Frontend Transport; Thu, 29 Dec 2016 05:15:06 +0000 Received: from Tophie.ap.freescale.net ([10.232.14.87]) by az84smr01.freescale.net (8.14.3/8.14.0) with ESMTP id uBT5DOHv010218; Wed, 28 Dec 2016 22:15:02 -0700 From: Shreyansh Jain To: CC: , , , , , Hemant Agrawal Date: Thu, 29 Dec 2016 10:46:34 +0530 Message-ID: <1482988612-6638-16-git-send-email-shreyansh.jain@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1482988612-6638-1-git-send-email-shreyansh.jain@nxp.com> References: <1482180853-18823-1-git-send-email-hemant.agrawal@nxp.com> <1482988612-6638-1-git-send-email-shreyansh.jain@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131274621066315109; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.158.2; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(7916002)(39410400002)(39400400002)(39850400002)(39450400003)(39840400002)(39860400002)(39380400002)(2980300002)(1109001)(1110001)(339900001)(189002)(199003)(2351001)(86362001)(575784001)(69596002)(551934003)(8936002)(33646002)(8676002)(305945005)(47776003)(356003)(81156014)(81166006)(97736004)(2950100002)(106466001)(6666003)(105606002)(92566002)(4326007)(626004)(2906002)(110136003)(6916009)(50986999)(76176999)(5660300001)(50226002)(68736007)(189998001)(50466002)(5003940100001)(36756003)(48376002)(38730400001)(85426001)(77096006)(104016004)(8656002)(7059030); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0301MB0745; H:az84smr01.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD011; 1:+L6LCBduAnpLKUDf6uul4lAO3S+LtmOwrphi/L/+SCqwszkv0+/nhpJ3bEHrkaHRxUS5vNh//sRTNXr9ayIciEQ+kc8XRcQRKdRN8ig6BGTs/6fJRHtqkPPeK4HIIxLSAbOdyA6i5BQ0qVmLVqqMdLjxxJFyXZ4Qm1OGour3jEVEvaoRuy23yTvBC3Ht0ZzjMe/Q0brBWdDPqRlVwORSEQX0mHKmgslpHqIwfdTMwqQWJNQn6aTWS0k6K7uML/W6r7/OrXj1o3bLi9hGsDUS71seNCCWj4cIXF6Jc0lyZTKKiDhsJZjG5nSXKaQeZ2wgD9LZ6XmKoT/xPYrq2fYAKV/O+KSOrB8BYD7kOvn4qIpuJsmxsYcj8eqBKSPZVn/MDHeGDRKGqNXiEU+4bteTA/SclGxsYnHu4NThf2bPWcnB/A2CqAdO/9JlpLqcPN8XGmvNIJpn3xoSiKGIPxazAfQsH6O9/eshiW1v2I15C8TEKCBToqU/Bedv9cTMo34Fw1DZSx1+ctc+YK/cvFRky9OOm2+ZwaF/a6Y/a33dGBTgvd0ZDJvhkaWWAmPWAmpWkHCZK/n6VnI8EZczfpggipx0D1hMeesAITd0m/Y76fAJi3555t2c0vc6zdiDiKllmdhdYBGekaggJqNqxV/WVKVko5RawtivWD+Z5TsG9fguvhPZpnQ3YIGFsNKyBUltJ8X3jJWH+zyhVIz5BJCV/j64MHZFKFCxUESnO6B7mIlAbcV8yXwuzrvcycBBvP5cRM7V+6g5rCiFVpcqDrUnmg== MIME-Version: 1.0 Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: 90dba36e-bdc5-43b1-18fc-08d42fa9a710 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY1PR0301MB0745; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 3:m9/0KuniB0umHh5I7OFN/TKZ+HQlkbzUhQlKTNnu9iQTiRjlCgk39sOZLBQ9c6mMcGesnKaJ9oHLhg+IBMG/6/KHKCNXnb18YOkKa5Xamd+wfqRmSrZPPjouDQ3xFj3f9QLkv3UriCt9CLrOYVHF+sm6bZJe94Li9Bil6O4QoslKhpfSJW8wsZz7R/IAzzWU45ddb0wT5pEm9s2688wiZIu12NDqLtkpyUsBmbjKoNd95dcjfgGW2P1cGhjwShnl8ETBVppSIWrd2SyHk6XqbmTlFPeu26DZ4K17a0TiTDHotu5YvCVepUpzmni0lcANv+xDQW9bEvF1zuaPGCXOXOd65ARjFrAqjZkvncfUm74= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 25:PpFk7ODXKdihTPM4MF/rpbOpZFvhVlMfAGVFx6N5T317feTdjhCRf1e+DCfOFRedN5QGqhArnmbgWgeeZHKoNf/4MAxA/uy2o2BxO4PwafEz8qQg+K71tYjsgHegRt3s6nIfcjERb4FrdpHHss97g/LqKMfJgftGFWglUFQoLKPIhHbqhQQtIUbKhrrdyUsKEeTO5TH06TQQNdmUT2hBDETzL3yae2feJwkhHF+WOT3uG52TXehdDmtI7DhiKz3iuirOXV0zpFuMhFHxhETo0WcJtQrcPdhvEZu++wTM1edCMGz9TfN/RUiqpehU6QxJ3l9WVm+kBcXPiOlt5EAryjFp8kONGOHylpmZUaYo0bxxWjmkS0QeZFCBfX1DsWxFm3QVYRdAN6Pc2SyonwWrwua2vuJ31JYzvFoOsB0aqQ5z+kYp0lU6keA57nCcisSSI+9ZLKwE2B/bwmMtMDDn4b81qeuBPqGSp+WQiFKztgQ6YQvntkKAd1PhpH2RAfCihovUqhxcJNwc4TY27WWHtUVniN6ThU0PKav7+iR0wrs9J+WC0hjrzKfykzdltI+U0qgNZO+ELvY7Kme7KH3vICfSdX4ZDxUtuTBMOZaoHcfbshLM0r0RaJIJRiqXJ1sOth4CktFboVMthwNm/mz6HljXeVeRYOwX4g7NEsFGHByvQxzZgzsQ7OvKVCZcW1APXZbNPWhzHNb64mkrYP5pJm1Z8OojTyTMEcS19gAERU0496G9vlVjCL8KZYlqm4+kssK79XZO0e4QPtqE3w4M9g== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 31:ndUGN2IM8+tjQjiP5mzLYbtjK+xcZ8k4ZJYBarUyZ/EX7uutTPzF8J9i1Hgg6BXuVu+C715x9KfteHtvClxXqGPyAkk1NVcRbqupUIwc/as117zinon+KI01X4Zr3Wwt8QWfqPOKg5XntyPIxZf3QRQfZ3ODfp2GQJp+moDmd8udf0soP1lwA+LeDYKIufKgR36hGiOc2CaehUaG3lx2yPxHWOGS+OsEf5YX3w22EZesQciQUOnPiLJl+XtfK7lNcVAHsjh/TdNexnlNuQ3rpZu/ZFLpNE4o+AU+ETspEOo= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095060)(601004)(2401047)(13024025)(13023025)(13015025)(13017025)(13018025)(8121501046)(5005006)(3002001)(10201501046)(6055026)(6096035)(20161123561025)(20161123559025)(20161123556025)(20161123565025)(20161123563025); SRVR:CY1PR0301MB0745; BCL:0; PCL:0; RULEID:(400006); SRVR:CY1PR0301MB0745; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 4:vhJDu1/PsLU2jVKxQh9/IVLrorVSHvhmMDus7Rjj8Qa3MWc9aMIthXhq22Q+rA53yN+LmnKbWUXLJpWp2IcMreTzB4EIy1ICrcItS/yxgWGbYTEzuuVzF6uilN2pCHvcU3J6kHB+GSPKYz7ogYEHcdOOp7O2WxmbUFjwYEBBPW84hLcluMGWFiUGnZlE4Sb2oLq/XF0xQU8ZvGiHZNO4yW+gHy1pi0CnM7tBMWAE5UG5OYFO2ps0r8TV3UBcGkzndVrxVKbapiDkzah1FjfJVGx7s4EeVGXhZhha8RZc9C9FKU4Mb7GUmALuQrMhyIih0uOb1ipn1EuOYazBj9bhExDV/uac1McKQK+ziK0eA1lprl6AFylBo42Um+jpuueuMkoWyQ3dt31hVH1AG7fNqNmopynA5Q5FLx1xhQTuiEkifhJjv9dMYcO+IEvMmhm7iwtMtEW6LihYkJ091cmB1ldpaV5kzWFDeVp4ZYZnpfe/JnuLGpQC5viZVn9aNwZmt7+bK/tXBbO5jPTZXBjjqd8E+h1zvoKhREo006CZ8NzvGNWFmb+xz03fAnE3WdAd4LItdKPLeUM1BJcIoIuLo1e3w8Qi0YDsU9sAsBuirtBaQrt/mcYgTrvtP5YmOWxbxQc9Pj1z0sOyls+BFS+QydzgsPNijnzKycWmi9RCnBLH6UWcfuBRpZcST9MRTVu3T1m3Av2J9UKm9xEE1ZSjTHfZX6q/m4C+E6+d2YERyWmO2m7xN3eWdoQxFb6xEABCW7y+0hZulEtT1ObWDkNH+w== X-Forefront-PRVS: 01713B2841 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB0745; 23:+XdUqQFSXNJ0a0tY0/zPMNSobb8Al2MAr3Y2vEA?= =?us-ascii?Q?hrq+eyYGVMnAhrStTaHi76Pak//kEErq8mM/VcIJAVy6YNwB5XRaI5fZM6GD?= =?us-ascii?Q?xBSsU68CpEv4PI4JOTXLlMFXfoIEOSYMebe7n42rx4foPZqHbdElv1lV2/0I?= =?us-ascii?Q?vlu6QY+i4KloGcOYZybqMeQgEoeEyRmUfjrGcM3aWDGQ7haXqggQrUpRGgTZ?= =?us-ascii?Q?5+MO2kRQ8xXF9wmPqgi1+vdhIU7iT5VRtsCLZ4FZe14PgmnhYp8uCha8Ztl1?= =?us-ascii?Q?JTQrVOXP6bdCy20d5oY6+NepOz39B2BHV+FeGDpBz5+MxjUbhZY/lbhAn9Az?= =?us-ascii?Q?3i8k0ls7MNW/QL4mUZDMYXH0zNT5+QGsm4j1nOdZlD/Ku9S/AmW/OKTTeQOW?= =?us-ascii?Q?DtOymRZmwi5s8G7QVAeg32OiR3VSwO34AL2asYhZUzN5VIWoNlz+XpHPdxQq?= =?us-ascii?Q?8cuvaB5YdyZWy1Vz5oDBH0Cj5+N+J2YUoDg2qXTw3wOhmtupsk7v/wVKJyC4?= =?us-ascii?Q?X8l63YM5ZNC5z14sQjoPos1XTRfRTL3wJDz3rj9XsZ5bRHTmDJeDyb6d/+xE?= =?us-ascii?Q?//nNq5tYTrDvgIGsb5Yracn5w+cFcnF18M4uMBg0vfdpui48NztDO7RISH+X?= =?us-ascii?Q?eO0fcLhXWezDO/SPYxTt3ONEBGCX2jtFVb21Y+WDLHSh9TKTtM1xIwng2TKs?= =?us-ascii?Q?w1LsCQT2H0VWF2MQ8dFwO5JMykBlPEhL2Rb0matzKSuEzAUWi6BX1vJSPj1l?= =?us-ascii?Q?Ky8Jwwqc5nxvgwN261TNwNTAlGwPq3tuAcWZOAYMrTJY6D3OjeG4CJq03Qj3?= =?us-ascii?Q?07STZ8/ZVlCnOb//Pla0I3ElNgtlCLPbrg8gNX6bvOYMFSzotnsRf5n2VNAY?= =?us-ascii?Q?HC1Z/K8YhI9M6JI50uwFLM4q00SJIeHJ5F2wFFztGuLxy4p76WlGUbc7yWHr?= =?us-ascii?Q?5k2K5nGm4Oa9GbmP9F0qhqsk37r3+FgnzaVp43mQ+eN08ABOCAgEjBQqtQks?= =?us-ascii?Q?rvtfdYfZTY4GCO0NEfci3R7hIiOFqAZI+tY/UOTtb2Vh4rtXqZKOiflIEmAr?= =?us-ascii?Q?g216ptmkI7Jm6EFLQbIZwOoyhFO5j3W6+8aeSCcDB4QLl8PgN/AXIpNvcUjV?= =?us-ascii?Q?ArnsigGQ1p/Grdz7bVCFbHnFPCCZ+U6FCc8LIHWLit4paeSO5SZZAH0rLIwg?= =?us-ascii?Q?OlJ5jS2CInypQ1pPRyFw4GP2AExFz0GTuyJ6+plsG21WImKXFYQ1uN7/W5zZ?= =?us-ascii?Q?vn0wjnAr2f50jV7GtS4b+iffl8ANmhI8EJUfaeOSSk9aCnFItFoGwVDeEuuo?= =?us-ascii?Q?pj8g4Y1fqMTt1En/a3iZ3pYf1VH8OIMY+aN7IGoBUJlWdH0ol6PNiILDxZtu?= =?us-ascii?Q?kC1fBAg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 6:l6EV/TAUpu4q7IfF4qiYxThss8qfxmbr9/4vDff/mGOQbizJOouzDbFibZRz7xe/09v5GJbcLJVsD+nmpOfp6nI0bba0gG6XGNtwA7bckbsc5rLUbZN7ibMUZ3WSO1w4qaxbUyI0kRiTim1V7+jMmFT585CoLrJPWlxRIeUCtpw1nEjmtxNX1mcXxzIDT2P4Sbh0nD8pNA9JckwWGOUx8Ew95QMO6p/KdOrLC8mDWkTfnGCcuSrECFZfe0rrr7YAEbawS3JNu2qmsIzY+1lbPehMqFaBX5aNwpAs50QXGPM0nz33iLB5b6bdqGh67EmHFBXJQcEL16RmYPS00swdTB/itWci2yp6UVkcvH4i1tBr77qKgfU2aI7Zt6MG5oyVeNwdUY39uCBva+Yhvr7c5+W7P49NAMB8C9NCvYIR9G+XfCFQaIGtTCOPLLZth0yh; 5:7WpJ6LXJxQXsU/6wrzAz8b5DEJQ28o4jDX00C1Czqu10+fW0E8eoeUUh87xiXHp5zIY2214bsnKmWe3Pp4GZn+6ForiQyGheN93tO6K0KTdkQHn8AACrspD8pqi+KeE0DaUhPkywyk1Omoc84qfB/902mwJG50g7I5TGaCjOx6J5XeyP96sM6bTz2fx4TGa6; 24:8eQ4p00sookir6sEPWfNk9cLl8NGgdbW5bBVpz7c1pMGAplIFj5XoEZHrBSk7YL3jzWIG7tOvXceHveKN3enwM9n8T8+zLuoIXn/t9YH+CM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB0745; 7:VdkUiM5F/gzTc2Efp/GgVoVQAsj5/4JeX3z13qE7hnNslzHmkklxq+5ueirXpvhYEdXJQ2pREap/dU+E5y512E3mh8qh6RBib8w0Y1xD4gc/1xQOHonujPvlyKiCZdp6JI7uMU4R1M+swR5wGz1HNZTbyZtr8F0Y9SwAsNogzxErZu7nvOW8adF9whD/YXq96Es5ur2ZdFMyxt8laeK0XVKqWjRNfDWiLHHTzttuc6xDavfLXlxEg0yfcEO/FXFNPEJaPSihxpM7x0+Axmcbhygvqm1MsP8RevIJIyY24pV6pWSNIOu/dIE0SO0hsyA+8oFfuKZ0Q9/gZZmEkAyFsbdRo/5uzvxUf4pPxehB6ImfhnGi9/SciEv+H7VizgE4lFqXM8EZKQBpZgbsfzOI030rQuHOpEED0CPvlxDPLNVCEeXVM/qGDRtjZW4aHkzzlaJPaZbEJJ3fw6IQsUXuog== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Dec 2016 05:15:06.2727 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.158.2]; Helo=[az84smr01.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0301MB0745 Subject: [dpdk-dev] [PATCH v3 15/33] drivers/common/dpaa2: dpio portal driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2016 05:15:09 -0000 From: Hemant Agrawal The portal driver is bound to DPIO objects discovered on the fsl-mc bus and provides services that: - allow other drivers, such as the Ethernet driver, to enqueue and dequeue frames for their respective objects A system will typically allocate 1 DPIO object per CPU to allow queuing operations to happen simultaneously across all CPUs. Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/Makefile | 3 + drivers/bus/fslmc/fslmc_vfio.c | 17 +- drivers/bus/fslmc/fslmc_vfio.h | 5 + drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 364 +++++++++++++++++++++++++ drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 60 ++++ drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 68 +++++ drivers/bus/fslmc/rte_pmd_fslmcbus_version.map | 2 + drivers/common/Makefile | 4 + 8 files changed, 522 insertions(+), 1 deletion(-) create mode 100644 drivers/bus/fslmc/portal/dpaa2_hw_dpio.c create mode 100644 drivers/bus/fslmc/portal/dpaa2_hw_dpio.h create mode 100644 drivers/bus/fslmc/portal/dpaa2_hw_pvt.h diff --git a/drivers/bus/fslmc/Makefile b/drivers/bus/fslmc/Makefile index b74c333..1b815dd 100644 --- a/drivers/bus/fslmc/Makefile +++ b/drivers/bus/fslmc/Makefile @@ -46,6 +46,7 @@ CFLAGS += "-Wno-strict-aliasing" CFLAGS += -I$(RTE_SDK)/drivers/bus/fslmc CFLAGS += -I$(RTE_SDK)/drivers/bus/fslmc/mc +CFLAGS += -I$(RTE_SDK)/drivers/common/dpaa2/qbman/include CFLAGS += -I$(RTE_SDK)/lib/librte_eal/linuxapp/eal # versioning export map @@ -61,10 +62,12 @@ SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += \ mc/dpio.c \ mc/mc_sys.c +SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += portal/dpaa2_hw_dpio.c SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc_vfio.c SRCS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc_bus.c # library dependencies DEPDIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += lib/librte_eal +DEPDIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += lib/librte_pmd_dpaa2_qbman include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index b133b55..ed0a8b9 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -61,6 +61,9 @@ #include "rte_fslmc.h" #include "fslmc_vfio.h" +#include "portal/dpaa2_hw_pvt.h" +#include "portal/dpaa2_hw_dpio.h" + #define VFIO_MAX_CONTAINERS 1 #define FSLMC_VFIO_LOG(level, fmt, args...) \ @@ -259,12 +262,13 @@ int fslmc_vfio_process_group(struct rte_bus *bus) struct fslmc_vfio_device *vdev; struct vfio_device_info device_info = { .argsz = sizeof(device_info) }; char *temp_obj, *object_type, *mcp_obj, *dev_name; - int32_t object_id, i, dev_fd; + int32_t object_id, i, dev_fd, ret; DIR *d; struct dirent *dir; char path[PATH_MAX]; int64_t v_addr; int ndev_count; + int dpio_count = 0; struct fslmc_vfio_group *group = &vfio_groups[0]; static int process_once; @@ -407,9 +411,20 @@ int fslmc_vfio_process_group(struct rte_bus *bus) fslmc_bus_add_device(bus, dev); } + if (!strcmp(object_type, "dpio")) { + ret = dpaa2_create_dpio_device(vdev, + &device_info, + object_id); + if (!ret) + dpio_count++; + } } closedir(d); + ret = dpaa2_affine_qbman_swp(); + if (ret) + FSLMC_VFIO_LOG(DEBUG, "Error in affining qbman swp %d", ret); + return 0; FAILURE: diff --git a/drivers/bus/fslmc/fslmc_vfio.h b/drivers/bus/fslmc/fslmc_vfio.h index c5a42fe..e89d980 100644 --- a/drivers/bus/fslmc/fslmc_vfio.h +++ b/drivers/bus/fslmc/fslmc_vfio.h @@ -71,4 +71,9 @@ int vfio_dmamap_mem_region( int fslmc_vfio_setup_group(void); int fslmc_vfio_process_group(struct rte_bus *bus); +/* create dpio device */ +int dpaa2_create_dpio_device(struct fslmc_vfio_device *vdev, + struct vfio_device_info *obj_info, + int object_id); + #endif /* _FSLMC_VFIO_H_ */ diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c new file mode 100644 index 0000000..011bd9f --- /dev/null +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c @@ -0,0 +1,364 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright (c) 2016 NXP. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Freescale Semiconductor, Inc nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include "dpaa2_hw_pvt.h" +#include "dpaa2_hw_dpio.h" + +#define NUM_HOST_CPUS RTE_MAX_LCORE + +struct dpaa2_io_portal_t dpaa2_io_portal[RTE_MAX_LCORE]; +RTE_DEFINE_PER_LCORE(struct dpaa2_io_portal_t, _dpaa2_io); + +TAILQ_HEAD(dpio_device_list, dpaa2_dpio_dev); +static struct dpio_device_list *dpio_dev_list; /*!< DPIO device list */ +static uint32_t io_space_count; + +/*Stashing Macros default for LS208x*/ +static int dpaa2_core_cluster_base = 0x04; +static int dpaa2_cluster_sz = 2; + +/* For LS208X platform There are four clusters with following mapping: + * Cluster 1 (ID = x04) : CPU0, CPU1; + * Cluster 2 (ID = x05) : CPU2, CPU3; + * Cluster 3 (ID = x06) : CPU4, CPU5; + * Cluster 4 (ID = x07) : CPU6, CPU7; + */ +/* For LS108X platform There are two clusters with following mapping: + * Cluster 1 (ID = x02) : CPU0, CPU1, CPU2, CPU3; + * Cluster 2 (ID = x03) : CPU4, CPU5, CPU6, CPU7; + */ + +/* Set the STASH Destination depending on Current CPU ID. + * e.g. Valid values of SDEST are 4,5,6,7. Where, + * CPU 0-1 will have SDEST 4 + * CPU 2-3 will have SDEST 5.....and so on. + */ +static int +dpaa2_core_cluster_sdest(int cpu_id) +{ + int x = cpu_id / dpaa2_cluster_sz; + + if (x > 3) + x = 3; + + return dpaa2_core_cluster_base + x; +} + +static int +configure_dpio_qbman_swp(struct dpaa2_dpio_dev *dpio_dev) +{ + struct qbman_swp_desc p_des; + struct dpio_attr attr; + + dpio_dev->dpio = malloc(sizeof(struct fsl_mc_io)); + if (!dpio_dev->dpio) { + PMD_INIT_LOG(ERR, "Memory allocation failure\n"); + return -1; + } + + PMD_DRV_LOG(DEBUG, "\t Allocated DPIO Portal[%p]", dpio_dev->dpio); + dpio_dev->dpio->regs = dpio_dev->mc_portal; + if (dpio_open(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->hw_id, + &dpio_dev->token)) { + PMD_INIT_LOG(ERR, "Failed to allocate IO space\n"); + free(dpio_dev->dpio); + return -1; + } + + if (dpio_reset(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token)) { + PMD_INIT_LOG(ERR, "Failed to reset dpio\n"); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); + free(dpio_dev->dpio); + return -1; + } + + if (dpio_enable(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token)) { + PMD_INIT_LOG(ERR, "Failed to Enable dpio\n"); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); + free(dpio_dev->dpio); + return -1; + } + + if (dpio_get_attributes(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token, &attr)) { + PMD_INIT_LOG(ERR, "DPIO Get attribute failed\n"); + dpio_disable(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); + free(dpio_dev->dpio); + return -1; + } + + PMD_INIT_LOG(DEBUG, "Qbman Portal ID %d", attr.qbman_portal_id); + PMD_INIT_LOG(DEBUG, "Portal CE adr 0x%lX", attr.qbman_portal_ce_offset); + PMD_INIT_LOG(DEBUG, "Portal CI adr 0x%lX", attr.qbman_portal_ci_offset); + + /* Configure & setup SW portal */ + p_des.block = NULL; + p_des.idx = attr.qbman_portal_id; + p_des.cena_bar = (void *)(dpio_dev->qbman_portal_ce_paddr); + p_des.cinh_bar = (void *)(dpio_dev->qbman_portal_ci_paddr); + p_des.irq = -1; + p_des.qman_version = attr.qbman_version; + + dpio_dev->sw_portal = qbman_swp_init(&p_des); + if (dpio_dev->sw_portal == NULL) { + PMD_DRV_LOG(ERR, " QBMan SW Portal Init failed\n"); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token); + free(dpio_dev->dpio); + return -1; + } + + PMD_INIT_LOG(DEBUG, "QBMan SW Portal 0x%p\n", dpio_dev->sw_portal); + + return 0; +} + +static int +dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev) +{ + int sdest; + int cpu_id, ret; + + /* Set the Stashing Destination */ + cpu_id = rte_lcore_id(); + if (cpu_id < 0) { + cpu_id = rte_get_master_lcore(); + if (cpu_id < 0) { + RTE_LOG(ERR, PMD, "\tGetting CPU Index failed\n"); + return -1; + } + } + /* Set the STASH Destination depending on Current CPU ID. + * Valid values of SDEST are 4,5,6,7. Where, + * CPU 0-1 will have SDEST 4 + * CPU 2-3 will have SDEST 5.....and so on. + */ + + sdest = dpaa2_core_cluster_sdest(cpu_id); + PMD_DRV_LOG(DEBUG, "Portal= %d CPU= %u SDEST= %d", + dpio_dev->index, cpu_id, sdest); + + ret = dpio_set_stashing_destination(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token, sdest); + if (ret) { + PMD_DRV_LOG(ERR, "%d ERROR in SDEST\n", ret); + return -1; + } + + return 0; +} + +static inline struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(void) +{ + struct dpaa2_dpio_dev *dpio_dev = NULL; + int ret; + + /* Get DPIO dev handle from list using index */ + TAILQ_FOREACH(dpio_dev, dpio_dev_list, next) { + if (dpio_dev && rte_atomic16_test_and_set(&dpio_dev->ref_count)) + break; + } + if (!dpio_dev) + return NULL; + + PMD_DRV_LOG(DEBUG, "New Portal=0x%x (%d) affined thread - %lu", + dpio_dev, dpio_dev->index, syscall(SYS_gettid)); + + ret = dpaa2_configure_stashing(dpio_dev); + if (ret) + PMD_DRV_LOG(ERR, "dpaa2_configure_stashing failed"); + + return dpio_dev; +} + +int +dpaa2_affine_qbman_swp(void) +{ + unsigned int lcore_id = rte_lcore_id(); + uint64_t tid = syscall(SYS_gettid); + + if (lcore_id == LCORE_ID_ANY) + lcore_id = rte_get_master_lcore(); + /* if the core id is not supported */ + else if (lcore_id >= RTE_MAX_LCORE) + return -1; + + if (dpaa2_io_portal[lcore_id].dpio_dev) { + PMD_DRV_LOG(INFO, "DPAA Portal=0x%x (%d) is being shared" + " between thread %lu and current %lu", + dpaa2_io_portal[lcore_id].dpio_dev, + dpaa2_io_portal[lcore_id].dpio_dev->index, + dpaa2_io_portal[lcore_id].net_tid, + tid); + RTE_PER_LCORE(_dpaa2_io).dpio_dev + = dpaa2_io_portal[lcore_id].dpio_dev; + rte_atomic16_inc(&dpaa2_io_portal + [lcore_id].dpio_dev->ref_count); + dpaa2_io_portal[lcore_id].net_tid = tid; + + PMD_DRV_LOG(DEBUG, "Old Portal=0x%x (%d) affined thread - %lu", + dpaa2_io_portal[lcore_id].dpio_dev, + dpaa2_io_portal[lcore_id].dpio_dev->index, + tid); + return 0; + } + + /* Populate the dpaa2_io_portal structure */ + dpaa2_io_portal[lcore_id].dpio_dev = dpaa2_get_qbman_swp(); + + if (dpaa2_io_portal[lcore_id].dpio_dev) { + RTE_PER_LCORE(_dpaa2_io).dpio_dev + = dpaa2_io_portal[lcore_id].dpio_dev; + dpaa2_io_portal[lcore_id].net_tid = tid; + + return 0; + } else { + return -1; + } +} + +int +dpaa2_create_dpio_device(struct fslmc_vfio_device *vdev, + struct vfio_device_info *obj_info, + int object_id) +{ + struct dpaa2_dpio_dev *dpio_dev; + struct vfio_region_info reg_info = { .argsz = sizeof(reg_info)}; + + if (obj_info->num_regions < NUM_DPIO_REGIONS) { + PMD_INIT_LOG(ERR, "ERROR, Not sufficient number " + "of DPIO regions.\n"); + return -1; + } + + if (!dpio_dev_list) { + dpio_dev_list = malloc(sizeof(struct dpio_device_list)); + if (!dpio_dev_list) { + PMD_INIT_LOG(ERR, "Memory alloc failed in DPIO list\n"); + return -1; + } + + /* Initialize the DPIO List */ + TAILQ_INIT(dpio_dev_list); + } + + dpio_dev = malloc(sizeof(struct dpaa2_dpio_dev)); + if (!dpio_dev) { + PMD_INIT_LOG(ERR, "Memory allocation failed for DPIO Device\n"); + return -1; + } + + PMD_DRV_LOG(INFO, "\t Aloocated DPIO [%p]", dpio_dev); + dpio_dev->dpio = NULL; + dpio_dev->hw_id = object_id; + dpio_dev->vfio_fd = vdev->fd; + rte_atomic16_init(&dpio_dev->ref_count); + /* Using single portal for all devices */ + dpio_dev->mc_portal = mcp_ptr_list[MC_PORTAL_INDEX]; + + reg_info.index = 0; + if (ioctl(dpio_dev->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, ®_info)) { + PMD_INIT_LOG(ERR, "vfio: error getting region info\n"); + return -1; + } + + PMD_DRV_LOG(DEBUG, "\t Region Offset = %llx", reg_info.offset); + PMD_DRV_LOG(DEBUG, "\t Region Size = %llx", reg_info.size); + dpio_dev->ce_size = reg_info.size; + dpio_dev->qbman_portal_ce_paddr = (uint64_t)mmap(NULL, reg_info.size, + PROT_WRITE | PROT_READ, MAP_SHARED, + dpio_dev->vfio_fd, reg_info.offset); + + /* Create Mapping for QBMan Cache Enabled area. This is a fix for + * SMMU fault for DQRR statshing transaction. + */ + if (vfio_dmamap_mem_region(dpio_dev->qbman_portal_ce_paddr, + reg_info.offset, reg_info.size)) { + PMD_INIT_LOG(ERR, "DMAMAP for Portal CE area failed.\n"); + return -1; + } + + reg_info.index = 1; + if (ioctl(dpio_dev->vfio_fd, VFIO_DEVICE_GET_REGION_INFO, ®_info)) { + PMD_INIT_LOG(ERR, "vfio: error getting region info\n"); + return -1; + } + + PMD_DRV_LOG(DEBUG, "\t Region Offset = %llx", reg_info.offset); + PMD_DRV_LOG(DEBUG, "\t Region Size = %llx", reg_info.size); + dpio_dev->ci_size = reg_info.size; + dpio_dev->qbman_portal_ci_paddr = (uint64_t)mmap(NULL, reg_info.size, + PROT_WRITE | PROT_READ, MAP_SHARED, + dpio_dev->vfio_fd, reg_info.offset); + + if (configure_dpio_qbman_swp(dpio_dev)) { + PMD_INIT_LOG(ERR, + "Fail to configure the dpio qbman portal for %d\n", + dpio_dev->hw_id); + return -1; + } + + io_space_count++; + dpio_dev->index = io_space_count; + TAILQ_INSERT_HEAD(dpio_dev_list, dpio_dev, next); + + return 0; +} diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h new file mode 100644 index 0000000..682f3fa --- /dev/null +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.h @@ -0,0 +1,60 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright (c) 2016 NXP. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Freescale Semiconductor, Inc nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _DPAA2_HW_DPIO_H_ +#define _DPAA2_HW_DPIO_H_ + +#include +#include + +struct dpaa2_io_portal_t { + struct dpaa2_dpio_dev *dpio_dev; + struct dpaa2_dpio_dev *sec_dpio_dev; + uint64_t net_tid; + uint64_t sec_tid; +}; + +/*! Global per thread DPIO portal */ +RTE_DECLARE_PER_LCORE(struct dpaa2_io_portal_t, _dpaa2_io); + +#define DPAA2_PER_LCORE_DPIO RTE_PER_LCORE(_dpaa2_io).dpio_dev +#define DPAA2_PER_LCORE_PORTAL DPAA2_PER_LCORE_DPIO->sw_portal + +#define DPAA2_PER_LCORE_SEC_DPIO RTE_PER_LCORE(_dpaa2_io).sec_dpio_dev +#define DPAA2_PER_LCORE_SEC_PORTAL DPAA2_PER_LCORE_SEC_DPIO->sw_portal + +/* Affine a DPIO portal to current processing thread */ +int dpaa2_affine_qbman_swp(void); + + +#endif /* _DPAA2_HW_DPIO_H_ */ diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h new file mode 100644 index 0000000..ef3eb71 --- /dev/null +++ b/drivers/bus/fslmc/portal/dpaa2_hw_pvt.h @@ -0,0 +1,68 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright (c) 2016 NXP. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Freescale Semiconductor, Inc nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _DPAA2_HW_PVT_H_ +#define _DPAA2_HW_PVT_H_ + +#include +#include + + +#define MC_PORTAL_INDEX 0 +#define NUM_DPIO_REGIONS 2 + +struct dpaa2_dpio_dev { + TAILQ_ENTRY(dpaa2_dpio_dev) next; + /**< Pointer to Next device instance */ + uint16_t index; /**< Index of a instance in the list */ + rte_atomic16_t ref_count; + /**< How many thread contexts are sharing this.*/ + struct fsl_mc_io *dpio; /** handle to DPIO portal object */ + uint16_t token; + struct qbman_swp *sw_portal; /** SW portal object */ + const struct qbman_result *dqrr[4]; + /**< DQRR Entry for this SW portal */ + void *mc_portal; /**< MC Portal for configuring this device */ + uintptr_t qbman_portal_ce_paddr; + /**< Physical address of Cache Enabled Area */ + uintptr_t ce_size; /**< Size of the CE region */ + uintptr_t qbman_portal_ci_paddr; + /**< Physical address of Cache Inhibit Area */ + uintptr_t ci_size; /**< Size of the CI region */ + int32_t vfio_fd; /**< File descriptor received via VFIO */ + int32_t hw_id; /**< An unique ID of this DPIO device instance */ +}; + +/*! Global MCP list */ +extern void *(*mcp_ptr_list); +#endif diff --git a/drivers/bus/fslmc/rte_pmd_fslmcbus_version.map b/drivers/bus/fslmc/rte_pmd_fslmcbus_version.map index 411200c..4236377 100644 --- a/drivers/bus/fslmc/rte_pmd_fslmcbus_version.map +++ b/drivers/bus/fslmc/rte_pmd_fslmcbus_version.map @@ -1,6 +1,7 @@ DPDK_17.02 { global: + dpaa2_affine_qbman_swp; dpbp_disable; dpbp_enable; dpbp_get_attributes; @@ -46,6 +47,7 @@ DPDK_17.02 { dpseci_reset; dpseci_set_rx_queue; mcp_ptr_list; + per_lcore__dpaa2_io; rte_fslmc_driver_register; rte_fslmc_driver_unregister; vfio_dmamap_mem_region; diff --git a/drivers/common/Makefile b/drivers/common/Makefile index 76ec2d1..434280f 100644 --- a/drivers/common/Makefile +++ b/drivers/common/Makefile @@ -33,6 +33,10 @@ include $(RTE_SDK)/mk/rte.vars.mk CONFIG_RTE_LIBRTE_DPAA2_COMMON = $(CONFIG_RTE_LIBRTE_DPAA2_PMD) +ifneq ($(CONFIG_RTE_LIBRTE_DPAA2_COMMON),y) +CONFIG_RTE_LIBRTE_DPAA2_COMMON = $(CONFIG_RTE_LIBRTE_FSLMC_BUS) +endif + DIRS-$(CONFIG_RTE_LIBRTE_DPAA2_COMMON) += dpaa2 include $(RTE_SDK)/mk/rte.subdir.mk -- 2.7.4