From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0087.outbound.protection.outlook.com [104.47.37.87]) by dpdk.org (Postfix) with ESMTP id D2444FA3C for ; Mon, 5 Dec 2016 08:29:04 +0100 (CET) Received: from CY1PR03CA0002.namprd03.prod.outlook.com (10.174.128.12) by MWHPR03MB2479.namprd03.prod.outlook.com (10.169.200.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.761.9; Mon, 5 Dec 2016 07:29:03 +0000 Received: from BY2FFO11FD017.protection.gbl (2a01:111:f400:7c0c::105) by CY1PR03CA0002.outlook.office365.com (2603:10b6:600::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.761.9 via Frontend Transport; Mon, 5 Dec 2016 07:29:02 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=fail action=none header.from=nxp.com;nxp.com; dkim=none (message not signed) header.d=none; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11FD017.mail.protection.outlook.com (10.1.14.105) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.734.4 via Frontend Transport; Mon, 5 Dec 2016 07:29:02 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:; UpperCasedChecksum:; SizeAsReceived:713; Count:10 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id uB57SdmY016006; Mon, 5 Dec 2016 00:28:59 -0700 From: Akhil Goyal To: CC: , , , , Akhil Goyal Date: Mon, 5 Dec 2016 18:25:38 +0530 Message-ID: <20161205125540.6419-7-akhil.goyal@nxp.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161205125540.6419-1-akhil.goyal@nxp.com> References: <20161205125540.6419-1-akhil.goyal@nxp.com> X-IncomingHeaderCount: 10 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131253965428627852; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(7916002)(2980300002)(1110001)(1109001)(336004)(339900001)(189002)(199003)(2950100002)(92566002)(6916009)(110136003)(6666003)(356003)(8666005)(36756003)(104016004)(5660300001)(77096006)(626004)(97736004)(85426001)(39840400001)(39860400001)(50226002)(7846002)(305945005)(33646002)(5003940100001)(575784001)(50466002)(50986999)(48376002)(4326007)(81156014)(2906002)(81166006)(189998001)(1076002)(8936002)(86362001)(47776003)(2351001)(39450400002)(39400400001)(39850400001)(38730400001)(39410400001)(39380400001)(106466001)(68736007)(105606002)(76176999)(5890100001)(8676002)(7059030)(569005); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR03MB2479; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD017; 1:l6npL12ukKBTYhuMMDsNK2SC8acPUm6vFxgO3MyHtkWx5pW5SLwM20oebyh6gd78mTjLt97jQ146C1m5eH4uqOu8PC3KR8eHLNIr5zXCgoVyPc8r5IrWWLER1IxyDVY4L6rbsypP8H+EwDAxUbfvAkLOVqL7z+tE1glTmYCPnAfHkelHGZXv3zrKtJoAZtIEHHAvJt3cUPFqQvaca06D44VAl6GroEdrgy1J4wDd33sxk76WrTcYpjz5WAcWRwM2Nor8qg9JJdXWSmaAsVQux9UQXVINbd19CvCxvAbu1mf/DaePWhdP2rUUZf1RCGBqWt2Qdq1QdSl9lbxRrqVINoqnDFvoaL7+QOS9Nxj/j7SZ0oWnQgK8QeyaWHEU7p1O/PVMa1aFyevHlkRi55PjTpbR8wsmG1tmT4nqvY0R+o7tQzs4hhF8BeA/4NyV+0KqwgRAHU7i5NZGfrUH12d4KvZJlIWNuq12lh2WwMrC2ZTF0l/2rwA9f291aC887RWL2W1VNOLuFN1Igzad8RFqSoRQh1Unj9vUICVX0TT58VODL/am3AwNlacvz9bhPARnwYD9BsKQbiR8dsrJfLAI6fXtE4CbX0ekzo0+eQoYaYFmnsTT38esyUVcASnX63VKM1PhEIlKFVNnGqZBwNS6LClt3V6DDpl9/5puyJ5XrJSGVJ9dOtMYvnNhaXR8Mhp4iN5NK65s6tueyIO2ru876+ByiYMUs+MwfM5OYfsr44EH9IfZMwee1ywABGBUWJ9lhYvIoDQTPn3c9pHlu1rYSw== MIME-Version: 1.0 Content-Type: text/plain X-MS-Office365-Filtering-Correlation-Id: a5d993d0-3856-4ce8-b82b-08d41ce06330 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:MWHPR03MB2479; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2479; 3:vYcov6TDgnRMKSrgdXa5Y99st25j+N5R2Io60Kdj4Zo6xfnpjAkzy6xbYUJLUseuhdd74TOtxjmDwxGnzkb0LBQ32009qQ9nfiQ58qEo2AyM0b3MUBxL9FF06c9hZ422UNGH4+BWVkjGmd8Uh+Z4fBag59jWcHHO3GlOZOLoMUF4JiTDIjjrYJQs31BPaqLV5wZVDsY4vmdcJcQXsufi3rhHuzQvYHnCPYvpdI8qqx3xn3JgxTJXMZfK3FchiUH5HdCUz3iBJuiZhYrc9VxYvgIMonw2wPjygD4uBUjIpE3CO4O1d4kPJFLZuXO+duuNOFbfJn4gHFiPWjtr3dRbdsUKUhBiw6OPQu1FHRv4vrBDgpbjMXxGH/yX65PKulvH X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2479; 25:M+QLMxfhfpFtigU6SLN7dI2jjyk7466gDEq0I2D0CFmyHUIS5jj5VgbUu187w0dPlxa23dcY3mVvvqmIqzR1aZER4Tx3UbCbT41ytCaEX+5sQYWwSTbzH9g5o2qe8NCxDCtnYF63cBOd0Kb42br1M6JxqU0K/2ST/OnkbbYYWxQOuuvRLrMGQ2TTR7BPkoZ6kdF5MtNt3JDTdN+M11tNsNFlWsSYPg+t1iJVocRlOLixWl6DR5Wgv+RUg0Q8UjawamIBwpzdUuFCjvwAxMbtjd6PVNQLYn/BQoRVuxtzoJP52bNdQI3gzAYYE3xOS3LsdTADWf0X3pZqK4dg0ZOCsArSetrF1+tS0Helg9DHr61iNzdWy0+6MI2iZ3Ro9FyQJKjldQbULHICibRHGQ+9Aea325Fa50aaS7CVG8hQWRmuGex+Js5Ft9M8Dfyb9FxyxelMGqppHZlD0NHvx+tTlTTm6rmVgSJQkUd5Lpj1t07EJMO0LJnImFK4q3K2L0BJJ4J3M7fJmrw7eeAna/kIl08sYJtxJG48jYHTKtzc+WaA0txI6yaK19uQOOogB8LZugp1vRq3dzmo5UCcLuF3ogFs/HKT1f+ju6rydd1hQDtBpATnoFZiNXkpu5bb/gBAKggi7VDLLvPbbnFo3pEE9kPOoyYUXuiCkpKGbNWP3RHw1y3G8njXlFItDw1le/cDgH4HGNgdm2czFKeal8yoKU4AT+1q6U+S9n8QFdPkWEvT5K92dm2OmQU0TE3Pfe3IvGFlR4+uNQt/PgV4WKUQkIfxQBo047N/eiRt0l8yfu8= X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2479; 31:77hyRuo5o9+SnOL5Xj6hToZ9MATQnlVuQ/fF/garqWLz3RRRyty56lNIwrnUJ8X4XYH95InX0ykd8+vgY++Q1MvCNVdAbu5Luf4ZRzwAAcXc+/Mcgi0s7NNwZD8D/RQLMeCdfawgH6XuQWZj5vPpl5ra6ntyGJH/gYFlDNAN/BzJnhPKueUUU9XES/395hwdyBLc9vZD3V6g7RAUL6OFJXZjdzxsGbIwcVOPDtL/tw77HUwrTO8wHaPufKvgtkWy9cd/xqsGQWiPQypkdIYoaV8lc4N6Dfrc/piKOVpRrNc= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095060)(601004)(2401047)(13024025)(13023025)(13018025)(13017025)(13015025)(5005006)(8121501046)(3002001)(10201501046)(6055026)(6096035)(20161123565025)(20161123563025)(20161123556025)(20161123561025)(20161123559025); SRVR:MWHPR03MB2479; BCL:0; PCL:0; RULEID:(400006); SRVR:MWHPR03MB2479; X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2479; 4:jHis7K1Ar6aBjrXCO4N395VacsM4VfWgnXf7WuQI1fRsSaXmYQa6wNKkxSOBejfczspjJEH8zmb7DDD4C4izr6zCFIScBD/o6aj3bqJGl5lFnqrZUqJxjOyVCBn2ztZljPUIo8Mm5BH0DSaOl0XCXjTn6ZZoDR/0w/bvWnnUYmEOJV7k67Z4mRxTJOUrJw9JeKuOL8nbNmA2hWM2bfxiavA9rVLgwalpSqkOoYVvelDq6gw3bZmLtcNLmvzVU1ZlL+wfzlDTkApX1P7Ot5fKNdlzNS82ud+ovNs4O7pJsfku5oYxwGnvjFhtCXWuqh4zBN2yqRQ5hTvSz5tQm+pDqtqNjUj7qrXOzR6H2hOQY4FUPifkAb1O6bZjcjfo1KVenE88uV8GYyxfluGfLBlDF5Gh3oxuXA9JPquhQHktq94TzZVSVnq/oQOur2wiKQ1bLT1MyKUjdCSdRzlw7guNfK7m/BjzDbkdclsrVl6FdG6X2C11qpdeb7vkk8g501y9CuP6Tc3yMaxEwPudZDaGBgai8fjkgJtjVkbe6mpBAoYnHzW5f+G9Ln/v4CggBbfVe7DaJyDmAO14E6X5c4UD26gdX49yAmXzrXDYwssy/1552gjoO8uhldzCNpYl1bftwrwa22EMIANXVo+uFpj5rJMerUH3wo5pPYxyBF57uO0atmCIruzZEc64mnC3SAkloodDH+y9o/P/mXCApJ5c9AOKy05e/CglwKvrjP1KWPp+ggrIZ7UzPyWMKlL4QXvzNaO0ebmxVCONUSr2HiGMMA== X-Forefront-PRVS: 0147E151B5 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR03MB2479; 23:i9Mhnzwml4Cs8RcLpJkzOZpOREpS5M6hGawpFLTxR?= =?us-ascii?Q?t0F6BUK74Qe9KpWBwW3E+fX5kQXkB069wAQdjLW0izuRVkMMQJ1Ij50bgiNx?= =?us-ascii?Q?ZccSBN9oMiAKXngaxBdQb5Fiunr+4HJPnVTwYkh23XB/80yJMTZCDsISbJSy?= =?us-ascii?Q?CR72eKsAWeMXJUCTc6lSngifjayuKYSSIUCFulbkoJU8p8QsVQKbqBa+AhmD?= =?us-ascii?Q?qhyoflkqgL4X+Fnjuy+VciCKADjWLOnQRKoil/4WVSipO8Qp/r380lpuQTCn?= =?us-ascii?Q?CUCbrOZLI2TBj9Vwo/2rxw4ugxaBYCpGfprsaJDkZhaoXw0Cxr51sbjg+1jf?= =?us-ascii?Q?77UcvWTn6jtup10KlCBDtKE0yhtda1WDWIbxmFDYhFDbjHj8HY13PvYXElO1?= =?us-ascii?Q?EuSn0AahUu8qhGnZkY5YVyJic5pMkvDu+7YEG2TBMELR7ZFmFCqxlOfkOhNb?= =?us-ascii?Q?uok+OKwsXLeJW5aNx4hZBDQ4wMzLcU9XjL7aH+m2MhOmie4uCI/mDOgqWZ7G?= =?us-ascii?Q?OhrBWP+FT8tRU1/4RpVVnd5fRNg3uh7yXQ60lkbvyIkckDeu5tHOPudQRFIU?= =?us-ascii?Q?X6AKvYJMB0bNbJuQTlBw0ED8DWX34HNRdNovGeKYLN8BwCfSMlmggwK0KUNp?= =?us-ascii?Q?fN7SXbgGDe7+IIyWnSbmGzExzvDH8Wy5vAS/KmfPrQzR2N2FbSrjfeHHmJWE?= =?us-ascii?Q?6pRduCajFzv0fpbCPvU1XDsgHsUm3OSdML8hzHCLJ/AAazdFgSf6T1ntOFya?= =?us-ascii?Q?3WuMQfAIAFnxAzNZIu8CiTulyAaQdeAFIAABz+OjdsFrkgymtS3Ra9UkrwrM?= =?us-ascii?Q?URT6oxm6NT+Z2H7uQ0WWY6who8R8dt9Cybxdl/9H8qZCoanGRWdSNQPHUEXE?= =?us-ascii?Q?sKpcnABnxCunIf/YgkkTwlnWFtJIlWNp1f4c4Vfz+Hcx0T1jkf5he8ojwFb/?= =?us-ascii?Q?OFsdXbCsUeCqO762ntLod2ylRwbl7N5VZATNsaNmVr1uhPPhw/oJ+DFxO8fm?= =?us-ascii?Q?aHJxmnx8lc03WWPHsHSeU25thfHN01gWoaTtgN4tcIlNC/w13GLIIUr4WMic?= =?us-ascii?Q?j/LZsoCoK45zvniOXZdMf0qciurILx/nQO82qAIfX8p7lhSVHz1LybXCTvtH?= =?us-ascii?Q?bZqoAKQJ9gV6aYI1QhE3fdYtBC4xH2CaOyWhgPEETA8LTGT/P5nzWf5rGkuJ?= =?us-ascii?Q?eC+S8yz5wI8eb/ZwsrnZD9UR703S71gXgekVCM6BUzkkjsbLdjHjnHqaKnLu?= =?us-ascii?Q?FLNzN4tBQirhGv0OD6aOBpONaRHm7zXAamvWtlgbGPJCaYB/1kgBHfyQSM52?= =?us-ascii?Q?KBhPGQWOpez81AiYaonajwAL5D+LxH8PLhm5h+RpiEdUqeoAknMoZhi87Zgn?= =?us-ascii?Q?7bmci9XJoXX/V9kL+q3rM5bUj7ikMJCHs2GV2z5DV1aXuw2?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2479; 6:zM2Grai6etPPeRHB1ikTKEsoo6URN1aA8d30k3pXVc+y47ECdf0/e6MkW85PvgUL6oP78HDsTcJc0/aIQBJsS7acol1YA/RAE+cKxApCEHImuXFDwpIQ6QChw+f7IIiNrxTpeUml/aIDkByII0XNLfV9ryVB03e+mB+cLC3lVZwTD7zeoVXFtzFxA3xhdBGU8zOE89KHkSIQyXfbJMqH+ZyGgcz5RvwaBgwzub5r3b/DxYBwvL8KPswRJtvjHgZHM8gPhhqhJW+zBZ5geKWySVHiD2hIz2U5rXRhg95znJHFVjwFti7MNSj2qpHU+xqep26AJHTkP7ahQbcQQfSlB7bwayYrFcHVHUUgUUbhnor6Ttd1mZdnRnwbSN3/881vvbLVU+PtJc5VtgXvUYSFMBrbww9sF8/hFVtsgvVu/fZJfhQ6dOJit6Jd/224HS2u; 5:oIciNHvrc8TxeBpdR2mNcyJUsUWkcLNJivu2Vg8xvbJXEbESmGj/pwZTZNxH/tcTYeVAK2/IvKJFO9eT1UUYz0M1UqDWabW82PLrHhayLLUc/x6xkg9i+mOuxbRSe5qOuHWTCCGltd2q+0bOjVa19jmldEH00Liywi/NxkVW2zOPLdLctT5HxyfgYF7IiL/v; 24:PPR0HNuE74e/26Yxj4E/HAKdZwGTT0KIOtvi5IRi6uAjeqVkpMB68TrQgfE3n4t6mCvl6iaEKFJc+MmyXnQPCQZR2xsyejMUZIIbsceNsJo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR03MB2479; 7:yTpoEvZrlA5FwlOLvjme0TWibAVMpGtTmGDm3FmvsW6M+vDafVfMLSwoDX6p2jGbOfEPKQIGYPo819ad3QWu/VFscaftJSLnwXMpXqIv1JO56/5EIfjuzTVHDyEj7ZiCuD7/Zn3g1vm/ZScdm35/tSNP3XRLPIv0EX8Xde1RX2mLnFNSAKWjh5ycJRMRrJHksDCJ4mxhDPK4BMGu+yWDaAjqGDlxx+8Yf/fZ0NQIt2bMdsz6GxM/iD2BPol4pVEanew4o1KVD8z0B9HgGc7YaFgTDax9Dyf0BHAUZ5Fhog9W6qMZZT6aW+QPZg0jyyeNQ68NyOuVVkR+Ip8Wg51y74ciGdjZnd/GGP+BdyHPEuJoycfIzBdVPJoJXU9kIsdx2zrLHyj5E5h1dK5rtSiBbmEjQtrpwS5Zarsk1fEk5Q7lX4mtM4ls7CgJ3TJUFIqQiUEUNImSWGdyYYupAolxcw== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2016 07:29:02.6599 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2479 Subject: [dpdk-dev] [PATCH 6/8] crypto/dpaa2_sec: add sec procssing functionality X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Dec 2016 07:29:05 -0000 Signed-off-by: Akhil Goyal Reviewed-by: Hemant Agrawal --- config/defconfig_arm64-dpaa2-linuxapp-gcc | 6 + drivers/crypto/dpaa2_sec/Makefile | 1 - drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 1337 +++++++++++++++++++++++++++ drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h | 516 +++++++++++ drivers/net/dpaa2/base/dpaa2_hw_pvt.h | 25 + 5 files changed, 1884 insertions(+), 1 deletion(-) create mode 100644 drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h diff --git a/config/defconfig_arm64-dpaa2-linuxapp-gcc b/config/defconfig_arm64-dpaa2-linuxapp-gcc index 5338010..5a79374 100644 --- a/config/defconfig_arm64-dpaa2-linuxapp-gcc +++ b/config/defconfig_arm64-dpaa2-linuxapp-gcc @@ -59,3 +59,9 @@ CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=y CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_INIT=n CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_RX=n + +# +# Number of sessions to create in the session memory pool +# on a single DPAA2 SEC device. +# +CONFIG_RTE_DPAA2_SEC_PMD_MAX_NB_SESSIONS=2048 diff --git a/drivers/crypto/dpaa2_sec/Makefile b/drivers/crypto/dpaa2_sec/Makefile index 2cb0611..f8da122 100644 --- a/drivers/crypto/dpaa2_sec/Makefile +++ b/drivers/crypto/dpaa2_sec/Makefile @@ -45,7 +45,6 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) endif CFLAGS += -D _GNU_SOURCE -CFLAGS +=-Wno-unused-function CFLAGS += -I$(RTE_SDK)/drivers/net/dpaa2/ CFLAGS += -I$(RTE_SDK)/drivers/common/dpaa2/mc diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 83b9b61..f249e48 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -48,12 +48,1322 @@ #include #include +#include +#include + +#include "dpaa2_sec_priv.h" #include "dpaa2_sec_logs.h" + +/* RTA header files */ +#include +#include + #define FSL_VENDOR_ID 0x1957 #define FSL_DEVICE_ID 0x410 #define FSL_SUBSYSTEM_SEC 1 #define FSL_MC_DPSECI_DEVID 3 +#define NO_PREFETCH 0 +#define TDES_CBC_IV_LEN 8 +#define AES_CBC_IV_LEN 16 +enum rta_sec_era rta_sec_era = RTA_SEC_ERA_8; +extern struct dpaa2_bp_info bpid_info[MAX_BPID]; + +static inline void print_fd(const struct qbman_fd *fd) +{ + printf("addr_lo: %u\n", fd->simple.addr_lo); + printf("addr_hi: %u\n", fd->simple.addr_hi); + printf("len: %u\n", fd->simple.len); + printf("bpid: %u\n", DPAA2_GET_FD_BPID(fd)); + printf("fi_bpid_off: %u\n", fd->simple.bpid_offset); + printf("frc: %u\n", fd->simple.frc); + printf("ctrl: %u\n", fd->simple.ctrl); + printf("flc_lo: %u\n", fd->simple.flc_lo); + printf("flc_hi: %u\n\n", fd->simple.flc_hi); +} + +static inline void print_fle(const struct qbman_fle *fle) +{ + printf("addr_lo: %u\n", fle->addr_lo); + printf("addr_hi: %u\n", fle->addr_hi); + printf("len: %u\n", fle->length); + printf("fi_bpid_off: %u\n", fle->fin_bpid_offset); + printf("frc: %u\n", fle->frc); +} + +static inline int build_authenc_fd(dpaa2_sec_session *sess, + struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct ctxt_priv *priv = sess->ctxt; + struct qbman_fle *fle, *sge; + struct sec_flow_context *flc; + uint32_t auth_only_len = sym_op->auth.data.length - + sym_op->cipher.data.length; + int icv_len = sym_op->auth.digest.length; + uint8_t *old_icv; + uint32_t mem_len = (7 * sizeof(struct qbman_fle)) + icv_len; + + PMD_INIT_FUNC_TRACE(); + + /* TODO we are using the first FLE entry to store Mbuf. + Currently we donot know which FLE has the mbuf stored. + So while retreiving we can go back 1 FLE from the FD -ADDR + to get the MBUF Addr from the previous FLE. + We can have a better approach to use the inline Mbuf*/ + /* todo - we can use some mempool to avoid malloc here */ + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for SGE\n"); + return -1; + } + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + sge = fle + 2; + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + DPAA2_SET_FLE_BPID(sge + 2, bpid); + DPAA2_SET_FLE_BPID(sge + 3, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + DPAA2_SET_FLE_IVP((sge + 2)); + DPAA2_SET_FLE_IVP((sge + 3)); + } + + /* Save the shared descriptor */ + flc = &priv->flc_desc[0].flc; + /* Configure FD as a FRAME LIST */ + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + PMD_TX_LOG(DEBUG, "auth_off: 0x%x/length %d, digest-len=%d\n" + "cipher_off: 0x%x/length %d, iv-len=%d data_off: 0x%x\n", + sym_op->auth.data.offset, + sym_op->auth.data.length, + sym_op->auth.digest.length, + sym_op->cipher.data.offset, + sym_op->cipher.data.length, + sym_op->cipher.iv.length, + sym_op->m_src->data_off); + + /* Configure Output FLE with Scatter/Gather Entry */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + if (auth_only_len) + DPAA2_SET_FLE_INTERNAL_JD(fle, auth_only_len); + fle->length = (sess->dir == DIR_ENC) ? + (sym_op->cipher.data.length + icv_len) : sym_op->cipher.data.length; + + DPAA2_SET_FLE_SG_EXT(fle); + + /* Configure Output SGE for Encap/Decap */ + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->cipher.data.offset + sym_op->m_src->data_off); + sge->length = sym_op->cipher.data.length; + + if (sess->dir == DIR_ENC) { + sge++; + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->auth.digest.data)); + sge->length = sym_op->auth.digest.length; + DPAA2_SET_FD_LEN(fd, (sym_op->auth.data.length + sym_op->cipher.iv.length)); + } + DPAA2_SET_FLE_FIN(sge); + + sge++; + fle++; + + /* Configure Input FLE with Scatter/Gather Entry */ + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + DPAA2_SET_FLE_SG_EXT(fle); + DPAA2_SET_FLE_FIN(fle); + fle->length = (sess->dir == DIR_ENC) ? + (sym_op->auth.data.length + sym_op->cipher.iv.length) : + (sym_op->auth.data.length + sym_op->cipher.iv.length + + sym_op->auth.digest.length); + + /* Configure Input SGE for Encap/Decap */ + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->cipher.iv.data)); + sge->length = sym_op->cipher.iv.length; + sge++; + + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->auth.data.offset + sym_op->m_src->data_off); + sge->length = sym_op->auth.data.length; + if (sess->dir == DIR_DEC) { + sge++; + old_icv = (uint8_t *)(sge + 1); + memcpy(old_icv, sym_op->auth.digest.data, sym_op->auth.digest.length); + memset(sym_op->auth.digest.data, 0, sym_op->auth.digest.length); + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(old_icv)); + sge->length = sym_op->auth.digest.length; + DPAA2_SET_FD_LEN(fd, (sym_op->auth.data.length + + sym_op->auth.digest.length + sym_op->cipher.iv.length)); + } + DPAA2_SET_FLE_FIN(sge); + if (auth_only_len) { + DPAA2_SET_FLE_INTERNAL_JD(fle, auth_only_len); + DPAA2_SET_FD_INTERNAL_JD(fd, auth_only_len); + } + return 0; +} + +static inline int build_auth_fd( + dpaa2_sec_session *sess, + struct rte_crypto_op *op, + struct qbman_fd *fd, + uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct qbman_fle *fle, *sge; + uint32_t mem_len = (sess->dir == DIR_ENC) ? (3 * sizeof(struct qbman_fle)) : + (5 * sizeof(struct qbman_fle) + sym_op->auth.digest.length); + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + uint8_t *old_digest; + + PMD_INIT_FUNC_TRACE(); + + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for FLE\n"); + return -1; + } + /* TODO we are using the first FLE entry to store Mbuf. + Currently we donot know which FLE has the mbuf stored. + So while retreiving we can go back 1 FLE from the FD -ADDR + to get the MBUF Addr from the previous FLE. + We can have a better approach to use the inline Mbuf*/ + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + } + flc = &priv->flc_desc[DESC_INITFINAL].flc; + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sym_op->auth.digest.data)); + fle->length = sym_op->auth.digest.length; + + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_COMPOUND_FMT(fd); + fle++; + + if (sess->dir == DIR_ENC) { + DPAA2_SET_FLE_ADDR(fle, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(fle, sym_op->auth.data.offset + sym_op->m_src->data_off); + DPAA2_SET_FD_LEN(fd, sym_op->auth.data.length); + fle->length = sym_op->auth.data.length; + } else { + sge = fle + 2; + DPAA2_SET_FLE_SG_EXT(fle); + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + } else { + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + } + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->auth.data.offset + + sym_op->m_src->data_off); + + DPAA2_SET_FD_LEN(fd, sym_op->auth.data.length + + sym_op->auth.digest.length); + sge->length = sym_op->auth.data.length; + sge++; + old_digest = (uint8_t *)(sge + 1); + rte_memcpy(old_digest, sym_op->auth.digest.data, + sym_op->auth.digest.length); + memset(sym_op->auth.digest.data, 0, sym_op->auth.digest.length); + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(old_digest)); + sge->length = sym_op->auth.digest.length; + fle->length = sym_op->auth.data.length + + sym_op->auth.digest.length; + DPAA2_SET_FLE_FIN(sge); + } + DPAA2_SET_FLE_FIN(fle); + + return 0; +} + +static int build_cipher_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + struct rte_crypto_sym_op *sym_op = op->sym; + struct qbman_fle *fle, *sge; + uint32_t mem_len = (5 * sizeof(struct qbman_fle)); + struct sec_flow_context *flc; + struct ctxt_priv *priv = sess->ctxt; + + PMD_INIT_FUNC_TRACE(); + + /* todo - we can use some mempool to avoid malloc here */ + fle = rte_zmalloc(NULL, mem_len, RTE_CACHE_LINE_SIZE); + if (!fle) { + RTE_LOG(ERR, PMD, "Memory alloc failed for SGE\n"); + return -1; + } + /* TODO we are using the first FLE entry to store Mbuf. + Currently we donot know which FLE has the mbuf stored. + So while retreiving we can go back 1 FLE from the FD -ADDR + to get the MBUF Addr from the previous FLE. + We can have a better approach to use the inline Mbuf*/ + DPAA2_SET_FLE_ADDR(fle, DPAA2_OP_VADDR_TO_IOVA(op)); + fle = fle + 1; + sge = fle + 2; + + if (likely(bpid < MAX_BPID)) { + DPAA2_SET_FD_BPID(fd, bpid); + DPAA2_SET_FLE_BPID(fle, bpid); + DPAA2_SET_FLE_BPID(fle + 1, bpid); + DPAA2_SET_FLE_BPID(sge, bpid); + DPAA2_SET_FLE_BPID(sge + 1, bpid); + } else { + DPAA2_SET_FD_IVP(fd); + DPAA2_SET_FLE_IVP(fle); + DPAA2_SET_FLE_IVP((fle + 1)); + DPAA2_SET_FLE_IVP(sge); + DPAA2_SET_FLE_IVP((sge + 1)); + } + + flc = &priv->flc_desc[0].flc; + DPAA2_SET_FD_ADDR(fd, DPAA2_VADDR_TO_IOVA(fle)); + DPAA2_SET_FD_LEN(fd, sym_op->cipher.data.length + sym_op->cipher.iv.length); + DPAA2_SET_FD_COMPOUND_FMT(fd); + DPAA2_SET_FD_FLC(fd, DPAA2_VADDR_TO_IOVA(flc)); + + PMD_TX_LOG(DEBUG, "cipher_off: 0x%x/length %d,ivlen=%d data_off: 0x%x", + sym_op->cipher.data.offset, + sym_op->cipher.data.length, + sym_op->cipher.iv.length, + sym_op->m_src->data_off); + + DPAA2_SET_FLE_ADDR(fle, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(fle, sym_op->cipher.data.offset + sym_op->m_src->data_off); + + /*todo - check the length stuff, idealy this should be only cipher data length */ + fle->length = sym_op->cipher.data.length + sym_op->cipher.iv.length; + + PMD_TX_LOG(DEBUG, "1 - flc = %p, fle = %p FLEaddr = %x-%x, length %d", + flc, fle, fle->addr_hi, fle->addr_lo, fle->length); + + fle++; + + DPAA2_SET_FLE_ADDR(fle, DPAA2_VADDR_TO_IOVA(sge)); + fle->length = sym_op->cipher.data.length + sym_op->cipher.iv.length; + + DPAA2_SET_FLE_SG_EXT(fle); + + DPAA2_SET_FLE_ADDR(sge, DPAA2_VADDR_TO_IOVA(sym_op->cipher.iv.data)); + sge->length = sym_op->cipher.iv.length; + + sge++; + DPAA2_SET_FLE_ADDR(sge, DPAA2_MBUF_VADDR_TO_IOVA(sym_op->m_src)); + DPAA2_SET_FLE_OFFSET(sge, sym_op->cipher.data.offset + sym_op->m_src->data_off); + + sge->length = sym_op->cipher.data.length; + DPAA2_SET_FLE_FIN(sge); + DPAA2_SET_FLE_FIN(fle); + + PMD_TX_LOG(DEBUG, "fdaddr =%p bpid =%d meta =%d off =%d, len =%d", + (void *)DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_BPID(fd), + bpid_info[bpid].meta_data_size, + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + return 0; +} + +static inline int +build_sec_fd(dpaa2_sec_session *sess, struct rte_crypto_op *op, + struct qbman_fd *fd, uint16_t bpid) +{ + int ret = -1; + + PMD_INIT_FUNC_TRACE(); + + switch (sess->ctxt_type) { + case DPAA2_SEC_CIPHER: + ret = build_cipher_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_AUTH: + ret = build_auth_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_CIPHER_HASH: + ret = build_authenc_fd(sess, op, fd, bpid); + break; + case DPAA2_SEC_HASH_CIPHER: + default: + RTE_LOG(ERR, PMD, "error: Unsupported session\n"); + } + return ret; +} + +static uint16_t +dpaa2_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and VQ*/ + uint32_t loop; + int32_t ret; + struct qbman_fd fd_arr[MAX_TX_RING_SLOTS]; + uint32_t frames_to_send; + struct qbman_eq_desc eqdesc; + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_swp *swp; + uint16_t num_tx = 0; + /*todo - need to support multiple buffer pools */ + uint16_t bpid; + struct rte_mempool *mb_pool; + dpaa2_sec_session *sess; + + if (unlikely(nb_ops == 0)) + return 0; + + if (ops[0]->sym->sess_type != RTE_CRYPTO_SYM_OP_WITH_SESSION) { + RTE_LOG(ERR, PMD, "sessionless crypto op not supported\n"); + return 0; + } + /*Prepare enqueue descriptor*/ + qbman_eq_desc_clear(&eqdesc); + qbman_eq_desc_set_no_orp(&eqdesc, DPAA2_EQ_RESP_ERR_FQ); + qbman_eq_desc_set_response(&eqdesc, 0, 0); + qbman_eq_desc_set_fq(&eqdesc, dpaa2_qp->tx_vq.fqid); + + if (!DPAA2_PER_LCORE_SEC_DPIO) { + ret = dpaa2_affine_qbman_swp_sec(); + if (ret) { + RTE_LOG(ERR, PMD, "Failure in affining portal\n"); + return 0; + } + } + swp = DPAA2_PER_LCORE_SEC_PORTAL; + + while (nb_ops) { + frames_to_send = (nb_ops >> 3) ? MAX_TX_RING_SLOTS : nb_ops; + + for (loop = 0; loop < frames_to_send; loop++) { + /*Clear the unused FD fields before sending*/ + memset(&fd_arr[loop], 0, sizeof(struct qbman_fd)); + sess = (dpaa2_sec_session *)(*ops)->sym->session->_private; + mb_pool = (*ops)->sym->m_src->pool; + bpid = mempool_to_bpid(mb_pool); + ret = build_sec_fd(sess, *ops, &fd_arr[loop], bpid); + if (ret) { + PMD_DRV_LOG(ERR, "error: Improper packet" + " contents for crypto operation\n"); + goto skip_tx; + } + ops++; + } + loop = 0; + while (loop < frames_to_send) { + loop += qbman_swp_send_multiple(swp, &eqdesc, + &fd_arr[loop], + frames_to_send - loop); + } + + num_tx += frames_to_send; + nb_ops -= frames_to_send; + } +skip_tx: + dpaa2_qp->tx_vq.tx_pkts += num_tx; + dpaa2_qp->tx_vq.err_pkts += nb_ops; + return num_tx; +} + +static inline +struct rte_crypto_op *sec_fd_to_mbuf( + const struct qbman_fd *fd) +{ + struct qbman_fle *fle; + struct rte_crypto_op *op; + + fle = (struct qbman_fle *)DPAA2_IOVA_TO_VADDR(DPAA2_GET_FD_ADDR(fd)); + + PMD_RX_LOG(DEBUG, "FLE addr = %x - %x, offset = %x", + fle->addr_hi, fle->addr_lo, fle->fin_bpid_offset); + + /* TODO we are using the first FLE entry to store Mbuf. + Currently we donot know which FLE has the mbuf stored. + So while retreiving we can go back 1 FLE from the FD -ADDR + to get the MBUF Addr from the previous FLE. + We can have a better approach to use the inline Mbuf*/ + + if (unlikely(DPAA2_GET_FD_IVP(fd))) { + /* TODO complete it. */ + RTE_LOG(ERR, PMD, "error: Non inline buffer - WHAT to DO?"); + return NULL; + } + op = (struct rte_crypto_op *)DPAA2_IOVA_TO_VADDR( + DPAA2_GET_FLE_ADDR((fle - 1))); + + /* Prefeth op */ + rte_prefetch0(op->sym->m_src); + + PMD_RX_LOG(DEBUG, "mbuf %p BMAN buf addr %p", + (void *)op->sym->m_src, op->sym->m_src->buf_addr); + + PMD_RX_LOG(DEBUG, "fdaddr =%p bpid =%d meta =%d off =%d, len =%d", + (void *)DPAA2_GET_FD_ADDR(fd), + DPAA2_GET_FD_BPID(fd), + bpid_info[DPAA2_GET_FD_BPID(fd)].meta_data_size, + DPAA2_GET_FD_OFFSET(fd), + DPAA2_GET_FD_LEN(fd)); + + /* free the fle memory */ + rte_free(fle - 1); + + return op; +} + +static uint16_t +dpaa2_sec_dequeue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function is responsible to receive frames for a given device and VQ*/ + struct dpaa2_sec_qp *dpaa2_qp = (struct dpaa2_sec_qp *)qp; + struct qbman_result *dq_storage; + uint32_t fqid = dpaa2_qp->rx_vq.fqid; + int ret, num_rx = 0; + uint8_t is_last = 0, status; + struct qbman_swp *swp; + const struct qbman_fd *fd; + struct qbman_pull_desc pulldesc; + + if (!DPAA2_PER_LCORE_SEC_DPIO) { + ret = dpaa2_affine_qbman_swp_sec(); + if (ret) { + RTE_LOG(ERR, PMD, "Failure in affining portal\n"); + return 0; + } + } + swp = DPAA2_PER_LCORE_SEC_PORTAL; + dq_storage = dpaa2_qp->rx_vq.q_storage->dq_storage[0]; + + qbman_pull_desc_clear(&pulldesc); + qbman_pull_desc_set_numframes(&pulldesc, + (nb_ops > DPAA2_DQRR_RING_SIZE) ? + DPAA2_DQRR_RING_SIZE : nb_ops); + qbman_pull_desc_set_fq(&pulldesc, fqid); + qbman_pull_desc_set_storage(&pulldesc, dq_storage, + (dma_addr_t)DPAA2_VADDR_TO_IOVA(dq_storage), 1); + + /*Issue a volatile dequeue command. */ + while (1) { + if (qbman_swp_pull(swp, &pulldesc)) { + RTE_LOG(WARNING, PMD, "SEC VDQ command is not issued." + "QBMAN is busy\n"); + /* Portal was busy, try again */ + continue; + } + break; + }; + + /* Receive the packets till Last Dequeue entry is found with + respect to the above issues PULL command. + */ + while (!is_last) { + /*Check if the previous issued command is completed. + *Also seems like the SWP is shared between the Ethernet Driver + *and the SEC driver.*/ + while (!qbman_check_command_complete(swp, dq_storage)) + ; + + /* Loop until the dq_storage is updated with + * new token by QBMAN */ + while (!qbman_result_has_new_result(swp, dq_storage)) + ; + /* Check whether Last Pull command is Expired and + setting Condition for Loop termination */ + if (qbman_result_DQ_is_pull_complete(dq_storage)) { + is_last = 1; + /* Check for valid frame. */ + status = (uint8_t)qbman_result_DQ_flags(dq_storage); + if (unlikely((status & QBMAN_DQ_STAT_VALIDFRAME) == 0)) { + PMD_RX_LOG(DEBUG, "No frame is delivered"); + continue; + } + } + + fd = qbman_result_DQ_fd(dq_storage); + ops[num_rx] = sec_fd_to_mbuf(fd); + + if (unlikely(fd->simple.frc)) { + /* TODO Parse SEC errors */ + RTE_LOG(ERR, PMD, "SEC returned Error - %x\n", fd->simple.frc); + ops[num_rx]->status = RTE_CRYPTO_OP_STATUS_ERROR; + } else { + ops[num_rx]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; + } + + num_rx++; + dq_storage++; + } /* End of Packet Rx loop */ + + dpaa2_qp->rx_vq.rx_pkts += num_rx; + + PMD_RX_LOG(DEBUG, "SEC Received %d Packets", num_rx); + /*Return the total number of packets received to DPAA2 app*/ + return num_rx; +} +/** Release queue pair */ +static int +dpaa2_sec_queue_pair_release(struct rte_cryptodev *dev, uint16_t queue_pair_id) +{ + struct dpaa2_sec_qp *qp = + (struct dpaa2_sec_qp *)dev->data->queue_pairs[queue_pair_id]; + + PMD_INIT_FUNC_TRACE(); + + if (qp->rx_vq.q_storage) { + dpaa2_free_dq_storage(qp->rx_vq.q_storage); + rte_free(qp->rx_vq.q_storage); + } + rte_free(qp); + + dev->data->queue_pairs[queue_pair_id] = NULL; + + return 0; +} + +/** Setup a queue pair */ +static int +dpaa2_sec_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, + __rte_unused const struct rte_cryptodev_qp_conf *qp_conf, + __rte_unused int socket_id) +{ + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + struct dpaa2_sec_qp *qp; + struct fsl_mc_io *dpseci = (struct fsl_mc_io *)priv->hw; + struct dpseci_rx_queue_cfg cfg; + int32_t retcode; + + PMD_INIT_FUNC_TRACE(); + + /* If qp is already in use free ring memory and qp metadata. */ + if (dev->data->queue_pairs[qp_id] != NULL) { + PMD_DRV_LOG(INFO, "QP already setup"); + return 0; + } + + PMD_DRV_LOG(DEBUG, "dev =%p, queue =%d, conf =%p", + dev, qp_id, qp_conf); + + memset(&cfg, 0, sizeof(struct dpseci_rx_queue_cfg)); + + qp = rte_malloc(NULL, sizeof(struct dpaa2_sec_qp), + RTE_CACHE_LINE_SIZE); + if (!qp) { + RTE_LOG(ERR, PMD, "malloc failed for rx/tx queues\n"); + return -1; + } + + qp->rx_vq.dev = dev; + qp->tx_vq.dev = dev; + qp->rx_vq.q_storage = rte_malloc("sec dq storage", + sizeof(struct queue_storage_info_t), + RTE_CACHE_LINE_SIZE); + if (!qp->rx_vq.q_storage) { + RTE_LOG(ERR, PMD, "malloc failed for q_storage\n"); + return -1; + } + memset(qp->rx_vq.q_storage, 0, sizeof(struct queue_storage_info_t)); + + if (dpaa2_alloc_dq_storage(qp->rx_vq.q_storage)) { + RTE_LOG(ERR, PMD, "dpaa2_alloc_dq_storage failed\n"); + return -1; + } + + dev->data->queue_pairs[qp_id] = qp; + + cfg.options = cfg.options | DPSECI_QUEUE_OPT_USER_CTX; + cfg.user_ctx = (uint64_t)(&qp->rx_vq); + retcode = dpseci_set_rx_queue(dpseci, CMD_PRI_LOW, priv->token, + qp_id, &cfg); + return retcode; +} + +/** Start queue pair */ +static int +dpaa2_sec_queue_pair_start(__rte_unused struct rte_cryptodev *dev, + __rte_unused uint16_t queue_pair_id) +{ + PMD_INIT_FUNC_TRACE(); + + return 0; +} + +/** Stop queue pair */ +static int +dpaa2_sec_queue_pair_stop(__rte_unused struct rte_cryptodev *dev, + __rte_unused uint16_t queue_pair_id) +{ + PMD_INIT_FUNC_TRACE(); + + return 0; +} + +/** Return the number of allocated queue pairs */ +static uint32_t +dpaa2_sec_queue_pair_count(struct rte_cryptodev *dev) +{ + PMD_INIT_FUNC_TRACE(); + + return dev->data->nb_queue_pairs; +} + +/** Returns the size of the aesni gcm session structure */ +static unsigned +dpaa2_sec_session_get_size(struct rte_cryptodev *dev __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); + + return sizeof(dpaa2_sec_session); +} + +static void +dpaa2_sec_session_initialize(struct rte_mempool *mp __rte_unused, + void *sess __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); +} + +static int dpaa2_sec_cipher_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_cipher_ctxt *ctxt = &session->ext_params.cipher_ctxt; + struct alginfo cipherdata; + unsigned int bufsize, i; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + + PMD_INIT_FUNC_TRACE(); + + /* For SEC CIPHER only one descriptor is required. */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[0].flc; + + session->cipher_key.data = rte_zmalloc(NULL, xform->cipher.key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for cipher key"); + return -1; + } + session->cipher_key.length = xform->cipher.key.length; + + memcpy(session->cipher_key.data, xform->cipher.key.data, xform->cipher.key.length); + cipherdata.key = (uint64_t)session->cipher_key.data; + cipherdata.keylen = session->cipher_key.length; + cipherdata.key_enc_flags = 0; + cipherdata.key_type = RTA_DATA_IMM; + + switch (xform->cipher.algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_AES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + ctxt->iv.length = AES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_3DES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + ctxt->iv.length = TDES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + case RTE_CRYPTO_CIPHER_3DES_CTR: + case RTE_CRYPTO_CIPHER_AES_GCM: + case RTE_CRYPTO_CIPHER_AES_CCM: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_XTS: + case RTE_CRYPTO_CIPHER_AES_F8: + case RTE_CRYPTO_CIPHER_ARC4: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + case RTE_CRYPTO_CIPHER_NULL: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u", + xform->cipher.algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + xform->cipher.algo); + goto error_out; + } + session->dir = (xform->cipher.op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + bufsize = cnstr_shdsc_blkcipher(priv->flc_desc[0].desc, 1, 0, + &cipherdata, NULL, ctxt->iv.length, + session->dir); + flc->dhr = 0; + flc->bpv0 = 0x1; + flc->mode_bits = 0x8000; + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + for (i = 0; i < bufsize; i++) + PMD_DRV_LOG(DEBUG, "DESC[%d]:0x%x\n", + i, priv->flc_desc[0].desc[i]); + + return 0; + +error_out: + rte_free(session->cipher_key.data); + return -1; +} + +static int dpaa2_sec_auth_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_auth_ctxt *ctxt = &session->ext_params.auth_ctxt; + struct alginfo authdata; + unsigned int bufsize; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + + PMD_INIT_FUNC_TRACE(); + + /* For SEC AUTH three descriptors are required for various stages */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + 3 * + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[DESC_INITFINAL].flc; + + session->auth_key.data = rte_zmalloc(NULL, xform->auth.key.length, + RTE_CACHE_LINE_SIZE); + session->auth_key.length = xform->auth.key.length; + + memcpy(session->auth_key.data, xform->auth.key.data, + xform->auth.key.length); + authdata.key = (uint64_t)session->auth_key.data; + authdata.keylen = session->auth_key.length; + authdata.key_enc_flags = 0; + authdata.key_type = RTA_DATA_IMM; + + switch (xform->auth.algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA1; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + authdata.algtype = OP_ALG_ALGSEL_MD5; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA256; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA384; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA512; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA224; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA224_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_AES_GCM: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_NULL: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_CCM: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CMAC: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u", + xform->auth.algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + xform->auth.algo); + goto error_out; + } + session->dir = (xform->auth.op == RTE_CRYPTO_AUTH_OP_GENERATE) ? + DIR_ENC : DIR_DEC; + + bufsize = cnstr_shdsc_hmac(priv->flc_desc[DESC_INITFINAL].desc, + 1, 0, &authdata, !session->dir, ctxt->trunc_len); + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + return 0; + +error_out: + rte_free(session->auth_key.data); + return -1; +} + +static int dpaa2_sec_aead_init(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, + dpaa2_sec_session *session) +{ + struct dpaa2_sec_aead_ctxt *ctxt = &session->ext_params.aead_ctxt; + struct alginfo authdata, cipherdata; + unsigned int bufsize; + struct ctxt_priv *priv; + struct sec_flow_context *flc; + struct rte_crypto_cipher_xform *cipher_xform; + struct rte_crypto_auth_xform *auth_xform; + int err; + + PMD_INIT_FUNC_TRACE(); + + if (session->ext_params.aead_ctxt.auth_cipher_text) { + cipher_xform = &xform->cipher; + auth_xform = &xform->next->auth; + session->ctxt_type = + (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DPAA2_SEC_CIPHER_HASH : DPAA2_SEC_HASH_CIPHER; + } else { + cipher_xform = &xform->next->cipher; + auth_xform = &xform->auth; + session->ctxt_type = + (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DPAA2_SEC_HASH_CIPHER : DPAA2_SEC_CIPHER_HASH; + } + /* For SEC AEAD only one descriptor is required */ + priv = (struct ctxt_priv *)rte_zmalloc(NULL, + sizeof(struct ctxt_priv) + sizeof(struct sec_flc_desc), + RTE_CACHE_LINE_SIZE); + if (priv == NULL) { + RTE_LOG(ERR, PMD, "No Memory for priv CTXT"); + return -1; + } + + flc = &priv->flc_desc[0].flc; + + session->cipher_key.data = rte_zmalloc(NULL, cipher_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for cipher key"); + return -1; + } + session->cipher_key.length = cipher_xform->key.length; + session->auth_key.data = rte_zmalloc(NULL, auth_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->auth_key.data == NULL) { + RTE_LOG(ERR, PMD, "No Memory for auth key"); + goto error_out; + } + session->auth_key.length = auth_xform->key.length; + memcpy(session->cipher_key.data, cipher_xform->key.data, + cipher_xform->key.length); + memcpy(session->auth_key.data, auth_xform->key.data, + auth_xform->key.length); + + ctxt->trunc_len = auth_xform->digest_length; + authdata.key = (uint64_t)session->auth_key.data; + authdata.keylen = session->auth_key.length; + authdata.key_enc_flags = 0; + authdata.key_type = RTA_DATA_IMM; + + switch (auth_xform->algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA1; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + authdata.algtype = OP_ALG_ALGSEL_MD5; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA224; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA224_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA256; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA384; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + authdata.algtype = OP_ALG_ALGSEL_SHA512; + authdata.algmode = OP_ALG_AAI_HMAC; + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_AES_GCM: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_NULL: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_CCM: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CMAC: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u", + auth_xform->algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + auth_xform->algo); + goto error_out; + } + cipherdata.key = (uint64_t)session->cipher_key.data; + cipherdata.keylen = session->cipher_key.length; + cipherdata.key_enc_flags = 0; + cipherdata.key_type = RTA_DATA_IMM; + + switch (cipher_xform->algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_AES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + ctxt->iv.length = AES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + cipherdata.algtype = OP_ALG_ALGSEL_3DES; + cipherdata.algmode = OP_ALG_AAI_CBC; + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + ctxt->iv.length = TDES_CBC_IV_LEN; + break; + case RTE_CRYPTO_CIPHER_AES_GCM: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_NULL: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_AES_CTR: + case RTE_CRYPTO_CIPHER_AES_CCM: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u", + cipher_xform->algo); + goto error_out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + cipher_xform->algo); + goto error_out; + } + session->dir = (cipher_xform->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT) ? + DIR_ENC : DIR_DEC; + + priv->flc_desc[0].desc[0] = authdata.keylen; + priv->flc_desc[0].desc[1] = cipherdata.keylen; + err = rta_inline_query(IPSEC_AUTH_VAR_BASE_DESC_LEN, + 0, (unsigned *)priv->flc_desc[0].desc, + &priv->flc_desc[0].desc[2], 2); + + if (err < 0) { + PMD_DRV_LOG(ERR, "Crypto: Incorrect key lengths"); + goto error_out; + } + + if (priv->flc_desc[0].desc[2] & 1) + authdata.key_type = RTA_DATA_PTR; + else + authdata.key_type = RTA_DATA_IMM; + + if (priv->flc_desc[0].desc[2] & (1<<1)) + cipherdata.key_type = RTA_DATA_PTR; + else + cipherdata.key_type = RTA_DATA_IMM; + priv->flc_desc[0].desc[0] = 0; + priv->flc_desc[0].desc[1] = 0; + priv->flc_desc[0].desc[2] = 0; + + if (session->ctxt_type == DPAA2_SEC_CIPHER_HASH) { + bufsize = cnstr_shdsc_authenc(priv->flc_desc[0].desc, 1, + 0, &cipherdata, &authdata, ctxt->iv.length, + ctxt->auth_only_len, ctxt->trunc_len, + session->dir); + } else { + RTE_LOG(ERR, PMD, "Hash before cipher not supported"); + goto error_out; + } + + flc->word1_sdl = (uint8_t)bufsize; + flc->word2_rflc_31_0 = lower_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + flc->word3_rflc_63_32 = upper_32_bits( + (uint64_t)&(((struct dpaa2_sec_qp *) + dev->data->queue_pairs[0])->rx_vq)); + session->ctxt = priv; + + return 0; + +error_out: + rte_free(session->cipher_key.data); + rte_free(session->auth_key.data); + return -1; +} + +static void * +dpaa2_sec_session_configure(struct rte_cryptodev *dev, + struct rte_crypto_sym_xform *xform, void *sess) +{ + dpaa2_sec_session *session = sess; + + PMD_INIT_FUNC_TRACE(); + + if (unlikely(sess == NULL)) { + RTE_LOG(ERR, PMD, "invalid session struct"); + return NULL; + } + /* Cipher Only */ + if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && xform->next == NULL) { + session->ctxt_type = DPAA2_SEC_CIPHER; + dpaa2_sec_cipher_init(dev, xform, session); + + /* Authentication Only */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next == NULL) { + session->ctxt_type = DPAA2_SEC_AUTH; + dpaa2_sec_auth_init(dev, xform, session); + + /* Cipher then Authenticate */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER && + xform->next->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + session->ext_params.aead_ctxt.auth_cipher_text = true; + dpaa2_sec_aead_init(dev, xform, session); + + /* Authenticate then Cipher */ + } else if (xform->type == RTE_CRYPTO_SYM_XFORM_AUTH && + xform->next->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + session->ext_params.aead_ctxt.auth_cipher_text = false; + dpaa2_sec_aead_init(dev, xform, session); + } else { + RTE_LOG(ERR, PMD, "Invalid crypto type"); + return NULL; + } + + return session; +} + +/** Clear the memory of session so it doesn't leave key material behind */ +static void +dpaa2_sec_session_clear(struct rte_cryptodev *dev __rte_unused, void *sess) +{ + PMD_INIT_FUNC_TRACE(); + + if (sess) + memset(sess, 0, sizeof(dpaa2_sec_session)); +} + +static int +dpaa2_sec_dev_configure(struct rte_cryptodev *dev __rte_unused) +{ + PMD_INIT_FUNC_TRACE(); + + return -ENOTSUP; +} + +static int +dpaa2_sec_dev_start(struct rte_cryptodev *dev) +{ + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + struct fsl_mc_io *dpseci = (struct fsl_mc_io *)priv->hw; + struct dpseci_attr attr; + struct dpaa2_queue *dpaa2_q; + struct dpaa2_sec_qp **qp = (struct dpaa2_sec_qp **) + dev->data->queue_pairs; + struct dpseci_rx_queue_attr rx_attr; + struct dpseci_tx_queue_attr tx_attr; + int ret, i; + + PMD_INIT_FUNC_TRACE(); + + memset(&attr, 0, sizeof(struct dpseci_attr)); + + ret = dpseci_enable(dpseci, CMD_PRI_LOW, priv->token); + if (ret) { + PMD_INIT_LOG(ERR, "DPSECI with HW_ID = %d ENABLE FAILED\n", + priv->hw_id); + goto get_attr_failure; + } + ret = dpseci_get_attributes(dpseci, CMD_PRI_LOW, priv->token, &attr); + if (ret) { + PMD_INIT_LOG(ERR, "DPSEC ATTRIBUTE READ FAILED, disabling DPSEC\n"); + goto get_attr_failure; + } + for (i = 0; i < attr.num_rx_queues && qp[i]; i++) { + dpaa2_q = &qp[i]->rx_vq; + dpseci_get_rx_queue(dpseci, CMD_PRI_LOW, priv->token, i, + &rx_attr); + dpaa2_q->fqid = rx_attr.fqid; + PMD_INIT_LOG(DEBUG, "rx_fqid: %d", dpaa2_q->fqid); + } + for (i = 0; i < attr.num_tx_queues && qp[i]; i++) { + dpaa2_q = &qp[i]->tx_vq; + dpseci_get_tx_queue(dpseci, CMD_PRI_LOW, priv->token, i, + &tx_attr); + dpaa2_q->fqid = tx_attr.fqid; + PMD_INIT_LOG(DEBUG, "tx_fqid: %d", dpaa2_q->fqid); + } + + return 0; +get_attr_failure: + dpseci_disable(dpseci, CMD_PRI_LOW, priv->token); + return -1; +} + +static void +dpaa2_sec_dev_stop(struct rte_cryptodev *dev) +{ + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + struct fsl_mc_io *dpseci = (struct fsl_mc_io *)priv->hw; + int ret; + + PMD_INIT_FUNC_TRACE(); + + ret = dpseci_disable(dpseci, CMD_PRI_LOW, priv->token); + if (ret) { + PMD_INIT_LOG(ERR, "Failure in disabling dpseci %d device", + priv->hw_id); + return; + } + + ret = dpseci_reset(dpseci, CMD_PRI_LOW, priv->token); + if (ret < 0) { + PMD_INIT_LOG(ERR, "SEC Device cannot be reset:Error = %0x\n", + ret); + return; + } +} + +static int +dpaa2_sec_dev_close(struct rte_cryptodev *dev) +{ + struct dpaa2_sec_dev_private *priv = dev->data->dev_private; + struct fsl_mc_io *dpseci = (struct fsl_mc_io *)priv->hw; + int ret; + + PMD_INIT_FUNC_TRACE(); + + /*Function is reverse of dpaa2_sec_dev_init. + * It does the following: + * 1. Detach a DPSECI from attached resources i.e. buffer pools, dpbp_id. + * 2. Close the DPSECI device + * 3. Free the allocated resources. + */ + + /*Close the device at underlying layer*/ + ret = dpseci_close(dpseci, CMD_PRI_LOW, priv->token); + if (ret) { + PMD_INIT_LOG(ERR, "Failure closing dpseci device with" + " error code %d\n", ret); + return -1; + } + + /*Free the allocated memory for ethernet private data and dpseci*/ + priv->hw = NULL; + free(dpseci); + + return 0; +} + +static void +dpaa2_sec_dev_infos_get(struct rte_cryptodev *dev, struct rte_cryptodev_info *info) +{ + struct dpaa2_sec_dev_private *internals = dev->data->dev_private; + + PMD_INIT_FUNC_TRACE(); + if (info != NULL) { + info->max_nb_queue_pairs = internals->max_nb_queue_pairs; + info->feature_flags = dev->feature_flags; + info->capabilities = dpaa2_sec_capabilities; + info->sym.max_nb_sessions = internals->max_nb_sessions; + info->dev_type = RTE_CRYPTODEV_DPAA2_SEC_PMD; + } +} + +static struct rte_cryptodev_ops crypto_ops = { + .dev_configure = dpaa2_sec_dev_configure, + .dev_start = dpaa2_sec_dev_start, + .dev_stop = dpaa2_sec_dev_stop, + .dev_close = dpaa2_sec_dev_close, + .dev_infos_get = dpaa2_sec_dev_infos_get, + .queue_pair_setup = dpaa2_sec_queue_pair_setup, + .queue_pair_release = dpaa2_sec_queue_pair_release, + .queue_pair_start = dpaa2_sec_queue_pair_start, + .queue_pair_stop = dpaa2_sec_queue_pair_stop, + .queue_pair_count = dpaa2_sec_queue_pair_count, + .session_get_size = dpaa2_sec_session_get_size, + .session_initialize = dpaa2_sec_session_initialize, + .session_configure = dpaa2_sec_session_configure, + .session_clear = dpaa2_sec_session_clear, +}; + static int dpaa2_sec_uninit(__attribute__((unused)) const struct rte_cryptodev_driver *crypto_drv, struct rte_cryptodev *dev) @@ -71,13 +1381,29 @@ static int dpaa2_sec_dev_init(__attribute__((unused)) struct rte_cryptodev_driver *crypto_drv, struct rte_cryptodev *dev) { + struct dpaa2_sec_dev_private *internals; struct fsl_mc_io *dpseci; uint16_t token; + struct dpseci_attr attr; int retcode, hw_id = dev->pci_dev->addr.devid; + PMD_INIT_FUNC_TRACE(); + PMD_INIT_LOG(DEBUG, "Found crypto device at %02x:%02x.%x", + dev->pci_dev->addr.bus, + dev->pci_dev->addr.devid, + dev->pci_dev->addr.function); dev->dev_type = RTE_CRYPTODEV_DPAA2_SEC_PMD; + dev->dev_ops = &crypto_ops; + + dev->enqueue_burst = dpaa2_sec_enqueue_burst; + dev->dequeue_burst = dpaa2_sec_dequeue_burst; + dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | + RTE_CRYPTODEV_FF_HW_ACCELERATED | + RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING; + internals = dev->data->dev_private; + internals->max_nb_sessions = RTE_DPAA2_SEC_PMD_MAX_NB_SESSIONS; /* * For secondary processes, we don't initialise any further as primary @@ -104,8 +1430,18 @@ dpaa2_sec_dev_init(__attribute__((unused)) struct rte_cryptodev_driver *crypto_d retcode); goto init_error; } + retcode = dpseci_get_attributes(dpseci, CMD_PRI_LOW, token, &attr); + if (retcode != 0) { + PMD_INIT_LOG(ERR, "Cannot get dpsec device attributed: Error = %x", + retcode); + goto init_error; + } sprintf(dev->data->name, "dpsec-%u", hw_id); + internals->max_nb_queue_pairs = attr.num_tx_queues; + dev->data->nb_queue_pairs = internals->max_nb_queue_pairs; + internals->hw = dpseci; + internals->token = token; PMD_INIT_LOG(DEBUG, "driver %s: created\n", dev->data->name); return 0; @@ -133,6 +1469,7 @@ static struct rte_cryptodev_driver rte_dpaa2_sec_pmd = { }, .cryptodev_init = dpaa2_sec_dev_init, .cryptodev_uninit = dpaa2_sec_uninit, + .dev_private_size = sizeof(struct dpaa2_sec_dev_private), }; RTE_PMD_REGISTER_PCI(CRYPTODEV_NAME_DPAA2_SEC_PMD, rte_dpaa2_sec_pmd.pci_drv); diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h new file mode 100644 index 0000000..01fae77 --- /dev/null +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_priv.h @@ -0,0 +1,516 @@ +/*- + * BSD LICENSE + * + * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. + * Copyright (c) 2016 NXP. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Freescale Semiconductor, Inc nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_DPAA2_SEC_PMD_PRIVATE_H_ +#define _RTE_DPAA2_SEC_PMD_PRIVATE_H_ + +#define MAX_QUEUES 64 +#define MAX_DESC_SIZE 64 +/** private data structure for each DPAA2_SEC device */ +struct dpaa2_sec_dev_private { + void *mc_portal; /**< MC Portal for configuring this device */ + void *hw; /**< Hardware handle for this device.Used by NADK framework */ + int32_t hw_id; /**< An unique ID of this device instance */ + int32_t vfio_fd; /**< File descriptor received via VFIO */ + uint16_t token; /**< Token required by DPxxx objects */ + unsigned max_nb_queue_pairs; + + unsigned max_nb_sessions; + /**< Max number of sessions supported by device */ +}; + +struct dpaa2_sec_qp { + struct dpaa2_queue rx_vq; + struct dpaa2_queue tx_vq; +}; + +enum shr_desc_type { + DESC_UPDATE, + DESC_FINAL, + DESC_INITFINAL, +}; + +#define DIR_ENC 1 +#define DIR_DEC 0 + +/* SEC Flow Context Descriptor */ +struct sec_flow_context { + /* word 0 */ + uint16_t word0_sdid; /* 11-0 SDID */ + uint16_t word0_res; /* 31-12 reserved */ + + /* word 1 */ + uint8_t word1_sdl; /* 5-0 SDL */ + /* 7-6 reserved */ + + uint8_t word1_bits_15_8; /* 11-8 CRID */ + /* 14-12 reserved */ + /* 15 CRJD */ + + uint8_t word1_bits23_16; /* 16 EWS */ + /* 17 DAC */ + /* 18,19,20 ? */ + /* 23-21 reserved */ + + uint8_t word1_bits31_24; /* 24 RSC */ + /* 25 RBMT */ + /* 31-26 reserved */ + + /* word 2 RFLC[31-0] */ + uint32_t word2_rflc_31_0; + + /* word 3 RFLC[63-32] */ + uint32_t word3_rflc_63_32; + + /* word 4 */ + uint16_t word4_iicid; /* 15-0 IICID */ + uint16_t word4_oicid; /* 31-16 OICID */ + + /* word 5 */ + uint32_t word5_ofqid:24; /* 23-0 OFQID */ + uint32_t word5_31_24:8; + /* 24 OSC */ + /* 25 OBMT */ + /* 29-26 reserved */ + /* 31-30 ICR */ + + /* word 6 */ + uint32_t word6_oflc_31_0; + + /* word 7 */ + uint32_t word7_oflc_63_32; + + /* Word 8-15 storage profiles */ + uint16_t dl; /**< DataLength(correction) */ + uint16_t reserved; /**< reserved */ + uint16_t dhr; /**< DataHeadRoom(correction) */ + uint16_t mode_bits; /**< mode bits */ + uint16_t bpv0; /**< buffer pool0 valid */ + uint16_t bpid0; /**< Bypass Memory Translation */ + uint16_t bpv1; /**< buffer pool1 valid */ + uint16_t bpid1; /**< Bypass Memory Translation */ + uint64_t word_12_15[2]; /**< word 12-15 are reserved */ +}; + +struct sec_flc_desc { + struct sec_flow_context flc; + uint32_t desc[MAX_DESC_SIZE]; +}; + +struct ctxt_priv { + struct sec_flc_desc flc_desc[0]; +}; + +enum dpaa2_sec_op_type { + DPAA2_SEC_NONE, /*!< No Cipher operations*/ + DPAA2_SEC_CIPHER,/*!< CIPHER operations */ + DPAA2_SEC_AUTH, /*!< Authentication Operations */ + DPAA2_SEC_CIPHER_HASH, /*!< Authenticated Encryption with associated data */ + DPAA2_SEC_HASH_CIPHER, /*!< Encryption with Authenticated associated data */ + DPAA2_SEC_IPSEC, /*!< IPSEC protocol operations*/ + DPAA2_SEC_PDCP, /*!< PDCP protocol operations*/ + DPAA2_SEC_PKC, /*!< Public Key Cryptographic Operations */ + DPAA2_SEC_MAX +}; + +struct dpaa2_sec_cipher_ctxt { + struct { + uint8_t *data; + uint16_t length; + } iv; /**< Initialisation vector parameters */ + uint8_t *init_counter; /*!< Set initial counter for CTR mode */ +}; + +struct dpaa2_sec_auth_ctxt { + uint8_t trunc_len; /*!< Length for output ICV, should + * be 0 if no truncation required */ +}; + +struct dpaa2_sec_aead_ctxt { + struct { + uint8_t *data; + uint16_t length; + } iv; /**< Initialisation vector parameters */ + uint16_t auth_only_len; /*!< Length of data for Auth only */ + uint8_t auth_cipher_text; /**< Authenticate/cipher ordering */ + uint8_t trunc_len; /*!< Length for output ICV, should + * be 0 if no truncation required */ +}; + +typedef struct dpaa2_sec_session_entry { + void *ctxt; + uint8_t ctxt_type; + uint8_t dir; /*!< Operation Direction */ + enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ + enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ + struct { + uint8_t *data; /**< pointer to key data */ + size_t length; /**< key length in bytes */ + } cipher_key; + struct { + uint8_t *data; /**< pointer to key data */ + size_t length; /**< key length in bytes */ + } auth_key; + uint8_t status; + union { + struct dpaa2_sec_cipher_ctxt cipher_ctxt; + struct dpaa2_sec_auth_ctxt auth_ctxt; + struct dpaa2_sec_aead_ctxt aead_ctxt; +/* struct nadk_ipsec_ctxt ipsec_ctxt; + struct nadk_pdcp_ctxt pdcp_ctxt; + struct nadk_null_sec_ctxt null_sec_ctxt;*/ + } ext_params; +} dpaa2_sec_session; + +static const struct rte_cryptodev_capabilities dpaa2_sec_capabilities[] = { + { /* MD5 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_MD5_HMAC, + .block_size = 64, + .key_size = { + .min = 64, + .max = 64, + .increment = 0 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* SHA1 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA1_HMAC, + .block_size = 64, + .key_size = { + .min = 64, + .max = 64, + .increment = 0 + }, + .digest_size = { + .min = 20, + .max = 20, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* SHA224 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA224_HMAC, + .block_size = 64, + .key_size = { + .min = 64, + .max = 64, + .increment = 0 + }, + .digest_size = { + .min = 28, + .max = 28, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* SHA256 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA256_HMAC, + .block_size = 64, + .key_size = { + .min = 64, + .max = 64, + .increment = 0 + }, + .digest_size = { + .min = 32, + .max = 32, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* SHA384 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA384_HMAC, + .block_size = 128, + .key_size = { + .min = 128, + .max = 128, + .increment = 0 + }, + .digest_size = { + .min = 48, + .max = 48, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* SHA512 HMAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_SHA512_HMAC, + .block_size = 128, + .key_size = { + .min = 128, + .max = 128, + .increment = 0 + }, + .digest_size = { + .min = 64, + .max = 64, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* AES XCBC MAC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_AES_XCBC_MAC, + .block_size = 16, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .digest_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .aad_size = { 0 } + }, } + }, } + }, + { /* AES GCM (AUTH) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + {.auth = { + .algo = RTE_CRYPTO_AUTH_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .digest_size = { + .min = 8, + .max = 16, + .increment = 4 + }, + .aad_size = { + .min = 8, + .max = 12, + .increment = 4 + } + }, } + }, } + }, + { /* AES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* AES CTR */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_CTR, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* AES GCM (CIPHER) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + .cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_GCM, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + } + } + }, + + { /* 3DES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_3DES_CBC, + .block_size = 8, + .key_size = { + .min = 16, + .max = 24, + .increment = 8 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, + { /* 3DES CTR */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_3DES_CTR, + .block_size = 8, + .key_size = { + .min = 16, + .max = 24, + .increment = 8 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, + { /* SNOW3G (UIA2) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_AUTH, + .auth = { + .algo = RTE_CRYPTO_AUTH_SNOW3G_UIA2, + .block_size = 16, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .digest_size = { + .min = 4, + .max = 4, + .increment = 0 + }, + .aad_size = { + .min = 16, + .max = 16, + .increment = 0 + } + } + } + }, + { /* SNOW3G (UEA2) */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + .cipher = { + .algo = RTE_CRYPTO_CIPHER_SNOW3G_UEA2, + .block_size = 16, + .key_size = { + .min = 16, + .max = 16, + .increment = 0 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + } + } + }, + RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() +}; +#endif /* _RTE_DPAA2_SEC_PMD_PRIVATE_H_ */ diff --git a/drivers/net/dpaa2/base/dpaa2_hw_pvt.h b/drivers/net/dpaa2/base/dpaa2_hw_pvt.h index a1afa23..ee3fdc9 100644 --- a/drivers/net/dpaa2/base/dpaa2_hw_pvt.h +++ b/drivers/net/dpaa2/base/dpaa2_hw_pvt.h @@ -140,8 +140,10 @@ struct qbman_fle { } while (0) #define DPAA2_SET_FD_LEN(fd, length) fd->simple.len = length #define DPAA2_SET_FD_BPID(fd, bpid) (fd->simple.bpid_offset |= bpid) +#define DPAA2_SET_FD_IVP(fd) ((fd->simple.bpid_offset |= 0x00004000)) #define DPAA2_SET_FD_OFFSET(fd, offset) \ ((fd->simple.bpid_offset |= (uint32_t)(offset) << 16)) +#define DPAA2_SET_FD_INTERNAL_JD(fd, len) fd->simple.frc = (0x80000000 | (len)) #define DPAA2_RESET_FD_CTRL(fd) fd->simple.ctrl = 0 #define DPAA2_SET_FD_ASAL(fd, asal) (fd->simple.ctrl |= (asal << 16)) @@ -149,12 +151,32 @@ struct qbman_fle { fd->simple.flc_lo = lower_32_bits((uint64_t)addr); \ fd->simple.flc_hi = upper_32_bits((uint64_t)addr); \ } while (0) +#define DPAA2_SET_FLE_INTERNAL_JD(fle, len) (fle->frc = (0x80000000 | (len))) +#define DPAA2_GET_FLE_ADDR(fle) \ + (uint64_t)((((uint64_t)(fle->addr_hi)) << 32) + fle->addr_lo) +#define DPAA2_SET_FLE_ADDR(fle, addr) do { \ + fle->addr_lo = lower_32_bits((uint64_t)addr); \ + fle->addr_hi = upper_32_bits((uint64_t)addr); \ +} while (0) +#define DPAA2_SET_FLE_OFFSET(fle, offset) \ + ((fle)->fin_bpid_offset |= (uint32_t)(offset) << 16) +#define DPAA2_SET_FLE_BPID(fle, bpid) ((fle)->fin_bpid_offset |= (uint64_t)bpid) +#define DPAA2_GET_FLE_BPID(fle, bpid) (fle->fin_bpid_offset & 0x000000ff) +#define DPAA2_SET_FLE_FIN(fle) (fle->fin_bpid_offset |= (uint64_t)1 << 31) +#define DPAA2_SET_FLE_IVP(fle) (((fle)->fin_bpid_offset |= 0x00004000)) +#define DPAA2_SET_FD_COMPOUND_FMT(fd) \ + (fd->simple.bpid_offset |= (uint32_t)1 << 28) #define DPAA2_GET_FD_ADDR(fd) \ ((uint64_t)((((uint64_t)(fd->simple.addr_hi)) << 32) + fd->simple.addr_lo)) #define DPAA2_GET_FD_LEN(fd) (fd->simple.len) #define DPAA2_GET_FD_BPID(fd) ((fd->simple.bpid_offset & 0x00003FFF)) +#define DPAA2_GET_FD_IVP(fd) ((fd->simple.bpid_offset & 0x00004000) >> 14) #define DPAA2_GET_FD_OFFSET(fd) ((fd->simple.bpid_offset & 0x0FFF0000) >> 16) +#define DPAA2_SET_FLE_SG_EXT(fle) (fle->fin_bpid_offset |= (uint64_t)1 << 29) +#define DPAA2_IS_SET_FLE_SG_EXT(fle) \ + ((fle->fin_bpid_offset & ((uint64_t)1 << 29)) ? 1 : 0) + #define DPAA2_INLINE_MBUF_FROM_BUF(buf, meta_data_size) \ ((struct rte_mbuf *)((uint64_t)buf - meta_data_size)) @@ -207,6 +229,7 @@ static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) */ #define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) (mbuf->buf_physaddr) +#define DPAA2_OP_VADDR_TO_IOVA(op) (op->phys_addr) /** * macro to convert Virtual address to IOVA @@ -227,6 +250,8 @@ static phys_addr_t dpaa2_mem_vtop(uint64_t vaddr) #else /* RTE_LIBRTE_DPAA2_USE_PHYS_IOVA */ #define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) (mbuf->buf_addr) +#define DPAA2_OP_VADDR_TO_IOVA(op) (op) + #define DPAA2_VADDR_TO_IOVA(_vaddr) (_vaddr) #define DPAA2_IOVA_TO_VADDR(_iova) (_iova) #define DPAA2_MODIFY_IOVA_TO_VADDR(_mem, _type) -- 2.9.3