From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0074.outbound.protection.outlook.com [104.47.33.74]) by dpdk.org (Postfix) with ESMTP id AB5741B143 for ; Fri, 5 Oct 2018 15:02:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pqLBfQiRcFtkKIGZEozDflOCGjtiHYbrM25Ct2HKgIY=; b=im3ecfnJMBj6B2GlppazB3f9ZNnkgUXHWSAiSyD60quAoidvig3kkryJ2gxaiD/AlFZ9uzf4t4QOh0zyq0kZNi1lU0nvR0Sqe/qzm6fparwbBSloqpBkqtVlk9A0nuLKaN8VS8NS4tuksnVSmU9eXEhJ2chD1thpm+huKwezhDU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.com (115.113.156.2) by SN6PR07MB4910.namprd07.prod.outlook.com (2603:10b6:805:39::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.21; Fri, 5 Oct 2018 13:02:11 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Srisivasubramanian S , Jerin Jacob , Narayana Prasad , dev@dpdk.org, Ankur Dwivedi , Anoob Joseph , Murthy NSSR , Nithin Dabilpuram , Ragothaman Jayaraman , Tejasree Kondoj Date: Fri, 5 Oct 2018 18:29:16 +0530 Message-Id: <1538744363-30340-26-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1536033560-21541-1-git-send-email-ajoseph@caviumnetworks.com> <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: MA1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:22::22) To SN6PR07MB4910.namprd07.prod.outlook.com (2603:10b6:805:39::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c3fe9d74-e18a-425a-2606-08d62ac2c5cb X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:SN6PR07MB4910; X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4910; 3:9KwVdxG4hi/gHROUC5Vwecq3cRVlgZ9o4E8vkoADuegQhkRRxPRz/Z09JE0RaabA7OpfsZ39arkeM2ELdsZPYBO66yfgYs9E7EJR6e4g2bq5GrPTNKlxZ7v746niQjJNecUD/pSPzFhqasCIJIMmK4fhNPu7/CGZk55dRPMUp6ustz8JkZj/4NZwdn/Ro0DZBBjykv6wIDPsf1JzlYuVml10hYyzIz40iXWDANoaZ5KzZdMZxPmGZZuN/0EqmbOQ; 25:9/MHwbW0CssSe6pfjCNNPWOEOT+SkzTLJUStFDroafBlr3zXoI54ME9XVPOiAi1VRa1IIn7UEviRj1BnJko55kWnOxA5wQwk5K6/XD1rz115dbgZubwWfe6qusfPmI1gjyJvaO5k4GHKOkgq+Csk0I7BBJgxL3I+joSLjNq2IEa1fTTdzrIAjs0e0d+7+VbvZ+0KkZ/oAoRfNMBGxbnDl3BF5wb0QP0zaL4wdAW9DsCnu9rm5ZQ7hEe+VA7vuox2/cKxPADMCLSXuo94T5etjbYsuyu0hj0yvxR2lhJws+bh2Tfg1+vGMSEJ/6E2+riqlA5y21cFwwdtQXQxB7iC5A==; 31:8K5nZOqyu7l8qsl9u6O7/f7qTTOup9BmUoNoMKRyb/w+jVt32nFe9L2IqTwnIUqsKl6/PU87/t2loUeJdwdLmmUeNowPO8mlGkZYjIPO0mA6b5CK+yLWNi4AFLhXAj4VhLs/YeWtxmAarDLlGA1g9AAhn1elrQYfmZPcUbZyWAvN1K1jkmv2y/RFsn6/sg/N9hjynlG7kBuH06FNFNOCKSlUaoDGSNDL7e1j0L6iIlQ= X-MS-TrafficTypeDiagnostic: SN6PR07MB4910: X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4910; 20:An4iIXDYrlI3w5iQXw3iDyk0ZkNu9cr+oJ3QW9e8zptc1przEm+y4UvGpLRnm+2bQMIAVqXSYcm5OObXPLLWJE5M2jZDATs7sD0k8Q03IkKUQDVIJqZj1OxVLt4MmHVsChbAQ6ppsahs3neZ6My/zt6dN+vWz0enP3xk6sMLgFVznYBEoJh8MU1LeaWO5CyQDmQ1BwR32bnXOKQEsv/1BRNM8nL/CF+Gf/MFCMIhXVpCOa0x3MoJq8Uis5oNjrz9llq+/wzPQMRfQcKxZzijMth/Td0Dn8pGLoISDA2R4t1V7olBJhB9HZdWRMvRT+5sAVUg8SgxCDg6vb40gjojH4ygenBtesUs8sqExOjF5sNg3ehz2CBBMMsQoDbvV3H2tAFFFHcQTEVn128RzpBDjvjfiKbYcP1KZL3YXqJz2XPkWPXENyHp8l/lKPO4bwaR4CnFSAoRSFDavjwAo/LAGYWBLTXbH/JEzdwgOSn4cpbvyi/rP8+6SfqFONpNRLpuNp4ULYKvPqdZvFLGu0eA/Sb3h2IFMyyhmIFIdXqpF+MoEp0v9K0QKsuQ++9zP15QQMGJQB1Pf3f6DNAKpR2YPjprZYOK1GOyzI2yeyGIPHw=; 4:CWSc5c9J/CLUu4fQNafXkwY2Y70aTG9TA6yBDucS3z816hicHLqwt/3CyMeNceOtog2S8i2tTNpkqsOYplPHOarXJI0URb/0W/c9Wokd2onpX75QU0deG7oc++rGkaGbM5GDhKQlT/f4+Paxs+gDfrkGxuGr5T2AYCoggecmqI/ItXYCMhnTm3Ir3XNmqW+GdY7ALitLWK0ra/CtLk2Bchf9JcWvcxrX0Xh7CWBQwApXw0H1UKHKRQcGdQgm5mrzJ+os2MvLT6xJWuwpu5gG8w== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(3231355)(944501410)(52105095)(10201501046)(93006095)(149066)(150057)(6041310)(20161123564045)(20161123558120)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051); SRVR:SN6PR07MB4910; BCL:0; PCL:0; RULEID:; SRVR:SN6PR07MB4910; X-Forefront-PRVS: 0816F1D86E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(39860400002)(346002)(136003)(396003)(366004)(199004)(189003)(51416003)(76176011)(11346002)(50226002)(25786009)(8936002)(48376002)(305945005)(186003)(16526019)(110136005)(6486002)(3846002)(446003)(52116002)(6116002)(81156014)(8676002)(97736004)(5660300001)(50466002)(4326008)(956004)(42882007)(2616005)(68736007)(478600001)(54906003)(7736002)(81166006)(16586007)(107886003)(26005)(55236004)(66066001)(386003)(47776003)(6512007)(72206003)(44832011)(476003)(316002)(14444005)(36756003)(4744004)(105586002)(106356001)(486006)(6506007)(2906002)(53936002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR07MB4910; H:ajoseph83.caveonetworks.com.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN6PR07MB4910; 23:hIZZBIOIjFM7KiC+4JIB30gZA+4W9352PkyVu4Chx?= =?us-ascii?Q?nA3VmIA/DTr4yd4KbwLXEW29aNyLmg+6Cn2bynBg9JnWYi0whrSgx6UTIRt/?= =?us-ascii?Q?WbRqIR+oNoyoK3K8x6iJs7Fe+qy/9NAru5JTy+bJE+bjG/3pTk+y6QG1gmS/?= =?us-ascii?Q?X2XwckvW4Ai8yc295anSQhZuL//JQZIbbok/D4/H3C2/CQt1KMu1NnwE+PEi?= =?us-ascii?Q?kqQzkZ1byjIS3NzO9nwpe2PWIX1u0kQDITFuvW01e2rJjCNrUsLqiIcvCtcG?= =?us-ascii?Q?/UDke4Gd9Rw22QR+aG0y7biJ/sTnam9S5+mdaMVF+onsHAEm7LlwqifI71pI?= =?us-ascii?Q?YmVy9r1hLUto6P0qCxiXoW5iIQQbsoDkJdJ0ywSQuoT+yTi55/d+GtE2J42w?= =?us-ascii?Q?gXPSSfUibMR9SoKgzmTsgbbuJYIBJU9BznFqHcZxFc7Tz4girRhLz2hpuGlW?= =?us-ascii?Q?D21+3aU4mmIEytqhFFDT4hjgEEtyS5PkVqt7GBXEKoUv6gTJb5XAYaxjLGoa?= =?us-ascii?Q?2ErX5QL6WN5ZVk4pdfaWSMXIKFUxSG+nPUKxCknH1ghzUGGEy8Ty47SpwsQ7?= =?us-ascii?Q?sVby/dGV+X363y/rHLmkv4BqeI/MMmZ691f31uYrNOGnCxa68v5IGQRsDoYl?= =?us-ascii?Q?XmE2b4B4+zErtJQNB3s+WHipeTlZcotfiib8iFCqMR1nt3RpdWIC3kF+Qd6E?= =?us-ascii?Q?PhUDrcXsOnIIZ2EX3/Gv2fGXpZFJwFWCtslYGJZAjA1inE4O9Fd2exYdmA5O?= =?us-ascii?Q?sfK2qoBJa8+75TWDIrkq4GrKM3UVxTkeeDAZ88V8Fr25nSfp4vm8a1H6dtso?= =?us-ascii?Q?oSu/YGe40Z+AmLNMgUZBOL005EK559IGoM5DSw937jx1crL8t7fBc02P7kbL?= =?us-ascii?Q?P3HEfJp1dB1YzNrImOpxa5jjiDoSDFvtbGO+tibn9U/2fBnZtbM4ANWC3ZkI?= =?us-ascii?Q?WtA5XXqnrLRrvMuwCFM2qc3CZvv984pnvUF5JXtyDtF9d5xe+SZf221q8nNf?= =?us-ascii?Q?jJsY2Hscfl2jQqG7GpE3/6rsSKdzDNWyNPU0W/jCWlOnrBXKMvzDO/f7yZRM?= =?us-ascii?Q?awJualZLMgV/xbCQXjUvtTfrutycKW5mnlFdE1fmxCJtR0dmmRqASnodwmLR?= =?us-ascii?Q?AnwK1RScj94K2e9NY686r2yDFGnaE0er94fJct3JcCOEbZhlBMA25qjTB1iu?= =?us-ascii?Q?vvpnV4+8WoiLepazsoZuEELOBK8OMNgmwzeLWs1sV3szMzWihPpwboQrSTxj?= =?us-ascii?Q?OX+gPNnNhXeSuPQWLB0h1kauEpqki19+saxF6/FYOs9azbaIIl8aBjPRVoef?= =?us-ascii?B?Zz09?= X-Microsoft-Antispam-Message-Info: NRASBuZ6RLS/Mo8p1ciBpV510f09InsE7wRsVhldDVfuOz6kmHWVBfH3Slc00qqfYcpcjz3KtGj7W59Tbx02Z9KjraVZtJkRDBpaB1FYGpLZROnS7mh3tyG2ES5+jh9q4hX2LPi4uDbw35r985jmrkm8nIsxFO3W0X7aWeITCyZjqg+8VeGzp6/ZDVcPJdVAR1dytZJ5HRJotWnqXrapfHQCHC3VRw6wtXZSgfqnYqXXGBYo8wVBXWpnlOaHzw9LimyFnKFcPJedIn+yoYsaCSwvbG9HChBbsIArkk8yOsZY/twmQ8Yqt+C4umc6T6VtQZvaTLt1dvRAEJkavQ1b59+epWcaxYvmXG5EEzUMpbQ= X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4910; 6:5rbsn6/41aM8ZpEwzGLvu0wa/iqb58xzmAsRFm1YAvNl5OQODryY50ye7KLrRDiGtEmG4KxCBKlTx7oNdlcFrfGI577xBLVD0wv3rIlJ0mewRtLhaUtTbGVgjdSRPCG/iCeh3ATb37XaSK9Bpw5OLoK04REKQiZI151tJDVa+Ius0FOm1CtIBXBKCXRIKvTU85hv0OEPptiXdJcXU3N1tGzIii47dK32TCCfvfOFjqPpth/uXLhjJZreXPqaZP6AWPaVjQk9SPCJBiavuvDvdO7oNjKohXRDEZ5sADYEVLLGrrYqcSIa/0FTt61JB+HmDnkeLSBeq7Lp6gGUk/s+/a6I5oKIZGCfy7qZU2VSlYVHpKi9WSYPzXjZXLjtNj6zZ6tuVWyg1QNjjDnKjq2oc1S39ZwKpyWFQrC0d6k/Bbrva5mpF5HRk9VGYF29mz6TTwEVLiZ6eOxMW2/PwUcfsw==; 5:zESBYxiqXjIzc8AlHX/2/81NqNYJKFFHJfGmQYqmHcLfxrWR5fF+2T8cLg4ZFGxZrMI2m1LfZhrcWSqMLyDTzQx9c6J1L9Kp0xvWRX29XhRSz4ieAVQ8mJN92Qz4hR20jJDGjOsQXxu5bGkc7sSnEpAJmluUek3tkkb3OnvZiVc=; 7:0XNNSJAi0kBRGrOeqhFqJ7VWm7NITBI11iOC+rTeGGwzN4wQPZkR4fMnsbGqbwOFPzVMtd767zmW5nPL+gF/dtWXluwOR0mABfARXINhhQBoHSvNUc2pyjSMfHbN4AR+xSFe5AUaRxXej627aL8S+Y1iNRM7Ng1y0NjFDOrHnAfoxLDYntFh3BaPBbCMoOpQcTqy7N9sUUzJJsZtzqo7/5x0bB3yX17nxAecLGamGHUsHQPe9/ysXHSiT01eJ55Q SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2018 13:02:11.1573 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c3fe9d74-e18a-425a-2606-08d62ac2c5cb X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR07MB4910 Subject: [dpdk-dev] [PATCH v3 25/32] common/cpt: add support for hash X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2018 13:02:19 -0000 From: Srisivasubramanian S Adding microcode interface for supporting verify and authentication Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian S Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_mcode_defines.h | 7 + drivers/common/cpt/cpt_request_mgr.h | 2 + drivers/common/cpt/cpt_ucode.h | 415 ++++++++++++++++++++++++++++++++- 3 files changed, 423 insertions(+), 1 deletion(-) diff --git a/drivers/common/cpt/cpt_mcode_defines.h b/drivers/common/cpt/cpt_mcode_defines.h index 263fc47..60be8b3 100644 --- a/drivers/common/cpt/cpt_mcode_defines.h +++ b/drivers/common/cpt/cpt_mcode_defines.h @@ -18,6 +18,7 @@ #define CPT_MAJOR_OP_HMAC 0x35 #define CPT_MAJOR_OP_ZUC_SNOW3G 0x37 #define CPT_MAJOR_OP_KASUMI 0x38 +#define CPT_MAJOR_OP_MISC 0x01 #define CPT_BYTE_16 16 #define CPT_BYTE_24 24 @@ -341,6 +342,12 @@ typedef struct fc_params { } fc_params_t; +/* + * Parameters for digest + * generate requests + * Only src_iov, op, ctx_buf, mac_buf, prep_req + * meta_buf, auth_data_len are used for digest gen. + */ typedef struct fc_params digest_params_t; /* Cipher Algorithms */ diff --git a/drivers/common/cpt/cpt_request_mgr.h b/drivers/common/cpt/cpt_request_mgr.h index 58a87c3..4d21f46 100644 --- a/drivers/common/cpt/cpt_request_mgr.h +++ b/drivers/common/cpt/cpt_request_mgr.h @@ -74,6 +74,8 @@ cpt_pmd_crypto_operation(struct cpt_instance *instance, if (likely(cpt_op & CPT_OP_CIPHER_MASK)) prep_req = fill_fc_params(op, sess, &mdata, &ret); + else + prep_req = fill_digest_params(op, sess, &mdata, &ret); if (unlikely(!prep_req)) { CPT_LOG_DP_ERR("prep cryto req : op %p, cpt_op 0x%x " diff --git a/drivers/common/cpt/cpt_ucode.h b/drivers/common/cpt/cpt_ucode.h index 05cf95c..4d892f2 100644 --- a/drivers/common/cpt/cpt_ucode.h +++ b/drivers/common/cpt/cpt_ucode.h @@ -4,7 +4,6 @@ #ifndef _CPT_UCODE_H_ #define _CPT_UCODE_H_ - #include #include "cpt_common.h" @@ -460,6 +459,214 @@ fill_sg_comp_from_iov(sg_comp_t *list, } static __rte_always_inline int +cpt_digest_gen_prep(uint32_t flags, + uint64_t d_lens, + digest_params_t *params, + void *op, + void **prep_req) +{ + struct cpt_request_info *req; + uint32_t size, i; + int32_t m_size; + uint16_t data_len, mac_len, key_len; + auth_type_t hash_type; + buf_ptr_t *meta_p; + struct cpt_ctx *ctx; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + void *c_vaddr, *m_vaddr; + uint64_t c_dma, m_dma; + opcode_info_t opcode; + + if (!params || !params->ctx_buf.vaddr) + return ERR_BAD_INPUT_ARG; + + ctx = params->ctx_buf.vaddr; + meta_p = ¶ms->meta_buf; + + if (!meta_p->vaddr || !meta_p->dma_addr) + return ERR_BAD_INPUT_ARG; + + if (meta_p->size < sizeof(struct cpt_request_info)) + return ERR_BAD_INPUT_ARG; + + m_vaddr = meta_p->vaddr; + m_dma = meta_p->dma_addr; + m_size = meta_p->size; + + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + hash_type = ctx->hash_type; + mac_len = ctx->mac_len; + key_len = ctx->auth_key_len; + data_len = AUTH_DLEN(d_lens); + + /*GP op header */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param2 = rte_cpu_to_be_16(((uint16_t)hash_type << 8)); + if (ctx->hmac) { + opcode.s.major = CPT_MAJOR_OP_HMAC | CPT_DMA_MODE; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(key_len); + vq_cmd_w0.s.dlen = + rte_cpu_to_be_16((data_len + ROUNDUP8(key_len))); + } else { + opcode.s.major = CPT_MAJOR_OP_HASH | CPT_DMA_MODE; + vq_cmd_w0.s.param1 = 0; + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(data_len); + } + + opcode.s.minor = 0; + + /* Null auth only case enters the if */ + if (unlikely(!hash_type && !ctx->enc_cipher)) { + opcode.s.major = CPT_MAJOR_OP_MISC; + /* Minor op is passthrough */ + opcode.s.minor = 0x03; + /* Send out completion code only */ + vq_cmd_w0.s.param2 = 0x1; + } + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input gather list + */ + + i = 0; + + if (ctx->hmac) { + uint64_t k_dma = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, auth_key); + /* Key */ + i = fill_sg_comp(gather_comp, i, k_dma, ROUNDUP8(key_len)); + } + + /* input data */ + size = data_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, + 0, &size, NULL, 0); + if (size) { + CPT_LOG_DP_DEBUG("Insufficient dst IOV size, short" + " by %dB", size); + return ERR_BAD_INPUT_ARG; + } + } else { + /* + * Looks like we need to support zero data + * gather ptr in case of hash & hmac + */ + i++; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Gather list + */ + + i = 0; + scatter_comp = (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + if (flags & VALID_MAC_BUF) { + if (params->mac_buf.size < mac_len) + return ERR_BAD_INPUT_ARG; + + size = mac_len; + i = fill_sg_comp_from_buf_min(scatter_comp, i, + ¶ms->mac_buf, &size); + } else { + size = mac_len; + i = fill_sg_comp_from_iov(scatter_comp, i, + params->src_iov, data_len, + &size, NULL, 0); + if (size) { + CPT_LOG_DP_DEBUG("Insufficient dst IOV size, short by" + " %dB", size); + return ERR_BAD_INPUT_ARG; + } + } + + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + +static __rte_always_inline int cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, @@ -2319,6 +2526,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, } else if (fc_type == KASUMI) { ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == HASH_HMAC) { + ret = cpt_digest_gen_prep(flags, d_lens, fc_params, op, + &prep_req); } else { ret = ERR_EIO; } @@ -3206,4 +3416,207 @@ instance_session_cfg(struct rte_crypto_sym_xform *xform, void *sess) return -1; } +static __rte_always_inline void +find_kasumif9_direction_and_length(uint8_t *src, + uint32_t counter_num_bytes, + uint32_t *addr_length_in_bits, + uint8_t *addr_direction) +{ + uint8_t found = 0; + while (!found && counter_num_bytes > 0) { + counter_num_bytes--; + if (src[counter_num_bytes] == 0x00) + continue; + if (src[counter_num_bytes] == 0x80) { + *addr_direction = src[counter_num_bytes - 1] & 0x1; + *addr_length_in_bits = counter_num_bytes * 8 - 1; + found = 1; + } else { + int i = 0; + uint8_t last_byte = src[counter_num_bytes]; + for (i = 0; i < 8 && found == 0; i++) { + if (last_byte & (1 << i)) { + *addr_direction = (last_byte >> (i+1)) + & 0x1; + if (i != 6) + *addr_length_in_bits = + counter_num_bytes * 8 + + (8 - (i + 2)); + else + *addr_length_in_bits = + counter_num_bytes * 8; + found = 1; + } + } + } + } +} + +/* + * This handles all auth only except AES_GMAC + */ +static __rte_always_inline void * +fill_digest_params(struct rte_crypto_op *cop, + struct cpt_sess_misc *sess, + void **mdata_ptr, + int *op_ret) +{ + uint32_t space = 0; + struct rte_crypto_sym_op *sym_op = cop->sym; + void *mdata; + phys_addr_t mphys; + uint64_t *op; + uint32_t auth_range_off; + uint32_t flags = 0; + uint64_t d_offs = 0, d_lens; + void *prep_req = NULL; + struct rte_mbuf *m_src, *m_dst; + uint16_t auth_op = sess->cpt_op & CPT_OP_AUTH_MASK; + uint8_t zsk_flag = sess->zsk_flag; + uint16_t mac_len = sess->mac_len; + fc_params_t params; + char src[SRC_IOV_SIZE]; + uint8_t iv_buf[16]; + memset(¶ms, 0, sizeof(fc_params_t)); + struct cptvf_meta_info *cpt_m_info = + (struct cptvf_meta_info *)(*mdata_ptr); + + m_src = sym_op->m_src; + + /* For just digest lets force mempool alloc */ + mdata = alloc_op_meta(NULL, ¶ms.meta_buf, cpt_m_info->cptvf_op_mlen, + cpt_m_info->cptvf_meta_pool); + if (mdata == NULL) { + CPT_LOG_DP_ERR("Error allocating meta buffer for request"); + *op_ret = -ENOMEM; + return NULL; + } + + mphys = params.meta_buf.dma_addr; + + op = mdata; + op[0] = (uintptr_t)mdata; + op[1] = (uintptr_t)cop; + op[2] = op[3] = 0; /* Used to indicate auth verify */ + space += 4 * sizeof(uint64_t); + + auth_range_off = sym_op->auth.data.offset; + + flags = VALID_MAC_BUF; + params.src_iov = (void *)src; + if (unlikely(zsk_flag)) { + /* + * Since for Zuc, Kasumi, Snow3g offsets are in bits + * we will send pass through even for auth only case, + * let MC handle it + */ + d_offs = auth_range_off; + auth_range_off = 0; + params.auth_iv_buf = rte_crypto_op_ctod_offset(cop, + uint8_t *, sess->auth_iv_offset); + if (zsk_flag == K_F9) { + uint32_t length_in_bits, num_bytes; + uint8_t *src, direction = 0; + uint32_t counter_num_bytes; + + memcpy(iv_buf, rte_pktmbuf_mtod(cop->sym->m_src, + uint8_t *), 8); + /* + * This is kasumi f9, take direction from + * source buffer + */ + length_in_bits = cop->sym->auth.data.length; + num_bytes = (length_in_bits >> 3); + counter_num_bytes = num_bytes; + src = rte_pktmbuf_mtod(cop->sym->m_src, uint8_t *); + find_kasumif9_direction_and_length(src, + counter_num_bytes, + &length_in_bits, + &direction); + length_in_bits -= 64; + cop->sym->auth.data.offset += 64; + d_offs = cop->sym->auth.data.offset; + auth_range_off = d_offs / 8; + cop->sym->auth.data.length = length_in_bits; + + /* Store it at end of auth iv */ + iv_buf[8] = direction; + params.auth_iv_buf = iv_buf; + } + } + + d_lens = sym_op->auth.data.length; + + params.ctx_buf.vaddr = SESS_PRIV(sess); + params.ctx_buf.dma_addr = sess->ctx_dma_addr; + + if (auth_op == CPT_OP_AUTH_GENERATE) { + if (sym_op->auth.digest.data) { + /* + * Digest to be generated + * in separate buffer + */ + params.mac_buf.size = + sess->mac_len; + params.mac_buf.vaddr = + sym_op->auth.digest.data; + params.mac_buf.dma_addr = + sym_op->auth.digest.phys_addr; + } else { + uint32_t off = sym_op->auth.data.offset + + sym_op->auth.data.length; + int32_t dlen, space; + + m_dst = sym_op->m_dst ? + sym_op->m_dst : sym_op->m_src; + dlen = rte_pktmbuf_pkt_len(m_dst); + + space = off + mac_len - dlen; + if (space > 0) + if (!rte_pktmbuf_append(m_dst, space)) { + CPT_LOG_DP_ERR("Failed to extend " + "mbuf by %uB", space); + goto err; + } + + params.mac_buf.vaddr = + rte_pktmbuf_mtod_offset(m_dst, void *, off); + params.mac_buf.dma_addr = + rte_pktmbuf_mtophys_offset(m_dst, off); + params.mac_buf.size = mac_len; + } + } else { + /* Need space for storing generated mac */ + params.mac_buf.vaddr = (uint8_t *)mdata + space; + params.mac_buf.dma_addr = mphys + space; + params.mac_buf.size = mac_len; + space += RTE_ALIGN_CEIL(mac_len, 8); + op[2] = (uintptr_t)params.mac_buf.vaddr; + op[3] = mac_len; + } + + params.meta_buf.vaddr = (uint8_t *)mdata + space; + params.meta_buf.dma_addr = mphys + space; + params.meta_buf.size -= space; + + /* Out of place processing */ + params.src_iov = (void *)src; + + /*Store SG I/O in the api for reuse */ + if (prepare_iov_from_pkt(m_src, params.src_iov, auth_range_off)) { + CPT_LOG_DP_ERR("Prepare src iov failed"); + *op_ret = -1; + goto err; + } + + prep_req = cpt_fc_enc_hmac_prep(flags, d_offs, d_lens, + ¶ms, op, op_ret); + *mdata_ptr = mdata; + return prep_req; +err: + if (unlikely(!prep_req)) + free_op_meta(mdata, cpt_m_info->cptvf_meta_pool); + return NULL; +} + #endif /*_CPT_UCODE_H_ */ -- 2.7.4