From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0051.outbound.protection.outlook.com [104.47.41.51]) by dpdk.org (Postfix) with ESMTP id 0F6DF1B142 for ; Tue, 9 Oct 2018 11:09:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n5122eWj/Rqh9xtYkk8pqS6sQPXUtfPUAMd5yKklTAI=; b=HnC0/1cRJ9OVh0uH2kz4+vV3FZaOax9jGMULaC/On4LJIKcHi40TbBAj8m98YaGJr2so0eVUNgNjqgJk/b/OU/Uqgchmj4QHXzuc/zbxiNDWeh15iSUBTvXoJhW3n4R4C3JOxTVSOhLKayF0bOEa6oTl4uVIfUC6CTFjrhNtdoQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.com (115.113.156.2) by BN7PR07MB4897.namprd07.prod.outlook.com (2603:10b6:406:ef::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.28; Tue, 9 Oct 2018 09:09:40 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Srisivasubramanian S , Jerin Jacob , Narayana Prasad , Ankur Dwivedi , Anoob Joseph , Murthy NSSR , Nithin Dabilpuram , Ragothaman Jayaraman , Tejasree Kondoj , dev@dpdk.org Date: Tue, 9 Oct 2018 14:37:49 +0530 Message-Id: <1539076076-19786-17-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539076076-19786-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> <1539076076-19786-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: PN1PR0101CA0004.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:e::14) To BN7PR07MB4897.namprd07.prod.outlook.com (2603:10b6:406:ef::26) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4a9c2ddf-d020-40c3-e964-08d62dc6f438 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BN7PR07MB4897; X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 3:GdBuJ/q+nXycH/uv+w4pEQR2hJrejGmQOhf933Hsl7j4OmyF0sDkUZZpV+XREr7jRyq+ji1kjFzWsgKDm+cUcnMpQgdLqfc4K+t+M73SjvQMcdSxBu+i6fZvlRRrpD8JJ7jlGiLZWRg6vN/ckYpNZI5Tejmb4h7+0sDyoYNflMrDyrMzgaUyZJ2jWHgXrGIlr8+9NuftGYwPLLAQ/uuV3bsG6OXOkqVT7el1kdEhnhylyI7lkN0YYvtiL6Qc6z2y; 25:jzIC3k7O3UjvEIvn3SZdaQ2N6Wri27DMGSuOF1Q4cieHaCoKQgIYSfE+bEG23kljXWo52LwRGzWjdOKRfSiA3/60xinrLZL1LObi43c+w3Zjkjj+y3F/QCiyBonuHMX39ZY8PwjQq9OwGTEIeqywmTtsRVqndu1OamyNUNOvX5r0NsE90jB3V/aYOsaERjaJNkh0K+zFtNTvF2rQsOg80DSwF5fQDx8edq84K1SkEinlRgUUtKwzyXCojjLQ6X6PJ//bWUCYXJcmIIRFL/ARjB1z21XOR3pczK9f39TMMyX6p7JFk8UqhdM6btnVX7p+u3Y7mGok1ZtWjLltSn4TdA==; 31:QxFfvbyN5iX2hBHrs4+isRTsa4KOhkeB2bb0bz+RjqPFCNce7b2afsZ3Q7+T557D4qWA04M1EkK3kZYtv1mQAAk7cFMfpQS4mkF7ORm3MdW1o5Zn6ZyAD3gJDSGNJ1onBt03S766wjlGx6+1lr5xZ4duU1PugBSCNp2pBMCOk5uIl29nX1CuxQEyqyc7fTK5ei0wZGW5uqHLN9dR60NAAIPv52bgW4ALnz6BYw4Q4vY= X-MS-TrafficTypeDiagnostic: BN7PR07MB4897: X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 20:1snS0C646xIwrNBON9PClQDIAEDYJEZWxUz9iO8Yp5KFxSsJk2hdLfKH+5Dw2rhl9dJ3R41gAMqL7KUwiz1g/YwHsJ/g0Q+yK2+tUjWo6UoPv9L1MxoOGmTsG1X9fjSzIFWDZSYOvKW8tpREn5ij0qa+sR4Qr4DE7bfkf8Wb4q/Yv9qxDCmPSY7+SERmnsxOjYy7JApaqQXlJ/iT2SO30+yihNHwDIiBIfqofw7DFdddlWJxSdcGrvg/yPxva/HP032PuN+juAf1tkMnWiJFf3m5cSg/hjgS6yQw4bCLmgjmf34zz6hI/AhhrXSFFPYqwfu8O5mtPkxSU+ARPr9/VD1iW3kpE4QE5lNAojdFJN2OMLXz7TOEPbUqOmdqhW8zX9BDwb1nJ0ktlpGwnxnOcAuXEV9olTkN6+L/jJrAp9x0iIJy4wfmrd7dnMt7+zI5fGF9Jc5fIJ+fXeQT3as7lT7jJ+FgoZGPkiHWNeQLhVyo5Noj1vupYDsXyuYMAsvjyGeXOhwAABwuOfe5msir7QLAUCORgRjNtRPj/qa237fW3Cww2kIls5YQHER2ebaS08h71eLhiqSvCHRj0BrdNWRTVbWxThHaST7uMma7RMw=; 4:03zkA7xkuLWj2so6CD0RohWnvwzL2iZvHGddafUHEoHtvIpr7h3X6oGzJVtcKpdk/NC0KZXwXcdB7ScodKYh+kwCPygJd6i2nA7STOnNYFHB+Q2h2Pc7/6jUNvr9O7nLY3YwKzil3mzbkISINOJ+/IgHDAzETUkxlaFF2Hs9IKh7IfTqZ80CNzNmFRwYM+vCXsr7zRyNsYuBtUWeyu0ZdlvTcphvfr5HdSlZL9tIH73KbxxyvO0LDYzVoiIqWYNI1pjeBalqincOObY63d6Ofw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(93006095)(10201501046)(3231355)(944501410)(52105095)(149066)(150057)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(20161123558120)(201708071742011)(7699051); SRVR:BN7PR07MB4897; BCL:0; PCL:0; RULEID:; SRVR:BN7PR07MB4897; X-Forefront-PRVS: 08200063E9 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(366004)(376002)(136003)(346002)(39860400002)(199004)(189003)(42882007)(47776003)(106356001)(186003)(5660300001)(26005)(68736007)(16526019)(8936002)(25786009)(6116002)(2906002)(81156014)(50226002)(305945005)(81166006)(7736002)(3846002)(105586002)(478600001)(8676002)(54906003)(110136005)(48376002)(16586007)(14444005)(6486002)(6506007)(50466002)(316002)(53936002)(386003)(51416003)(52116002)(76176011)(11346002)(6512007)(44832011)(446003)(36756003)(55236004)(956004)(2616005)(4744004)(486006)(4326008)(66066001)(97736004)(72206003)(476003)(32563001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN7PR07MB4897; H:ajoseph83.caveonetworks.com.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN7PR07MB4897; 23:Y5uqONh2LAhECPkL7Wpim5ZOs44PVUlUZfCI6qdQy?= =?us-ascii?Q?0EZLzfEycCP9a0YlyvdEFucVjH6Gd4rn9imhPIQ/8I08eC/Gxd3V391J42BF?= =?us-ascii?Q?liKUo2FvynUu2K0Qn4rNynXAABt/izqw/0oWw6Iv8PutkVs9UmItJBHV6NEG?= =?us-ascii?Q?Qfz0quu10cXuTKaAxaffXmV9MLnzjh3cEhodsFwVhsR5cIQktHW7Z9qpXy5s?= =?us-ascii?Q?kfBQ40iWw07RYGrEV6GNVO03qTvYg2v1T9cIoEt8LfyFmNifQO83uydknB9L?= =?us-ascii?Q?SIWSMXWLIPM9LBtJAX4FUymx0YFkayItavyGj+BI7L3AjOS4f7EOxHf1q6TU?= =?us-ascii?Q?U62Ju+vqkJlZaXJNNnrWp3SVMMG2C42k3dTwSMp5urAAoU13Pa0GiltR44xn?= =?us-ascii?Q?ylWoODRSbDRRAcKOAZCLdpLlCUBFWQJZRGcmFNXtIeBYsxjRapsV1YhwhQxE?= =?us-ascii?Q?ydXBIg0VciCvFH10r8wcPbEA5/UtGURoKo4+HigXJqwWAjfPteYW2aGeDOhe?= =?us-ascii?Q?sgxELiU91nmPRlAdF5280wB9PhGqmPOQFDb916EmId0njNZPXcTKcjsEhF+t?= =?us-ascii?Q?61g4HqGyW02Xv/EjTqRSva/TAOzs7x7TEieBAPesUdczzcXlpVlBWP1pmBo1?= =?us-ascii?Q?KARXc771pjy0/k7YWy7GrqgM5mqCJhkFjXzvDHE66z8sOoXvuXtXSFSxQAKq?= =?us-ascii?Q?1Ql4z4o823Jmo4WlHdd7zxJsnrp3BhzJmhs3ZNbJzK3TrRyzIinUTJbeWtIY?= =?us-ascii?Q?C1HMbtiEDKyPBfAKO3jRRUYmCm6Rf8OH8K4OC3gnbfyH4nT7LqyP6H8iB/AK?= =?us-ascii?Q?k8T5138pXARbGDuZeV/7gUt/uTUPdvA6kbqZxkrlw8VfTZTqmzpXvN7BznIF?= =?us-ascii?Q?YJ2w1KiCbWsZur1rttLg+OSVKJUoJrfGVEOmeMZ/eUC146taiR6q+ifPY5mG?= =?us-ascii?Q?lLG6ctAeTw4N9qElQ5KhrdmY4K3bLVAeyR2iMqfoL6FY1KB4DhKc+5jHc9z/?= =?us-ascii?Q?1Rk7HXDwWFxA5hLoqkfrhqNOPYp4ONbBRQ2e5lui+QgAM4YatpgxogoiOsO6?= =?us-ascii?Q?UTGh9ADX4hbGa8MoMN7hvSPtMakeC46O//rSbAiV2bzyOMILdWh9Oi+JYkUh?= =?us-ascii?Q?Vb93d692BKFP56PTjpF2ibl9pCVQhsOspWkVohbB330jJpji//rbPIOgJVZ+?= =?us-ascii?Q?/JbRNQm7t2rOLZRXnRIeWdV4Bl9pnh9v6VAwJ+SH+Jvmnm9Qi+hXoz/oSg/d?= =?us-ascii?Q?WyKcmuS6+BgVZB6aI4DaedeofNWSdrMPbban1sqBQtgJOx16t/mrgYWfiaKy?= =?us-ascii?B?Zz09?= X-Microsoft-Antispam-Message-Info: 7e5L7WbPk5jmHmEgKrQZG2olgBbjieyzZAe8lYhCEM+p4YvIvinPqxIgt/1EyAOj6LeuES/jbUlSCTHzfvkns6cLu3f6rQcSsmNnipPEopFt5HAfkMzsFmsRr1ii6sGMD3D1qAPHrlHNawa2wbEm1j9DWzobsWv7N0ErJ47w8kQOjULn88adpchPU6WT0SMCDs2AticiGWSAjsMXAuJaa+rzS97TUCQWhbg3q+fHmQzM/Q2paJl64N/6ecejxzxSIowO7V8oLBLDup9DeMFXtp6mDpFz0YSsSaIG7VU4rbf62WZWYQ8RNYqpwJWbQoNMNzk7Y+xDvr8vUwzkyEdpsGKqdHMrykszDjZ7+h+25jA= X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 6:NV0vCgJeuS3dK0ewGxG1m7v9MVgESyzyyodBNdQ7fWZ4fsdt92vyc9NkAK9Lm/rWXmgh8nqCZpyLA8ia2ib2cOWKev9kMLAWqeMtYZPtOcj5GlY6k8EszfC+ikgtRjtQ0iPmReeieaYgSRHfn4m4zXPo9s31njUt5PVHJKTt4qbo9PPR5yGkfknIjqJJjhHy+jVoqHUwWREEAsOsQF8YeS6L3J/0PQmuAy4eqBsdpbLQGsiVOaKnNPl7xc7wVmZ5hj1QNVPMfasRCQYYnBeHVUma//VO4S5JOa2Vr4u9FyTdHXc3T0zrHWznGJ32XJTzSol6vtgpEqazslD7WztgO85K7VzmPKH6XJsq2jtTpsdJ8+y7e3cY/8xdJ+SbW+8fItyJ97FZeiu8RgGToW1Xmr0bqRLZKOEkoLIe0WfNN0sXDiHUbzEumZUYe396fiwqM9pAcnC6z2+escPDoZckyw==; 5:6OUA/hHH67F/sNmCsukujpC2yDX+zJvVjs0YnSHZrWAE8+ghxWF9EV9YzbFXDCNN/gZUiBHfv1DqZ3MOvtc0H1JEDM6hm6XyNm75eb9asrRgrfatzxZqGOsGtqh3R8RHA4qrEYVe2PgRvJy64naOtoOB9/J/vInm2JlmLhcCM0I=; 7:hHIa2qGBC4+/9BxdmOJRxSAp4pJIH3/JlCEZG9LWCjDDpJCioRPxl/swafHE7H1QJmsHfgiw6yGe+si83qs9hhqNgOoeNYKF31jvqGdBc4JrZfsi/rRELEsrK8WGD41dtiOevYHAEujqqelLidJpTYDI36xZHzZKchn+FZ9d8E+SokrDfeTXiyXJi5UFLcZRWBuhPwITHGPsiCnRaB/CbwzZXxsMKuYqTL7kfNPMlv/4wLn9+i9+2ti3wdq/RJtK SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2018 09:09:40.7823 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4a9c2ddf-d020-40c3-e964-08d62dc6f438 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR07MB4897 Subject: [dpdk-dev] [PATCH v4 16/23] common/cpt: support zuc and snow3g X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2018 09:09:47 -0000 From: Srisivasubramanian S Adding microcode interface for supporting ZUC and SNOW3G. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian S Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_ucode.h | 596 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 596 insertions(+) diff --git a/drivers/common/cpt/cpt_ucode.h b/drivers/common/cpt/cpt_ucode.h index 9085667..5d7743c 100644 --- a/drivers/common/cpt/cpt_ucode.h +++ b/drivers/common/cpt/cpt_ucode.h @@ -1226,6 +1226,596 @@ cpt_dec_hmac_prep(uint32_t flags, return 0; } +static __rte_always_inline int +cpt_zuc_snow3g_enc_prep(uint32_t req_flags, + uint64_t d_offs, + uint64_t d_lens, + fc_params_t *params, + void *op, + void **prep_req) +{ + uint32_t size; + int32_t inputlen, outputlen; + struct cpt_ctx *cpt_ctx; + uint32_t mac_len = 0; + uint8_t snow3g, j; + struct cpt_request_info *req; + buf_ptr_t *buf_p; + uint32_t encr_offset = 0, auth_offset = 0; + uint32_t encr_data_len = 0, auth_data_len = 0; + int flags, iv_len = 16, m_size; + void *m_vaddr, *c_vaddr; + uint64_t m_dma, c_dma, offset_ctrl; + uint64_t *offset_vaddr, offset_dma; + uint32_t *iv_s, iv[4]; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + opcode_info_t opcode; + + buf_p = ¶ms->meta_buf; + m_vaddr = buf_p->vaddr; + m_dma = buf_p->dma_addr; + m_size = buf_p->size; + + cpt_ctx = params->ctx_buf.vaddr; + flags = cpt_ctx->zsk_flags; + mac_len = cpt_ctx->mac_len; + snow3g = cpt_ctx->snow3g; + + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Reserve memory for cpt request info */ + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + opcode.s.major = CPT_MAJOR_OP_ZUC_SNOW3G; + + /* indicates CPTR ctx, operation type, KEY & IV mode from DPTR */ + opcode.s.minor = ((1 << 6) | (snow3g << 5) | (0 << 4) | + (0 << 3) | (flags & 0x7)); + + if (flags == 0x1) { + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + auth_data_len = AUTH_DLEN(d_lens); + + /* EIA3 or UIA2 */ + auth_offset = AUTH_OFFSET(d_offs); + auth_offset = auth_offset / 8; + + /* consider iv len */ + auth_offset += iv_len; + + inputlen = auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8); + outputlen = mac_len; + + offset_ctrl = rte_cpu_to_be_64((uint64_t)auth_offset); + + } else { + /* EEA3 or UEA2 */ + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + encr_data_len = ENCR_DLEN(d_lens); + + encr_offset = ENCR_OFFSET(d_offs); + encr_offset = encr_offset / 8; + /* consider iv len */ + encr_offset += iv_len; + + inputlen = encr_offset + (RTE_ALIGN(encr_data_len, 8) / 8); + outputlen = inputlen; + + /* iv offset is 0 */ + offset_ctrl = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + } + + /* IV */ + iv_s = (flags == 0x1) ? params->auth_iv_buf : + params->iv_buf; + + if (snow3g) { + /* + * DPDK seems to provide it in form of IV3 IV2 IV1 IV0 + * and BigEndian, MC needs it as IV0 IV1 IV2 IV3 + */ + + for (j = 0; j < 4; j++) + iv[j] = iv_s[3 - j]; + } else { + /* ZUC doesn't need a swap */ + for (j = 0; j < 4; j++) + iv[j] = iv_s[j]; + } + + /* + * GP op header, lengths are expected in bits. + */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(encr_data_len); + vq_cmd_w0.s.param2 = rte_cpu_to_be_16(auth_data_len); + + /* + * In 83XX since we have a limitation of + * IV & Offset control word not part of instruction + * and need to be part of Data Buffer, we check if + * head room is there and then only do the Direct mode processing + */ + if (likely((req_flags & SINGLE_BUF_INPLACE) && + (req_flags & SINGLE_BUF_HEADTAILROOM))) { + void *dm_vaddr = params->bufs[0].vaddr; + uint64_t dm_dma_addr = params->bufs[0].dma_addr; + /* + * This flag indicates that there is 24 bytes head room and + * 8 bytes tail room available, so that we get to do + * DIRECT MODE with limitation + */ + + offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - + OFF_CTRL_LEN - iv_len); + offset_dma = dm_dma_addr - OFF_CTRL_LEN - iv_len; + + /* DPTR */ + req->ist.ei1 = offset_dma; + /* RPTR should just exclude offset control word */ + req->ist.ei2 = dm_dma_addr - iv_len; + req->alternate_caddr = (uint64_t *)((uint8_t *)dm_vaddr + + outputlen - iv_len); + + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(inputlen + OFF_CTRL_LEN); + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + if (likely(iv_len)) { + uint32_t *iv_d = (uint32_t *)((uint8_t *)offset_vaddr + + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + } + + *offset_vaddr = offset_ctrl; + } else { + uint32_t i, g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t *iv_d; + + /* save space for iv */ + offset_vaddr = m_vaddr; + offset_dma = m_dma; + + m_vaddr = (uint8_t *)m_vaddr + OFF_CTRL_LEN + iv_len; + m_dma += OFF_CTRL_LEN + iv_len; + m_size -= OFF_CTRL_LEN + iv_len; + + opcode.s.major |= CPT_DMA_MODE; + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input Gather List + */ + i = 0; + + /* Offset control word followed by iv */ + + i = fill_sg_comp(gather_comp, i, offset_dma, + OFF_CTRL_LEN + iv_len); + + /* iv offset is 0 */ + *offset_vaddr = offset_ctrl; + + iv_d = (uint32_t *)((uint8_t *)offset_vaddr + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + + /* input data */ + size = inputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, + params->src_iov, + 0, &size, NULL, 0); + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Scatter List + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + if (flags == 0x1) { + /* IV in SLIST only for EEA3 & UEA2 */ + iv_len = 0; + } + + if (iv_len) { + i = fill_sg_comp(scatter_comp, i, + offset_dma + OFF_CTRL_LEN, iv_len); + } + + /* Add output data */ + if (req_flags & VALID_MAC_BUF) { + size = outputlen - iv_len - mac_len; + if (size) { + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + + /* mac data */ + if (mac_len) { + i = fill_sg_comp_from_buf(scatter_comp, i, + ¶ms->mac_buf); + } + } else { + /* Output including mac */ + size = outputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + } + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + } + + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + vq_cmd_w3.s.grp = 0; + vq_cmd_w3.s.cptr = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, zs_ctx); + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + +static __rte_always_inline int +cpt_zuc_snow3g_dec_prep(uint32_t req_flags, + uint64_t d_offs, + uint64_t d_lens, + fc_params_t *params, + void *op, + void **prep_req) +{ + uint32_t size; + int32_t inputlen = 0, outputlen; + struct cpt_ctx *cpt_ctx; + uint8_t snow3g, iv_len = 16; + struct cpt_request_info *req; + buf_ptr_t *buf_p; + uint32_t encr_offset; + uint32_t encr_data_len; + int flags, m_size; + void *m_vaddr, *c_vaddr; + uint64_t m_dma, c_dma; + uint64_t *offset_vaddr, offset_dma; + uint32_t *iv_s, iv[4], j; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + opcode_info_t opcode; + + buf_p = ¶ms->meta_buf; + m_vaddr = buf_p->vaddr; + m_dma = buf_p->dma_addr; + m_size = buf_p->size; + + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + encr_offset = ENCR_OFFSET(d_offs) / 8; + encr_data_len = ENCR_DLEN(d_lens); + + cpt_ctx = params->ctx_buf.vaddr; + flags = cpt_ctx->zsk_flags; + snow3g = cpt_ctx->snow3g; + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Reserve memory for cpt request info */ + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + opcode.s.major = CPT_MAJOR_OP_ZUC_SNOW3G; + + /* indicates CPTR ctx, operation type, KEY & IV mode from DPTR */ + opcode.s.minor = ((1 << 6) | (snow3g << 5) | (0 << 4) | + (0 << 3) | (flags & 0x7)); + + /* consider iv len */ + encr_offset += iv_len; + + inputlen = encr_offset + + (RTE_ALIGN(encr_data_len, 8) / 8); + outputlen = inputlen; + + /* IV */ + iv_s = params->iv_buf; + if (snow3g) { + /* + * DPDK seems to provide it in form of IV3 IV2 IV1 IV0 + * and BigEndian, MC needs it as IV0 IV1 IV2 IV3 + */ + + for (j = 0; j < 4; j++) + iv[j] = iv_s[3 - j]; + } else { + /* ZUC doesn't need a swap */ + for (j = 0; j < 4; j++) + iv[j] = iv_s[j]; + } + + /* + * GP op header, lengths are expected in bits. + */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(encr_data_len); + + /* + * In 83XX since we have a limitation of + * IV & Offset control word not part of instruction + * and need to be part of Data Buffer, we check if + * head room is there and then only do the Direct mode processing + */ + if (likely((req_flags & SINGLE_BUF_INPLACE) && + (req_flags & SINGLE_BUF_HEADTAILROOM))) { + void *dm_vaddr = params->bufs[0].vaddr; + uint64_t dm_dma_addr = params->bufs[0].dma_addr; + /* + * This flag indicates that there is 24 bytes head room and + * 8 bytes tail room available, so that we get to do + * DIRECT MODE with limitation + */ + + offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - + OFF_CTRL_LEN - iv_len); + offset_dma = dm_dma_addr - OFF_CTRL_LEN - iv_len; + + /* DPTR */ + req->ist.ei1 = offset_dma; + /* RPTR should just exclude offset control word */ + req->ist.ei2 = dm_dma_addr - iv_len; + req->alternate_caddr = (uint64_t *)((uint8_t *)dm_vaddr + + outputlen - iv_len); + + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(inputlen + OFF_CTRL_LEN); + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + if (likely(iv_len)) { + uint32_t *iv_d = (uint32_t *)((uint8_t *)offset_vaddr + + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + } + + /* iv offset is 0 */ + *offset_vaddr = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + } else { + uint32_t i, g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t *iv_d; + + /* save space for offset and iv... */ + offset_vaddr = m_vaddr; + offset_dma = m_dma; + + m_vaddr = (uint8_t *)m_vaddr + OFF_CTRL_LEN + iv_len; + m_dma += OFF_CTRL_LEN + iv_len; + m_size -= OFF_CTRL_LEN + iv_len; + + opcode.s.major |= CPT_DMA_MODE; + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input Gather List + */ + i = 0; + + /* Offset control word */ + + /* iv offset is 0 */ + *offset_vaddr = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + + i = fill_sg_comp(gather_comp, i, offset_dma, + OFF_CTRL_LEN + iv_len); + + iv_d = (uint32_t *)((uint8_t *)offset_vaddr + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + + /* Add input data */ + size = inputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, + params->src_iov, + 0, &size, NULL, 0); + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Scatter List + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + /* IV */ + i = fill_sg_comp(scatter_comp, i, + offset_dma + OFF_CTRL_LEN, + iv_len); + + /* Add output data */ + size = outputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + } + + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + vq_cmd_w3.s.grp = 0; + vq_cmd_w3.s.cptr = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, zs_ctx); + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + static __rte_always_inline void * cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, @@ -1243,6 +1833,9 @@ cpt_fc_dec_hmac_prep(uint32_t flags, if (likely(fc_type == FC_GEN)) { ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == ZUC_SNOW3G) { + ret = cpt_zuc_snow3g_dec_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { /* * For AUTH_ONLY case, @@ -1273,6 +1866,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, if (likely(fc_type == FC_GEN)) { ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == ZUC_SNOW3G) { + ret = cpt_zuc_snow3g_enc_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { ret = ERR_EIO; } -- 2.7.4