From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0040.outbound.protection.outlook.com [104.47.33.40]) by dpdk.org (Postfix) with ESMTP id 803C01B1C3 for ; Fri, 5 Oct 2018 15:02:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n5122eWj/Rqh9xtYkk8pqS6sQPXUtfPUAMd5yKklTAI=; b=HOOrgsqXd1IkVsRvE82KRZcbFbSRN7MKSX0N5JmfOw5PQbdyNUbCi06sxAE4lAB+bOYQE1gsEsv/nIfDlxs1DslFuQysEphseJeUvPFOd68ie5oWWkHPyK5No8bMIwkdRYFEenEmOCy1DBmK9V0mlHFItLMwwcXd3sYRzRahc8U= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.com (115.113.156.2) by SN6PR07MB4910.namprd07.prod.outlook.com (2603:10b6:805:39::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.21; Fri, 5 Oct 2018 13:02:02 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Srisivasubramanian S , Jerin Jacob , Narayana Prasad , dev@dpdk.org, Ankur Dwivedi , Anoob Joseph , Murthy NSSR , Nithin Dabilpuram , Ragothaman Jayaraman , Tejasree Kondoj Date: Fri, 5 Oct 2018 18:29:14 +0530 Message-Id: <1538744363-30340-24-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1536033560-21541-1-git-send-email-ajoseph@caviumnetworks.com> <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: MA1PR0101CA0036.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:22::22) To SN6PR07MB4910.namprd07.prod.outlook.com (2603:10b6:805:39::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: da1f21ba-0901-42b6-c463-08d62ac2c083 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:SN6PR07MB4910; X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4910; 3:OrlajzI+qcGD8bFGZaCHfoEywJ1nkdVlLwzIt7bgNtGIdW9bHMvHLYI6x6X4YbaFMoAoN7RxuIODz31nuvdHKzPh3zIKO7PcPgrKRnWnUsYe+v1ysVA4bwyJFF3USAmxx+VUJmyB55koKzx6HsE4nhHOmvgFhPFDmV4kBiLbSzy+LI/brOLG8wyzOimKyF7Ew3dsMhfpo1rzFmDwVuEAQgqHxTjlNgn0wD926Yu50wbee+icBwJyv+IaA3pqN8OX; 25:nsdYjG2sT92oooDYdSsHf8YYIdJfGr93mSQrABwwmWipZcLZmXBPo0cr10cSOmmaCZfoghykCvv3f7qy1N8DmksuOupv1/QjjqhuDtfBkBdE3rpbuWJo6uqGUqeNmx7NmQm0FBzSure5Z3spUjPKDxiscF+N/bbf1FIoaz6ZMXHEKjM6Dj4uMF2NuVmAbMtMr6jECoHaxeY2fjZ0wXYkVgEDXDVUKOwtKbLAODUjUh0HJwVbOsUK7ZnX6PBoaYKN4PC/ruDUMLcQhftV9X2OJPTJ++sU6M+srabmFIM59HrU4O7Q24oOiNaxiG6DlC94/pIpl5swBiqNM+ez82f8QA==; 31:TCDXw1JoVyATpf7yQF+HYRyoINOev5uHMeqpiNt0lLF18qNf/pSxpO2PvXj1LBlAcNYqXDdexrIxRattWit5b9Z1nj96TZ3CEz8+zfJHqXtJznPdo0lURdFs9cOJ5lubWGf4WbwTmmZYMhE/+KktyBZzPc5pZ88z8i7JKGYVqYB9Apquxjmce31GZSFxOpYqtXppTZi3nRyEp9mQ69oE6InxpNl2a5phKl82ZN0kpDI= X-MS-TrafficTypeDiagnostic: SN6PR07MB4910: X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4910; 20:4LlxNnaDMdev54S6pOj3gXFRJXZ6wBil7bTmVmbChS9m4fh+3L2WYFqq5sven0lWPw+njWagq5nl0yYEjlM/k6pT3vPxUTooWkQ1yE+E+t7ERzMh6GYHHEVoHouyhQdx737Rn+Djxwj2TQOyrYQQ52H8r1xSEpG6EgVI+FAS+/su79MOSlhqdYUBpQJLd6647cq0OBQj7ZI0lxzDSSpkqCpIgZRaCwBSzqhoNd6HflijtbYN8TNBx0nFE7joHGqBAyaXpXNDqB0Vkk/Q9vew2W/TB4ZkfMlJvX3udbMXXeGEx+J76XEEwET3lNY0uhgYAKtOxnU1av2w/75tEPTxjtwcm+2KfBcFsm2dO3mrBzk31kHIbrTcfRCQCIgONgjANbfG37d55CmKuc5q/nbGjtCs43CaCKj5S3SB6BP1snlR1et9tVZ7Zchb4Ec6bwc9pvcR8LtXS0Xs0nqmlb6ghbNNMwA1LOKTAiHCV8wk1i2zPobBYhJfRSLFRNqSjy0rqDmYn878fg3vS2OOjsseGT0hOEy7MetmsdkHHFOoJnHqCLL4YnNJkn/76YSdHCU+Fakne5X4Pij/EXuZVWLtAJqmd3KrM8fIWg/5c/2HwWc=; 4:4VxRdGx5XFX6dwhS0q/CZfzo61ggZnmhAIy3ak6xWzWgPKur2LT9tRHvwUHaCQ8RBtU+DG8xuKSMxl6SIkMBdyJtuLhvEtENaJA90tqI/FL9tr2j1zexnBu1Ush1J2fWjOhYqzxVMHYXNPAdLNdGS5hShPPtp3kol2ampPzkrsf3rZ/tCzv8y1Q3oPEPjH+J62U/8aQYCd92mj6Tw+2PB+S4FjZGRzKQJ1Wf1cCFbNMgZRmzPsoSO0HS+Gi/kfUIgPsPnpGCSG6IfDiwiS48zw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(3231355)(944501410)(52105095)(10201501046)(93006095)(149066)(150057)(6041310)(20161123564045)(20161123558120)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051); SRVR:SN6PR07MB4910; BCL:0; PCL:0; RULEID:; SRVR:SN6PR07MB4910; X-Forefront-PRVS: 0816F1D86E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(39860400002)(346002)(136003)(396003)(366004)(199004)(189003)(51416003)(76176011)(11346002)(50226002)(25786009)(8936002)(48376002)(305945005)(186003)(16526019)(110136005)(6486002)(3846002)(446003)(52116002)(6116002)(81156014)(8676002)(97736004)(5660300001)(50466002)(4326008)(956004)(42882007)(2616005)(68736007)(478600001)(54906003)(7736002)(81166006)(6666003)(16586007)(107886003)(26005)(55236004)(66066001)(386003)(47776003)(6512007)(72206003)(44832011)(476003)(316002)(14444005)(36756003)(4744004)(105586002)(106356001)(486006)(6506007)(2906002)(53936002)(32563001); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR07MB4910; H:ajoseph83.caveonetworks.com.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN6PR07MB4910; 23:q9PJbh6t0pnPaRUeuHYuPcGHcLG8Xe1oMMN/6kDs8?= =?us-ascii?Q?nincG2rTYBzyZfcjn5qkMVvb0HR2FuJ1Ap/Z8S535Y9Q0O+zBGtIje2/h2KE?= =?us-ascii?Q?duRRTJ46cf7Wy1ocEB+iVgJtwsyLgGlPXMPrf5lLUqxdtBY7vjRtQvTDKF01?= =?us-ascii?Q?t6loEzUhtg9Ezu7wUh7vf3q0fxmERXhRKqqX1vhqYZWtQXUDoG4ee4lxIxn/?= =?us-ascii?Q?81W3nrWQZk960yQraSg/c/fTFEVobi2iPetxgXmo/LPmzCqYeR8b1xZ9VsFp?= =?us-ascii?Q?SrwvPZgAcQYGhNFGrSxmZILfSpkz9tfnnlhaNL9GAshSc9iOvYhTrkbgZMw9?= =?us-ascii?Q?XPrH/S5N1gM3sglbdZFX2OyKu/F61jErA1OrhGXOOnfY6BMVGFZ1kF1lh9DY?= =?us-ascii?Q?ZB7ldXFbKSx5k2zvQwEtBZBqTknHFFLrRWe18qE8Nzj1EOJd3UbDsY6nNNPV?= =?us-ascii?Q?6DUmCLcUPtuD4HdIegl0UgVm+rlFCtlZtw0mKqCmSNEnaAAh7qvnlm1ZnZnb?= =?us-ascii?Q?Vt0usZJ4REU6hJ97/r/SZRMI0b/oZX0O0LW+LcpWPOi9NXEDLvPehL9vJN1j?= =?us-ascii?Q?5Z8UY952TyZWiU1NGtvJgkOUKhCauPfUMQ5gnR9CDvTES9apcbyZKobmlsdG?= =?us-ascii?Q?XB7qNauNEClm76l1eZ//iV8HGntcpDN7JfUoRo1SpC909miVUk2LXalN15UQ?= =?us-ascii?Q?gHq5phEonuhXlHT4XpAGhzLAPvG4p65yfGhr/CI2uxLR1ZxoykZKmygQDtH/?= =?us-ascii?Q?j68vo7cjbleCEA0wNjsMH3wC5Tfz5kNRoYH0rN/DnHEkyxXQAJ0m5XRditRc?= =?us-ascii?Q?PF4ny8bfmz5bxuTb5ROxYm3gKzYnqX40dxN6ExcB8Ph7tpuF/6f/myMWtfLh?= =?us-ascii?Q?FkZd7Mbd5IuVbGF0YMU++R1k3l9T+nG3RsegQWofwu71DHUlYW7Y2uxpH9Tu?= =?us-ascii?Q?LX0Di1ZN6SJUEbLE97R1we/qgklqVo4xnIZ2pdWbzi90glEcgaq86wTOGq5n?= =?us-ascii?Q?E2khOMFqcVK/U9E076CaopbRxgR444vZE3ZC4wQ0ZD4Chll4GGp+UR0DWuIh?= =?us-ascii?Q?nb1SpESyn89nze4Y7jgXww4b7batBYwqbZOGh7TW/SBF3MoS5RTNfAr7dV7C?= =?us-ascii?Q?ZZrWVFhg/Czgh6uJgwqGiLUg1jo5tLl9FqJk2rFCoKvpTLO3ubNQhqpEU4UH?= =?us-ascii?Q?pcykElxnypkELQKEqU9gSP5VD1USFz40sjHPMVvsPM5KDXE9lIY/hL3Kzrlp?= =?us-ascii?Q?ViLSqxAdOTHYEev1elpQsaMzQigTUOv3cDc3vtvBOwxPiwxUhTEsQthGpS7Q?= =?us-ascii?Q?U8MZKd8dJVLldgh/cycrmC5bdZW3hYIk1J3sP1glyHx?= X-Microsoft-Antispam-Message-Info: Wgrubecvpyqfa5TwXOKPwpIBQIgP/XFKbuww//pFnKSe7zm1JUYY6GRHF3Df/zr5nqNgR0uUztEKPF8aC11vn2ST90xItqSFLYRhiZijccqRPxIvlcFw64ocWrzrS5HPidmOCTFSVLeL6yDpAyoc0kWb1KCiKRb1MkcBdvIR3+01BZ92ku7M/wToPzLWYohKKtqtoY3em+dkzSbI9AmvHlOkdDYZewbx9GB6EX2iBfgGIIONl5EGkfik5z7Obmxok/YteZwjDBdyDBNrQSPYW/PotLrSL8fU7tPl7h+8QvxIbt5b3U+vwkibDhfJEvRZssdRtWE4JcrX5rXnr7kmAv39MsWgJRf/NpcCZ8u0CBs= X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4910; 6:sDa1eoMKLd+OpsDPIiW8IOppSngzW9ytI5qhkgmm/LeSj9EDPmiDn+ba4NaQ07PYceBfqu5n8mSwQDtWhZ8BnNAcrCbfjs+7brPXAmVF8bhiZFj6kJLfToAyENx+yn57PBwuKDXwBvR5khZt3WkGnlowiMnJ0ueXZrjvl8S1G7LbHDE2A+ma59iQpr20Y+V/Cjmk2OVEDGt3QrseIMlN6ci4OU6YbQlI37OwhzzOD5M0rujIeYtPnP2h+NsC8O/l3LrT08GvftRNQjebPBxMeDoaLcHRb1G90flCqHdzN1HLb7aJYIjWNcMcwZipqjPIEWLfkQaveOUG+POErPbXv16VTuK3m1A98ZMvGTtrWy2/VdgF+eEUmAxhbRspHQw0u+mj2lBbkAopbkbg/xG5ItCbwWY6D6Ivo1df3av/jJFvJv3n3cm58ZZfbrd/Os2NsG/x3qje3lMzuo48dP74wA==; 5:yFF8kJ6ZHvuCFN3ZYeRINIRS8EVu3jOAF94UwUDBOUTMCW8ap6HLc7a9fMz13N4/UR09iXApmVXL/adX7Z4xP2hyrloKjp2aQUl36rNoHA35y6fcSz2c/Jx+qnHVnhhnF+pPIcB3lHbJJJUKqmDs3yXcAnXpb3hMPNzvki9zysQ=; 7:GGmw4lqWQ/ErpJVNYkAdkyX062YA8X11KvW50QdRnqvd6sw+T044NVjVBh/ChSa/FUKoS/nCVdH8tznBy/biZoZ23g+6Rccfqsc3oxiaF1ZZHUtYUddrkKmrteEzZAIlD9fbRxmg7WFaT4meBV/pX1Wi8lz5RrM+bUXKWX9BMGO+Oi2L4YzsdyiW9c99K1kJeVznuDMjuRjngT+Nlh9Nl9jHGDAzUvUfVfV/oA5AdKaxEql8OhJNa3nD7kTAen8W SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2018 13:02:02.2118 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: da1f21ba-0901-42b6-c463-08d62ac2c083 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR07MB4910 Subject: [dpdk-dev] [PATCH v3 23/32] common/cpt: add support for zuc and snow3g X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2018 13:02:08 -0000 From: Srisivasubramanian S Adding microcode interface for supporting ZUC and SNOW3G. Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian S Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_ucode.h | 596 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 596 insertions(+) diff --git a/drivers/common/cpt/cpt_ucode.h b/drivers/common/cpt/cpt_ucode.h index 9085667..5d7743c 100644 --- a/drivers/common/cpt/cpt_ucode.h +++ b/drivers/common/cpt/cpt_ucode.h @@ -1226,6 +1226,596 @@ cpt_dec_hmac_prep(uint32_t flags, return 0; } +static __rte_always_inline int +cpt_zuc_snow3g_enc_prep(uint32_t req_flags, + uint64_t d_offs, + uint64_t d_lens, + fc_params_t *params, + void *op, + void **prep_req) +{ + uint32_t size; + int32_t inputlen, outputlen; + struct cpt_ctx *cpt_ctx; + uint32_t mac_len = 0; + uint8_t snow3g, j; + struct cpt_request_info *req; + buf_ptr_t *buf_p; + uint32_t encr_offset = 0, auth_offset = 0; + uint32_t encr_data_len = 0, auth_data_len = 0; + int flags, iv_len = 16, m_size; + void *m_vaddr, *c_vaddr; + uint64_t m_dma, c_dma, offset_ctrl; + uint64_t *offset_vaddr, offset_dma; + uint32_t *iv_s, iv[4]; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + opcode_info_t opcode; + + buf_p = ¶ms->meta_buf; + m_vaddr = buf_p->vaddr; + m_dma = buf_p->dma_addr; + m_size = buf_p->size; + + cpt_ctx = params->ctx_buf.vaddr; + flags = cpt_ctx->zsk_flags; + mac_len = cpt_ctx->mac_len; + snow3g = cpt_ctx->snow3g; + + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Reserve memory for cpt request info */ + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + opcode.s.major = CPT_MAJOR_OP_ZUC_SNOW3G; + + /* indicates CPTR ctx, operation type, KEY & IV mode from DPTR */ + opcode.s.minor = ((1 << 6) | (snow3g << 5) | (0 << 4) | + (0 << 3) | (flags & 0x7)); + + if (flags == 0x1) { + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + auth_data_len = AUTH_DLEN(d_lens); + + /* EIA3 or UIA2 */ + auth_offset = AUTH_OFFSET(d_offs); + auth_offset = auth_offset / 8; + + /* consider iv len */ + auth_offset += iv_len; + + inputlen = auth_offset + (RTE_ALIGN(auth_data_len, 8) / 8); + outputlen = mac_len; + + offset_ctrl = rte_cpu_to_be_64((uint64_t)auth_offset); + + } else { + /* EEA3 or UEA2 */ + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + encr_data_len = ENCR_DLEN(d_lens); + + encr_offset = ENCR_OFFSET(d_offs); + encr_offset = encr_offset / 8; + /* consider iv len */ + encr_offset += iv_len; + + inputlen = encr_offset + (RTE_ALIGN(encr_data_len, 8) / 8); + outputlen = inputlen; + + /* iv offset is 0 */ + offset_ctrl = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + } + + /* IV */ + iv_s = (flags == 0x1) ? params->auth_iv_buf : + params->iv_buf; + + if (snow3g) { + /* + * DPDK seems to provide it in form of IV3 IV2 IV1 IV0 + * and BigEndian, MC needs it as IV0 IV1 IV2 IV3 + */ + + for (j = 0; j < 4; j++) + iv[j] = iv_s[3 - j]; + } else { + /* ZUC doesn't need a swap */ + for (j = 0; j < 4; j++) + iv[j] = iv_s[j]; + } + + /* + * GP op header, lengths are expected in bits. + */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(encr_data_len); + vq_cmd_w0.s.param2 = rte_cpu_to_be_16(auth_data_len); + + /* + * In 83XX since we have a limitation of + * IV & Offset control word not part of instruction + * and need to be part of Data Buffer, we check if + * head room is there and then only do the Direct mode processing + */ + if (likely((req_flags & SINGLE_BUF_INPLACE) && + (req_flags & SINGLE_BUF_HEADTAILROOM))) { + void *dm_vaddr = params->bufs[0].vaddr; + uint64_t dm_dma_addr = params->bufs[0].dma_addr; + /* + * This flag indicates that there is 24 bytes head room and + * 8 bytes tail room available, so that we get to do + * DIRECT MODE with limitation + */ + + offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - + OFF_CTRL_LEN - iv_len); + offset_dma = dm_dma_addr - OFF_CTRL_LEN - iv_len; + + /* DPTR */ + req->ist.ei1 = offset_dma; + /* RPTR should just exclude offset control word */ + req->ist.ei2 = dm_dma_addr - iv_len; + req->alternate_caddr = (uint64_t *)((uint8_t *)dm_vaddr + + outputlen - iv_len); + + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(inputlen + OFF_CTRL_LEN); + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + if (likely(iv_len)) { + uint32_t *iv_d = (uint32_t *)((uint8_t *)offset_vaddr + + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + } + + *offset_vaddr = offset_ctrl; + } else { + uint32_t i, g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t *iv_d; + + /* save space for iv */ + offset_vaddr = m_vaddr; + offset_dma = m_dma; + + m_vaddr = (uint8_t *)m_vaddr + OFF_CTRL_LEN + iv_len; + m_dma += OFF_CTRL_LEN + iv_len; + m_size -= OFF_CTRL_LEN + iv_len; + + opcode.s.major |= CPT_DMA_MODE; + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input Gather List + */ + i = 0; + + /* Offset control word followed by iv */ + + i = fill_sg_comp(gather_comp, i, offset_dma, + OFF_CTRL_LEN + iv_len); + + /* iv offset is 0 */ + *offset_vaddr = offset_ctrl; + + iv_d = (uint32_t *)((uint8_t *)offset_vaddr + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + + /* input data */ + size = inputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, + params->src_iov, + 0, &size, NULL, 0); + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Scatter List + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + if (flags == 0x1) { + /* IV in SLIST only for EEA3 & UEA2 */ + iv_len = 0; + } + + if (iv_len) { + i = fill_sg_comp(scatter_comp, i, + offset_dma + OFF_CTRL_LEN, iv_len); + } + + /* Add output data */ + if (req_flags & VALID_MAC_BUF) { + size = outputlen - iv_len - mac_len; + if (size) { + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + + /* mac data */ + if (mac_len) { + i = fill_sg_comp_from_buf(scatter_comp, i, + ¶ms->mac_buf); + } + } else { + /* Output including mac */ + size = outputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + } + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + } + + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + vq_cmd_w3.s.grp = 0; + vq_cmd_w3.s.cptr = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, zs_ctx); + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + +static __rte_always_inline int +cpt_zuc_snow3g_dec_prep(uint32_t req_flags, + uint64_t d_offs, + uint64_t d_lens, + fc_params_t *params, + void *op, + void **prep_req) +{ + uint32_t size; + int32_t inputlen = 0, outputlen; + struct cpt_ctx *cpt_ctx; + uint8_t snow3g, iv_len = 16; + struct cpt_request_info *req; + buf_ptr_t *buf_p; + uint32_t encr_offset; + uint32_t encr_data_len; + int flags, m_size; + void *m_vaddr, *c_vaddr; + uint64_t m_dma, c_dma; + uint64_t *offset_vaddr, offset_dma; + uint32_t *iv_s, iv[4], j; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + opcode_info_t opcode; + + buf_p = ¶ms->meta_buf; + m_vaddr = buf_p->vaddr; + m_dma = buf_p->dma_addr; + m_size = buf_p->size; + + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + encr_offset = ENCR_OFFSET(d_offs) / 8; + encr_data_len = ENCR_DLEN(d_lens); + + cpt_ctx = params->ctx_buf.vaddr; + flags = cpt_ctx->zsk_flags; + snow3g = cpt_ctx->snow3g; + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Reserve memory for cpt request info */ + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + opcode.s.major = CPT_MAJOR_OP_ZUC_SNOW3G; + + /* indicates CPTR ctx, operation type, KEY & IV mode from DPTR */ + opcode.s.minor = ((1 << 6) | (snow3g << 5) | (0 << 4) | + (0 << 3) | (flags & 0x7)); + + /* consider iv len */ + encr_offset += iv_len; + + inputlen = encr_offset + + (RTE_ALIGN(encr_data_len, 8) / 8); + outputlen = inputlen; + + /* IV */ + iv_s = params->iv_buf; + if (snow3g) { + /* + * DPDK seems to provide it in form of IV3 IV2 IV1 IV0 + * and BigEndian, MC needs it as IV0 IV1 IV2 IV3 + */ + + for (j = 0; j < 4; j++) + iv[j] = iv_s[3 - j]; + } else { + /* ZUC doesn't need a swap */ + for (j = 0; j < 4; j++) + iv[j] = iv_s[j]; + } + + /* + * GP op header, lengths are expected in bits. + */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(encr_data_len); + + /* + * In 83XX since we have a limitation of + * IV & Offset control word not part of instruction + * and need to be part of Data Buffer, we check if + * head room is there and then only do the Direct mode processing + */ + if (likely((req_flags & SINGLE_BUF_INPLACE) && + (req_flags & SINGLE_BUF_HEADTAILROOM))) { + void *dm_vaddr = params->bufs[0].vaddr; + uint64_t dm_dma_addr = params->bufs[0].dma_addr; + /* + * This flag indicates that there is 24 bytes head room and + * 8 bytes tail room available, so that we get to do + * DIRECT MODE with limitation + */ + + offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - + OFF_CTRL_LEN - iv_len); + offset_dma = dm_dma_addr - OFF_CTRL_LEN - iv_len; + + /* DPTR */ + req->ist.ei1 = offset_dma; + /* RPTR should just exclude offset control word */ + req->ist.ei2 = dm_dma_addr - iv_len; + req->alternate_caddr = (uint64_t *)((uint8_t *)dm_vaddr + + outputlen - iv_len); + + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(inputlen + OFF_CTRL_LEN); + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + if (likely(iv_len)) { + uint32_t *iv_d = (uint32_t *)((uint8_t *)offset_vaddr + + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + } + + /* iv offset is 0 */ + *offset_vaddr = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + } else { + uint32_t i, g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t *iv_d; + + /* save space for offset and iv... */ + offset_vaddr = m_vaddr; + offset_dma = m_dma; + + m_vaddr = (uint8_t *)m_vaddr + OFF_CTRL_LEN + iv_len; + m_dma += OFF_CTRL_LEN + iv_len; + m_size -= OFF_CTRL_LEN + iv_len; + + opcode.s.major |= CPT_DMA_MODE; + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input Gather List + */ + i = 0; + + /* Offset control word */ + + /* iv offset is 0 */ + *offset_vaddr = rte_cpu_to_be_64((uint64_t)encr_offset << 16); + + i = fill_sg_comp(gather_comp, i, offset_dma, + OFF_CTRL_LEN + iv_len); + + iv_d = (uint32_t *)((uint8_t *)offset_vaddr + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + + /* Add input data */ + size = inputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, + params->src_iov, + 0, &size, NULL, 0); + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Scatter List + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + /* IV */ + i = fill_sg_comp(scatter_comp, i, + offset_dma + OFF_CTRL_LEN, + iv_len); + + /* Add output data */ + size = outputlen - iv_len; + if (size) { + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + } + + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + vq_cmd_w3.s.grp = 0; + vq_cmd_w3.s.cptr = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, zs_ctx); + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + static __rte_always_inline void * cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, @@ -1243,6 +1833,9 @@ cpt_fc_dec_hmac_prep(uint32_t flags, if (likely(fc_type == FC_GEN)) { ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == ZUC_SNOW3G) { + ret = cpt_zuc_snow3g_dec_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { /* * For AUTH_ONLY case, @@ -1273,6 +1866,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, if (likely(fc_type == FC_GEN)) { ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == ZUC_SNOW3G) { + ret = cpt_zuc_snow3g_enc_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { ret = ERR_EIO; } -- 2.7.4