From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0040.outbound.protection.outlook.com [104.47.36.40]) by dpdk.org (Postfix) with ESMTP id 1B0625F17 for ; Fri, 8 Jun 2018 18:49:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yHQ9VcoE2gh7kJJy0Foitz9lEJLAEi9Us1kMzqQOjhc=; b=UxNk43JOQgyhJTMmDqQL6Gdgy1gP6E8a3dY0jEYYaeY0Fll01TIB5HD4uNz/j3KSSNYbTyl/C2pYsI6Oolfxhc03zbpf+EnePkr/xibTF4KLeCPL4DvSz85FbKS2GKuTAMJa/wdEt3H/TDGbwxQQfiRc+QW///TQF90cIkLHJhU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.caveonetworks.com (115.113.156.2) by SN6PR07MB4911.namprd07.prod.outlook.com (2603:10b6:805:3c::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.15; Fri, 8 Jun 2018 16:49:05 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Murthy NSSR , Ankur Dwivedi , Jerin Jacob , Narayana Prasad , Nithin Dabilpuram , Ragothaman Jayaraman , Srisivasubramanian Srinivasan , dev@dpdk.org Date: Fri, 8 Jun 2018 22:15:16 +0530 Message-Id: <1528476325-15585-8-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1528476325-15585-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1528476325-15585-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: BM1PR01CA0071.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1::11) To SN6PR07MB4911.namprd07.prod.outlook.com (2603:10b6:805:3c::29) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:SN6PR07MB4911; X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 3:bYfJdQibFnaHCoe829FphgEtfXnstIFdNk0TA4XLLpZn1E4coRJFGvOsBazM5cvG1l3s+9UT6Mmhx+vgT3z0ScEaW8amfaCskgBWoHepcg2oaVhAqSBb83zHmIVodEp9PV35GIp32RyuI4ehAWU+q0AEFO1O7xjTJVD2jpNTVkhN090r8dErepPdHEODvA5kvUzOVJFrrgxdnr/2TJJakb7EYnj8Q6/AEAj6duVh12GHxSc+xBtHW4alOlfGPjOE; 25:wED9bacacmXk2/w5RLhhwHIfqT8GJTizV2NsL2JtZwOxWECU+xFrMIMaOOaG4t+krM0Gd/eP4CNyCkoVfJCf4w0NfTpfmBkosoJ3mvfUnTicnlVlns79VM8OGe5q1Kb3juJkyvMr6PgWcDz8539nIm9hL26EUfNDl4zk7ECgMluudImhXTo+jeCFemer+heMH0M5mvRWUfsb1Aa1NNdCOKjmmjOH/TjLYvn+BwONPQQxM6CACcmdiBffncWe+V9RNefIYWplxpcCxjhC/zh4QatRVhvXf41ZOr9QKzLzIdq1kySmHDvXm+eTyAkEaqZ1iv5xb/469wfr9fykyz8x7w==; 31:ai4026JtNN0to/Y9ns/SufNiD6WTk3bU7XwzIXcmlOrWCwueHsRyOZrSoVzZlEN2PLY0Lir2wvDnSozXQaMhQ9VGi6HE8J7Y3ycNmtYjLGQoeDOv5J73NtdM45ZDaHzgsWA/jiqhbW0S+liyfYnD6Awff854qSDIL8fGrh8V1630br0aBAFrrwluEQnzkf+KCMkIt9WtFoqGIyZuvhYJvBQjxbURvKm3s6FHwgxPLfU= X-MS-TrafficTypeDiagnostic: SN6PR07MB4911: X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 20:lzxEoMZmnDCzUQD3tElt4jG27ylq1UORC2h9DVH0EeuUpAzVdwuPVq318gowmNY5YqzVueSmvSYv6q9qkuyDea/9FCIzVLq5BXPg6OjooSFZf5ORzNBnv/LTeCYopeP2tGEEBVQkQpJict/PIOUGWo6rDlOq6HSmbQiRVogq946kbvzI/NYIiYeqQtv+vtikG76wjDY0dJraPcSJkTyipO3ubABp1E15FSfqHecbXnq1BCMrvE8sdovx9WtcxT02+mYeU4cU8otMW9brPV00mYCfe91abfrrJV+uCBDXbYsUPitrToZf8kVGDi34YW2ftQYHVI+IXvubA4EAgHd8ZLRim4e0bcLpG1WXychbnDPsulDh40ApiJHfavmC45q3hws38NBO11fKFy4eQp3ycze1Aq3U7sgh42km5OUHU69twGJFDte3l496NUsDFwcU+KGSAdAYnBtph9x9SK/XhphQij8TTDqfuiR+OrJe0PMfLuQOcJ07HGUXTBnpDQ8utwFBdpv6EUiF8TEgjIh6SEYRUpVAYSynaEhHyV07nFqevcMzixzW4y5IjLvG2iSF3wYVXTUBAMdCrzUB3Pc1dbjBoNrYdWoUmup8PCyKP5U=; 4:EinhGb2j4h2scLdz44IFaXgZ0xVEL9ROfMqPq2xmB/L8VqqI5fKR/VsOLLoJFjhwoWBVlRDmvxIfuIkHDcdz3SrT7bKg4LTOpL0fF94VMqx+CzDGkmwX6ysD35swKgFdnvk/6l306UUUhl4zcHBgdqLI+F5VQilcWeWFCCc9jadvmNdc5bpHcjuYDwy1ZqHQbr4sVkheqiQaK2u6bXoi9CRc//pPVnhFWaC14gfKGfJgVYbKsReVIS/ArU4Z6wlbaSBjAjIgGsBQ//aI1P3WPA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(3002001)(3231254)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:SN6PR07MB4911; BCL:0; PCL:0; RULEID:; SRVR:SN6PR07MB4911; X-Forefront-PRVS: 06973FFAD3 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(396003)(346002)(39380400002)(366004)(39860400002)(189003)(199004)(26005)(52116002)(956004)(305945005)(8676002)(81156014)(50226002)(8936002)(81166006)(7736002)(68736007)(105586002)(106356001)(3846002)(6116002)(72206003)(478600001)(6486002)(97736004)(6506007)(16586007)(6512007)(53936002)(47776003)(486006)(25786009)(5660300001)(50466002)(48376002)(6666003)(4326008)(54906003)(76176011)(36756003)(110136005)(66066001)(8656006)(2906002)(316002)(55236004)(11346002)(16526019)(386003)(186003)(59450400001)(446003)(476003)(42882007)(44832011)(2616005)(51416003)(32563001); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR07MB4911; H:ajoseph83.caveonetworks.com.caveonetworks.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN6PR07MB4911; 23:xzbEENu4kl1vAF2+b5oN1RCS7gNVeFzjOCWJJ9U87?= =?us-ascii?Q?FBPS/q9K0pnNH/iGHqm0ty6Q30FR31FYvSPrciMH7V7wqZIBqujztNsQddMF?= =?us-ascii?Q?oiaQPvl5xORw0oQ9D9cRni6v9apItkEZW6ZqyXDx1OGBs0NqccaTsqXRval2?= =?us-ascii?Q?QHdINtBQhvO1cXvXhSreFWfMTLQLMcTVOciNUGHud41T/pOt7yq5gzgf02jw?= =?us-ascii?Q?hX0nZtJHexn6CFjtp9+75kL7iyocGrN83lD6u3rDDg3Qi2ghj8eGcBHBCq60?= =?us-ascii?Q?HJaGPlZGKWK1dI/9jvngKYmSFsYLVlgEB9uhwFre6rfGdL9YeKCP/tdhJW6f?= =?us-ascii?Q?Sgf2vJ3wbu5azI8tXKyvS7yyP42kIs0lLuHrJh+g0qNlswH+qlWVrjH9VDzi?= =?us-ascii?Q?qecLd9XH2ulPi5PREcWXCHcmaMP+HkpKHYtFhKmt+b1jAPsllVwFOjsnVzFM?= =?us-ascii?Q?O/senwF0fviCsNw+pxQO95bpAF6/s2a7uR6W3KM++se6mcV+kusJ7/6tOzzM?= =?us-ascii?Q?SIBjlhjLxKUxSf+EOY2Fm652lGuBRN47qrFmhZVuJoDsgomeEArdmB0wC6iY?= =?us-ascii?Q?FDOV3ZrILhjtl5oUVLoQ1PeRtsgpixRIBGcc0rgtAL3fhLLc2O2hG8snsoK9?= =?us-ascii?Q?vQ09SHEMf0nvZ1T14rzvaTL1ToXONCTR9epDTWWhMI3jjzBsZQIm5Uz94zKY?= =?us-ascii?Q?v6aVQOhbR4cKSNxXKWL/C7JN4f8nkL//PIACplQN14t8VJjjISPgzZ6QM9NQ?= =?us-ascii?Q?Bs4rvKskjSuiwgwzOLqHcE2Va3La1Un+tcAJSllLaHoH12C8/uTbk6YfNRmA?= =?us-ascii?Q?/0fehBZAoN75Cp+A2jZZH9K3L0gjMZlT9aHrj0jhBYPFMj2Qab8MSuShmKeC?= =?us-ascii?Q?StJvcPR6+Hy0ASbdYw+rqc4oB+TwIvfU2O5CALS21i+4V2SzzA2etjdzcRfC?= =?us-ascii?Q?PG6jzVCntpsr9bDOYESnNLCDKL0CPDrpsLbj2q85w6kxv3XYoCIFMKygMuD6?= =?us-ascii?Q?s5YDzJNVYVdH1r1KHgiZUU1ZTf708eCoSHToRSUpoVt4ze5Zu6rccEgYW+/P?= =?us-ascii?Q?0YGTOLwcwgcRfzyGp2Dyzj1OaY7kh0N1gWk06JOGqb1dNlh2T4wFqNQXFWYp?= =?us-ascii?Q?bY2mkzKAKpyLUdZY1sRZsGEd3UDXjzkhuLw5YNI9NEolIyVNclX3NYEhjuw3?= =?us-ascii?Q?IM52z2mCjE7itIGqxiHaolSiSIJGfM1T4FHYBjUMFf2A7BfaN3QfsY3/YpPF?= =?us-ascii?Q?oXhPHvWJz2ebjwACjRQsdqjYx8B4CXocapUICsmbamq7nUarIK0lGxVs2aPb?= =?us-ascii?Q?v12wFXeFrpAhD9i/rkhUmJfirlVsXkQmpvtFJUEfa81?= X-Microsoft-Antispam-Message-Info: nmW+SuZgl62NQvMmboUVUMoGNUBt4jhu/OKPUFUL4Pgo0atbHotgilD4DQK8W1hn+KJqZGX55RUzc2JahyamzDrck7DV9FuSeovW0+6Um6V93vw70HoNt9r3Xa2iMLbEE2eNgswgKxXGVYHbZA46oPypnbA98lZyBQEN0Jj/bgh2NWXiAu8SPTAC1/MBfKMn X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 6:915TpE6lYaWg6nTjpx5Bqk/ytk3/kEHs4hNuS/bzVlCBhF/+0qRMJdDlakIUmlxZyN1T4JZDrnwCufZM9m/Ge6h8quY8hdE3KqJi10P0+y7jTKaClhXHzJPmvRmGMj9y+lNW9StHCHAHxs8+i4CLh0LS8rcUwi+dVYgZSkHlsBO3HgCRADUeTGHx8vN5TCEHQjqFsgOMZUHIQdFr1ETlS6ICDPNmr6rr1JfRgKji5H3lRUactb/0v9hw/r5LV3MkRDrY9hVCmCvVOnY+25Uf4zmDmGianw8dGk1nyphgEP6UpngIIryxQlCnkV61NR4KLqAuIAbfSnuFcMdOlVMh0iClCbHNrKsWwj8epvgMAcs2SDbXD4H4wC3v7sxNZbvSmIOHHNbJOoR6FJuFLg2lvQXomfxZTTCE6o7Aa471aXVbnAlT5AFFt77vzR7O2KdKXzSlPoGphgty1r+6XJ+XZw==; 5:0FujIaVh4WTZCArRDVRLfjEVDh03mLiPdRQz+4rCLqHdRh35L9NGURAmMLbGtXaAikt9LtXol3hbbHZFoLQMx3WcOWvRz25we+7hVQEpo960qQ+ehkM0d/HWA9ebInlp/BOFlisqW5vpFqsbrrDu6kTxtdCNOm5VSdYTMQwBd7g=; 24:o3OHssylIWInppTK9xHqvmqnWrAwBmJjRHu0FOooxj/B2WcgZYdR1YlZgixW7VKlq0bmDK6f7TCe1lBSAo9JIkILT8hRfrmdgFCHiFqm2hk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN6PR07MB4911; 7:Vlob0IoYXwB3vQXN3Vc/tspLrxSAT0XpHqPQQ0yWzW19Lb78p0quhS2kfh5+dnVOooboWfKb9R/ufFl2X/jlfSIB31h47Cf+/vCKrcQ3qVYVm8aAZh6ocQEISGjq/n2zgkykSpq6MsGD/uuDlrPTgex0NhY8Z75nDjD7EJQICPM2RjBzwPD/OlAB6jeXIfeit6C5w2pQNAOUwgKjhLwe61X2yiS/ai58bW/buyeCKGNAbSLiWp9pclkxFNj+7QjE X-MS-Office365-Filtering-Correlation-Id: 33065d93-ccd4-43b3-b9bc-08d5cd5fc182 X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2018 16:49:05.9632 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 33065d93-ccd4-43b3-b9bc-08d5cd5fc182 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR07MB4911 Subject: [dpdk-dev] [PATCH 07/16] crypto/cpt/base: add request prepare API for ZUC and SNOW3G X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2018 16:49:11 -0000 From: Murthy NSSR These functions help in preparing symmetric crypto requests for the ZUC, SNOW3G algorithms of both cipher and auth but not cipher+auth in single xform. Signed-off-by: Ankur Dwivedi Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian Srinivasan --- drivers/crypto/cpt/base/cpt_ops.c | 667 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 667 insertions(+) diff --git a/drivers/crypto/cpt/base/cpt_ops.c b/drivers/crypto/cpt/base/cpt_ops.c index 31f8064..ff7522a 100644 --- a/drivers/crypto/cpt/base/cpt_ops.c +++ b/drivers/crypto/cpt/base/cpt_ops.c @@ -1272,6 +1272,667 @@ static inline int __attribute__((always_inline)) return 0; } +static int +cpt_zuc_snow3g_enc_prep(uint32_t req_flags, + uint64_t d_offs, + uint64_t d_lens, + fc_params_t *params, + void *op, + void **prep_req) +{ + uint32_t size; + int32_t inputlen, outputlen; + struct cpt_ctx *cpt_ctx; + uint32_t mac_len = 0; + uint8_t snow3g, j; + cpt_request_info_t *req; + buf_ptr_t *buf_p; + uint32_t encr_offset = 0, auth_offset = 0; + uint32_t encr_data_len = 0, auth_data_len = 0; + int flags, iv_len = 16, m_size; + void *m_vaddr, *c_vaddr; + uint64_t m_dma, c_dma, offset_ctrl; + uint64_t *offset_vaddr, offset_dma; + uint32_t *iv_s, iv[4]; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + opcode_info_t opcode; + + buf_p = ¶ms->meta_buf; +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params || !buf_p->vaddr || !buf_p->size) + return ERR_BAD_INPUT_ARG; +#endif + m_vaddr = buf_p->vaddr; + m_dma = buf_p->dma_addr; + m_size = buf_p->size; + + cpt_ctx = params->ctx_buf.vaddr; + flags = cpt_ctx->zsk_flags; + mac_len = cpt_ctx->mac_len; + snow3g = cpt_ctx->snow3g; + + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Reserve memory for cpt request info */ + req = m_vaddr; + + size = sizeof(cpt_request_info_t); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Initialising ctrl and opcode + * fields for cpt request + */ + + req->se_req = SE_CORE_REQ; + req->dma_mode = CTRL_DMA_MODE_SGIO; + + opcode.s.major = MAJOR_OP_ZUC_SNOW3G; + + /* indicates CPTR ctx, operation type, KEY & IV mode from DPTR */ + opcode.s.minor = ((1 << 6) | (snow3g << 5) | (0 << 4) | + (0 << 3) | (flags & 0x7)); + + if (flags == 0x1) { + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + auth_data_len = AUTH_DLEN(d_lens); + + /* EIA3 or UIA2 */ + auth_offset = AUTH_OFFSET(d_offs); + auth_offset = auth_offset / 8; + + /* consider iv len */ + auth_offset += iv_len; + + inputlen = auth_offset + + (RTE_ALIGN(auth_data_len, 8) / 8); + outputlen = mac_len; + + offset_ctrl = htobe64((uint64_t)auth_offset); + + } else { + /* EEA3 or UEA2 */ + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + encr_data_len = ENCR_DLEN(d_lens); + + + encr_offset = ENCR_OFFSET(d_offs); + encr_offset = encr_offset / 8; + /* consider iv len */ + encr_offset += iv_len; + + inputlen = encr_offset + + (RTE_ALIGN(encr_data_len, 8) / 8); + outputlen = inputlen; + + /* iv offset is 0 */ + offset_ctrl = htobe64((uint64_t)encr_offset << 16); + } + + /* IV */ + iv_s = (flags == 0x1) ? params->auth_iv_buf : + params->iv_buf; + + if (snow3g) { + /* + * DPDK seems to provide it in form of IV3 IV2 IV1 IV0 + * and BigEndian, MC needs it as IV0 IV1 IV2 IV3 + */ + + for (j = 0; j < 4; j++) + iv[j] = iv_s[3 - j]; + } else { + /* ZUC doesn't need a swap */ + for (j = 0; j < 4; j++) + iv[j] = iv_s[j]; + } + + /* + * GP op header, lengths are expected in bits. + */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param1 = htobe16(encr_data_len); + vq_cmd_w0.s.param2 = htobe16(auth_data_len); + + /* + * In 83XX since we have a limitation of + * IV & Offset control word not part of instruction + * and need to be part of Data Buffer, we check if + * head room is there and then only do the Direct mode processing + */ + if (likely((req_flags & SINGLE_BUF_INPLACE) && + (req_flags & SINGLE_BUF_HEADTAILROOM))) { + void *dm_vaddr = params->bufs[0].vaddr; + uint64_t dm_dma_addr = params->bufs[0].dma_addr; + /* + * This flag indicates that there is 24 bytes head room and + * 8 bytes tail room available, so that we get to do + * DIRECT MODE with limitation + */ + + offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - + OFF_CTRL_LEN - iv_len); + offset_dma = dm_dma_addr - OFF_CTRL_LEN - iv_len; + + /* DPTR */ + req->ist.ei1 = offset_dma; + /* RPTR should just exclude offset control word */ + req->ist.ei2 = dm_dma_addr - iv_len; + req->alternate_caddr = (uint64_t *)((uint8_t *)dm_vaddr + + outputlen - iv_len); + + vq_cmd_w0.s.dlen = htobe16(inputlen + OFF_CTRL_LEN); + + vq_cmd_w0.s.opcode = htobe16(opcode.flags); + + if (likely(iv_len)) { + uint32_t *iv_d = (uint32_t *)((uint8_t *)offset_vaddr + + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + } + + *offset_vaddr = offset_ctrl; + } else { + uint32_t i, g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t *iv_d; + + /*save space for iv */ + offset_vaddr = m_vaddr; + offset_dma = m_dma; + + m_vaddr = (uint8_t *)m_vaddr + OFF_CTRL_LEN + iv_len; + m_dma += OFF_CTRL_LEN + iv_len; + m_size -= OFF_CTRL_LEN + iv_len; + + opcode.s.major |= DMA_MODE; + + vq_cmd_w0.s.opcode = htobe16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input Gather List + */ + i = 0; + + /* Offset control word followed by iv */ + + i = fill_sg_comp(gather_comp, i, offset_dma, + offset_vaddr, OFF_CTRL_LEN + iv_len); + + /* iv offset is 0 */ + *offset_vaddr = offset_ctrl; + + iv_d = (uint32_t *)((uint8_t *)offset_vaddr + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + + /* input data */ + size = inputlen - iv_len; + if (size) { +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params->src_iov) + return ERR_BAD_INPUT_ARG; +#endif + + i = fill_sg_comp_from_iov(gather_comp, i, + params->src_iov, + 0, &size, NULL, 0); + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[2] = htobe16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Scatter List + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + if (flags == 0x1) { + /* IV in SLIST only for EEA3 & UEA2 */ + iv_len = 0; + } + + if (iv_len) { + i = fill_sg_comp(scatter_comp, i, + offset_dma + OFF_CTRL_LEN, + (uint8_t *)offset_vaddr + OFF_CTRL_LEN, + iv_len); + } + + /* Add output data */ + if (req_flags & VALID_MAC_BUF) { + size = outputlen - iv_len - mac_len; + if (size) { +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params->dst_iov) + return ERR_BAD_INPUT_ARG; +#endif + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + + /* mac data */ + if (mac_len) { + i = fill_sg_comp_from_buf(scatter_comp, i, + ¶ms->mac_buf); + } + } else { + /* Output including mac */ + size = outputlen - iv_len; + if (size) { +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params->dst_iov) + return ERR_BAD_INPUT_ARG; +#endif + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + } + ((uint16_t *)in_buffer)[3] = htobe16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = htobe16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + } + + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = htobe64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + vq_cmd_w3.s.grp = 0; + vq_cmd_w3.s.cptr = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, zs_ctx); + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!(m_size >= 0)) { + PMD_TX_LOG(ERR, "!!! Buffer Overflow %d\n", + m_size); + abort(); + } +#endif + *prep_req = req; + return 0; +} + +static inline int +cpt_zuc_snow3g_dec_prep(uint32_t req_flags, + uint64_t d_offs, + uint64_t d_lens, + fc_params_t *params, + void *op, + void **prep_req) +{ + uint32_t size; + int32_t inputlen = 0, outputlen; + struct cpt_ctx *cpt_ctx; + uint8_t snow3g, iv_len = 16; + cpt_request_info_t *req; + buf_ptr_t *buf_p; + uint32_t encr_offset; + uint32_t encr_data_len; + int flags, m_size; + void *m_vaddr, *c_vaddr; + uint64_t m_dma, c_dma; + uint64_t *offset_vaddr, offset_dma; + uint32_t *iv_s, iv[4], j; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + opcode_info_t opcode; + + (void)req_flags; + buf_p = ¶ms->meta_buf; +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params || !buf_p->vaddr || !buf_p->size) + return ERR_BAD_INPUT_ARG; +#endif + m_vaddr = buf_p->vaddr; + m_dma = buf_p->dma_addr; + m_size = buf_p->size; + + /* + * Microcode expects offsets in bytes + * TODO: Rounding off + */ + encr_offset = ENCR_OFFSET(d_offs) / 8; + encr_data_len = ENCR_DLEN(d_lens); + + cpt_ctx = params->ctx_buf.vaddr; + flags = cpt_ctx->zsk_flags; + snow3g = cpt_ctx->snow3g; +#ifdef CPTVF_STRICT_PARAM_CHECK + if (flags != 0) { + /* Dec not supported for EIA3 or UIA2 */ + return ERR_BAD_INPUT_ARG; + } +#endif + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Reserve memory for cpt request info */ + req = m_vaddr; + + size = sizeof(cpt_request_info_t); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Initialising ctrl and opcode + * fields for cpt request + */ + + req->se_req = SE_CORE_REQ; + req->dma_mode = CTRL_DMA_MODE_SGIO; + + opcode.s.major = MAJOR_OP_ZUC_SNOW3G; + + /* indicates CPTR ctx, operation type, KEY & IV mode from DPTR */ + opcode.s.minor = ((1 << 6) | (snow3g << 5) | (0 << 4) | + (0 << 3) | (flags & 0x7)); + + /* consider iv len */ + encr_offset += iv_len; + + inputlen = encr_offset + + (RTE_ALIGN(encr_data_len, 8) / 8); + outputlen = inputlen; + + /* IV */ + iv_s = params->iv_buf; + if (snow3g) { + /* + * DPDK seems to provide it in form of IV3 IV2 IV1 IV0 + * and BigEndian, MC needs it as IV0 IV1 IV2 IV3 + */ + + for (j = 0; j < 4; j++) + iv[j] = iv_s[3 - j]; + } else { + /* ZUC doesn't need a swap */ + for (j = 0; j < 4; j++) + iv[j] = iv_s[j]; + } + + /* + * GP op header, lengths are expected in bits. + */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param1 = htobe16(encr_data_len); + + /* + * In 83XX since we have a limitation of + * IV & Offset control word not part of instruction + * and need to be part of Data Buffer, we check if + * head room is there and then only do the Direct mode processing + */ + if (likely((req_flags & SINGLE_BUF_INPLACE) && + (req_flags & SINGLE_BUF_HEADTAILROOM))) { + void *dm_vaddr = params->bufs[0].vaddr; + uint64_t dm_dma_addr = params->bufs[0].dma_addr; + /* + * This flag indicates that there is 24 bytes head room and + * 8 bytes tail room available, so that we get to do + * DIRECT MODE with limitation + */ + + offset_vaddr = (uint64_t *)((uint8_t *)dm_vaddr - + OFF_CTRL_LEN - iv_len); + offset_dma = dm_dma_addr - OFF_CTRL_LEN - iv_len; + + /* DPTR */ + req->ist.ei1 = offset_dma; + /* RPTR should just exclude offset control word */ + req->ist.ei2 = dm_dma_addr - iv_len; + req->alternate_caddr = (uint64_t *)((uint8_t *)dm_vaddr + + outputlen - iv_len); + /* *req->alternate_caddr = ((uint64_t)0xdeadbeefdeadbeef) */ + + vq_cmd_w0.s.dlen = htobe16(inputlen + OFF_CTRL_LEN); + + vq_cmd_w0.s.opcode = htobe16(opcode.flags); + + if (likely(iv_len)) { + uint32_t *iv_d = (uint32_t *)((uint8_t *)offset_vaddr + + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + } + + /* iv offset is 0 */ + *offset_vaddr = htobe64((uint64_t)encr_offset << 16); + } else { + uint32_t i, g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t *iv_d; + + /* save space for offset and iv... */ + offset_vaddr = m_vaddr; + offset_dma = m_dma; + + m_vaddr = (uint8_t *)m_vaddr + OFF_CTRL_LEN + iv_len; + m_dma += OFF_CTRL_LEN + iv_len; + m_size -= OFF_CTRL_LEN + iv_len; + + opcode.s.major |= DMA_MODE; + + vq_cmd_w0.s.opcode = htobe16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input Gather List + */ + i = 0; + + /* Offset control word */ + + /* iv offset is 0 */ + *offset_vaddr = htobe64((uint64_t)encr_offset << 16); + + i = fill_sg_comp(gather_comp, i, offset_dma, offset_vaddr, + OFF_CTRL_LEN + iv_len); + + iv_d = (uint32_t *)((uint8_t *)offset_vaddr + OFF_CTRL_LEN); + memcpy(iv_d, iv, 16); + + /* Add input data */ + size = inputlen - iv_len; + if (size) { +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params->src_iov) + return ERR_BAD_INPUT_ARG; +#endif + i = fill_sg_comp_from_iov(gather_comp, i, + params->src_iov, + 0, &size, NULL, 0); + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[2] = htobe16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Scatter List + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + /* IV */ + i = fill_sg_comp(scatter_comp, i, + offset_dma + OFF_CTRL_LEN, + (uint8_t *)offset_vaddr + OFF_CTRL_LEN, + iv_len); + + /* Add output data */ + size = outputlen - iv_len; + if (size) { +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!params->dst_iov) + return ERR_BAD_INPUT_ARG; +#endif + i = fill_sg_comp_from_iov(scatter_comp, i, + params->dst_iov, 0, + &size, NULL, 0); + + if (size) + return ERR_BAD_INPUT_ARG; + } + ((uint16_t *)in_buffer)[3] = htobe16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = htobe16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + } + + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = htobe64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + vq_cmd_w3.s.grp = 0; + vq_cmd_w3.s.cptr = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, zs_ctx); + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + +#ifdef CPTVF_STRICT_PARAM_CHECK + if (!(m_size >= 0)) { + PMD_TX_LOG(ERR, "!!! Buffer Overflow %d\n", + m_size); + abort(); + } +#endif + *prep_req = req; + return 0; +} + void * cpt_fc_dec_hmac_prep(uint32_t flags, uint64_t d_offs, @@ -1289,6 +1950,9 @@ static inline int __attribute__((always_inline)) if (likely(fc_type == FC_GEN)) { ret = cpt_dec_hmac_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == ZUC_SNOW3G) { + ret = cpt_zuc_snow3g_dec_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { /* * For AUTH_ONLY case, @@ -1319,6 +1983,9 @@ static inline int __attribute__((always_inline)) if (likely(fc_type == FC_GEN)) { ret = cpt_enc_hmac_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == ZUC_SNOW3G) { + ret = cpt_zuc_snow3g_enc_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { ret = ERR_EIO; } -- 1.9.3