From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0056.outbound.protection.outlook.com [104.47.41.56]) by dpdk.org (Postfix) with ESMTP id F22B31B397 for ; Tue, 9 Oct 2018 11:09:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pqLBfQiRcFtkKIGZEozDflOCGjtiHYbrM25Ct2HKgIY=; b=TSRdmcCPmb6FNcbVCStnpTJUbRUP2MhE1vzxVUYs68BmD1Rh+VBW3VEM1j2AxpuMJC8MngDtMt3n/i0/HkJrQBQNUAZxzD+zBGSaec+Z6yAOOPhuqU6KxkctT8igb4WJLA2fcjKr8oQ6Nb5P1PHn9Wymktllpn6kuAzlf+j9wZk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.com (115.113.156.2) by BN7PR07MB4897.namprd07.prod.outlook.com (2603:10b6:406:ef::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.28; Tue, 9 Oct 2018 09:09:49 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Srisivasubramanian S , Jerin Jacob , Narayana Prasad , Ankur Dwivedi , Anoob Joseph , Murthy NSSR , Nithin Dabilpuram , Ragothaman Jayaraman , Tejasree Kondoj , dev@dpdk.org Date: Tue, 9 Oct 2018 14:37:51 +0530 Message-Id: <1539076076-19786-19-git-send-email-anoob.joseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539076076-19786-1-git-send-email-anoob.joseph@caviumnetworks.com> References: <1538744363-30340-1-git-send-email-anoob.joseph@caviumnetworks.com> <1539076076-19786-1-git-send-email-anoob.joseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: PN1PR0101CA0004.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:e::14) To BN7PR07MB4897.namprd07.prod.outlook.com (2603:10b6:406:ef::26) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e79de655-e847-4802-829c-08d62dc6f991 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BN7PR07MB4897; X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 3:W5aJLLR/ApLQV5eKJZpL8t8cVFKYPRxOuaJM7Vzergl91A14QKV15QyoGQLMthdLLRqXebmAr7tLANl8I2qk5HtAchn1F0preKeuZz4q+mPKHlqixaoAL9AJCBxFVvfRjmKP0N+/iZ0Sr4PB+kfmV9KdqGRFbI5LiiyrLf999OW3kFfyPqevkQL8IqMWzr4ZCoFbFvCaQA569XKNHq5DddCPwZioePi9t9yrEyi6n62TGWCSL3tVPNPOxyQU2Aca; 25:911GovWk/9cW8BjnKOtCaxyse0fluhSm6YIw+YtSan/ugMQvy8aKR6w1XtuJh5NDhpsy6nSZNV6vvukGWHNEYPPAUqsx8KZCwZFf+t+N8+8F27qkyuxsFMs7PLoTIFRZ2nLgFNEj6RcqlNOZnDLLTb6iLgidAZCbTb30aRjaFZnKnZQRmoj6RIHRFSXySSHpNMMuG719Mo5++/5QsXo5GRgQ3lNvwup/z9tzy/LAzQDzlepC9DGEosT2x1CsONzzf5rmrunVk+021hNeR9rfxx+cKyzHeUGtvkNtWWzNp+pMd+6b3dA8JQnKrBDNTaS3Wng7ZpqGphgKTpe7/pcQNw==; 31:k5T3KFRQKZ5zAvS6Nxbb4GilLBAaKoyJn/q9QrpPiKiWP9ke8Gw249+a6rGHknCvgHw01d9KqFvomcDFHMzIzo0cyxq9f3ePJs5M41Yx+uAglkd5kGorjLIkVtMgdgvpaJsXUiWo85ghmOLIKHnSHZ1VY2l38yH+4AV3XeClOU1A7yXR1qjdGeH8FERRWHWLsmxd/2jodXL3VcVxwpfxJO7dGNvJy6tWzotnOrKnsBE= X-MS-TrafficTypeDiagnostic: BN7PR07MB4897: X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 20:Rlop7LGWWyLmNgVM8+UH67NbuILGJ6b0ZPmk5yRpD108phx1l2gCXfKZPnQo1K++AgL2MPQlLAbXSlUcZ6ArAekq7NPT+Tz51YmaKZOiLRB3/PueOUWLBlBsIQCrXlul1O0ZTltY8cOwH90zfeYIvAGqpyktkBwNQ+mLWjHC+dN12Cb02IMea3vbqD92i2SSrNKtwZ7+A4xyAAN1p09JItxy9d3/sffGoiEys3VVQ+ed5RyaBjSLdcI5KgGSQwD4+AIpu7DjG5CM3Dwk+jsCrwlmi04biyHmMs0B4VR8vPgrHF6g307Bewhgdh1PUgVmMfmSrDvoQ2HuUjAOHBblDGoIUgqXxnTuXqdtAhJV4FywtthoF1kqN1EdEtwvpyXeJi+zEtdvJXKt5jfe51bu1znKSJjEfzmXRi8PLHNbfDk7PaygysMvc27jE4yAbjt48jh2+9OdbKTqzsnbvI23iBHk6iYJxExVLQFHg3HZbXoUKoapr9hN/xcpSc8SLz4m2C1Qv2E2DSeMH/P5lr2lFnZ40wUlQoCchk5exSvkxyt2ztm/1vMVuKeyjexOK7pfSMNi8A1lNoaXm+hwNv340PsHjb3XzQUGNu3a1It6HRA=; 4:8TGvv4DFtTvhvhXSbsQfIjmzCvAjzrlsWWRbtl5YlYwPgoxGvIB0kgshcw/3k0QX3sc7bv7zjyGftxoGGHSPl+KZJIUHcciE8CpJN9Wcnsi8eDnOhLO/fY9J7dThfY1JDyFqRb+NqQKFS1d+wqb72wtipKAzojcvFewGRM/5of466NE3xCSjiTTJHdRq3e4a3hTsjW5lCD9LcH06z79NofHtip5/NyBezkwF9A5mGr9vp0iAxMmkvST6WMo6ix7cN6A+yUn/zoKsmAaw8TQ9+w== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(93006095)(10201501046)(3231355)(944501410)(52105095)(149066)(150057)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(20161123558120)(201708071742011)(7699051); SRVR:BN7PR07MB4897; BCL:0; PCL:0; RULEID:; SRVR:BN7PR07MB4897; X-Forefront-PRVS: 08200063E9 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(366004)(376002)(136003)(346002)(39860400002)(199004)(189003)(42882007)(47776003)(106356001)(6666003)(186003)(5660300001)(26005)(68736007)(16526019)(8936002)(25786009)(6116002)(2906002)(81156014)(50226002)(305945005)(81166006)(7736002)(3846002)(105586002)(478600001)(8676002)(54906003)(110136005)(48376002)(16586007)(14444005)(6486002)(6506007)(50466002)(316002)(53936002)(386003)(51416003)(52116002)(76176011)(11346002)(6512007)(44832011)(446003)(36756003)(55236004)(956004)(2616005)(4744004)(486006)(4326008)(66066001)(97736004)(72206003)(476003); DIR:OUT; SFP:1101; SCL:1; SRVR:BN7PR07MB4897; H:ajoseph83.caveonetworks.com.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN7PR07MB4897; 23:4HlL69XTkL3B4KY4mI9gteDZqMVsnsDsXrS2UncE+?= =?us-ascii?Q?1CHgxwpgTspL9/KhKf1u7PyZ3Rox8uKpCy8aceDNpnCesStkfL4L5tHrW/Hz?= =?us-ascii?Q?KFOUy3TOF26CCfpD8P65OB2kQm9xNpqtlWfdR1E5GFNXBUhPY1aBsZ/yttke?= =?us-ascii?Q?aXSTN4wKcr1LKHvDtbT/q1eDEBYJ4TRB4J2zZtMwLbYosV18sRvDosV16BB/?= =?us-ascii?Q?iZ+laSghMNy5EPaMElLoQrBZ7mYBRlkjS2StwA06zsYprlSOCm8Y+F5qpgac?= =?us-ascii?Q?MRV9InLa+FXIzlOrWZL4ZddkjELmuL6sHXHCf4aZPVDBdrXZaB1uq+37PnU5?= =?us-ascii?Q?DmZeSxzqeDILRmH2E+C3RwFmP78WYTzcovi3yZJnKiMFVPLDEA6KtJQoqIIp?= =?us-ascii?Q?lQlLxuy9gk1xGs8Qt9CCIETX/lJqnkBgGZdpCIHYSKUYOP8IYy8+1IwL+aTW?= =?us-ascii?Q?OPWZKUd3ulgFtLXujob4FRFUzqce2nw/thyqbyxRw0sY+WjS2pxFhceM0+UN?= =?us-ascii?Q?IcfKoaAnU/m7stOU032en2edkmknyM4i2E0pEBTyDb63sIMz4Q9vcFPtdwjR?= =?us-ascii?Q?1dA0UdaoURuX/JTe/mJu/n5FgsCkeYr3T5XcFQZoy7oVxbzq+pe3B52Ox3I5?= =?us-ascii?Q?UlzaRTD6RCX3nOhnYJGHia0Dx8AORWT0C4bhVarIy5MxPR+XYOBlkGnYKixa?= =?us-ascii?Q?mSdSZyHE6J7NMWKeqTblLk+SCLKdVWEy6ylHGDjR2ZZuQAQdjcRsZiyujEBn?= =?us-ascii?Q?f0+nGR6MjbE56IvB+gOdy4Czx1jPOsv+iyo+wQWsZAdQkq5r3baEuud5ZufX?= =?us-ascii?Q?+El8ziiPrjOs/AROdvmbdFMpiEiYHp32ukDjLOqGrf3CPEL4LdqpJPyLZuga?= =?us-ascii?Q?SA2JS3+fX6RL0GCyc1Or5gBf2aQbcMCsr8va3Gw12wpoPYR4tvsJSyCW4ZNO?= =?us-ascii?Q?EcmaFRiQSSteLgz9eJHNzDOUYaZIaib9gBPNfNmCif4Y7eOqe7slnl5myHvN?= =?us-ascii?Q?4q76YL/yZ1wu/9AfZ5ZB83bnHaGLXzdxrg2NjiNNasB/ORTaoC1zeUWsekrf?= =?us-ascii?Q?ZmQATEOMcWvpx+9jFN1hhMF7228j3ts+SGuW9sWXeeHDwq4mLLubnAgCi6zr?= =?us-ascii?Q?GPOXTUEP7YpYRNfD/J+tfODjLHw43/M4qSNhn0eCKKoS93RXfWb3CgRUNw5y?= =?us-ascii?Q?XSUnGX7lTfjMfQK8hyDXwYNxZ8d37wP0402wEJbLs2b09o8TLJe4/kMGxrhZ?= =?us-ascii?Q?c0z0VUsI/cu2SMoFm5rwcCr+U4ohz1jZO95eNa9ArsZucrlgSkr0oq+mhE7c?= =?us-ascii?B?dz09?= X-Microsoft-Antispam-Message-Info: 2EUE3UQWWlhuSkqLXdSsjcmqLtiQBN0Xtc91XQNdpi6Kn9oC0VAjYShELAkLNzzhzlvHsiQMGT7aq3kHHp3M1NjrhASw/NgQU9AbMkisgKgVFpqCgK5hLvO97Gh6nkiTjarUHSXSJWjC7dJ+xxj+n8sZ3A5I638KHLJ+abPa3QB4vhmO7AjBsZtdpBUFagCGGufyZj5OfC1zl2he4LlmQ1UcDy7vznao+my12FYRk1PqwpWj0ZF2cDPfeaSksFzw2eF8rz3aDYVK6lJuGAKH1L69RLkS6/qXmN8w9p0pkMcSCOf5lmTcytjDbJLtzLRlw8PP9oM+iGzGDn+oP6GcR3TqWp1BC/sLfk/qT5MYI8U= X-Microsoft-Exchange-Diagnostics: 1; BN7PR07MB4897; 6:Yhjfbjm16jHyAkNl1CctyibCwlUU2rHHfLTbpwJJ2UjmU3CX52kfDEgcntQBx+qq7BXlBxlBHtbfR8SH9VYOshDRJ29hIwtCpd3rEKrBJRNZHs+zPQjygkm3X1pDwRRcptchck4IOz1iH3EKE5w/B7cY3b30QXS9eed2hSZ4bQHAqnhbuBvtM2FJUJvCantRyez2HS0xmtoYg8j2y9aW0ZBiA9zuIo991zUD2GEQXtYbLVebZikQ2bypRa6qiI17hlmhAyXO36nTuWhldky+k8W+JccaWTwKmOU/qIn80qpDzLIbk9DgFYLrouaLgpFJ7iisSJuYYxRiBHPa/A5OGqMC/0Cyq49kpNOq7UmD5UHQASwg3Uhz1W9ioCCXviToe099EoE4I28N2mMVjphhVAi+8bZeXmeXqNYsUHxtha3/JuyqWGnnP3fs9vitJ4cZlDaInJA1v5A5IaLmaXHCcQ==; 5:VZoqh9JBlbAlapYIJsuHxA89nJV4GA5DEsdrweB3GmiNUGBCxUT7Bobz+R9svxMZZJs9vMPKx45ehbf/B9LgDiTSnh+xHFkJyWV3Ev8MbVk4cMlQmJ4dKL+71ekRqij1MKmyM348ufb97HeQg4no9NdXWSYkkGSqrMxpidPP/PU=; 7:uefZAeZZI6GSuwQs8w7AfbcsnJ6sum8gsywmjZ/vnuRVTGNaueH9JRR1iq1JJBem/us4TmZMfDggg5zXS8Q5n1cEGejCj8sQ3wcE50LJ5uzRH/MWf3G5M6tbBJPmY5PmDWmUGMM72qI8IA+pZyCv/DEazNDTVgVGbS3IcaNkpvUFaqukMEsbjt7NOpfvc2se/7qq3o+x0ThGPb538HK6pRgwyVsv6GHwAhoPgNltqZPhIPR5sVuYky0wpNdEPZJl SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2018 09:09:49.9777 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e79de655-e847-4802-829c-08d62dc6f991 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR07MB4897 Subject: [dpdk-dev] [PATCH v4 18/23] common/cpt: support hash X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2018 09:09:55 -0000 From: Srisivasubramanian S Adding microcode interface for supporting verify and authentication Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian S Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_mcode_defines.h | 7 + drivers/common/cpt/cpt_request_mgr.h | 2 + drivers/common/cpt/cpt_ucode.h | 415 ++++++++++++++++++++++++++++++++- 3 files changed, 423 insertions(+), 1 deletion(-) diff --git a/drivers/common/cpt/cpt_mcode_defines.h b/drivers/common/cpt/cpt_mcode_defines.h index 263fc47..60be8b3 100644 --- a/drivers/common/cpt/cpt_mcode_defines.h +++ b/drivers/common/cpt/cpt_mcode_defines.h @@ -18,6 +18,7 @@ #define CPT_MAJOR_OP_HMAC 0x35 #define CPT_MAJOR_OP_ZUC_SNOW3G 0x37 #define CPT_MAJOR_OP_KASUMI 0x38 +#define CPT_MAJOR_OP_MISC 0x01 #define CPT_BYTE_16 16 #define CPT_BYTE_24 24 @@ -341,6 +342,12 @@ typedef struct fc_params { } fc_params_t; +/* + * Parameters for digest + * generate requests + * Only src_iov, op, ctx_buf, mac_buf, prep_req + * meta_buf, auth_data_len are used for digest gen. + */ typedef struct fc_params digest_params_t; /* Cipher Algorithms */ diff --git a/drivers/common/cpt/cpt_request_mgr.h b/drivers/common/cpt/cpt_request_mgr.h index 58a87c3..4d21f46 100644 --- a/drivers/common/cpt/cpt_request_mgr.h +++ b/drivers/common/cpt/cpt_request_mgr.h @@ -74,6 +74,8 @@ cpt_pmd_crypto_operation(struct cpt_instance *instance, if (likely(cpt_op & CPT_OP_CIPHER_MASK)) prep_req = fill_fc_params(op, sess, &mdata, &ret); + else + prep_req = fill_digest_params(op, sess, &mdata, &ret); if (unlikely(!prep_req)) { CPT_LOG_DP_ERR("prep cryto req : op %p, cpt_op 0x%x " diff --git a/drivers/common/cpt/cpt_ucode.h b/drivers/common/cpt/cpt_ucode.h index 05cf95c..4d892f2 100644 --- a/drivers/common/cpt/cpt_ucode.h +++ b/drivers/common/cpt/cpt_ucode.h @@ -4,7 +4,6 @@ #ifndef _CPT_UCODE_H_ #define _CPT_UCODE_H_ - #include #include "cpt_common.h" @@ -460,6 +459,214 @@ fill_sg_comp_from_iov(sg_comp_t *list, } static __rte_always_inline int +cpt_digest_gen_prep(uint32_t flags, + uint64_t d_lens, + digest_params_t *params, + void *op, + void **prep_req) +{ + struct cpt_request_info *req; + uint32_t size, i; + int32_t m_size; + uint16_t data_len, mac_len, key_len; + auth_type_t hash_type; + buf_ptr_t *meta_p; + struct cpt_ctx *ctx; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + void *c_vaddr, *m_vaddr; + uint64_t c_dma, m_dma; + opcode_info_t opcode; + + if (!params || !params->ctx_buf.vaddr) + return ERR_BAD_INPUT_ARG; + + ctx = params->ctx_buf.vaddr; + meta_p = ¶ms->meta_buf; + + if (!meta_p->vaddr || !meta_p->dma_addr) + return ERR_BAD_INPUT_ARG; + + if (meta_p->size < sizeof(struct cpt_request_info)) + return ERR_BAD_INPUT_ARG; + + m_vaddr = meta_p->vaddr; + m_dma = meta_p->dma_addr; + m_size = meta_p->size; + + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + hash_type = ctx->hash_type; + mac_len = ctx->mac_len; + key_len = ctx->auth_key_len; + data_len = AUTH_DLEN(d_lens); + + /*GP op header */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param2 = rte_cpu_to_be_16(((uint16_t)hash_type << 8)); + if (ctx->hmac) { + opcode.s.major = CPT_MAJOR_OP_HMAC | CPT_DMA_MODE; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(key_len); + vq_cmd_w0.s.dlen = + rte_cpu_to_be_16((data_len + ROUNDUP8(key_len))); + } else { + opcode.s.major = CPT_MAJOR_OP_HASH | CPT_DMA_MODE; + vq_cmd_w0.s.param1 = 0; + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(data_len); + } + + opcode.s.minor = 0; + + /* Null auth only case enters the if */ + if (unlikely(!hash_type && !ctx->enc_cipher)) { + opcode.s.major = CPT_MAJOR_OP_MISC; + /* Minor op is passthrough */ + opcode.s.minor = 0x03; + /* Send out completion code only */ + vq_cmd_w0.s.param2 = 0x1; + } + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input gather list + */ + + i = 0; + + if (ctx->hmac) { + uint64_t k_dma = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, auth_key); + /* Key */ + i = fill_sg_comp(gather_comp, i, k_dma, ROUNDUP8(key_len)); + } + + /* input data */ + size = data_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, + 0, &size, NULL, 0); + if (size) { + CPT_LOG_DP_DEBUG("Insufficient dst IOV size, short" + " by %dB", size); + return ERR_BAD_INPUT_ARG; + } + } else { + /* + * Looks like we need to support zero data + * gather ptr in case of hash & hmac + */ + i++; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + /* + * Output Gather list + */ + + i = 0; + scatter_comp = (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + if (flags & VALID_MAC_BUF) { + if (params->mac_buf.size < mac_len) + return ERR_BAD_INPUT_ARG; + + size = mac_len; + i = fill_sg_comp_from_buf_min(scatter_comp, i, + ¶ms->mac_buf, &size); + } else { + size = mac_len; + i = fill_sg_comp_from_iov(scatter_comp, i, + params->src_iov, data_len, + &size, NULL, 0); + if (size) { + CPT_LOG_DP_DEBUG("Insufficient dst IOV size, short by" + " %dB", size); + return ERR_BAD_INPUT_ARG; + } + } + + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + +static __rte_always_inline int cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, @@ -2319,6 +2526,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, } else if (fc_type == KASUMI) { ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == HASH_HMAC) { + ret = cpt_digest_gen_prep(flags, d_lens, fc_params, op, + &prep_req); } else { ret = ERR_EIO; } @@ -3206,4 +3416,207 @@ instance_session_cfg(struct rte_crypto_sym_xform *xform, void *sess) return -1; } +static __rte_always_inline void +find_kasumif9_direction_and_length(uint8_t *src, + uint32_t counter_num_bytes, + uint32_t *addr_length_in_bits, + uint8_t *addr_direction) +{ + uint8_t found = 0; + while (!found && counter_num_bytes > 0) { + counter_num_bytes--; + if (src[counter_num_bytes] == 0x00) + continue; + if (src[counter_num_bytes] == 0x80) { + *addr_direction = src[counter_num_bytes - 1] & 0x1; + *addr_length_in_bits = counter_num_bytes * 8 - 1; + found = 1; + } else { + int i = 0; + uint8_t last_byte = src[counter_num_bytes]; + for (i = 0; i < 8 && found == 0; i++) { + if (last_byte & (1 << i)) { + *addr_direction = (last_byte >> (i+1)) + & 0x1; + if (i != 6) + *addr_length_in_bits = + counter_num_bytes * 8 + + (8 - (i + 2)); + else + *addr_length_in_bits = + counter_num_bytes * 8; + found = 1; + } + } + } + } +} + +/* + * This handles all auth only except AES_GMAC + */ +static __rte_always_inline void * +fill_digest_params(struct rte_crypto_op *cop, + struct cpt_sess_misc *sess, + void **mdata_ptr, + int *op_ret) +{ + uint32_t space = 0; + struct rte_crypto_sym_op *sym_op = cop->sym; + void *mdata; + phys_addr_t mphys; + uint64_t *op; + uint32_t auth_range_off; + uint32_t flags = 0; + uint64_t d_offs = 0, d_lens; + void *prep_req = NULL; + struct rte_mbuf *m_src, *m_dst; + uint16_t auth_op = sess->cpt_op & CPT_OP_AUTH_MASK; + uint8_t zsk_flag = sess->zsk_flag; + uint16_t mac_len = sess->mac_len; + fc_params_t params; + char src[SRC_IOV_SIZE]; + uint8_t iv_buf[16]; + memset(¶ms, 0, sizeof(fc_params_t)); + struct cptvf_meta_info *cpt_m_info = + (struct cptvf_meta_info *)(*mdata_ptr); + + m_src = sym_op->m_src; + + /* For just digest lets force mempool alloc */ + mdata = alloc_op_meta(NULL, ¶ms.meta_buf, cpt_m_info->cptvf_op_mlen, + cpt_m_info->cptvf_meta_pool); + if (mdata == NULL) { + CPT_LOG_DP_ERR("Error allocating meta buffer for request"); + *op_ret = -ENOMEM; + return NULL; + } + + mphys = params.meta_buf.dma_addr; + + op = mdata; + op[0] = (uintptr_t)mdata; + op[1] = (uintptr_t)cop; + op[2] = op[3] = 0; /* Used to indicate auth verify */ + space += 4 * sizeof(uint64_t); + + auth_range_off = sym_op->auth.data.offset; + + flags = VALID_MAC_BUF; + params.src_iov = (void *)src; + if (unlikely(zsk_flag)) { + /* + * Since for Zuc, Kasumi, Snow3g offsets are in bits + * we will send pass through even for auth only case, + * let MC handle it + */ + d_offs = auth_range_off; + auth_range_off = 0; + params.auth_iv_buf = rte_crypto_op_ctod_offset(cop, + uint8_t *, sess->auth_iv_offset); + if (zsk_flag == K_F9) { + uint32_t length_in_bits, num_bytes; + uint8_t *src, direction = 0; + uint32_t counter_num_bytes; + + memcpy(iv_buf, rte_pktmbuf_mtod(cop->sym->m_src, + uint8_t *), 8); + /* + * This is kasumi f9, take direction from + * source buffer + */ + length_in_bits = cop->sym->auth.data.length; + num_bytes = (length_in_bits >> 3); + counter_num_bytes = num_bytes; + src = rte_pktmbuf_mtod(cop->sym->m_src, uint8_t *); + find_kasumif9_direction_and_length(src, + counter_num_bytes, + &length_in_bits, + &direction); + length_in_bits -= 64; + cop->sym->auth.data.offset += 64; + d_offs = cop->sym->auth.data.offset; + auth_range_off = d_offs / 8; + cop->sym->auth.data.length = length_in_bits; + + /* Store it at end of auth iv */ + iv_buf[8] = direction; + params.auth_iv_buf = iv_buf; + } + } + + d_lens = sym_op->auth.data.length; + + params.ctx_buf.vaddr = SESS_PRIV(sess); + params.ctx_buf.dma_addr = sess->ctx_dma_addr; + + if (auth_op == CPT_OP_AUTH_GENERATE) { + if (sym_op->auth.digest.data) { + /* + * Digest to be generated + * in separate buffer + */ + params.mac_buf.size = + sess->mac_len; + params.mac_buf.vaddr = + sym_op->auth.digest.data; + params.mac_buf.dma_addr = + sym_op->auth.digest.phys_addr; + } else { + uint32_t off = sym_op->auth.data.offset + + sym_op->auth.data.length; + int32_t dlen, space; + + m_dst = sym_op->m_dst ? + sym_op->m_dst : sym_op->m_src; + dlen = rte_pktmbuf_pkt_len(m_dst); + + space = off + mac_len - dlen; + if (space > 0) + if (!rte_pktmbuf_append(m_dst, space)) { + CPT_LOG_DP_ERR("Failed to extend " + "mbuf by %uB", space); + goto err; + } + + params.mac_buf.vaddr = + rte_pktmbuf_mtod_offset(m_dst, void *, off); + params.mac_buf.dma_addr = + rte_pktmbuf_mtophys_offset(m_dst, off); + params.mac_buf.size = mac_len; + } + } else { + /* Need space for storing generated mac */ + params.mac_buf.vaddr = (uint8_t *)mdata + space; + params.mac_buf.dma_addr = mphys + space; + params.mac_buf.size = mac_len; + space += RTE_ALIGN_CEIL(mac_len, 8); + op[2] = (uintptr_t)params.mac_buf.vaddr; + op[3] = mac_len; + } + + params.meta_buf.vaddr = (uint8_t *)mdata + space; + params.meta_buf.dma_addr = mphys + space; + params.meta_buf.size -= space; + + /* Out of place processing */ + params.src_iov = (void *)src; + + /*Store SG I/O in the api for reuse */ + if (prepare_iov_from_pkt(m_src, params.src_iov, auth_range_off)) { + CPT_LOG_DP_ERR("Prepare src iov failed"); + *op_ret = -1; + goto err; + } + + prep_req = cpt_fc_enc_hmac_prep(flags, d_offs, d_lens, + ¶ms, op, op_ret); + *mdata_ptr = mdata; + return prep_req; +err: + if (unlikely(!prep_req)) + free_op_meta(mdata, cpt_m_info->cptvf_meta_pool); + return NULL; +} + #endif /*_CPT_UCODE_H_ */ -- 2.7.4