From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0055.outbound.protection.outlook.com [104.47.37.55]) by dpdk.org (Postfix) with ESMTP id 908231AEE9 for ; Tue, 4 Sep 2018 06:03:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ab4TDxZl7GxMm+8tN/ry5HUSiqNUl9H/ZOJCA+c85HQ=; b=bBIRBY8sZCmSt7BklCY6NYM5puwJlA5/eot0TtlVOtGO/8Ja+z9W5rLNJCE9JQpbFemq8WwT3MwNk3A+0MpcAXH/CqMrqPmd01FZq8P+Hf8+gtPSbGtrvNpk3SBBnh8LMZp5xu05V8JtpM46Uesw6V0Rj9oWFaeDt4t31aFfrXM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Anoob.Joseph@cavium.com; Received: from ajoseph83.caveonetworks.com.com (115.113.156.2) by BYAPR07MB4902.namprd07.prod.outlook.com (2603:10b6:a02:ef::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1101.15; Tue, 4 Sep 2018 04:03:39 +0000 From: Anoob Joseph To: Akhil Goyal , Pablo de Lara , Thomas Monjalon Cc: Srisivasubramanian S , Jerin Jacob , Narayana Prasad , dev@dpdk.org, Ankur Dwivedi , Anoob Joseph , Murthy NSSR , Nithin Dabilpuram , Ragothaman Jayaraman , Tejasree Kondoj Date: Tue, 4 Sep 2018 09:29:13 +0530 Message-Id: <1536033560-21541-27-git-send-email-ajoseph@caviumnetworks.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536033560-21541-1-git-send-email-ajoseph@caviumnetworks.com> References: <1528476325-15585-1-git-send-email-anoob.joseph@caviumnetworks.com> <1536033560-21541-1-git-send-email-ajoseph@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [115.113.156.2] X-ClientProxiedBy: BMXPR01CA0023.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::33) To BYAPR07MB4902.namprd07.prod.outlook.com (2603:10b6:a02:ef::25) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7bcb7fd4-08cf-4980-d48c-08d6121b67bc X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BYAPR07MB4902; X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4902; 3:CEPdsfU/srum9ucHx7JF3CcL6VS5+eqtGvDNs8stRbZMfIs5/3+dtP2g/J4qjUsPCo5CSlt2FK6YZ05X5lZuXTkvEBu/5UQHp0EiiZV8ZWa2qYrQ8esUZZqiS3APIRtHd86OYbIpGOww2NUhSQ8O3xctuv5iCiX9LvqWn990/cFN/kmxKbAsoFAN6ytJOxHBujaY8XPbgnhDFSWCUP5RLi8LAUXM3RlPEJCYV6BqT3FAtEe1qcS1fQT/6fR9UsIP; 25:HkVMBQ3U928l5jIxEkk0oAwNvAi9fY8I4dOOCiLIoy3rCJ/kESTA7ylS9D8liCEdcizWiU4mGUkgJEY9PPdBOnab8jSmQWt/1VGTvKgvyuBUzfJWKwnsv8Zisru9BwPz64mPwXvx6wJLPvjjpHi8ZMd2EhX6zvtAuAnI3m24px4uZIecU3UzWY0Lg01ucwH433AP1evgyqlvQq/twI1t1PxYtUk/sKe5AaaDaVn7C2FweOC5Az18FDpsGqDZkoqtCLLsjKv0Iz1jsvWTshXof+nlx37kNuvkCEfHqowmNRWPxzhZPfacCaExkk96WXlWlg2g7SiSsvFfrYk1G5A0uw==; 31:9HwaWPtVw9eGRV36FypKBietXK1ZIFC1Yk9B2S9nAJR7r4BMUnFNTkH9vSVMO+CTmDe+QR/QDlSWsn4nW6xoUYSZXi0xZKNDddn8tXmBTjMjRNdymociee0+4r29oNuX0AOJDH3AbOtpHTtbFQeo4G8q+PGAXkHhm5p1J29fjcFfmx7UuUAa37jEE41JD3zGY/bOvhgx2FFcckp7pbXa/E1s3z2S1QBA+OBY1PhXHRk= X-MS-TrafficTypeDiagnostic: BYAPR07MB4902: X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4902; 20:k4bVlQ6Nn0LXQJvTArkInvxiQzJQZt/yrTmIHVgw/OwA2Q5TcJAbc+qzsmvKP2Y445ltle3zqhHTIu/a23+94iTrI58NPHAcrt8fXn+XZWKklLcYyxoxwWq0DvKZ9Vxb0TFUoaPxhlhdEiqvGGQ6QQMbJFSZkfO3PA7Lz9+JY5Hf3VPzmF6wbdoxi6KWBylv8mbl7dX5m+zg1SO4y+DY0NMWAsaBVKAFUBxDAx7zz5TzvuzPgDM5qTXHH/2sSQj5gz/Tbs38vE06+GklNkqqhoI37ixi/R67DPTRwJ+OpfdVHDo1Nu4liBgs8+PyWa1XX1cEzaVmnCxdYtJwHFbyJTjSoCEERj6dqI8ZGwpcVV1C0hTEj7hD3eL8sHCyT5IIxxzDlilQcNbQiyka8/n0Z2aL0cOxKkstYKEncdCGcrFPf9+ZBotKB7JUkmVN12H0kwHvVR1ozWJ4e9jelFZXgQLIDxPF7spm6snGgxakonqKIsnRRJ0VqSbCRYj8lwcw0BpPvBenDNyEAa8TLkrbV2FWNhkWYtw1Q7HJZC4kGtPrlmpIYyAQIFLq7QUXuRhJdqB7yBDxs8nW5GRFr2la/RNWnDDMMN4hdLD1jS7bFZU=; 4:ecbAlpAoi63Ysmyt62xtYVvPHRimCgeLVnOk274ZfJvA21kikCHfLi1EX/a0wxxS4G11l73ILH3D1ds0pnnMzwFBGKT4sh+s33oDwy/Y6VUoD7ql95T78MQV+Gierz7nhHT6NGsSu8xt8FtK6WlkXtLzL2cE3a5/2MphLus5zKfkmolQi00yZRB3M8piXd5XVBIg/jf9ZF5IeiArwTF5D4jUC7b56F5GMBw5uJyQMHBfd6++amR9YntDb6TKVZH/sKh3NcQGJs5S+RIiZcvbEQ== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(3231311)(944501410)(52105095)(3002001)(149027)(150027)(6041310)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(201708071742011)(7699016); SRVR:BYAPR07MB4902; BCL:0; PCL:0; RULEID:; SRVR:BYAPR07MB4902; X-Forefront-PRVS: 0785459C39 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(396003)(136003)(346002)(376002)(39860400002)(199004)(189003)(26005)(51416003)(52116002)(76176011)(81166006)(305945005)(81156014)(7736002)(105586002)(316002)(6506007)(106356001)(6512007)(8936002)(446003)(54906003)(25786009)(6666003)(68736007)(50226002)(110136005)(53936002)(16586007)(42882007)(386003)(5660300001)(2906002)(72206003)(50466002)(6486002)(36756003)(6116002)(107886003)(2616005)(14444005)(186003)(3846002)(478600001)(66066001)(476003)(8676002)(47776003)(486006)(575784001)(956004)(48376002)(11346002)(4326008)(97736004)(16526019); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR07MB4902; H:ajoseph83.caveonetworks.com.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BYAPR07MB4902; 23:X5WA6UlOPrGBCh75LlMfZ4TyiUkJmezRk+RO9TsJ/?= =?us-ascii?Q?J+QhJMTeq9j3E9e4BUSaD0ELQ0hAziZOGpFuAWlgDWadp1S5nrMkiKnMwCVW?= =?us-ascii?Q?PytRVfudGApQFkCkvaLx2gxLftXs9mXkPRr9GP3pjayHc2OzuUQT3w48B/e2?= =?us-ascii?Q?IJ0plR6F++X4OWlG7CmOTaR3Jwm9/9awEnktDrXaJQBDNtaLB9J1XAAPIKAz?= =?us-ascii?Q?rjIMrpXUqF6LJY6ccw3WuRHYt/2B37+Wg7Zy+umholyrvPFDknHFzzj7r7Da?= =?us-ascii?Q?YdsdE9U4dftG00T7HYwUjb5D5wjx//4l1FVAMphAHoojXH61SF2v85Ral2gv?= =?us-ascii?Q?TJpgRWp9bggpHht7oIHynjLdEEfsta7VTWvU8LhkQ2rnhfjs8vtB0BpuUceg?= =?us-ascii?Q?54V0yphn/ttruOrfUnlRHsxvluw9g1NFcrhLT6VXj2PeSvYPRrXDM9UqBePM?= =?us-ascii?Q?OHwUV/Dqtes4DveMHTcn5JL7G5sMtGS+ZND1W416TuE54JMIQEH/9y9BE4Oj?= =?us-ascii?Q?1bzmO/MbNF6CSe3LbTL+RuX+IqPhZapx92bGB1a9nVHV4+466dtyprfZoKLB?= =?us-ascii?Q?62DrYokwJdaLKOEcqltl1hF5/pHGOacGrXvOVYbzwLh99QwYKHJ5Y1dWj07D?= =?us-ascii?Q?yDlpuDP2y26DDodLOOTZtiXIIAGcl3gWgmXD9y6rCuXx7oCgknEU8L+LkcHx?= =?us-ascii?Q?zHD9Js97z+Sm5gTqPhhIxcUSgQsB+U55RAADFPYTwYD8hkWFhSLBdsM6F/bM?= =?us-ascii?Q?4CyIZL0HuH+/pB058y0qA05kMjbIE6JJ9Axmf/2AlDEUqc852cYOylw9id4s?= =?us-ascii?Q?/U64+1pGY7GAx0lqc/EjpQr/VN6mGZr2k1P5jguyvBrkD0BL2srHKgzpeVbX?= =?us-ascii?Q?mceGFmVld70qEID3xiJCqKKy7volZ9nn4KA0KklI/naAq1iSvfwgKq1+9MEe?= =?us-ascii?Q?1/rAEN3z5v7Po4g65fzDOUTnAvZx7F3ZzlMx1El8LRtgzkauFrfd0D/c+FuW?= =?us-ascii?Q?gHhEWjs2WK5NIR2g5h7MkFa/BgGYNtIWCx9bQfKjOAtE3jG1TxoURVM6DUMp?= =?us-ascii?Q?nS/LJFUA1NH6cNBLmMzE/AWsypZ1RSQDjUeFowZ6pF+5z7+97/a6hrK5HDuz?= =?us-ascii?Q?QOFFeCnyXsBG+4xh78EVFh79x071mmL6GaS/cDhlpQs4XJSOXd3nnELLp5af?= =?us-ascii?Q?KE75ppdF9U0MpT6raVCdlZ3A2pn6kntlScf2BI4epqu8cIqR4yiXJiEn9gpB?= =?us-ascii?Q?7Cg+SAffZPMrEO9I7NCJLOLxyAzHmcjN+ukvY9m?= X-Microsoft-Antispam-Message-Info: UqkITQT3npAq2y7hskooJdTK76nIDksq1ygUk5mxrXYSsD3vlathmAv2Y9J05Fzm4oDnXfdWELYQRMVngtR4ec30lQ0Nv9OxYNN6uO+BaPN3+Y/GXJ+54JYIknME+xR81bNCYWCyveqjGNJTUqK58YtoXcpaFrTIvjtgKTWkzNUmnthqiWsINwSZ+3zL58SagMQAZGnErqz5LQB+JvsdNRXQEJP2Sq5OBgWwQxS+69/VEox7OTprGbNN1SFtI0WT6FRCqU8Bm1Kx+eZ86vO24sp4SzTAyv63E/fuxvAjEIefLMRXD/J4tQ7WGm0j3JSt40ATNvvfE7tfWAMjgsdaF2OlKT1xC6WURzi3V5kKe+U= X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4902; 6:glZoSEZWBae/+TO58E1WOLBu/tuc9pvOZAglzABiiAEuOLG6m9xHuYFIbuN9Nrq3icwhFrJWyG/s/vhDcySnUI8t0Hem9cQW8YD8LJTYNUmhLV2WC7/NULwMaCV0Ri6FD7ydD624KZwqdX/FFIpE4OIQCiHynWqA0q958E69J3TTUUUkzXHF1fY+e4YYEmOvL2il4+mjF8VVtkIEWl/Ut+6ig+0+eo1fkexS1XM3RZhDcV0xXFSsbkYKT9zm58vdqM1mKT9j50LeQAQr9D9L7xiDUtoK11b2i5bdfhQGkvtzqvtBwjRAW6VKNQ0AKhNybNBjzgBOjQuwZibPV4Gd8Yi+eFoF+0xHD+XUi4oAhSfIIdVkHBkqv1i33cajwYgmAza0u74mBLu+NI02NnxMEnSN2ITIPM4ajEh4HNN9nWkKm38epZX9yyPHCqi2Ij5xZbjq5PCvtL4vtulOcypMMA==; 5:gAFypzPflZv5OvrWyoPgRkEhNAF3+6C+xyboVCXky5Si0HRZSjzfQqWvQY7SlllXo0Jbpl90dtoxISEbW6QjxiX0HKGxF8vxp96nfiDtOoSc9bJNhI4slVAvnSia8c9kEWOl2qVBVSq0Idu5zlkvKbn2lCI6HTbOIIpFIYFg75E=; 7:zEDTV6uwsxM8zsCvRqMK7dVLVxb9liinnCaB4kXfvr2d3+XzXXgQcVf7tqt8kFRw0A7W5yRJAY/tKp7cQK5kVFenhMFb1fFDbrxxEdkkVvzrc1aT+6wIYoWk+EWZWZI5AtUIGqBBLSEyusJAuan6ZQ7IL3GD4I+UvUPk57GSJf2fcGJYwkdZaNjXIcYqpG62SrpmDXrzunFFbBSvvJxujPZ5Qj1HqSbVr+SIkS8z3DgO+J4uaRESiZbBYTblsC+u SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2018 04:03:39.5067 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7bcb7fd4-08cf-4980-d48c-08d6121b67bc X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR07MB4902 Subject: [dpdk-dev] [PATCH v2 26/33] common/cpt: add support for hash X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2018 04:03:45 -0000 From: Srisivasubramanian S Adding microcode interface for supporting verify and authentication Signed-off-by: Ankur Dwivedi Signed-off-by: Anoob Joseph Signed-off-by: Murthy NSSR Signed-off-by: Nithin Dabilpuram Signed-off-by: Ragothaman Jayaraman Signed-off-by: Srisivasubramanian S Signed-off-by: Tejasree Kondoj --- drivers/common/cpt/cpt_mcode_defines.h | 6 + drivers/common/cpt/cpt_request_mgr.h | 2 + drivers/common/cpt/cpt_ucode.h | 424 ++++++++++++++++++++++++++++++++- 3 files changed, 431 insertions(+), 1 deletion(-) diff --git a/drivers/common/cpt/cpt_mcode_defines.h b/drivers/common/cpt/cpt_mcode_defines.h index 17b0de3..b580289 100644 --- a/drivers/common/cpt/cpt_mcode_defines.h +++ b/drivers/common/cpt/cpt_mcode_defines.h @@ -334,6 +334,12 @@ typedef struct fc_params { } fc_params_t; +/* + * Parameters for digest + * generate requests + * Only src_iov, op, ctx_buf, mac_buf, prep_req + * meta_buf, auth_data_len are used for digest gen. + */ typedef struct fc_params digest_params_t; /* Cipher Algorithms */ diff --git a/drivers/common/cpt/cpt_request_mgr.h b/drivers/common/cpt/cpt_request_mgr.h index c082e21..d843ee7 100644 --- a/drivers/common/cpt/cpt_request_mgr.h +++ b/drivers/common/cpt/cpt_request_mgr.h @@ -74,6 +74,8 @@ cpt_pmd_crypto_operation(struct cpt_instance *instance, if (likely(cpt_op & CPT_OP_CIPHER_MASK)) prep_req = fill_fc_params(op, sess, &mdata, &ret); + else + prep_req = fill_digest_params(op, sess, &mdata, &ret); if (unlikely(!prep_req)) { CPT_LOG_DP_ERR("prep cryto req : op %p, cpt_op 0x%x " diff --git a/drivers/common/cpt/cpt_ucode.h b/drivers/common/cpt/cpt_ucode.h index f618203..7d2b3ef 100644 --- a/drivers/common/cpt/cpt_ucode.h +++ b/drivers/common/cpt/cpt_ucode.h @@ -4,7 +4,6 @@ #ifndef _CPT_UCODE_H_ #define _CPT_UCODE_H_ - #include #include "cpt_common.h" @@ -367,6 +366,218 @@ fill_sg_comp_from_iov(sg_comp_t *list, } static __rte_always_inline int +cpt_digest_gen_prep(uint32_t flags, + uint64_t d_offs, + uint64_t d_lens, + digest_params_t *params, + void *op, + void **prep_req) +{ + struct cpt_request_info *req; + uint32_t size, i; + int32_t m_size; + uint16_t data_len, mac_len, key_len; + auth_type_t hash_type; + buf_ptr_t *meta_p; + struct cpt_ctx *ctx; + sg_comp_t *gather_comp; + sg_comp_t *scatter_comp; + uint8_t *in_buffer; + uint32_t g_size_bytes, s_size_bytes; + uint64_t dptr_dma, rptr_dma; + vq_cmd_word0_t vq_cmd_w0; + vq_cmd_word3_t vq_cmd_w3; + void *c_vaddr, *m_vaddr; + uint64_t c_dma, m_dma; + opcode_info_t opcode; + + if (!params || !params->ctx_buf.vaddr) + return ERR_BAD_INPUT_ARG; + + (void)d_offs; + ctx = params->ctx_buf.vaddr; + meta_p = ¶ms->meta_buf; + + if (!meta_p->vaddr || !meta_p->dma_addr) + return ERR_BAD_INPUT_ARG; + + if (meta_p->size < sizeof(struct cpt_request_info)) + return ERR_BAD_INPUT_ARG; + + m_vaddr = meta_p->vaddr; + m_dma = meta_p->dma_addr; + m_size = meta_p->size; + + /* + * Save initial space that followed app data for completion code & + * alternate completion code to fall in same cache line as app data + */ + m_vaddr = (uint8_t *)m_vaddr + COMPLETION_CODE_SIZE; + m_dma += COMPLETION_CODE_SIZE; + size = (uint8_t *)RTE_PTR_ALIGN((uint8_t *)m_vaddr, 16) - + (uint8_t *)m_vaddr; + c_vaddr = (uint8_t *)m_vaddr + size; + c_dma = m_dma + size; + size += sizeof(cpt_res_s_t); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + req = m_vaddr; + + size = sizeof(struct cpt_request_info); + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* Initialising ctrl and opcode fields */ + + req->dma_mode = CTRL_DMA_MODE_SGIO; + req->se_req = SE_CORE_REQ; + + hash_type = ctx->hash_type; + mac_len = ctx->mac_len; + key_len = ctx->auth_key_len; + data_len = AUTH_DLEN(d_lens); + + /*GP op header */ + vq_cmd_w0.u64 = 0; + vq_cmd_w0.s.param2 = rte_cpu_to_be_16(((uint16_t)hash_type << 8)); + if (ctx->hmac) { + opcode.s.major = CPT_MAJOR_OP_HMAC | CPT_DMA_MODE; + vq_cmd_w0.s.param1 = rte_cpu_to_be_16(key_len); + vq_cmd_w0.s.dlen = + rte_cpu_to_be_16((data_len + ROUNDUP8(key_len))); + } else { + opcode.s.major = CPT_MAJOR_OP_HASH | CPT_DMA_MODE; + vq_cmd_w0.s.param1 = 0; + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(data_len); + /* This is required for null auth only case */ + if (!hash_type && !ctx->enc_cipher) + vq_cmd_w0.s.param2 = rte_cpu_to_be_16(data_len); + } + + opcode.s.minor = 0; + + vq_cmd_w0.s.opcode = rte_cpu_to_be_16(opcode.flags); + + /* DPTR has SG list */ + in_buffer = m_vaddr; + dptr_dma = m_dma; + + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* TODO Add error check if space will be sufficient */ + gather_comp = (sg_comp_t *)((uint8_t *)m_vaddr + 8); + + /* + * Input gather list + */ + + i = 0; + + if (ctx->hmac) { + uint64_t k_dma = params->ctx_buf.dma_addr + + offsetof(struct cpt_ctx, auth_key); + /* Key */ + i = fill_sg_comp(gather_comp, i, k_dma, + ctx->auth_key, ROUNDUP8(key_len)); + } + + /* input data */ + size = data_len; + if (size) { + i = fill_sg_comp_from_iov(gather_comp, i, params->src_iov, + 0, &size, NULL, 0); + if (size) { + CPT_LOG_DP_DEBUG("Insufficient dst IOV size, short" + " by %dB\n", size); + return ERR_BAD_INPUT_ARG; + } + } else { + /* + * Looks like we need to support zero data + * gather ptr in case of hash & hmac + */ + i++; + } + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + g_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + + /* + * Output Gather list + */ + + i = 0; + scatter_comp = + (sg_comp_t *)((uint8_t *)gather_comp + g_size_bytes); + + if (flags & VALID_MAC_BUF) { + if (params->mac_buf.size < mac_len) + return ERR_BAD_INPUT_ARG; + + size = mac_len; + i = fill_sg_comp_from_buf_min(scatter_comp, i, + ¶ms->mac_buf, &size); + } else { + size = mac_len; + i = fill_sg_comp_from_iov(scatter_comp, i, + params->src_iov, data_len, + &size, NULL, 0); + if (size) { + CPT_LOG_DP_DEBUG("Insufficient dst IOV size, short by" + " %dB\n", size); + return ERR_BAD_INPUT_ARG; + } + } + + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + s_size_bytes = ((i + 3) / 4) * sizeof(sg_comp_t); + + size = g_size_bytes + s_size_bytes + SG_LIST_HDR_SIZE; + + /* This is DPTR len incase of SG mode */ + vq_cmd_w0.s.dlen = rte_cpu_to_be_16(size); + + m_vaddr = (uint8_t *)m_vaddr + size; + m_dma += size; + m_size -= size; + + /* cpt alternate completion address saved earlier */ + req->alternate_caddr = (uint64_t *)((uint8_t *)c_vaddr - 8); + *req->alternate_caddr = ~((uint64_t)COMPLETION_CODE_INIT); + rptr_dma = c_dma - 8; + + req->ist.ei1 = dptr_dma; + req->ist.ei2 = rptr_dma; + /* First 16-bit swap then 64-bit swap */ + /* TODO: HACK: Reverse the vq_cmd and cpt_req bit field definitions + * to eliminate all the swapping + */ + vq_cmd_w0.u64 = rte_cpu_to_be_64(vq_cmd_w0.u64); + + /* vq command w3 */ + vq_cmd_w3.u64 = 0; + + /* 16 byte aligned cpt res address */ + req->completion_addr = (uint64_t *)((uint8_t *)c_vaddr); + *req->completion_addr = COMPLETION_CODE_INIT; + req->comp_baddr = c_dma; + + /* Fill microcode part of instruction */ + req->ist.ei0 = vq_cmd_w0.u64; + req->ist.ei3 = vq_cmd_w3.u64; + + req->op = op; + + *prep_req = req; + return 0; +} + +static __rte_always_inline int cpt_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, @@ -2306,6 +2517,9 @@ cpt_fc_enc_hmac_prep(uint32_t flags, uint64_t d_offs, uint64_t d_lens, } else if (fc_type == KASUMI) { ret = cpt_kasumi_enc_prep(flags, d_offs, d_lens, fc_params, op, &prep_req); + } else if (fc_type == HASH_HMAC) { + ret = cpt_digest_gen_prep(flags, d_offs, d_lens, + fc_params, op, &prep_req); } else { ret = ERR_EIO; } @@ -3195,4 +3409,212 @@ instance_session_cfg(struct rte_crypto_sym_xform *xform, void *sess) return -1; } +static __rte_always_inline void +find_kasumif9_direction_and_length(uint8_t *src, + uint32_t counter_num_bytes, + uint32_t *addr_length_in_bits, + uint8_t *addr_direction) +{ + uint8_t found = 0; + while (!found && counter_num_bytes > 0) { + counter_num_bytes--; + if (src[counter_num_bytes] == 0x00) + continue; + if (src[counter_num_bytes] == 0x80) { + *addr_direction = src[counter_num_bytes - 1] & 0x1; + *addr_length_in_bits = counter_num_bytes * 8 - 1; + found = 1; + } else { + int i = 0; + uint8_t last_byte = src[counter_num_bytes]; + for (i = 0; i < 8 && found == 0; i++) { + if (last_byte & (1 << i)) { + *addr_direction = (last_byte >> (i+1)) + & 0x1; + if (i != 6) + *addr_length_in_bits = + counter_num_bytes * 8 + + (8 - (i + 2)); + else + *addr_length_in_bits = + counter_num_bytes * 8; + + found = 1; + } + } + } + } +} +/* + * This handles all auth only except AES_GMAC + */ +static __rte_always_inline void * +fill_digest_params(struct rte_crypto_op *cop, + struct cpt_sess_misc *sess, + void **mdata_ptr, + int *op_ret) +{ + uint32_t space = 0; + struct rte_crypto_sym_op *sym_op = cop->sym; + void *mdata; + phys_addr_t mphys; + uint64_t *op; + uint32_t auth_range_off; + uint32_t flags = 0; + uint64_t d_offs = 0, d_lens; + void *prep_req = NULL; + struct rte_mbuf *m_src, *m_dst; + uint16_t auth_op = sess->cpt_op & CPT_OP_AUTH_MASK; + uint8_t zsk_flag = sess->zsk_flag; + uint16_t mac_len = sess->mac_len; + fc_params_t params; + char src[SRC_IOV_SIZE]; + uint8_t iv_buf[16]; + memset(¶ms, 0, sizeof(fc_params_t)); + struct cptvf_meta_info *cpt_m_info = + (struct cptvf_meta_info *)(*mdata_ptr); + + m_src = sym_op->m_src; + + /* For just digest lets force mempool alloc */ + mdata = alloc_op_meta(NULL, ¶ms.meta_buf, cpt_m_info->cptvf_op_mlen, + cpt_m_info->cptvf_meta_pool); + if (mdata == NULL) { + CPT_LOG_DP_ERR("Error allocating meta buffer for" + " request\n"); + *op_ret = -ENOMEM; + return NULL; + } + + mphys = params.meta_buf.dma_addr; + + op = mdata; + op[0] = (uintptr_t)mdata; + op[1] = (uintptr_t)cop; + op[2] = op[3] = 0; /* Used to indicate auth verify */ + space += 4 * sizeof(uint64_t); + + auth_range_off = sym_op->auth.data.offset; + + flags = VALID_MAC_BUF; + params.src_iov = (void *)src; + if (unlikely(zsk_flag)) { + /* + * Since for Zuc, Kasumi, Snow3g offsets are in bits + * we will send pass through even for auth only case, + * let MC handle it + */ + d_offs = auth_range_off; + auth_range_off = 0; + params.auth_iv_buf = rte_crypto_op_ctod_offset(cop, + uint8_t *, sess->auth_iv_offset); + if (zsk_flag == K_F9) { + uint32_t length_in_bits, num_bytes; + uint8_t *src, direction = 0; + uint32_t counter_num_bytes; + + memcpy(iv_buf, rte_pktmbuf_mtod(cop->sym->m_src, + uint8_t *), 8); + /* + * This is kasumi f9, take direction from + * source buffer + */ + length_in_bits = cop->sym->auth.data.length; + num_bytes = (length_in_bits >> 3); + counter_num_bytes = num_bytes; + src = rte_pktmbuf_mtod(cop->sym->m_src, uint8_t *); + find_kasumif9_direction_and_length(src, + counter_num_bytes, + &length_in_bits, + &direction); + length_in_bits -= 64; + cop->sym->auth.data.offset += 64; + d_offs = cop->sym->auth.data.offset; + auth_range_off = d_offs / 8; + cop->sym->auth.data.length = length_in_bits; + + /* Store it at end of auth iv */ + iv_buf[8] = direction; + params.auth_iv_buf = iv_buf; + } + } + + d_lens = sym_op->auth.data.length; + + params.ctx_buf.vaddr = SESS_PRIV(sess); + params.ctx_buf.dma_addr = sess->ctx_dma_addr; + + if (auth_op == CPT_OP_AUTH_GENERATE) { + if (sym_op->auth.digest.data) { + /* + * Digest to be generated + * in separate buffer + */ + params.mac_buf.size = + sess->mac_len; + params.mac_buf.vaddr = + sym_op->auth.digest.data; + params.mac_buf.dma_addr = + sym_op->auth.digest.phys_addr; + } else { + uint32_t off = sym_op->auth.data.offset + + sym_op->auth.data.length; + int32_t dlen, space; + + m_dst = sym_op->m_dst ? + sym_op->m_dst : sym_op->m_src; + dlen = rte_pktmbuf_pkt_len(m_dst); + + space = off + mac_len - dlen; + if (space > 0) + if (!rte_pktmbuf_append(m_dst, space)) { + CPT_LOG_DP_ERR("Failed to extend " + "mbuf by %uB\n", space); + goto err; + } + + params.mac_buf.vaddr = + rte_pktmbuf_mtod_offset(m_dst, + void *, off); + params.mac_buf.dma_addr = + rte_pktmbuf_mtophys_offset(m_dst, off); + params.mac_buf.size = mac_len; + } + } else { + /* Need space for storing generated mac */ + params.mac_buf.vaddr = + (uint8_t *)mdata + space; + params.mac_buf.dma_addr = mphys + space; + params.mac_buf.size = mac_len; + space += RTE_ALIGN_CEIL(mac_len, 8); + op[2] = (uintptr_t)params.mac_buf.vaddr; + op[3] = mac_len; + + } + + params.meta_buf.vaddr = (uint8_t *)mdata + space; + params.meta_buf.dma_addr = mphys + space; + params.meta_buf.size -= space; + + /* Out of place processing */ + params.src_iov = (void *)src; + + /*Store SG I/O in the api for reuse */ + if (prepare_iov_from_pkt(m_src, params.src_iov, + auth_range_off)) { + CPT_LOG_DP_ERR("Prepare src iov failed\n"); + *op_ret = -1; + goto err; + } + + prep_req = cpt_fc_enc_hmac_prep(flags, d_offs, d_lens, + ¶ms, op, op_ret); + *mdata_ptr = mdata; + return prep_req; +err: + if (unlikely(!prep_req)) + free_op_meta(mdata, cpt_m_info->cptvf_meta_pool); + return NULL; +} + #endif /*_CPT_UCODE_H_ */ -- 2.7.4