From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50059.outbound.protection.outlook.com [40.107.5.59]) by dpdk.org (Postfix) with ESMTP id 00D371B995 for ; Fri, 12 Oct 2018 16:41:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=osT6DYesv9+/5G8G9d+BL3b4qqVmZ8XrstpXFpHKPog=; b=NYDycdnONy8fpctEUILqz0n8Ci3G5aRmhMQLfCYanafcl1g9dtDNjX4KyfP9ERThn2PCz9kQOkoZ1w2WEt2ZHLCtiH5XZhCZYcwRSmJcBy6zPWzH8oD80LIDGUnyW/62QjQJ/QwHJIEAJi/+DCc6M5IvrnoUxbIsfIPr9ju6XEI= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=G.Singh@nxp.com; Received: from Tophie.ap.freescale.net (14.142.187.166) by HE1PR04MB1530.eurprd04.prod.outlook.com (2a01:111:e400:59a8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1207.23; Fri, 12 Oct 2018 14:41:51 +0000 From: Gagandeep Singh To: dev@dpdk.org, akhil.goyal@nxp.com Cc: Gagandeep Singh , Hemant Agrawal Date: Fri, 12 Oct 2018 20:10:52 +0530 Message-Id: <20181012144055.9461-12-g.singh@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181012144055.9461-1-g.singh@nxp.com> References: <20180913060846.29930-1-g.singh@nxp.com> <20181012144055.9461-1-g.singh@nxp.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: PN1PR0101CA0039.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:c::25) To HE1PR04MB1530.eurprd04.prod.outlook.com (2a01:111:e400:59a8::20) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fd11b171-c323-4f43-9afe-08d63050d9ee X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:HE1PR04MB1530; X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1530; 3:kOA+pLj9vhg2E4MypRknBO4u+bLLznON9a0V1rKSMGeTPyUu+vNLctMC9YztFNnuwev59gXZKcPAQgn+N1zhTMsI+J88nEBmBa4UucspbViOcR+G5uFqUnW0jNAeUGpxiEMFVT8y9BQsGBzR1zVYNM5c/Zf0JkuoKLuB/LX3wTesBiivyg5noGITEJqR1h7aCd6KPXwdP/2JJAp/dEvaOAHtQ51mOMa+qoSlr/i0RoN0g/z6fg5oGe/bvLIHqEHu; 25:3+nRoGADRtYc8WTJHQtXO0o/5mnwIPK+BcCy/fYu7TikK/fnEjbdPFwtXpShr7uEfVDG0qf8Nhg9Va4vJJm3Bh/KFse0tZ0VropL4f6RtszKldoDxyf8/acgu3AXh88GU0p6kBP7d+C1rb7o89O26sbTGWKVBusiGCzWtDxwMyvp6m3ZbsE3jgvaooCX9hMPh5w0G42/4Mh3zBGAgHoGPVJC/OW+3Zim7TJpzDHHdV9i8oasL4QExivWnlj1diFVXbJBZJM7FthaDuDk9sih4sEIrsE2QF49NwgeJ/+nCWEXkHm6rWs2Qb8fb8DiJMXkChd+SL2GhLWWmKB7ux184w==; 31:t2DxkEfKtGtnGczEz8y+mlLUEcjv+zK+hG6dIrI9fzaonZIMIZZhx3DGWXKGGlkUli3xPRDFhFeJFXbOfoeu6B4qA/AUFpDxsnOsbqYTNRDtPsNW3kVGu490Fc7j4ugS2jRyno9h0zhFwCj+LRphKtI7DsSILiu5vWKlXf5qUYpk2NiIE3dF7qOh8CDtMMlLT0bCrDN9ahwnzO8QqscZ8fj/CtpluUM7+PqHNDVQSfc= X-MS-TrafficTypeDiagnostic: HE1PR04MB1530: X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1530; 20:ahkSYG3javfBHjvqFEauuggjXg2aWKWfbDZFqLL6hMeL0KzaH4hJ5xJPV0N1r5vDTZboHg2n9lvozTm7Qhi6g/cmL2KAPF+Tu7siGGJOqMwzs+F7MVuX1A5l8S9gWzNwwJxAz+z9f14iNCXIEMGRrB3GG9vWEHA36hH3E/7kYtWH0KiCANP1X80iqHhH/892YtQsZQ8RShCDLcQ5dSNXWhSHCjCvIY0iey7+XX5mULAops4At6Nh5vIAkNVY95Um544LWNix5wtvxL41Eg2xWtE+k4gf3nOH9qNO40wdwB0+GQigSGb8yRv5wTXzUXG95PLdJLll7qjzU7N3OFAgAGEj77WDreaq0se2KlNyjSfz8LlTvuojM8zXLfSD4gK9x3+xbLqBF1AVsKC0086ZvOMPRF0Y3blvYAh7gMFFsyKCfoKr96QZcSyLqQP407gMECJPiHnEozVdK25uTwBkn6YMqiy2ymsp+uUrA+K/W/dMncNJleD3J/Ebvdc0rNIu; 4:KE+Ggvzxa/kz7YiA8hS/O9FRyNX7/P3qphGyKs3u2M2e0nPl/iivC4VAYKPOs2pXy5VnFdkvtYlT7/qPky19bur3Pieyy5rkr2o69uiSxVJDADZKdy4D0YPdOyesrYwvVGrjvs+pQuWWhjHYHrhUJKcr7PotSLueRJndky4QpeahhNuT3xlVJpVhYMADzligiCNtFL2BGFfwDNtQ7+Ebmo7upQeRjfCFpvmXAMruBKdbdT3eO3kmO7gYf3T7b58h8WgznMOW/GG3Rjamdepqp1/CHoz69g/K2v3kP32Fi6VLSEVJnYqAVN4K2HrH4+ez X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(3231355)(944501410)(52105095)(10201501046)(93006095)(93001095)(6055026)(149066)(150057)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123558120)(20161123562045)(201708071742011)(7699051); SRVR:HE1PR04MB1530; BCL:0; PCL:0; RULEID:; SRVR:HE1PR04MB1530; X-Forefront-PRVS: 0823A5777B X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(346002)(376002)(366004)(396003)(136003)(39860400002)(189003)(199004)(316002)(305945005)(8936002)(386003)(55236004)(6636002)(6506007)(106356001)(25786009)(36756003)(4744004)(16586007)(5660300001)(72206003)(52116002)(53936002)(26005)(6512007)(478600001)(54906003)(68736007)(186003)(16526019)(50226002)(7736002)(105586002)(51416003)(97736004)(76176011)(78486009)(3846002)(2906002)(66066001)(1076002)(4326008)(956004)(446003)(2616005)(47776003)(11346002)(1006002)(6116002)(486006)(14444005)(476003)(8676002)(50466002)(48376002)(81156014)(86362001)(81166006)(6486002)(110426005); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR04MB1530; H:Tophie.ap.freescale.net; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR04MB1530; 23:QHPwBZVN3rwyw/tFUIW2HXJb6MClEWebIBwqzaUfn?= =?us-ascii?Q?+zHbTgjHp+rzfkzXT+cQbQZsoY+UBB2tGGCUa3igO3QhNF27bav9Iejqhv/9?= =?us-ascii?Q?5GU9jLL5M1xduOfmJKrLYJnbXByBx/BLaUCYqczqUfHn2PeEkR/DL5l2DgkV?= =?us-ascii?Q?a+8PG797+3syENjzaTliRVL9yJTYdOpad7ef6/wwad+3D5C52U4kpPkV0jxe?= =?us-ascii?Q?zfAMMP+RTj36n7++z31SwIocQHKA6Qz6EWB/Gxj1NOFUJlxaU/+jqKIexe0P?= =?us-ascii?Q?34aeihrsMXctMCA1ZXdFSeDxj0EjgqBw65e9tP80DTqa03E69IvQ3lUCAcF5?= =?us-ascii?Q?+D2x8q4vGCWdMhHnalXM6WLDu69qGPuIMDjdeNwhZIV31JIDDzGr4y2HD/Qt?= =?us-ascii?Q?5hY51pl4cPbH611kIuI6w6mKBA3WnmOgAJQkmj7zC5mzx6BDmsj9HhtjxFn0?= =?us-ascii?Q?z0e5ABSiAkfM6vo3NtAkwKJfg8BOTMaP8by9jZa0RTHa71LnS8cgUQ32G33a?= =?us-ascii?Q?xUmWpwcAMoaiBh5JAUyOclYBtRSG5rZEylAMDO/q4z6WYdk83Rm6JgAFyAx3?= =?us-ascii?Q?iXVxnuYB7ykZE9Nv37XNzYaXcVCiDhvWgaISnE8G1g4DyTPr1yJh/6xCz9XH?= =?us-ascii?Q?BBT8H/9kuSzXAerjgiiWquU2R52msw9b38r6DtuMjzhQFOJQSxcgGWX3i28L?= =?us-ascii?Q?01fu7lxqHpFOy8yQydPcSgzdK3aMaH3teHC1Rf4PKdO4XmayvkrZqm2reJGq?= =?us-ascii?Q?tvGcu32gv7WD9H267Ha+QUxJyaZt1vywRbU13qUWy4Z0SgHvLZsR8LY5PhiF?= =?us-ascii?Q?eTG0CkEDRV9wVvd6TgPKJblRJ4PxFUJXinnJ0faPDF/QgC4qGDQjG+vmIei1?= =?us-ascii?Q?ZZ19O6bN2uPFiH4an6qdLFVw5dF58b8AUQgRwARQ72p/KudX2TDmCBXTtJrQ?= =?us-ascii?Q?o9GQ+netRtoPM0ynLYeH75tlyKjY619RbzYivlqAF5Pa29ikqZ3iKKFHdFyY?= =?us-ascii?Q?wvuJRni/dr9OWaRXtoDtCHO8guqa5PoPhyRHY2xMXDildJXxY/7WGUadHIGW?= =?us-ascii?Q?/m3uLjdFDksf2/s78+7yUKiwM8+N8xx/T6FanScH1hSZ3lildcGWkyV1nVlr?= =?us-ascii?Q?opCJ5iARjlMQw0Kip4svwAGjf9W26fFLMNU2TvJ40a4+gsVEzq4MbcnLdkLk?= =?us-ascii?Q?i0PMqtdBhtw5w47wf9Rwqw1Kv4jkbCpKHtIGFWC/A8XnYLDJw/pwl2cYqCk8?= =?us-ascii?Q?i96XsvFlj0nYX3QIP/TU4CokGqVJyhJC2osKzq4YxEzGlUZ2dyc3k59I93RP?= =?us-ascii?Q?wB76MNwoC+mM54IqnWB8sBKRuZPLb19aADVZW66kU7u?= X-Microsoft-Antispam-Message-Info: RxssKZun8ehHdP8volWWA0wnRPzW8FOTINVl6fvWIX0BTx1YBFj5O9ga3ZV2N3fCk9+XXhmCFO/Plu81SqZJpkph/fyz8aELw+qPip6G/JI8t9p6mkm8ooh0JLQFuAGOHYZw2PHtkW/uK4PkXPopuQAVFEejubYSKVUqBTDOYHWqVLIyL+IxBetzhJOqH81nZz3r4kGI6u14GXKCrbaPqAcHAd7n38YbTDZoQosIdnFbKuvme60VPPry2NCpD4Z6nUSqiCeOWYVqN5iRVK9BqUKWuA2QwPlA+KJltCU0feL3pq88VTQ2Y+o+Ri81uSj5AZ4j31pnvEpurpgAC28Xeg6aWR6mNcjhQbpj1VIChG8= X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1530; 6:zz/kg4e1YmgiglFp6AqNVnErLhKBysjj0mcokiooZ7jRDPotrtxIRSP5SZ3XslmPHxL3opTnubE5RyCk1JVuvFVdyjdp1g9ZUluFh5Gfcg0zump7GHfIvKns9K04YAdxa8yVKmenkNlcKPrGokBdYmTYuJoBj81LqBD2qfY2wADddoGuSGaP1iEE0A8G7L2iAe3Oy7AT9UYibJVwleyBLcBS+91rj+JLFRsv3/Czok3+sFIjOL5HVEod51ZD/HOSsl1La4BIDaEIc5urkYRn45mDNu0ZFMxTVzBsbQp3ngC/nYLRNm2KLlW5YGU3SQQSyKqfkNa0cekbvGj654rd5wNW/54Pvn0lBHe3e8B1Mzkt0JvvQGPPAbu+dNg96O/CgE75m4CZDjSAvkKw3kBLAVjuAiDv78Di0TPp766mk/eG6520OdzmUb4lJZZiWNGQuEunXAwBa5iAHF7eKjKjsw==; 5:/auc8atWatX+yda7uY7zR9P+Zc1pqe1xjgDxm5CCbtTRpP2ppG/wnwue/uA0ZL2I+ft71CqUo5yDUXdHbsrjpnj4FWFeI4KTOeaTKK1G78L9zYM8d/Bk1GZH2IkKCvYGwhukvAVH+0wBx3jA3YAUgF55oFOOSQ2bmXhFyWJSXd0=; 7:tpK4TCVWygBJb91Np/i+Jn1t878NLL9ZQYYt23wE07fXOOQ4sfPovGJ4avubkz5fRi+ty/VNw+AyBKbMjSlim9OXNDL/42rpRqKj99dKpsB5fbLqytVvKI/IsblRTePnNpIGhpxZDvUiLvQeWWpBcR3mB9f7WHJPejjZ8eiviv4HH59gRRQYCorRwp7Mt3tM4JLivzi7cjPz9DoRdH7P1uwhT0rygkSNYySuPmLMy6a/7yzAZ5gmrAPpIGjdjJaN SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Oct 2018 14:41:51.9282 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fd11b171-c323-4f43-9afe-08d63050d9ee X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR04MB1530 Subject: [dpdk-dev] [PATCH v2 11/14] crypto/caam_jr: add scatter gather X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Oct 2018 14:41:55 -0000 This patch add the scatter gather feature for auth-only, cipher-only and cipher-auth operations Signed-off-by: Gagandeep Singh Signed-off-by: Hemant Agrawal --- drivers/crypto/caam_jr/caam_jr.c | 375 ++++++++++++++++++++++++++++++- 1 file changed, 374 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/caam_jr/caam_jr.c b/drivers/crypto/caam_jr/caam_jr.c index c0933a750..a4118cd11 100644 --- a/drivers/crypto/caam_jr/caam_jr.c +++ b/drivers/crypto/caam_jr/caam_jr.c @@ -623,6 +623,101 @@ caam_jr_dequeue_burst(void *qp, struct rte_crypto_op **ops, return num_rx; } +/** + * packet looks like: + * |<----data_len------->| + * |ip_header|ah_header|icv|payload| + * ^ + * | + * mbuf->pkt.data + */ +static inline struct caam_jr_op_ctx * +build_auth_only_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct rte_mbuf *mbuf = sym->m_src; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + int length; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + struct sec_job_descriptor_t *jobdescr; + uint8_t extra_segs; + + PMD_INIT_FUNC_TRACE(); + if (is_decode(ses)) + extra_segs = 2; + else + extra_segs = 1; + + if ((mbuf->nb_segs + extra_segs) > MAX_SG_ENTRIES) { + CAAM_JR_DP_ERR("Auth: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* output */ + SEC_JD_SET_OUT_PTR(jobdescr, (uint64_t)sym->auth.digest.phys_addr, + 0, ses->digest_length); + + /*input */ + sg = &ctx->sg[0]; + length = sym->auth.data.length; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + sym->auth.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->auth.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + + if (is_decode(ses)) { + /* digest verification case */ + sg++; + /* hash result or digest, save digest first */ + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); +#if CAAM_JR_DBG + rte_hexdump(stdout, "ICV", ctx->digest, ses->digest_length); +#endif + sg->ptr = cpu_to_caam64(caam_jr_vtop_ctx(ctx, ctx->digest)); + sg->len = cpu_to_caam32(ses->digest_length); + length += ses->digest_length; + } else { + length -= ses->digest_length; + } + + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_IN_PTR(jobdescr, + (uint64_t)caam_jr_vtop_ctx(ctx, &ctx->sg[0]), 0, length); + /* enabling sg list */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + return ctx; +} + static inline struct caam_jr_op_ctx * build_auth_only(struct rte_crypto_op *op, struct caam_jr_session *ses) { @@ -688,6 +783,123 @@ build_auth_only(struct rte_crypto_op *op, struct caam_jr_session *ses) return ctx; } +static inline struct caam_jr_op_ctx * +build_cipher_only_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct rte_mbuf *mbuf = sym->m_src; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg, *in_sg; + int length; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + uint8_t reg_segs; + + PMD_INIT_FUNC_TRACE(); + if (sym->m_dst) { + mbuf = sym->m_dst; + reg_segs = mbuf->nb_segs + sym->m_src->nb_segs + 2; + } else { + mbuf = sym->m_src; + reg_segs = mbuf->nb_segs * 2 + 2; + } + + if (reg_segs > MAX_SG_ENTRIES) { + CAAM_JR_DP_ERR("Cipher: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + +#if CAAM_JR_DBG + CAAM_JR_INFO("mbuf offset =%d, cipher offset = %d, length =%d+%d", + sym->m_src->data_off, sym->cipher.data.offset, + sym->cipher.data.length, ses->iv.length); +#endif + /* output */ + if (sym->m_dst) + mbuf = sym->m_dst; + else + mbuf = sym->m_src; + + sg = &ctx->sg[0]; + length = sym->cipher.data.length; + + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->cipher.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->cipher.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_OUT_PTR(jobdescr, + (uint64_t)caam_jr_vtop_ctx(ctx, &ctx->sg[0]), 0, + length); + /*enabling sg bit */ + (jobdescr)->seq_out.command.word |= 0x01000000; + + /*input */ + sg++; + mbuf = sym->m_src; + in_sg = sg; + + length = sym->cipher.data.length + ses->iv.length; + + /* IV */ + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + + /* 1st seg */ + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->cipher.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->cipher.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_vtop_ctx(ctx, in_sg), 0, + length); + /*enabling sg bit */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + return ctx; +} + static inline struct caam_jr_op_ctx * build_cipher_only(struct rte_crypto_op *op, struct caam_jr_session *ses) { @@ -752,6 +964,161 @@ build_cipher_only(struct rte_crypto_op *op, struct caam_jr_session *ses) return ctx; } +/* For decapsulation: + * Input: + * +----+----------------+--------------------------------+-----+ + * | IV | Auth-only data | Authenticated & Encrypted data | ICV | + * +----+----------------+--------------------------------+-----+ + * Output: + * +----+--------------------------+ + * | Decrypted & authenticated data | + * +----+--------------------------+ + */ + +static inline struct caam_jr_op_ctx * +build_cipher_auth_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg, *out_sg, *in_sg; + struct rte_mbuf *mbuf; + uint32_t length = 0; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t req_segs; + uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + uint32_t auth_only_len; + + PMD_INIT_FUNC_TRACE(); + auth_only_len = op->sym->auth.data.length - + op->sym->cipher.data.length; + + if (sym->m_dst) { + mbuf = sym->m_dst; + req_segs = mbuf->nb_segs + sym->m_src->nb_segs + 3; + } else { + mbuf = sym->m_src; + req_segs = mbuf->nb_segs * 2 + 3; + } + + if (req_segs > MAX_SG_ENTRIES) { + CAAM_JR_DP_ERR("Cipher-Auth: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* output */ + if (sym->m_dst) + mbuf = sym->m_dst; + else + mbuf = sym->m_src; + + out_sg = &ctx->sg[0]; + if (is_encode(ses)) + length = sym->auth.data.length + ses->digest_length; + else + length = sym->auth.data.length; + + sg = &ctx->sg[0]; + + /* 1st seg */ + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->auth.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->auth.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + + if (is_encode(ses)) { + /* set auth output */ + sg++; + sg->ptr = cpu_to_caam64(sym->auth.digest.phys_addr); + sg->len = cpu_to_caam32(ses->digest_length); + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_OUT_PTR(jobdescr, + (uint64_t)caam_jr_dma_vtop(out_sg), 0, length); + /* set sg bit */ + (jobdescr)->seq_out.command.word |= 0x01000000; + + /* input */ + sg++; + mbuf = sym->m_src; + in_sg = sg; + if (is_encode(ses)) + length = ses->iv.length + sym->auth.data.length; + else + length = ses->iv.length + sym->auth.data.length + + ses->digest_length; + + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + + sg++; + /* 1st seg */ + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->auth.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->auth.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + + if (is_decode(ses)) { + sg++; + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(ctx->digest)); + sg->len = cpu_to_caam32(ses->digest_length); + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_dma_vtop(in_sg), 0, + length); + /* set sg bit */ + (jobdescr)->seq_in.command.word |= 0x01000000; + /* Auth_only_len is set as 0 in descriptor and it is + * overwritten here in the jd which will update + * the DPOVRD reg. + */ + if (auth_only_len) + /* set sg bit */ + (jobdescr)->dpovrd = 0x80000000 | auth_only_len; + + return ctx; +} + static inline struct caam_jr_op_ctx * build_cipher_auth(struct rte_crypto_op *op, struct caam_jr_session *ses) { @@ -899,8 +1266,14 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) else if (is_cipher_only(ses)) ctx = build_cipher_only(op, ses); } else { - if (is_aead(ses)) + if (is_auth_cipher(ses)) + ctx = build_cipher_auth_sg(op, ses); + else if (is_aead(ses)) goto err1; + else if (is_auth_only(ses)) + ctx = build_auth_only_sg(op, ses); + else if (is_cipher_only(ses)) + ctx = build_cipher_only_sg(op, ses); } err1: if (unlikely(!ctx)) { -- 2.17.1