From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0043.outbound.protection.outlook.com [104.47.38.43]) by dpdk.org (Postfix) with ESMTP id 73063A49F for ; Thu, 11 Jan 2018 12:37:12 +0100 (CET) Received: from MWHPR03CA0021.namprd03.prod.outlook.com (2603:10b6:300:117::31) by SN2PR03MB2368.namprd03.prod.outlook.com (2603:10b6:804:e::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Thu, 11 Jan 2018 11:37:10 +0000 Received: from BL2FFO11FD015.protection.gbl (2a01:111:f400:7c09::139) by MWHPR03CA0021.outlook.office365.com (2603:10b6:300:117::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.386.5 via Frontend Transport; Thu, 11 Jan 2018 11:37:10 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; NXP1.onmicrosoft.com; dkim=none (message not signed) header.d=none;NXP1.onmicrosoft.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BL2FFO11FD015.mail.protection.outlook.com (10.173.160.223) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.20.345.12 via Frontend Transport; Thu, 11 Jan 2018 11:36:37 +0000 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id w0BBb0ss031996; Thu, 11 Jan 2018 04:37:07 -0700 From: Akhil Goyal To: CC: , , Akhil Goyal Date: Thu, 11 Jan 2018 17:03:04 +0530 Message-ID: <20180111113304.19709-4-akhil.goyal@nxp.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20180111113304.19709-1-akhil.goyal@nxp.com> References: <20171213135659.32648-1-akhil.goyal@nxp.com> <20180111113304.19709-1-akhil.goyal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131601441979720513; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(376002)(39860400002)(39380400002)(346002)(396003)(2980300002)(1109001)(1110001)(339900001)(199004)(189003)(51416003)(8656006)(6666003)(2950100002)(575784001)(6916009)(5660300001)(4326008)(85426001)(86362001)(77096006)(47776003)(81166006)(305945005)(106466001)(8676002)(81156014)(356003)(498600001)(50466002)(68736007)(97736004)(2906002)(36756003)(50226002)(1076002)(8936002)(105606002)(76176011)(59450400001)(48376002)(2351001)(316002)(104016004)(16586007)(54906003)(53936002)(5890100001)(296002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN2PR03MB2368; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD015; 1:sKeD5p+z2sWsnfiT1Qx/7PrKkJBH9UazPUd8xH5ucgL7JQqT4xkHJxABhC1hktItQkSVdkWilz8iJaQuOEsnlIeE7CcmXcvCzkGwA2GiM0gBy623Bh++nfAleMHsdJhP MIME-Version: 1.0 Content-Type: text/plain X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 05556146-5715-4067-6341-08d558e79398 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020065)(5600026)(4604075)(2017052603307); SRVR:SN2PR03MB2368; X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB2368; 3:8Kl7kpAw/R5LnE2SaN65qL6LMVCedDX0imzLrSB0NqZ5kFrNYNbEjHikDrux/rX+vCUE7W8uhQv9owyNcChVfmk5qIIRc9Xrqm4ryZl/GYu+ARccc5j8dHmsuWozYFHTD/9dSnNAQ9b7/B3NNPOLCegzv74LKtDP77U/f1LfomXKrxXTLEOwUDBhtb/9XLVHThomWFeSq69ga0tSUWkniP+zsg8Avq8sp8DoBlxg/57nvs0XBxnYACW53OZizmZ+V2VVGdJb7hIkj2jpgMiMdVPjkQcPdFNfz61NQIpMBQJpUBQbOSUeC4vMESDkVfmiHUyCB2MIqOVmDdSScr/+3VYE1RCW9Wxgl1cBGRV359s=; 25:pXAiHgaZuvfJZ0UHn2ZL7PdvMSERi/Kq5rStqig5bD3FIob4d6PUxQcVCfSXVXx8+gKCj1JIzDkzkbu2gVkMal/6lnLiJ3d3gjV8ESaryCael/F2IBe9fMnDxQ8SHpN+WQbOzHlRCdfIoFUSZEzCoqa+nvvbypB1n+/YLJkDwQ42eFlf8REqKjgTunQ3/YdZy+jOveauDr2se8nnZbaBaX98MiN4s4SNprXWQP9RERoiaZ5q55sU9EdIRD7Sb9+iCJM8RzHn+BaIb2SZTw4geIfBgi1pz6G6Be28uA3VXJzSVaBj3hb5m9p9s5AZL8Ya5axMncqgB92TWZlQcKzbug== X-MS-TrafficTypeDiagnostic: SN2PR03MB2368: X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB2368; 31:SVLOWlWmHOzePKFKL8CGsHHaqW+yiaQBfjWIdbSIUu7nJarbttPLgUH0gM3cq6/h+T8bwp9ivIToE3oguEaO3h3m1TAI6cdumJyLIiOowi2mSRadZKLiPS3UPbOuYxPKpw6mmGmV2tSay6ehzgmDB0LvAa8LdddNVSzCv+mgNRRXZxIMw1crg1HwBBQ+iORGweDdM7y62uFzbbcBCgolTOucN1auUQ+aEwcEeDyf7Tg=; 4:KfeRMtYJxu6kxlAelrQR80fgEvYGHegjmuQ3PxkK6dVkxSFHZ1QgcB1klAJTDFSssMYH5LOzcQwzIhmAh/9IyoMJ8Gu1eUu66HrJdbbaE54+HyV4OeoyUpBOCgJUgYzOxaU1ZnSDiyoqrsZT3ak2kBEBu/2dg9pYvDL5D8Il1fnLU22WP4Rbu1CxXrteJPwrZF2Iq6xfGbII/7Nf0NjyWpjfdlEfanVIKVChyvJmL55LvlU4saW09+bnBPPThIchCV8Ku/rLcE9/KLBhAcnmdYBCQxqgVnVfsv/hSePcxEOa8N0lxDJWc9Y7LG89cst0zW03+IibP3KUNJZvIRChE87MVtkD78Lgow42DcRSPjY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(192374486261705)(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(2401047)(5005006)(8121501046)(3002001)(3231023)(944501134)(93006095)(93001095)(10201501046)(6055026)(6096035)(20161123561025)(201703131430075)(201703131520075)(201703131448075)(201703131433075)(201703151042153)(20161123559100)(20161123556025)(20161123563025)(20161123565025)(201708071742011); SRVR:SN2PR03MB2368; BCL:0; PCL:0; RULEID:(100000803101)(100110400095)(400006); SRVR:SN2PR03MB2368; X-Forefront-PRVS: 0549E6FD50 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN2PR03MB2368; 23:aEgTZYfo06YK2KOp28rUJw+w2r3rH9U2058Qc20j9?= =?us-ascii?Q?u3jXHBHrd1F2j3axUcPYDSsRN/qBGCblDSP1v/KfZ0uekXxs3Y66GIHLbluD?= =?us-ascii?Q?r6+jlfSLEilBTosuuESzndxg+pX8KE9ll1N4mRbomFVHXMI8VeXKjvPUMS7n?= =?us-ascii?Q?P4e4vNLdH3TJzZFcyF7YaQ/bqCptfjqgfF0dEKX7zWw2f88819UIn75C47We?= =?us-ascii?Q?CczrgW/zuLJSdpmEfBTzuHRMr6JFRJN6phBLw2Tc+x9Eeao1cajSCQVKJoiS?= =?us-ascii?Q?2/pQLZIvaocbndkTb8mNHFvcl5a0mS209zxOGvzKfLCnvg8ofSfTd3aEMkMw?= =?us-ascii?Q?Y5D/j89At6xHSpzZtdV56ZkUIno7cZOFiHOoovsNlmYlx1Tvis58ztpRf8fF?= =?us-ascii?Q?jS1gEnE7zFf38JlYXlqGXJjdMePc2BmOumor6xYMpG83S396WK1cjXZ+1owL?= =?us-ascii?Q?odereHMR4APXL10mD2DRP9/BkFt03ouCEJgLbczRpj5aM5Al3xn1YeTeDgUe?= =?us-ascii?Q?R5XJn5u1NdkjSanQZsozVyzI8/eOFbBhX+m5QKaQ5t0oAp7Ng7+nxUmU6EQL?= =?us-ascii?Q?C/2uCGN2oI1gmFJZSYmS1y86yshaolbZGr83pCQSqzKMI6oEib/NF18s1m3K?= =?us-ascii?Q?CGpxiEJpNzfrXTBaj9UMwqpi9DlzsLTpJqXM/lJmqLEiJ1JLAhEZT/tKgQFH?= =?us-ascii?Q?rI53HCmw8D/s6JFr3uYDFfoNvKTcMBrG8b3F9KZE+n0cDCfeitdjEg7j/19n?= =?us-ascii?Q?9xxFVeo1sXCty9rgm4iWNpT6MR2NxDMD3vKSngfcvMuSlxjkRdtqPL5ISG1g?= =?us-ascii?Q?FpYPUbaNaeeuG/dOB3Lsz/EEFO5FBOUAEbNKZdJ9EnOx7aMG5qrDPD7QLFZ3?= =?us-ascii?Q?dZS3qUcQizqFnclEEtFpTkzALMF++17OICxHpOGlVje4gYQl2xGFRb66T4xd?= =?us-ascii?Q?68cM8CMK1hG8QVefhhJtAhkzrI4Z2oJNSgUrACyj9OyJdLg789gMza/YT/sF?= =?us-ascii?Q?Yfj2b1nQtfbyEZCvfiwHv3LC07CwSBV7HBKEWCk3aLWtxBMxiYewwggO4JJF?= =?us-ascii?Q?v9MSqRPySQnGoqrQDFDa7iPT/Xz5vROHpYEjAXmIugPi2NzjZIFfURS489Ke?= =?us-ascii?Q?mGlsyySYRwLxefKtpaCSztu79e9BIsRuPB6CwzFrxy3rZw+3sQQzLbdPWklq?= =?us-ascii?Q?ErtleDMsUue4OI=3D?= X-Microsoft-Exchange-Diagnostics: 1; SN2PR03MB2368; 6:Dzm8/9L/NeRZMksxBKDMhOI3esg5eJJ8b5a4Z/FGZNBBHBM6yDiP4tcpJx6NSI9jaa6RauP+dkUfedAsJ0/hN4zcf9ur4Zx+l90ufAJZG1xqtJ9o2S4xVAvVvlIMBCnke7CJ0Z+/SdraeIBfoLeQRdPjJuHtd5MWYOMjnVeDXzX+kGmReEEncPRpP44nDk9tmQQwROHQIjX9+JVYUcZjIapMKpIiGkwk/i3WZB7er868kHR+ytYVeukM3efZ2K+7RuXFhJUlk8HHuDEmDgkShWuz0L2WJ0ke1ZLGHneC+YVewnGaC0MXnHFHWYUudQhQGpVEsoSiQ6uNoumZIQSbTknZM7qqjlOAPj6Iq2wmbo0=; 5:OmeUSmz6PbY3n/Ig7FKa1+TexVpzqcp8BQCxb3qrlIEadoSHO281BNW6U1iK8eqA1A9I0BW0kW+UuO7U4JheVYWC3wn3xTUncTE0kHDmflyyFkbm3wHympbKCz18fVl/qaKNHosOaviUZRMdkZY7nZxE0m6SCLoq+ZrBft224xk=; 24:kgMIa3s4BDGvMjbu9qVkq7L7PiSPPn3sFN+jIJw5chtb3WmsFADWMLlh5xVD/78evYaK/Aq9WQYTuEPub7yANv2gUHHVj/dcK4hM1szu5AU=; 7:g8pXblDwx3iEZgisDFhEJbi3L1Z+64zLdgn5kCY4j3QF/vA0Na1DpPrASsld3I+Fn76ur0/Fl5l/YLcCTLF0zsvtJC3pAEsq0NkGNKB9a7clLTDwz17PCiYjgs2iP2hF/FlUa4ojlQbS7OmbCCfYc37MwxcJrNjyXyolFKrJG7IxqZRCKiPUpun8Bfrzgk7yo+WXAJegNFfPpEDlj68MmmABP0PsI4Ew/SPnTTZD8Y/KhY16luqO/3r4Ys+tKQSU SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2018 11:36:37.8004 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 05556146-5715-4067-6341-08d558e79398 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR03MB2368 Subject: [dpdk-dev] [PATCH v2 3/3] crypto/dpaa_sec: support ipsec protocol offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jan 2018 11:37:12 -0000 Signed-off-by: Akhil Goyal Acked-by: Hemant Agrawal --- doc/guides/cryptodevs/features/dpaa_sec.ini | 1 + drivers/crypto/dpaa_sec/dpaa_sec.c | 410 ++++++++++++++++++++++++++-- drivers/crypto/dpaa_sec/dpaa_sec.h | 62 ++++- 3 files changed, 450 insertions(+), 23 deletions(-) diff --git a/doc/guides/cryptodevs/features/dpaa_sec.ini b/doc/guides/cryptodevs/features/dpaa_sec.ini index 0e8f5b2..deab53a 100644 --- a/doc/guides/cryptodevs/features/dpaa_sec.ini +++ b/doc/guides/cryptodevs/features/dpaa_sec.ini @@ -7,6 +7,7 @@ Symmetric crypto = Y Sym operation chaining = Y HW Accelerated = Y +Protocol offload = Y ; ; Supported crypto algorithms of the 'dpaa_sec' crypto driver. diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.c b/drivers/crypto/dpaa_sec/dpaa_sec.c index f433f0a..d7b6f39 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -196,8 +197,19 @@ dqrr_out_fq_cb_rx(struct qman_portal *qm __always_unused, * sg[1] for input */ job = dpaa_mem_ptov(qm_fd_addr_get64(fd)); + ctx = container_of(job, struct dpaa_sec_op_ctx, job); ctx->fd_status = fd->status; + if (ctx->op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct qm_sg_entry *sg_out; + uint32_t len; + + sg_out = &job->sg[0]; + hw_sg_to_cpu(sg_out); + len = sg_out->length; + ctx->op->sym->m_src->pkt_len = len; + ctx->op->sym->m_src->data_len = len; + } dpaa_sec_ops[dpaa_sec_op_nb++] = ctx->op; dpaa_sec_op_ending(ctx); @@ -261,7 +273,13 @@ static inline int is_aead(dpaa_sec_session *ses) static inline int is_auth_cipher(dpaa_sec_session *ses) { return ((ses->cipher_alg != RTE_CRYPTO_CIPHER_NULL) && - (ses->auth_alg != RTE_CRYPTO_AUTH_NULL)); + (ses->auth_alg != RTE_CRYPTO_AUTH_NULL) && + (ses->proto_alg != RTE_SECURITY_PROTOCOL_IPSEC)); +} + +static inline int is_proto_ipsec(dpaa_sec_session *ses) +{ + return (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC); } static inline int is_encode(dpaa_sec_session *ses) @@ -282,27 +300,39 @@ caam_auth_alg(dpaa_sec_session *ses, struct alginfo *alginfo_a) ses->digest_length = 0; break; case RTE_CRYPTO_AUTH_MD5_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_MD5; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_MD5_96 : OP_ALG_ALGSEL_MD5; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA1_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA1; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA1_96 : OP_ALG_ALGSEL_SHA1; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA224_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA224; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA1_160 : OP_ALG_ALGSEL_SHA224; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA256_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA256; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA2_256_128 : OP_ALG_ALGSEL_SHA256; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA384_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA384; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA2_384_192 : OP_ALG_ALGSEL_SHA384; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; case RTE_CRYPTO_AUTH_SHA512_HMAC: - alginfo_a->algtype = OP_ALG_ALGSEL_SHA512; + alginfo_a->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_HMAC_SHA2_512_256 : OP_ALG_ALGSEL_SHA512; alginfo_a->algmode = OP_ALG_AAI_HMAC; break; default: @@ -317,15 +347,21 @@ caam_cipher_alg(dpaa_sec_session *ses, struct alginfo *alginfo_c) case RTE_CRYPTO_CIPHER_NULL: break; case RTE_CRYPTO_CIPHER_AES_CBC: - alginfo_c->algtype = OP_ALG_ALGSEL_AES; + alginfo_c->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_AES_CBC : OP_ALG_ALGSEL_AES; alginfo_c->algmode = OP_ALG_AAI_CBC; break; case RTE_CRYPTO_CIPHER_3DES_CBC: - alginfo_c->algtype = OP_ALG_ALGSEL_3DES; + alginfo_c->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_3DES : OP_ALG_ALGSEL_3DES; alginfo_c->algmode = OP_ALG_AAI_CBC; break; case RTE_CRYPTO_CIPHER_AES_CTR: - alginfo_c->algtype = OP_ALG_ALGSEL_AES; + alginfo_c->algtype = + (ses->proto_alg == RTE_SECURITY_PROTOCOL_IPSEC) ? + OP_PCL_IPSEC_AES_CTR : OP_ALG_ALGSEL_AES; alginfo_c->algmode = OP_ALG_AAI_CTR; break; default: @@ -471,14 +507,28 @@ dpaa_sec_prep_cdb(dpaa_sec_session *ses) cdb->sh_desc[0] = 0; cdb->sh_desc[1] = 0; cdb->sh_desc[2] = 0; - - /* Auth_only_len is set as 0 here and it will be overwritten - * in fd for each packet. - */ - shared_desc_len = cnstr_shdsc_authenc(cdb->sh_desc, - true, swap, &alginfo_c, &alginfo_a, - ses->iv.length, 0, - ses->digest_length, ses->dir); + if (is_proto_ipsec(ses)) { + if (ses->dir == DIR_ENC) { + shared_desc_len = cnstr_shdsc_ipsec_new_encap( + cdb->sh_desc, + true, swap, &ses->encap_pdb, + (uint8_t *)&ses->ip4_hdr, + &alginfo_c, &alginfo_a); + } else if (ses->dir == DIR_DEC) { + shared_desc_len = cnstr_shdsc_ipsec_new_decap( + cdb->sh_desc, + true, swap, &ses->decap_pdb, + &alginfo_c, &alginfo_a); + } + } else { + /* Auth_only_len is set as 0 here and it will be + * overwritten in fd for each packet. + */ + shared_desc_len = cnstr_shdsc_authenc(cdb->sh_desc, + true, swap, &alginfo_c, &alginfo_a, + ses->iv.length, 0, + ses->digest_length, ses->dir); + } } cdb->sh_hdr.hi.field.idlen = shared_desc_len; cdb->sh_hdr.hi.word = rte_cpu_to_be_32(cdb->sh_hdr.hi.word); @@ -860,6 +910,45 @@ build_cipher_auth(struct rte_crypto_op *op, dpaa_sec_session *ses) return cf; } +static inline struct dpaa_sec_job * +build_proto(struct rte_crypto_op *op, dpaa_sec_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct dpaa_sec_job *cf; + struct dpaa_sec_op_ctx *ctx; + struct qm_sg_entry *sg; + phys_addr_t src_start_addr, dst_start_addr; + + ctx = dpaa_sec_alloc_ctx(ses); + if (!ctx) + return NULL; + cf = &ctx->job; + ctx->op = op; + + src_start_addr = rte_pktmbuf_mtophys(sym->m_src); + + if (sym->m_dst) + dst_start_addr = rte_pktmbuf_mtophys(sym->m_dst); + else + dst_start_addr = src_start_addr; + + /* input */ + sg = &cf->sg[1]; + qm_sg_entry_set64(sg, src_start_addr); + sg->length = sym->m_src->pkt_len; + sg->final = 1; + cpu_to_hw_sg(sg); + + sym->m_src->packet_type &= ~RTE_PTYPE_L4_MASK; + /* output */ + sg = &cf->sg[0]; + qm_sg_entry_set64(sg, dst_start_addr); + sg->length = sym->m_src->buf_len - sym->m_src->data_off; + cpu_to_hw_sg(sg); + + return cf; +} + static int dpaa_sec_enqueue_op(struct rte_crypto_op *op, struct dpaa_sec_qp *qp) { @@ -870,8 +959,14 @@ dpaa_sec_enqueue_op(struct rte_crypto_op *op, struct dpaa_sec_qp *qp) uint32_t auth_only_len = op->sym->auth.data.length - op->sym->cipher.data.length; - ses = (dpaa_sec_session *)get_session_private_data(op->sym->session, - cryptodev_driver_id); + if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) + ses = (dpaa_sec_session *)get_session_private_data( + op->sym->session, cryptodev_driver_id); + else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) + ses = (dpaa_sec_session *)get_sec_session_private_data( + op->sym->sec_session); + else + return -ENOTSUP; if (unlikely(!ses->qp || ses->qp != qp)) { PMD_INIT_LOG(DEBUG, "sess->qp - %p qp %p", ses->qp, qp); @@ -895,6 +990,8 @@ dpaa_sec_enqueue_op(struct rte_crypto_op *op, struct dpaa_sec_qp *qp) auth_only_len = ses->auth_only_len; } else if (is_auth_cipher(ses)) { cf = build_cipher_auth(op, ses); + } else if (is_proto_ipsec(ses)) { + cf = build_proto(op, ses); } else { PMD_TX_LOG(ERR, "not supported sec op"); return -ENOTSUP; @@ -933,7 +1030,7 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, /*Prepare each packet which is to be sent*/ for (loop = 0; loop < nb_ops; loop++) { - if (ops[loop]->sess_type != RTE_CRYPTO_OP_WITH_SESSION) { + if (ops[loop]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { PMD_TX_LOG(ERR, "sessionless crypto op not supported"); return 0; } @@ -1343,6 +1440,236 @@ dpaa_sec_session_clear(struct rte_cryptodev *dev, } static int +dpaa_sec_set_ipsec_session(__rte_unused struct rte_cryptodev *dev, + struct rte_security_session_conf *conf, + void *sess) +{ + struct dpaa_sec_dev_private *internals = dev->data->dev_private; + struct rte_security_ipsec_xform *ipsec_xform = &conf->ipsec; + struct rte_crypto_auth_xform *auth_xform; + struct rte_crypto_cipher_xform *cipher_xform; + dpaa_sec_session *session = (dpaa_sec_session *)sess; + + PMD_INIT_FUNC_TRACE(); + + if (ipsec_xform->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + cipher_xform = &conf->crypto_xform->cipher; + auth_xform = &conf->crypto_xform->next->auth; + } else { + auth_xform = &conf->crypto_xform->auth; + cipher_xform = &conf->crypto_xform->next->cipher; + } + session->proto_alg = conf->protocol; + session->cipher_key.data = rte_zmalloc(NULL, + cipher_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->cipher_key.data == NULL && + cipher_xform->key.length > 0) { + RTE_LOG(ERR, PMD, "No Memory for cipher key\n"); + return -ENOMEM; + } + + session->cipher_key.length = cipher_xform->key.length; + session->auth_key.data = rte_zmalloc(NULL, + auth_xform->key.length, + RTE_CACHE_LINE_SIZE); + if (session->auth_key.data == NULL && + auth_xform->key.length > 0) { + RTE_LOG(ERR, PMD, "No Memory for auth key\n"); + rte_free(session->cipher_key.data); + return -ENOMEM; + } + session->auth_key.length = auth_xform->key.length; + memcpy(session->cipher_key.data, cipher_xform->key.data, + cipher_xform->key.length); + memcpy(session->auth_key.data, auth_xform->key.data, + auth_xform->key.length); + + switch (auth_xform->algo) { + case RTE_CRYPTO_AUTH_SHA1_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA1_HMAC; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_MD5_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA256_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA384_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + session->auth_alg = RTE_CRYPTO_AUTH_SHA512_HMAC; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + session->auth_alg = RTE_CRYPTO_AUTH_AES_CMAC; + break; + case RTE_CRYPTO_AUTH_NULL: + session->auth_alg = RTE_CRYPTO_AUTH_NULL; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + case RTE_CRYPTO_AUTH_AES_XCBC_MAC: + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + case RTE_CRYPTO_AUTH_SHA1: + case RTE_CRYPTO_AUTH_SHA256: + case RTE_CRYPTO_AUTH_SHA512: + case RTE_CRYPTO_AUTH_SHA224: + case RTE_CRYPTO_AUTH_SHA384: + case RTE_CRYPTO_AUTH_MD5: + case RTE_CRYPTO_AUTH_AES_GMAC: + case RTE_CRYPTO_AUTH_KASUMI_F9: + case RTE_CRYPTO_AUTH_AES_CBC_MAC: + case RTE_CRYPTO_AUTH_ZUC_EIA3: + RTE_LOG(ERR, PMD, "Crypto: Unsupported auth alg %u\n", + auth_xform->algo); + goto out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Auth specified %u\n", + auth_xform->algo); + goto out; + } + + switch (cipher_xform->algo) { + case RTE_CRYPTO_CIPHER_AES_CBC: + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CBC; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + session->cipher_alg = RTE_CRYPTO_CIPHER_3DES_CBC; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + session->cipher_alg = RTE_CRYPTO_CIPHER_AES_CTR; + break; + case RTE_CRYPTO_CIPHER_NULL: + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + case RTE_CRYPTO_CIPHER_3DES_ECB: + case RTE_CRYPTO_CIPHER_AES_ECB: + case RTE_CRYPTO_CIPHER_KASUMI_F8: + RTE_LOG(ERR, PMD, "Crypto: Unsupported Cipher alg %u\n", + cipher_xform->algo); + goto out; + default: + RTE_LOG(ERR, PMD, "Crypto: Undefined Cipher specified %u\n", + cipher_xform->algo); + goto out; + } + + if (ipsec_xform->direction == RTE_SECURITY_IPSEC_SA_DIR_EGRESS) { + memset(&session->encap_pdb, 0, sizeof(struct ipsec_encap_pdb) + + sizeof(session->ip4_hdr)); + session->ip4_hdr.ip_v = IPVERSION; + session->ip4_hdr.ip_hl = 5; + session->ip4_hdr.ip_len = rte_cpu_to_be_16( + sizeof(session->ip4_hdr)); + session->ip4_hdr.ip_tos = ipsec_xform->tunnel.ipv4.dscp; + session->ip4_hdr.ip_id = 0; + session->ip4_hdr.ip_off = 0; + session->ip4_hdr.ip_ttl = ipsec_xform->tunnel.ipv4.ttl; + session->ip4_hdr.ip_p = (ipsec_xform->proto == + RTE_SECURITY_IPSEC_SA_PROTO_ESP) ? IPPROTO_ESP + : IPPROTO_AH; + session->ip4_hdr.ip_sum = 0; + session->ip4_hdr.ip_src = ipsec_xform->tunnel.ipv4.src_ip; + session->ip4_hdr.ip_dst = ipsec_xform->tunnel.ipv4.dst_ip; + session->ip4_hdr.ip_sum = calc_chksum((uint16_t *) + (void *)&session->ip4_hdr, + sizeof(struct ip)); + + session->encap_pdb.options = + (IPVERSION << PDBNH_ESP_ENCAP_SHIFT) | + PDBOPTS_ESP_OIHI_PDB_INL | + PDBOPTS_ESP_IVSRC | + PDBHMO_ESP_ENCAP_DTTL; + session->encap_pdb.spi = ipsec_xform->spi; + session->encap_pdb.ip_hdr_len = sizeof(struct ip); + + session->dir = DIR_ENC; + } else if (ipsec_xform->direction == + RTE_SECURITY_IPSEC_SA_DIR_INGRESS) { + memset(&session->decap_pdb, 0, sizeof(struct ipsec_decap_pdb)); + session->decap_pdb.options = sizeof(struct ip) << 16; + session->dir = DIR_DEC; + } else + goto out; + session->ctx_pool = internals->ctx_pool; + session->inq = dpaa_sec_attach_rxq(internals); + if (session->inq == NULL) { + PMD_DRV_LOG(ERR, "unable to attach sec queue"); + goto out; + } + + + return 0; +out: + rte_free(session->auth_key.data); + rte_free(session->cipher_key.data); + memset(session, 0, sizeof(dpaa_sec_session)); + return -1; +} + +static int +dpaa_sec_security_session_create(void *dev, + struct rte_security_session_conf *conf, + struct rte_security_session *sess, + struct rte_mempool *mempool) +{ + void *sess_private_data; + struct rte_cryptodev *cdev = (struct rte_cryptodev *)dev; + int ret; + + if (rte_mempool_get(mempool, &sess_private_data)) { + CDEV_LOG_ERR( + "Couldn't get object from session mempool"); + return -ENOMEM; + } + + switch (conf->protocol) { + case RTE_SECURITY_PROTOCOL_IPSEC: + ret = dpaa_sec_set_ipsec_session(cdev, conf, + sess_private_data); + break; + case RTE_SECURITY_PROTOCOL_MACSEC: + return -ENOTSUP; + default: + return -EINVAL; + } + if (ret != 0) { + PMD_DRV_LOG(ERR, + "DPAA2 PMD: failed to configure session parameters"); + + /* Return session to mempool */ + rte_mempool_put(mempool, sess_private_data); + return ret; + } + + set_sec_session_private_data(sess, sess_private_data); + + return ret; +} + +/** Clear the memory of session so it doesn't leave key material behind */ +static int +dpaa_sec_security_session_destroy(void *dev __rte_unused, + struct rte_security_session *sess) +{ + PMD_INIT_FUNC_TRACE(); + void *sess_priv = get_sec_session_private_data(sess); + + dpaa_sec_session *s = (dpaa_sec_session *)sess_priv; + + if (sess_priv) { + struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); + + rte_free(s->cipher_key.data); + rte_free(s->auth_key.data); + memset(sess, 0, sizeof(dpaa_sec_session)); + set_sec_session_private_data(sess, NULL); + rte_mempool_put(sess_mp, sess_priv); + } + return 0; +} + + +static int dpaa_sec_dev_configure(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_config *config __rte_unused) { @@ -1408,6 +1735,21 @@ static struct rte_cryptodev_ops crypto_ops = { .qp_detach_session = dpaa_sec_qp_detach_sess, }; +static const struct rte_security_capability * +dpaa_sec_capabilities_get(void *device __rte_unused) +{ + return dpaa_sec_security_cap; +} + +struct rte_security_ops dpaa_sec_security_ops = { + .session_create = dpaa_sec_security_session_create, + .session_update = NULL, + .session_stats_get = NULL, + .session_destroy = dpaa_sec_security_session_destroy, + .set_pkt_metadata = NULL, + .capabilities_get = dpaa_sec_capabilities_get +}; + static int dpaa_sec_uninit(struct rte_cryptodev *dev) { @@ -1416,6 +1758,8 @@ dpaa_sec_uninit(struct rte_cryptodev *dev) if (dev == NULL) return -ENODEV; + rte_free(dev->security_ctx); + rte_mempool_free(internals->ctx_pool); rte_free(internals); @@ -1429,6 +1773,7 @@ static int dpaa_sec_dev_init(struct rte_cryptodev *cryptodev) { struct dpaa_sec_dev_private *internals; + struct rte_security_ctx *security_instance; struct dpaa_sec_qp *qp; uint32_t i, flags; int ret; @@ -1443,12 +1788,33 @@ dpaa_sec_dev_init(struct rte_cryptodev *cryptodev) cryptodev->dequeue_burst = dpaa_sec_dequeue_burst; cryptodev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | - RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING; + RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | + RTE_CRYPTODEV_FF_SECURITY; internals = cryptodev->data->dev_private; internals->max_nb_queue_pairs = RTE_DPAA_MAX_NB_SEC_QPS; internals->max_nb_sessions = RTE_DPAA_SEC_PMD_MAX_NB_SESSIONS; + /* + * For secondary processes, we don't initialise any further as primary + * has already done this work. Only check we don't need a different + * RX function + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + PMD_INIT_LOG(DEBUG, "Device already init by primary process"); + return 0; + } + + /* Initialize security_ctx only for primary process*/ + security_instance = rte_malloc("rte_security_instances_ops", + sizeof(struct rte_security_ctx), 0); + if (security_instance == NULL) + return -ENOMEM; + security_instance->device = (void *)cryptodev; + security_instance->ops = &dpaa_sec_security_ops; + security_instance->sess_cnt = 0; + cryptodev->security_ctx = security_instance; + for (i = 0; i < internals->max_nb_queue_pairs; i++) { /* init qman fq for queue pair */ qp = &internals->qps[i]; diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.h b/drivers/crypto/dpaa_sec/dpaa_sec.h index 93369e4..578c46a 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.h +++ b/drivers/crypto/dpaa_sec/dpaa_sec.h @@ -91,7 +91,8 @@ typedef struct dpaa_sec_session_entry { uint8_t dir; /*!< Operation Direction */ enum rte_crypto_cipher_algorithm cipher_alg; /*!< Cipher Algorithm*/ enum rte_crypto_auth_algorithm auth_alg; /*!< Authentication Algorithm*/ - enum rte_crypto_aead_algorithm aead_alg; /*!< Authentication Algorithm*/ + enum rte_crypto_aead_algorithm aead_alg; /*!< AEAD Algorithm*/ + enum rte_security_session_protocol proto_alg; /*!< Security Algorithm*/ union { struct { uint8_t *data; /**< pointer to key data */ @@ -114,6 +115,9 @@ typedef struct dpaa_sec_session_entry { } iv; /**< Initialisation vector parameters */ uint16_t auth_only_len; /*!< Length of data for Auth only */ uint32_t digest_length; + struct ipsec_encap_pdb encap_pdb; + struct ip ip4_hdr; + struct ipsec_decap_pdb decap_pdb; struct dpaa_sec_qp *qp; struct qman_fq *inq; struct sec_cdb cdb; /**< cmd block associated with qp */ @@ -378,4 +382,60 @@ static const struct rte_cryptodev_capabilities dpaa_sec_capabilities[] = { RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() }; +static const struct rte_security_capability dpaa_sec_security_cap[] = { + { /* IPsec Lookaside Protocol offload ESP Transport Egress */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_EGRESS, + .options = { 0 } + }, + .crypto_capabilities = dpaa_sec_capabilities + }, + { /* IPsec Lookaside Protocol offload ESP Tunnel Ingress */ + .action = RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL, + .protocol = RTE_SECURITY_PROTOCOL_IPSEC, + .ipsec = { + .proto = RTE_SECURITY_IPSEC_SA_PROTO_ESP, + .mode = RTE_SECURITY_IPSEC_SA_MODE_TUNNEL, + .direction = RTE_SECURITY_IPSEC_SA_DIR_INGRESS, + .options = { 0 } + }, + .crypto_capabilities = dpaa_sec_capabilities + }, + { + .action = RTE_SECURITY_ACTION_TYPE_NONE + } +}; + +/** + * Checksum + * + * @param buffer calculate chksum for buffer + * @param len buffer length + * + * @return checksum value in host cpu order + */ +static inline uint16_t +calc_chksum(void *buffer, int len) +{ + uint16_t *buf = (uint16_t *)buffer; + uint32_t sum = 0; + uint16_t result; + + for (sum = 0; len > 1; len -= 2) + sum += *buf++; + + if (len == 1) + sum += *(unsigned char *)buf; + + sum = (sum >> 16) + (sum & 0xFFFF); + sum += (sum >> 16); + result = ~sum; + + return result; +} + #endif /* _DPAA_SEC_H_ */ -- 2.9.3