From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20045.outbound.protection.outlook.com [40.107.2.45]) by dpdk.org (Postfix) with ESMTP id 5387E1B411 for ; Mon, 22 Oct 2018 16:57:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JmNsEu++aBJmKJUhq9OV0wlCp1weTR7JvaOfvZmVV64=; b=HVLYtIc5xubmPCoOStUDya/3jvkuu1K5XufKY4h5ZjKgssKyhq+B/5y3yN68aREtJMUbWjAnkpZOTNx7KpiDH9F6ZA+dm8LhXHQSdw1vafZoS4rJieKyGvnQes7CB58/44MW3xFf2y3JVJMTjMI3Vie43Zu3ozBwna7d0tIWWeU= Received: from HE1PR04MB1530.eurprd04.prod.outlook.com (10.164.48.20) by HE1PR04MB1305.eurprd04.prod.outlook.com (10.162.249.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1250.30; Mon, 22 Oct 2018 14:57:44 +0000 Received: from HE1PR04MB1530.eurprd04.prod.outlook.com ([fe80::95cd:198:d5c5:59a4]) by HE1PR04MB1530.eurprd04.prod.outlook.com ([fe80::95cd:198:d5c5:59a4%5]) with mapi id 15.20.1250.028; Mon, 22 Oct 2018 14:57:44 +0000 From: Gagandeep Singh To: "dev@dpdk.org" , Akhil Goyal CC: Gagandeep Singh , Hemant Agrawal Thread-Topic: [PATCH v5 10/15] crypto/caam_jr: add enqueue dequeue operations Thread-Index: AQHUaheWggFElCFVMUCRZAm91tTTFQ== Date: Mon, 22 Oct 2018 14:57:44 +0000 Message-ID: <20181022145644.29277-11-g.singh@nxp.com> References: <20181022141657.4727-1-g.singh@nxp.com> <20181022145644.29277-1-g.singh@nxp.com> In-Reply-To: <20181022145644.29277-1-g.singh@nxp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BM1PR0101CA0067.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:19::29) To HE1PR04MB1530.eurprd04.prod.outlook.com (2a01:111:e400:59a8::20) authentication-results: spf=none (sender IP is ) smtp.mailfrom=G.Singh@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [14.142.187.166] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR04MB1305; 6:Us6CdUogRHtKy9Iq0D1D/nhPfKW6KfK2QSVf66V9O4306yU2n5PhS0BZwzk5OC3I6o9J4WKzWBoNdLBbl/h9kcIvIffAcdIsxcATMrmrhVwD7HBTjOUm0zFoWHt/tKX6ErctocY36qab8crd94vddAHpxBJ9hSbJ4FqsRJcF58PVrhBb9iuLTfvonNU8T+hG8kzVhv23zLiVEzsLm1Q5HO8UU7nwPIz5lr1MrvDF49NwwntCPPFLWdqfiZfIkzS+XJDvH8ofjfiP9K58Hl6jfnCOrkDo7B5HkAtmeUR8MQLqlN0ytXy/HJH6i6VnhzHDyot/mdjZ2wPj3eevkEuzNnI/F01tUrfv61PT/0vMInLpSt06qKol00oQh+CPg4bzrckp8rKZozlj0zd9kJpHN2YlDmx0nPHfQxqVjOjEN2j8ZV/vngB4aO7trahxYQXnQYjNIrmezRk5VZGO5NAe5w==; 5:pejjhI+0LQIIZ+3yISrgZXreJYwSBuFQp1hxqGKvrY72LHtDZ52VFhAAZ8y8ZgsEBxZnR+KSVavdbuIUJGwDJbiPBB8A8hzGIDKs7fjNo+R8ZpEJWlc6QyDVBCHaLw3DW1oK46PvF20O5JqvR4XBiFurDigKJyp0X81DDy+BiXQ=; 7:4DSyJU8gQtnzmUJgNC9dWWKshj6GO5Oz1IjbLexqWS5hCAOb/IgdMBPOIcd9OIeQ/tvvgatnf60UdxRI8vCCaoIdnLAUrA2X7uk75OsMpIfzaL3FHuXMXcF2x02fai7qD2N4Y5fEU6jJW6gPecq3JeiZwmZ+O/EcTeFr8AnRJbjDqFjJUysnX210YmM6A/J8Ds/cdhyMeehi+4a6TkTIcv2I/8NDTqS4nzXowWB5FqKtA6noMFrCgxZQmze1s35W x-ms-office365-filtering-correlation-id: 75fb0af8-569c-44a8-dece-08d6382eb7fc x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:HE1PR04MB1305; x-ms-traffictypediagnostic: HE1PR04MB1305: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(185117386973197); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3002001)(3231355)(944501410)(52105095)(6055026)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:HE1PR04MB1305; BCL:0; PCL:0; RULEID:; SRVR:HE1PR04MB1305; x-forefront-prvs: 08331F819E x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(39860400002)(396003)(136003)(366004)(346002)(199004)(189003)(8676002)(256004)(81156014)(81166006)(11346002)(476003)(99286004)(446003)(316002)(52116002)(2616005)(486006)(106356001)(8936002)(105586002)(186003)(71200400001)(5250100002)(78486010)(3846002)(2501003)(4744004)(71190400001)(25786009)(2900100001)(305945005)(7736002)(97736004)(478600001)(4326008)(2906002)(72206003)(6636002)(53936002)(14454004)(66066001)(6512007)(5660300001)(6116002)(1076002)(6436002)(53946003)(6486002)(36756003)(14444005)(16200700003)(54906003)(6506007)(76176011)(55236004)(575784001)(386003)(68736007)(26005)(86362001)(102836004)(110136005)(134034003)(569006); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR04MB1305; H:HE1PR04MB1530.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: u2CaRSCHNWo9CvpgpkMFnLgdEwl2B+IgVOsUZQY0EpvHda+dlKdF6FPe9zrW7YsoMZRpnTsdSDWaSr/MWBmB0hIZtEUfnac4rKvP4v8zgQJRsrIuqIH1yDqLO7Z+3DB7hGaMp6TqYDRHFXEO/gG6iCeKUTVCclNzWs6XUH0A9fj9iTK7lfypOq1xFbES4jMX5S2ovCnovSDOyJFsC4NVguzAJ04MJv15UCErg42zrif2oFOzEAdgwdTzOfJ4WT8iHhpcClb6Y8HgIhUqeypqtQ1I+ykxLuE0WXiEw+de53jsn7JHzkluqKvPqOK9JDkCPE9wet/BOWk8lUCfvmz7077/n22cNW1tqFLws4k96JE= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 75fb0af8-569c-44a8-dece-08d6382eb7fc X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Oct 2018 14:57:44.5173 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR04MB1305 Subject: [dpdk-dev] [PATCH v5 10/15] crypto/caam_jr: add enqueue dequeue operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2018 14:57:49 -0000 This patch add support for : 1. creating run time sec hw decriptors for a given request. 2. enqueue operation to the caam jr ring 3. dequeue operation from the caam jr ring in poll mode 4. creating a crypto protocol descriptor for session - first time. Signed-off-by: Gagandeep Singh Signed-off-by: Hemant Agrawal Acked-by: Akhil Goyal --- drivers/crypto/caam_jr/caam_jr.c | 875 +++++++++++++++++++++++++- drivers/crypto/caam_jr/caam_jr_desc.h | 285 +++++++++ 2 files changed, 1158 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/caam_jr/caam_jr_desc.h diff --git a/drivers/crypto/caam_jr/caam_jr.c b/drivers/crypto/caam_jr/caam= _jr.c index 96b18be12..594ae8ded 100644 --- a/drivers/crypto/caam_jr/caam_jr.c +++ b/drivers/crypto/caam_jr/caam_jr.c @@ -20,6 +20,7 @@ #include #include #include +#include #include =20 /* RTA header files */ @@ -52,6 +53,343 @@ static enum sec_driver_state_e g_driver_state =3D SEC_D= RIVER_STATE_IDLE; static int g_job_rings_no; static int g_job_rings_max; =20 +struct sec_outring_entry { + phys_addr_t desc; /* Pointer to completed descriptor */ + uint32_t status; /* Status for completed descriptor */ +} __rte_packed; + +/* virtual address conversin when mempool support is available for ctx */ +static inline phys_addr_t +caam_jr_vtop_ctx(struct caam_jr_op_ctx *ctx, void *vaddr) +{ + PMD_INIT_FUNC_TRACE(); + return (size_t)vaddr - ctx->vtop_offset; +} + +static inline void +caam_jr_op_ending(struct caam_jr_op_ctx *ctx) +{ + PMD_INIT_FUNC_TRACE(); + /* report op status to sym->op and then free the ctx memeory */ + rte_mempool_put(ctx->ctx_pool, (void *)ctx); +} + +static inline struct caam_jr_op_ctx * +caam_jr_alloc_ctx(struct caam_jr_session *ses) +{ + struct caam_jr_op_ctx *ctx; + int ret; + + PMD_INIT_FUNC_TRACE(); + ret =3D rte_mempool_get(ses->ctx_pool, (void **)(&ctx)); + if (!ctx || ret) { + CAAM_JR_DP_WARN("Alloc sec descriptor failed!"); + return NULL; + } + /* + * Clear SG memory. There are 16 SG entries of 16 Bytes each. + * one call to dcbz_64() clear 64 bytes, hence calling it 4 times + * to clear all the SG entries. caam_jr_alloc_ctx() is called for + * each packet, memset is costlier than dcbz_64(). + */ + dcbz_64(&ctx->sg[SG_CACHELINE_0]); + dcbz_64(&ctx->sg[SG_CACHELINE_1]); + dcbz_64(&ctx->sg[SG_CACHELINE_2]); + dcbz_64(&ctx->sg[SG_CACHELINE_3]); + + ctx->ctx_pool =3D ses->ctx_pool; + ctx->vtop_offset =3D (size_t) ctx - rte_mempool_virt2iova(ctx); + + return ctx; +} + +static inline int +is_cipher_only(struct caam_jr_session *ses) +{ + PMD_INIT_FUNC_TRACE(); + return ((ses->cipher_alg !=3D RTE_CRYPTO_CIPHER_NULL) && + (ses->auth_alg =3D=3D RTE_CRYPTO_AUTH_NULL)); +} + +static inline int +is_auth_only(struct caam_jr_session *ses) +{ + PMD_INIT_FUNC_TRACE(); + return ((ses->cipher_alg =3D=3D RTE_CRYPTO_CIPHER_NULL) && + (ses->auth_alg !=3D RTE_CRYPTO_AUTH_NULL)); +} + +static inline int +is_aead(struct caam_jr_session *ses) +{ + PMD_INIT_FUNC_TRACE(); + return ((ses->cipher_alg =3D=3D 0) && + (ses->auth_alg =3D=3D 0) && + (ses->aead_alg !=3D 0)); +} + +static inline int +is_auth_cipher(struct caam_jr_session *ses) +{ + PMD_INIT_FUNC_TRACE(); + return ((ses->cipher_alg !=3D RTE_CRYPTO_CIPHER_NULL) && + (ses->auth_alg !=3D RTE_CRYPTO_AUTH_NULL)); +} + +static inline int +is_encode(struct caam_jr_session *ses) +{ + PMD_INIT_FUNC_TRACE(); + return ses->dir =3D=3D DIR_ENC; +} + +static inline int +is_decode(struct caam_jr_session *ses) +{ + PMD_INIT_FUNC_TRACE(); + return ses->dir =3D=3D DIR_DEC; +} + +static inline void +caam_auth_alg(struct caam_jr_session *ses, struct alginfo *alginfo_a) +{ + PMD_INIT_FUNC_TRACE(); + switch (ses->auth_alg) { + case RTE_CRYPTO_AUTH_NULL: + ses->digest_length =3D 0; + break; + case RTE_CRYPTO_AUTH_MD5_HMAC: + alginfo_a->algtype =3D OP_ALG_ALGSEL_MD5; + alginfo_a->algmode =3D OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA1_HMAC: + alginfo_a->algtype =3D OP_ALG_ALGSEL_SHA1; + alginfo_a->algmode =3D OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA224_HMAC: + alginfo_a->algtype =3D OP_ALG_ALGSEL_SHA224; + alginfo_a->algmode =3D OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA256_HMAC: + alginfo_a->algtype =3D OP_ALG_ALGSEL_SHA256; + alginfo_a->algmode =3D OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA384_HMAC: + alginfo_a->algtype =3D OP_ALG_ALGSEL_SHA384; + alginfo_a->algmode =3D OP_ALG_AAI_HMAC; + break; + case RTE_CRYPTO_AUTH_SHA512_HMAC: + alginfo_a->algtype =3D OP_ALG_ALGSEL_SHA512; + alginfo_a->algmode =3D OP_ALG_AAI_HMAC; + break; + default: + CAAM_JR_DEBUG("unsupported auth alg %u", ses->auth_alg); + } +} + +static inline void +caam_cipher_alg(struct caam_jr_session *ses, struct alginfo *alginfo_c) +{ + PMD_INIT_FUNC_TRACE(); + switch (ses->cipher_alg) { + case RTE_CRYPTO_CIPHER_NULL: + break; + case RTE_CRYPTO_CIPHER_AES_CBC: + alginfo_c->algtype =3D OP_ALG_ALGSEL_AES; + alginfo_c->algmode =3D OP_ALG_AAI_CBC; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + alginfo_c->algtype =3D OP_ALG_ALGSEL_3DES; + alginfo_c->algmode =3D OP_ALG_AAI_CBC; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + alginfo_c->algtype =3D OP_ALG_ALGSEL_AES; + alginfo_c->algmode =3D OP_ALG_AAI_CTR; + break; + default: + CAAM_JR_DEBUG("unsupported cipher alg %d", ses->cipher_alg); + } +} + +static inline void +caam_aead_alg(struct caam_jr_session *ses, struct alginfo *alginfo) +{ + PMD_INIT_FUNC_TRACE(); + switch (ses->aead_alg) { + case RTE_CRYPTO_AEAD_AES_GCM: + alginfo->algtype =3D OP_ALG_ALGSEL_AES; + alginfo->algmode =3D OP_ALG_AAI_GCM; + break; + default: + CAAM_JR_DEBUG("unsupported AEAD alg %d", ses->aead_alg); + } +} + +/* prepare command block of the session */ +static int +caam_jr_prep_cdb(struct caam_jr_session *ses) +{ + struct alginfo alginfo_c =3D {0}, alginfo_a =3D {0}, alginfo =3D {0}; + int32_t shared_desc_len =3D 0; + struct sec_cdb *cdb; + int err; +#if RTE_BYTE_ORDER =3D=3D RTE_BIG_ENDIAN + int swap =3D false; +#else + int swap =3D true; +#endif + + PMD_INIT_FUNC_TRACE(); + if (ses->cdb) + caam_jr_dma_free(ses->cdb); + + cdb =3D caam_jr_dma_mem_alloc(L1_CACHE_BYTES, sizeof(struct sec_cdb)); + if (!cdb) { + CAAM_JR_ERR("failed to allocate memory for cdb\n"); + return -1; + } + + ses->cdb =3D cdb; + + memset(cdb, 0, sizeof(struct sec_cdb)); + + if (is_cipher_only(ses)) { + caam_cipher_alg(ses, &alginfo_c); + if (alginfo_c.algtype =3D=3D (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported cipher alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_c.key =3D (size_t)ses->cipher_key.data; + alginfo_c.keylen =3D ses->cipher_key.length; + alginfo_c.key_enc_flags =3D 0; + alginfo_c.key_type =3D RTA_DATA_IMM; + + shared_desc_len =3D cnstr_shdsc_blkcipher( + cdb->sh_desc, true, + swap, &alginfo_c, + NULL, + ses->iv.length, + ses->dir); + } else if (is_auth_only(ses)) { + caam_auth_alg(ses, &alginfo_a); + if (alginfo_a.algtype =3D=3D (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported auth alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_a.key =3D (size_t)ses->auth_key.data; + alginfo_a.keylen =3D ses->auth_key.length; + alginfo_a.key_enc_flags =3D 0; + alginfo_a.key_type =3D RTA_DATA_IMM; + + shared_desc_len =3D cnstr_shdsc_hmac(cdb->sh_desc, true, + swap, &alginfo_a, + !ses->dir, + ses->digest_length); + } else if (is_aead(ses)) { + caam_aead_alg(ses, &alginfo); + if (alginfo.algtype =3D=3D (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported aead alg"); + rte_free(cdb); + return -ENOTSUP; + } + alginfo.key =3D (size_t)ses->aead_key.data; + alginfo.keylen =3D ses->aead_key.length; + alginfo.key_enc_flags =3D 0; + alginfo.key_type =3D RTA_DATA_IMM; + + if (ses->dir =3D=3D DIR_ENC) + shared_desc_len =3D cnstr_shdsc_gcm_encap( + cdb->sh_desc, true, swap, + &alginfo, + ses->iv.length, + ses->digest_length); + else + shared_desc_len =3D cnstr_shdsc_gcm_decap( + cdb->sh_desc, true, swap, + &alginfo, + ses->iv.length, + ses->digest_length); + } else { + caam_cipher_alg(ses, &alginfo_c); + if (alginfo_c.algtype =3D=3D (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported cipher alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_c.key =3D (size_t)ses->cipher_key.data; + alginfo_c.keylen =3D ses->cipher_key.length; + alginfo_c.key_enc_flags =3D 0; + alginfo_c.key_type =3D RTA_DATA_IMM; + + caam_auth_alg(ses, &alginfo_a); + if (alginfo_a.algtype =3D=3D (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported auth alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_a.key =3D (size_t)ses->auth_key.data; + alginfo_a.keylen =3D ses->auth_key.length; + alginfo_a.key_enc_flags =3D 0; + alginfo_a.key_type =3D RTA_DATA_IMM; + + cdb->sh_desc[0] =3D alginfo_c.keylen; + cdb->sh_desc[1] =3D alginfo_a.keylen; + err =3D rta_inline_query(IPSEC_AUTH_VAR_AES_DEC_BASE_DESC_LEN, + MIN_JOB_DESC_SIZE, + (unsigned int *)cdb->sh_desc, + &cdb->sh_desc[2], 2); + + if (err < 0) { + CAAM_JR_ERR("Crypto: Incorrect key lengths"); + rte_free(cdb); + return err; + } + if (cdb->sh_desc[2] & 1) + alginfo_c.key_type =3D RTA_DATA_IMM; + else { + alginfo_c.key =3D (size_t)caam_jr_mem_vtop( + (void *)(size_t)alginfo_c.key); + alginfo_c.key_type =3D RTA_DATA_PTR; + } + if (cdb->sh_desc[2] & (1<<1)) + alginfo_a.key_type =3D RTA_DATA_IMM; + else { + alginfo_a.key =3D (size_t)caam_jr_mem_vtop( + (void *)(size_t)alginfo_a.key); + alginfo_a.key_type =3D RTA_DATA_PTR; + } + cdb->sh_desc[0] =3D 0; + cdb->sh_desc[1] =3D 0; + cdb->sh_desc[2] =3D 0; + /* Auth_only_len is set as 0 here and it will be + * overwritten in fd for each packet. + */ + shared_desc_len =3D cnstr_shdsc_authenc(cdb->sh_desc, + true, swap, &alginfo_c, &alginfo_a, + ses->iv.length, 0, + ses->digest_length, ses->dir); + } + + if (shared_desc_len < 0) { + CAAM_JR_ERR("error in preparing command block"); + return shared_desc_len; + } + +#if CAAM_JR_DBG + SEC_DUMP_DESC(cdb->sh_desc); +#endif + + cdb->sh_hdr.hi.field.idlen =3D shared_desc_len; + + return 0; +} + /* @brief Poll the HW for already processed jobs in the JR * and silently discard the available jobs or notify them to UA * with indicated error code. @@ -100,6 +438,539 @@ hw_flush_job_ring(struct sec_job_ring_t *job_ring, } } =20 +/* @brief Poll the HW for already processed jobs in the JR + * and notify the available jobs to UA. + * + * @param [in] job_ring The job ring to poll. + * @param [in] limit The maximum number of jobs to notify. + * If set to negative value, all available jo= bs are + * notified. + * + * @retval >=3D0 for No of jobs notified to UA. + * @retval -1 for error + */ +static int +hw_poll_job_ring(struct sec_job_ring_t *job_ring, + struct rte_crypto_op **ops, int32_t limit, + struct caam_jr_qp *jr_qp) +{ + int32_t jobs_no_to_notify =3D 0; /* the number of done jobs to notify*/ + int32_t number_of_jobs_available =3D 0; + int32_t notified_descs_no =3D 0; + uint32_t sec_error_code =3D 0; + struct job_descriptor *current_desc; + phys_addr_t current_desc_addr; + phys_addr_t *temp_addr; + struct caam_jr_op_ctx *ctx; + + PMD_INIT_FUNC_TRACE(); + /* TODO check for ops have memory*/ + /* check here if any JR error that cannot be written + * in the output status word has occurred + */ + if (JR_REG_JRINT_JRE_EXTRACT(GET_JR_REG(JRINT, job_ring))) { + CAAM_JR_INFO("err received"); + sec_error_code =3D JR_REG_JRINT_ERR_TYPE_EXTRACT( + GET_JR_REG(JRINT, job_ring)); + if (unlikely(sec_error_code)) { + hw_job_ring_error_print(job_ring, sec_error_code); + return -1; + } + } + /* compute the number of jobs available in the job ring based on the + * producer and consumer index values. + */ + number_of_jobs_available =3D hw_get_no_finished_jobs(job_ring); + /* Compute the number of notifications that need to be raised to UA + * If limit > total number of done jobs -> notify all done jobs + * If limit =3D 0 -> error + * If limit < total number of done jobs -> notify a number + * of done jobs equal with limit + */ + jobs_no_to_notify =3D (limit > number_of_jobs_available) ? + number_of_jobs_available : limit; + CAAM_JR_DP_DEBUG( + "Jr[%p] pi[%d] ci[%d].limit =3D%d Available=3D%d.Jobs to notify=3D%d", + job_ring, job_ring->pidx, job_ring->cidx, + limit, number_of_jobs_available, jobs_no_to_notify); + + rte_smp_rmb(); + + while (jobs_no_to_notify > notified_descs_no) { + static uint64_t false_alarm; + static uint64_t real_poll; + + /* Get job status here */ + sec_error_code =3D job_ring->output_ring[job_ring->cidx].status; + /* Get completed descriptor */ + temp_addr =3D &(job_ring->output_ring[job_ring->cidx].desc); + current_desc_addr =3D (phys_addr_t)sec_read_addr(temp_addr); + + real_poll++; + /* todo check if it is false alarm no desc present */ + if (!current_desc_addr) { + false_alarm++; + printf("false alarm %" PRIu64 "real %" PRIu64 + " sec_err =3D0x%x cidx Index =3D0%d\n", + false_alarm, real_poll, + sec_error_code, job_ring->cidx); + rte_panic("CAAM JR descriptor NULL"); + return notified_descs_no; + } + current_desc =3D (struct job_descriptor *) + caam_jr_dma_ptov(current_desc_addr); + /* now increment the consumer index for the current job ring, + * AFTER saving job in temporary location! + */ + job_ring->cidx =3D SEC_CIRCULAR_COUNTER(job_ring->cidx, + SEC_JOB_RING_SIZE); + /* Signal that the job has been processed and the slot is free*/ + hw_remove_entries(job_ring, 1); + /*TODO for multiple ops, packets*/ + ctx =3D container_of(current_desc, struct caam_jr_op_ctx, jobdes); + if (unlikely(sec_error_code)) { + CAAM_JR_ERR("desc at cidx %d generated error 0x%x\n", + job_ring->cidx, sec_error_code); + hw_handle_job_ring_error(job_ring, sec_error_code); + //todo improve with exact errors + ctx->op->status =3D RTE_CRYPTO_OP_STATUS_ERROR; + jr_qp->rx_errs++; + } else { + ctx->op->status =3D RTE_CRYPTO_OP_STATUS_SUCCESS; +#if CAAM_JR_DBG + if (ctx->op->sym->m_dst) { + rte_hexdump(stdout, "PROCESSED", + rte_pktmbuf_mtod(ctx->op->sym->m_dst, void *), + rte_pktmbuf_data_len(ctx->op->sym->m_dst)); + } else { + rte_hexdump(stdout, "PROCESSED", + rte_pktmbuf_mtod(ctx->op->sym->m_src, void *), + rte_pktmbuf_data_len(ctx->op->sym->m_src)); + } +#endif + } + *ops =3D ctx->op; + caam_jr_op_ending(ctx); + ops++; + notified_descs_no++; + } + return notified_descs_no; +} + +static uint16_t +caam_jr_dequeue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + struct caam_jr_qp *jr_qp =3D (struct caam_jr_qp *)qp; + struct sec_job_ring_t *ring =3D jr_qp->ring; + int num_rx; + int ret; + + PMD_INIT_FUNC_TRACE(); + CAAM_JR_DP_DEBUG("Jr[%p]Polling. limit[%d]", ring, nb_ops); + + /* Poll job ring + * If nb_ops < 0 -> poll JR until no more notifications are available. + * If nb_ops > 0 -> poll JR until limit is reached. + */ + + /* Run hw poll job ring */ + num_rx =3D hw_poll_job_ring(ring, ops, nb_ops, jr_qp); + if (num_rx < 0) { + CAAM_JR_ERR("Error polling SEC engine (%d)", num_rx); + return 0; + } + + CAAM_JR_DP_DEBUG("Jr[%p].Jobs notified[%d]. ", ring, num_rx); + + if (ring->jr_mode =3D=3D SEC_NOTIFICATION_TYPE_NAPI) { + if (num_rx < nb_ops) { + ret =3D caam_jr_enable_irqs(ring->irq_fd); + SEC_ASSERT(ret =3D=3D 0, ret, + "Failed to enable irqs for job ring %p", ring); + } + } else if (ring->jr_mode =3D=3D SEC_NOTIFICATION_TYPE_IRQ) { + + /* Always enable IRQ generation when in pure IRQ mode */ + ret =3D caam_jr_enable_irqs(ring->irq_fd); + SEC_ASSERT(ret =3D=3D 0, ret, + "Failed to enable irqs for job ring %p", ring); + } + + jr_qp->rx_pkts +=3D num_rx; + + return num_rx; +} + +static inline struct caam_jr_op_ctx * +build_auth_only(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym =3D op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + rte_iova_t start_addr; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + struct sec_job_descriptor_t *jobdescr; + + PMD_INIT_FUNC_TRACE(); + ctx =3D caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op =3D op; + + cdb =3D ses->cdb; + sdesc_offset =3D (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + start_addr =3D rte_pktmbuf_iova(sym->m_src); + + jobdescr =3D (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* output */ + SEC_JD_SET_OUT_PTR(jobdescr, (uint64_t)sym->auth.digest.phys_addr, + 0, ses->digest_length); + + /*input */ + if (is_decode(ses)) { + sg =3D &ctx->sg[0]; + SEC_JD_SET_IN_PTR(jobdescr, + (uint64_t)caam_jr_vtop_ctx(ctx, sg), 0, + (sym->auth.data.length + ses->digest_length)); + /* enabling sg list */ + (jobdescr)->seq_in.command.word |=3D 0x01000000; + + /* hash result or digest, save digest first */ + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); + sg->ptr =3D cpu_to_caam64(start_addr + sym->auth.data.offset); + sg->len =3D cpu_to_caam32(sym->auth.data.length); + +#if CAAM_JR_DBG + rte_hexdump(stdout, "ICV", ctx->digest, ses->digest_length); +#endif + /* let's check digest by hw */ + sg++; + sg->ptr =3D cpu_to_caam64(caam_jr_vtop_ctx(ctx, ctx->digest)); + sg->len =3D cpu_to_caam32(ses->digest_length); + /* last element*/ + sg->len |=3D cpu_to_caam32(SEC4_SG_LEN_FIN); + } else { + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)start_addr, + sym->auth.data.offset, sym->auth.data.length); + } + return ctx; +} + +static inline struct caam_jr_op_ctx * +build_cipher_only(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym =3D op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + rte_iova_t src_start_addr, dst_start_addr; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t *IV_ptr =3D rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + + PMD_INIT_FUNC_TRACE(); + ctx =3D caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op =3D op; + cdb =3D ses->cdb; + sdesc_offset =3D (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + src_start_addr =3D rte_pktmbuf_iova(sym->m_src); + if (sym->m_dst) + dst_start_addr =3D rte_pktmbuf_iova(sym->m_dst); + else + dst_start_addr =3D src_start_addr; + + jobdescr =3D (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + +#if CAAM_JR_DBG + CAAM_JR_INFO("mbuf offset =3D%d, cipher offset =3D %d, length =3D%d+%d", + sym->m_src->data_off, sym->cipher.data.offset, + sym->cipher.data.length, ses->iv.length); +#endif + /* output */ + SEC_JD_SET_OUT_PTR(jobdescr, (uint64_t)dst_start_addr, + sym->cipher.data.offset, + sym->cipher.data.length + ses->iv.length); + + /*input */ + sg =3D &ctx->sg[0]; + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_vtop_ctx(ctx, sg), 0, + sym->cipher.data.length + ses->iv.length); + /*enabling sg bit */ + (jobdescr)->seq_in.command.word |=3D 0x01000000; + + sg->ptr =3D cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len =3D cpu_to_caam32(ses->iv.length); + + sg =3D &ctx->sg[1]; + sg->ptr =3D cpu_to_caam64(src_start_addr + sym->cipher.data.offset); + sg->len =3D cpu_to_caam32(sym->cipher.data.length); + /* last element*/ + sg->len |=3D cpu_to_caam32(SEC4_SG_LEN_FIN); + + return ctx; +} + +static inline struct caam_jr_op_ctx * +build_cipher_auth(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym =3D op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + rte_iova_t src_start_addr, dst_start_addr; + uint32_t length =3D 0; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t *IV_ptr =3D rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + uint32_t auth_only_len; + + PMD_INIT_FUNC_TRACE(); + auth_only_len =3D op->sym->auth.data.length - + op->sym->cipher.data.length; + + src_start_addr =3D rte_pktmbuf_iova(sym->m_src); + if (sym->m_dst) + dst_start_addr =3D rte_pktmbuf_iova(sym->m_dst); + else + dst_start_addr =3D src_start_addr; + + ctx =3D caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op =3D op; + cdb =3D ses->cdb; + sdesc_offset =3D (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr =3D (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + + /* input */ + sg =3D &ctx->sg[0]; + if (is_encode(ses)) { + sg->ptr =3D cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len =3D cpu_to_caam32(ses->iv.length); + length +=3D ses->iv.length; + + sg++; + sg->ptr =3D cpu_to_caam64(src_start_addr + sym->auth.data.offset); + sg->len =3D cpu_to_caam32(sym->auth.data.length); + length +=3D sym->auth.data.length; + /* last element*/ + sg->len |=3D cpu_to_caam32(SEC4_SG_LEN_FIN); + } else { + sg->ptr =3D cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len =3D cpu_to_caam32(ses->iv.length); + length +=3D ses->iv.length; + + sg++; + sg->ptr =3D cpu_to_caam64(src_start_addr + sym->auth.data.offset); + sg->len =3D cpu_to_caam32(sym->auth.data.length); + length +=3D sym->auth.data.length; + + rte_memcpy(ctx->digest, sym->auth.digest.data, + ses->digest_length); + sg++; + sg->ptr =3D cpu_to_caam64(caam_jr_dma_vtop(ctx->digest)); + sg->len =3D cpu_to_caam32(ses->digest_length); + length +=3D ses->digest_length; + /* last element*/ + sg->len |=3D cpu_to_caam32(SEC4_SG_LEN_FIN); + } + + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_dma_vtop(&ctx->sg[0]), 0, + length); + /* set sg bit */ + (jobdescr)->seq_in.command.word |=3D 0x01000000; + + /* output */ + sg =3D &ctx->sg[6]; + + sg->ptr =3D cpu_to_caam64(dst_start_addr + sym->cipher.data.offset); + sg->len =3D cpu_to_caam32(sym->cipher.data.length); + length =3D sym->cipher.data.length; + + if (is_encode(ses)) { + /* set auth output */ + sg++; + sg->ptr =3D cpu_to_caam64(sym->auth.digest.phys_addr); + sg->len =3D cpu_to_caam32(ses->digest_length); + length +=3D ses->digest_length; + } + /* last element*/ + sg->len |=3D cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_OUT_PTR(jobdescr, + (uint64_t)caam_jr_dma_vtop(&ctx->sg[6]), 0, length); + /* set sg bit */ + (jobdescr)->seq_out.command.word |=3D 0x01000000; + + /* Auth_only_len is set as 0 in descriptor and it is + * overwritten here in the jd which will update + * the DPOVRD reg. + */ + if (auth_only_len) + /* set sg bit */ + (jobdescr)->dpovrd =3D 0x80000000 | auth_only_len; + + return ctx; +} +static int +caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) +{ + struct sec_job_ring_t *ring =3D qp->ring; + struct caam_jr_session *ses; + struct caam_jr_op_ctx *ctx =3D NULL; + struct sec_job_descriptor_t *jobdescr __rte_unused; + + PMD_INIT_FUNC_TRACE(); + switch (op->sess_type) { + case RTE_CRYPTO_OP_WITH_SESSION: + ses =3D (struct caam_jr_session *) + get_sym_session_private_data(op->sym->session, + cryptodev_driver_id); + break; + default: + CAAM_JR_DP_ERR("sessionless crypto op not supported"); + qp->tx_errs++; + return -1; + } + + if (unlikely(!ses->qp || ses->qp !=3D qp)) { + CAAM_JR_DP_DEBUG("Old:sess->qp=3D%p New qp =3D %p\n", ses->qp, qp); + ses->qp =3D qp; + caam_jr_prep_cdb(ses); + } + + if (rte_pktmbuf_is_contiguous(op->sym->m_src)) { + if (is_auth_cipher(ses)) + ctx =3D build_cipher_auth(op, ses); + else if (is_aead(ses)) + goto err1; + else if (is_auth_only(ses)) + ctx =3D build_auth_only(op, ses); + else if (is_cipher_only(ses)) + ctx =3D build_cipher_only(op, ses); + } else { + if (is_aead(ses)) + goto err1; + } +err1: + if (unlikely(!ctx)) { + qp->tx_errs++; + CAAM_JR_ERR("not supported sec op"); + return -1; + } +#if CAAM_JR_DBG + if (is_decode(ses)) + rte_hexdump(stdout, "DECODE", + rte_pktmbuf_mtod(op->sym->m_src, void *), + rte_pktmbuf_data_len(op->sym->m_src)); + else + rte_hexdump(stdout, "ENCODE", + rte_pktmbuf_mtod(op->sym->m_src, void *), + rte_pktmbuf_data_len(op->sym->m_src)); + + printf("\n JD before conversion\n"); + for (int i =3D 0; i < 12; i++) + printf("\n 0x%08x", ctx->jobdes.desc[i]); +#endif + + CAAM_JR_DP_DEBUG("Jr[%p] pi[%d] ci[%d].Before sending desc", + ring, ring->pidx, ring->cidx); + + /* todo - do we want to retry */ + if (SEC_JOB_RING_IS_FULL(ring->pidx, ring->cidx, + SEC_JOB_RING_SIZE, SEC_JOB_RING_SIZE)) { + CAAM_JR_DP_DEBUG("Ring FULL Jr[%p] pi[%d] ci[%d].Size =3D %d", + ring, ring->pidx, ring->cidx, SEC_JOB_RING_SIZE); + caam_jr_op_ending(ctx); + qp->tx_ring_full++; + return -EBUSY; + } + +#if CORE_BYTE_ORDER !=3D CAAM_BYTE_ORDER + jobdescr =3D (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + jobdescr->deschdr.command.word =3D + cpu_to_caam32(jobdescr->deschdr.command.word); + jobdescr->sd_ptr =3D cpu_to_caam64(jobdescr->sd_ptr); + jobdescr->seq_out.command.word =3D + cpu_to_caam32(jobdescr->seq_out.command.word); + jobdescr->seq_out_ptr =3D cpu_to_caam64(jobdescr->seq_out_ptr); + jobdescr->out_ext_length =3D cpu_to_caam32(jobdescr->out_ext_length); + jobdescr->seq_in.command.word =3D + cpu_to_caam32(jobdescr->seq_in.command.word); + jobdescr->seq_in_ptr =3D cpu_to_caam64(jobdescr->seq_in_ptr); + jobdescr->in_ext_length =3D cpu_to_caam32(jobdescr->in_ext_length); + jobdescr->load_dpovrd.command.word =3D + cpu_to_caam32(jobdescr->load_dpovrd.command.word); + jobdescr->dpovrd =3D cpu_to_caam32(jobdescr->dpovrd); +#endif + + /* Set ptr in input ring to current descriptor */ + sec_write_addr(&ring->input_ring[ring->pidx], + (phys_addr_t)caam_jr_vtop_ctx(ctx, ctx->jobdes.desc)); + rte_smp_wmb(); + + /* Notify HW that a new job is enqueued */ + hw_enqueue_desc_on_job_ring(ring); + + /* increment the producer index for the current job ring */ + ring->pidx =3D SEC_CIRCULAR_COUNTER(ring->pidx, SEC_JOB_RING_SIZE); + + return 0; +} + +static uint16_t +caam_jr_enqueue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and queuepair */ + uint32_t loop; + int32_t ret; + struct caam_jr_qp *jr_qp =3D (struct caam_jr_qp *)qp; + uint16_t num_tx =3D 0; + + PMD_INIT_FUNC_TRACE(); + /*Prepare each packet which is to be sent*/ + for (loop =3D 0; loop < nb_ops; loop++) { + ret =3D caam_jr_enqueue_op(ops[loop], jr_qp); + if (!ret) + num_tx++; + } + + jr_qp->tx_pkts +=3D num_tx; + + return num_tx; +} + /* Release queue pair */ static int caam_jr_queue_pair_release(struct rte_cryptodev *dev, @@ -726,8 +1597,8 @@ caam_jr_dev_init(const char *name, dev->dev_ops =3D &caam_jr_ops; =20 /* register rx/tx burst functions for data path */ - dev->dequeue_burst =3D NULL; - dev->enqueue_burst =3D NULL; + dev->dequeue_burst =3D caam_jr_dequeue_burst; + dev->enqueue_burst =3D caam_jr_enqueue_burst; dev->feature_flags =3D RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | diff --git a/drivers/crypto/caam_jr/caam_jr_desc.h b/drivers/crypto/caam_jr= /caam_jr_desc.h new file mode 100644 index 000000000..6683ea835 --- /dev/null +++ b/drivers/crypto/caam_jr/caam_jr_desc.h @@ -0,0 +1,285 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2017-2018 NXP + */ + +#ifndef CAAM_JR_DESC_H +#define CAAM_JR_DESC_H + +#define CMD_HDR_CTYPE_SD 0x16 +#define CMD_HDR_CTYPE_JD 0x17 + +/* The maximum size of a SEC descriptor, in WORDs (32 bits). */ +#define MAX_DESC_SIZE_WORDS 64 + +/* + * Macros manipulating descriptors + */ +/* Macro for setting the SD pointer in a JD. Common for all protocols + * supported by the SEC driver. + */ +#define SEC_JD_SET_SD(descriptor, ptr, len) { \ + (descriptor)->sd_ptr =3D (ptr); \ + (descriptor)->deschdr.command.jd.shr_desc_len =3D (len); \ +} + +/* Macro for setting a pointer to the job which this descriptor processes. + * It eases the lookup procedure for identifying the descriptor that has + * completed. + */ +#define SEC_JD_SET_JOB_PTR(descriptor, ptr) \ + ((descriptor)->job_ptr =3D (ptr)) + +/* Macro for setting up a JD. The structure of the JD is common across all + * supported protocols, thus its structure is identical. + */ +#define SEC_JD_INIT(descriptor) ({ \ + /* CTYPE =3D job descriptor \ + * RSMS, DNR =3D 0 + * ONE =3D 1 + * Start Index =3D 0 + * ZRO,TD, MTD =3D 0 + * SHR =3D 1 (there's a shared descriptor referenced + * by this job descriptor,pointer in next word) + * REO =3D 1 (execute job descr. first, shared descriptor + * after) + * SHARE =3D DEFER + * Descriptor Length =3D 0 ( to be completed @ runtime ) */ \ + (descriptor)->deschdr.command.word =3D 0xB0801C0D; \ + /* + * CTYPE =3D SEQ OUT command * Scater Gather Flag =3D 0 + * (can be updated @ runtime) PRE =3D 0 * EXT =3D 1 + * (data length is in next word, following the * command) + * RTO =3D 0 */ \ + (descriptor)->seq_out.command.word =3D 0xF8400000; /**/ \ + /* + * CTYPE =3D SEQ IN command + * Scater Gather Flag =3D 0 (can be updated @ runtime) + * PRE =3D 0 + * EXT =3D 1 ( data length is in next word, following the + * command) + * RTO =3D 0 */ \ + (descriptor)->seq_in.command.word =3D 0xF0400000; /**/ \ + /* + * In order to be compatible with QI scenarios, the DPOVRD value + * loaded must be formated like this: + * DPOVRD_EN (1b) | Res| DPOVRD Value (right aligned). */ \ + (descriptor)->load_dpovrd.command.word =3D 0x16870004; \ + /* By default, DPOVRD mechanism is disabled, thus the value to be + * LOAD-ed through the above descriptor command will be + * 0x0000_0000. */ \ + (descriptor)->dpovrd =3D 0x00000000; \ +}) + +/* Macro for setting the pointer to the input buffer in the JD, according = to + * the parameters set by the user in the ::sec_packet_t structure. + */ +#define SEC_JD_SET_IN_PTR(descriptor, phys_addr, offset, length) { \ + (descriptor)->seq_in_ptr =3D (phys_addr) + (offset); \ + (descriptor)->in_ext_length =3D (length); \ +} + +/* Macro for setting the pointer to the output buffer in the JD, according= to + * the parameters set by the user in the ::sec_packet_t structure. + */ +#define SEC_JD_SET_OUT_PTR(descriptor, phys_addr, offset, length) { \ + (descriptor)->seq_out_ptr =3D (phys_addr) + (offset); \ + (descriptor)->out_ext_length =3D (length); \ +} + +/* Macro for setting the Scatter-Gather flag in the SEQ IN command. Used i= n + * case the input buffer is split in multiple buffers, according to the us= er + * specification. + */ +#define SEC_JD_SET_SG_IN(descriptor) \ + ((descriptor)->seq_in.command.field.sgf =3D 1) + +/* Macro for setting the Scatter-Gather flag in the SEQ OUT command. Used = in + * case the output buffer is split in multiple buffers, according to the u= ser + * specification. + */ +#define SEC_JD_SET_SG_OUT(descriptor) \ + ((descriptor)->seq_out.command.field.sgf =3D 1) + +#define SEC_JD_SET_DPOVRD(descriptor) \ + +/* Macro for retrieving a descriptor's length. Works for both SD and JD. *= / +#define SEC_GET_DESC_LEN(descriptor) \ + (((struct descriptor_header_s *)(descriptor))->command.sd.ctype =3D=3D \ + CMD_HDR_CTYPE_SD ? ((struct descriptor_header_s *) \ + (descriptor))->command.sd.desclen : \ + ((struct descriptor_header_s *)(descriptor))->command.jd.desclen) + +/* Helper macro for dumping the hex representation of a descriptor */ +#define SEC_DUMP_DESC(descriptor) { \ + int __i; \ + CAAM_JR_INFO("Des@ 0x%08x\n", (uint32_t)((uint32_t *)(descriptor)));\ + for (__i =3D 0; \ + __i < SEC_GET_DESC_LEN(descriptor); \ + __i++) { \ + printf("0x%08x: 0x%08x\n", \ + (uint32_t)(((uint32_t *)(descriptor)) + __i), \ + *(((uint32_t *)(descriptor)) + __i)); \ + } \ +} +/* Union describing a descriptor header. + */ +struct descriptor_header_s { + union { + uint32_t word; + struct { + /* 4 */ unsigned int ctype:5; + /* 5 */ unsigned int res1:2; + /* 7 */ unsigned int dnr:1; + /* 8 */ unsigned int one:1; + /* 9 */ unsigned int res2:1; + /* 10 */ unsigned int start_idx:6; + /* 16 */ unsigned int res3:2; + /* 18 */ unsigned int cif:1; + /* 19 */ unsigned int sc:1; + /* 20 */ unsigned int pd:1; + /* 21 */ unsigned int res4:1; + /* 22 */ unsigned int share:2; + /* 24 */ unsigned int res5:2; + /* 26 */ unsigned int desclen:6; + } sd; + struct { + /* TODO only below struct members are corrected, + * all others also need to be reversed please verify it + */ + /* 0 */ unsigned int desclen:7; + /* 7 */ unsigned int res4:1; + /* 8 */ unsigned int share:3; + /* 11 */ unsigned int reo:1; + /* 12 */ unsigned int shr:1; + /* 13 */ unsigned int mtd:1; + /* 14 */ unsigned int td:1; + /* 15 */ unsigned int zero:1; + /* 16 */ unsigned int shr_desc_len:6; + /* 22 */ unsigned int res2:1; + /* 23 */ unsigned int one:1; + /* 24 */ unsigned int dnr:1; + /* 25 */ unsigned int rsms:1; + /* 26 */ unsigned int res1:1; + /* 27 */ unsigned int ctype:5; + } jd; + } __rte_packed command; +} __rte_packed; + +/* Union describing a KEY command in a descriptor. + */ +struct key_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int cls:2; + unsigned int sgf:1; + unsigned int imm:1; + unsigned int enc:1; + unsigned int nwb:1; + unsigned int ekt:1; + unsigned int kdest:4; + unsigned int tk:1; + unsigned int rsvd1:5; + unsigned int length:10; + } __rte_packed field; + } __rte_packed command; +} __rte_packed; + +/* Union describing a PROTOCOL command + * in a descriptor. + */ +struct protocol_operation_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int optype:3; + unsigned char protid; + unsigned short protinfo; + } __rte_packed field; + } __rte_packed command; +} __rte_packed; + +/* Union describing a SEQIN command in a + * descriptor. + */ +struct seq_in_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int res1:1; + unsigned int inl:1; + unsigned int sgf:1; + unsigned int pre:1; + unsigned int ext:1; + unsigned int rto:1; + unsigned int rjd:1; + unsigned int res2:4; + unsigned int length:16; + } field; + } __rte_packed command; +} __rte_packed; + +/* Union describing a SEQOUT command in a + * descriptor. + */ +struct seq_out_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int res1:2; + unsigned int sgf:1; + unsigned int pre:1; + unsigned int ext:1; + unsigned int rto:1; + unsigned int res2:5; + unsigned int length:16; + } field; + } __rte_packed command; +} __rte_packed; + +struct load_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int class:2; + unsigned int sgf:1; + unsigned int imm:1; + unsigned int dst:7; + unsigned char offset; + unsigned char length; + } fields; + } __rte_packed command; +} __rte_packed; + +/* Structure encompassing a general shared descriptor of maximum + * size (64 WORDs). Usually, other specific shared descriptor structures + * will be type-casted to this one + * this one. + */ +struct sec_sd_t { + uint32_t rsvd[MAX_DESC_SIZE_WORDS]; +} __attribute__((packed, aligned(64))); + +/* Structure encompassing a job descriptor which processes + * a single packet from a context. The job descriptor references + * a shared descriptor from a SEC context. + */ +struct sec_job_descriptor_t { + struct descriptor_header_s deschdr; + dma_addr_t sd_ptr; + struct seq_out_command_s seq_out; + dma_addr_t seq_out_ptr; + uint32_t out_ext_length; + struct seq_in_command_s seq_in; + dma_addr_t seq_in_ptr; + uint32_t in_ext_length; + struct load_command_s load_dpovrd; + uint32_t dpovrd; +} __attribute__((packed, aligned(64))); + +#endif --=20 2.17.1