From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 99CCE45F2A; Tue, 24 Dec 2024 08:38:23 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 480824065A; Tue, 24 Dec 2024 08:38:13 +0100 (CET) Received: from mx0a-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3B016402AD for ; Tue, 24 Dec 2024 08:38:11 +0100 (CET) Received: from pps.filterd (m0431384.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BO5u8Gl022793; Mon, 23 Dec 2024 23:38:10 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=D XuYD52dpHpKAkZIQHDdCiDm/MwaV5Kqt6xLA1DzNBM=; b=MIcFx94QAcC40wvv3 afZJ56SlHEQmFzGQq/DmM40hD8GpS5sDQNocySIBBBU1TRAoTeIpXAj/Kn59ojP/ eEQgGSwnMNd92PbDJ4UkqcrOHzY6zmZzQbjMbbMWI2viRudEhhXEaIMQ1ZJQGvqn po1Ze113DNcZX3cFSAvrHGGg1TwKqsM/9hWErhoiNoaFkPo0yCoHQCE5x35t92Rn T577IkONBw2tuv+5saIzmt614rnX8X1nH7UpnnqSwuOp7x+q3JrohV5TNbk0H4nT 1U1FO8F1aWCfcJq5YxisQjYjF2QixnQfUp+1lkX8pIKvxB61OLcSBFuMurfQX6xA IouHg== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 43qq83r8gr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 23 Dec 2024 23:38:10 -0800 (PST) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 23 Dec 2024 23:38:08 -0800 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 23 Dec 2024 23:38:08 -0800 Received: from IN-lckQE5Rwctls.marvell.com (IN-lckQE5Rwctls.marvell.com [10.28.163.68]) by maili.marvell.com (Postfix) with ESMTP id A0A703F705D; Mon, 23 Dec 2024 23:38:05 -0800 (PST) From: Gowrishankar Muthukrishnan To: , Akhil Goyal , Maxime Coquelin , Chenbo Xia , Fan Zhang , Jay Zhou CC: , , Rajesh Mudimadugula , Gowrishankar Muthukrishnan Subject: [v1 07/16] vhost: add asymmetric RSA support Date: Tue, 24 Dec 2024 13:07:05 +0530 Message-ID: <54deff4d0a08b42b2d26231d578a80b30679e238.1735025264.git.gmuthukrishn@marvell.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: owv_8rW1fax0drO4OIQRiniiNz6y5T-n X-Proofpoint-ORIG-GUID: owv_8rW1fax0drO4OIQRiniiNz6y5T-n X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Support asymmetric RSA crypto operations in vhost-user. Signed-off-by: Gowrishankar Muthukrishnan --- lib/vhost/vhost_crypto.c | 504 ++++++++++++++++++++++++++++++++++++--- lib/vhost/vhost_user.h | 33 ++- 2 files changed, 498 insertions(+), 39 deletions(-) diff --git a/lib/vhost/vhost_crypto.c b/lib/vhost/vhost_crypto.c index 7caf6d9afa..6ce06ef42b 100644 --- a/lib/vhost/vhost_crypto.c +++ b/lib/vhost/vhost_crypto.c @@ -54,6 +54,14 @@ RTE_LOG_REGISTER_SUFFIX(vhost_crypto_logtype, crypto, INFO); */ #define vhost_crypto_desc vring_desc +struct vhost_crypto_session { + union { + struct rte_cryptodev_asym_session *asym; + struct rte_cryptodev_sym_session *sym; + }; + enum rte_crypto_op_type type; +}; + static int cipher_algo_transform(uint32_t virtio_cipher_algo, enum rte_crypto_cipher_algorithm *algo) @@ -206,8 +214,10 @@ struct __rte_cache_aligned vhost_crypto { uint64_t last_session_id; - uint64_t cache_session_id; - struct rte_cryptodev_sym_session *cache_session; + uint64_t cache_sym_session_id; + struct rte_cryptodev_sym_session *cache_sym_session; + uint64_t cache_asym_session_id; + struct rte_cryptodev_asym_session *cache_asym_session; /** socket id for the device */ int socket_id; @@ -237,7 +247,7 @@ struct vhost_crypto_data_req { static int transform_cipher_param(struct rte_crypto_sym_xform *xform, - VhostUserCryptoSessionParam *param) + VhostUserCryptoSymSessionParam *param) { int ret; @@ -273,7 +283,7 @@ transform_cipher_param(struct rte_crypto_sym_xform *xform, static int transform_chain_param(struct rte_crypto_sym_xform *xforms, - VhostUserCryptoSessionParam *param) + VhostUserCryptoSymSessionParam *param) { struct rte_crypto_sym_xform *xform_cipher, *xform_auth; int ret; @@ -334,17 +344,17 @@ transform_chain_param(struct rte_crypto_sym_xform *xforms, } static void -vhost_crypto_create_sess(struct vhost_crypto *vcrypto, +vhost_crypto_create_sym_sess(struct vhost_crypto *vcrypto, VhostUserCryptoSessionParam *sess_param) { struct rte_crypto_sym_xform xform1 = {0}, xform2 = {0}; struct rte_cryptodev_sym_session *session; int ret; - switch (sess_param->op_type) { + switch (sess_param->u.sym_sess.op_type) { case VIRTIO_CRYPTO_SYM_OP_NONE: case VIRTIO_CRYPTO_SYM_OP_CIPHER: - ret = transform_cipher_param(&xform1, sess_param); + ret = transform_cipher_param(&xform1, &sess_param->u.sym_sess); if (unlikely(ret)) { VC_LOG_ERR("Error transform session msg (%i)", ret); sess_param->session_id = ret; @@ -352,7 +362,7 @@ vhost_crypto_create_sess(struct vhost_crypto *vcrypto, } break; case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING: - if (unlikely(sess_param->hash_mode != + if (unlikely(sess_param->u.sym_sess.hash_mode != VIRTIO_CRYPTO_SYM_HASH_MODE_AUTH)) { sess_param->session_id = -VIRTIO_CRYPTO_NOTSUPP; VC_LOG_ERR("Error transform session message (%i)", @@ -362,7 +372,7 @@ vhost_crypto_create_sess(struct vhost_crypto *vcrypto, xform1.next = &xform2; - ret = transform_chain_param(&xform1, sess_param); + ret = transform_chain_param(&xform1, &sess_param->u.sym_sess); if (unlikely(ret)) { VC_LOG_ERR("Error transform session message (%i)", ret); sess_param->session_id = ret; @@ -402,22 +412,264 @@ vhost_crypto_create_sess(struct vhost_crypto *vcrypto, vcrypto->last_session_id++; } +static int +tlv_decode(uint8_t *tlv, uint8_t type, uint8_t **data, size_t *data_len) +{ + size_t tlen = -EINVAL, len; + + if (tlv[0] != type) + return -EINVAL; + + if (tlv[1] == 0x82) { + len = (tlv[2] << 8) | tlv[3]; + *data = rte_malloc(NULL, len, 0); + rte_memcpy(*data, &tlv[4], len); + tlen = len + 4; + } else if (tlv[1] == 0x81) { + len = tlv[2]; + *data = rte_malloc(NULL, len, 0); + rte_memcpy(*data, &tlv[3], len); + tlen = len + 3; + } else { + len = tlv[1]; + *data = rte_malloc(NULL, len, 0); + rte_memcpy(*data, &tlv[2], len); + tlen = len + 2; + } + + *data_len = len; + return tlen; +} + +static int +virtio_crypto_asym_rsa_der_to_xform(uint8_t *der, size_t der_len, + struct rte_crypto_asym_xform *xform) +{ + uint8_t *n = NULL, *e = NULL, *d = NULL, *p = NULL, *q = NULL, *dp = NULL, + *dq = NULL, *qinv = NULL, *v = NULL, *tlv; + size_t nlen, elen, dlen, plen, qlen, dplen, dqlen, qinvlen, vlen; + int len; + + RTE_SET_USED(der_len); + + if (der[0] != 0x30) + return -EINVAL; + + if (der[1] == 0x82) + tlv = &der[4]; + else if (der[1] == 0x81) + tlv = &der[3]; + else + return -EINVAL; + + len = tlv_decode(tlv, 0x02, &v, &vlen); + if (len < 0 || v[0] != 0x0 || vlen != 1) { + len = -EINVAL; + goto _error; + } + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &n, &nlen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &e, &elen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &d, &dlen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &p, &plen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &q, &qlen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &dp, &dplen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &dq, &dqlen); + if (len < 0) + goto _error; + + tlv = tlv + len; + len = tlv_decode(tlv, 0x02, &qinv, &qinvlen); + if (len < 0) + goto _error; + + xform->rsa.n.data = n; + xform->rsa.n.length = nlen; + xform->rsa.e.data = e; + xform->rsa.e.length = elen; + xform->rsa.d.data = d; + xform->rsa.d.length = dlen; + xform->rsa.qt.p.data = p; + xform->rsa.qt.p.length = plen; + xform->rsa.qt.q.data = q; + xform->rsa.qt.q.length = qlen; + xform->rsa.qt.dP.data = dp; + xform->rsa.qt.dP.length = dplen; + xform->rsa.qt.dQ.data = dq; + xform->rsa.qt.dQ.length = dqlen; + xform->rsa.qt.qInv.data = qinv; + xform->rsa.qt.qInv.length = qinvlen; + + RTE_ASSERT((tlv + len - &der[0]) == der_len); + return 0; +_error: + rte_free(v); + rte_free(n); + rte_free(e); + rte_free(d); + rte_free(p); + rte_free(q); + rte_free(dp); + rte_free(dq); + rte_free(qinv); + return len; +} + +static int +transform_rsa_param(struct rte_crypto_asym_xform *xform, + VhostUserCryptoAsymSessionParam *param) +{ + int ret = -EINVAL; + + ret = virtio_crypto_asym_rsa_der_to_xform(param->key_buf, param->key_len, xform); + if (ret < 0) + goto _error; + + switch (param->u.rsa.padding_algo) { + case VIRTIO_CRYPTO_RSA_RAW_PADDING: + xform->rsa.padding.type = RTE_CRYPTO_RSA_PADDING_NONE; + break; + case VIRTIO_CRYPTO_RSA_PKCS1_PADDING: + xform->rsa.padding.type = RTE_CRYPTO_RSA_PADDING_PKCS1_5; + break; + default: + VC_LOG_ERR("Unknown padding type"); + goto _error; + } + + xform->rsa.key_type = RTE_RSA_KEY_TYPE_QT; + xform->xform_type = RTE_CRYPTO_ASYM_XFORM_RSA; +_error: + return ret; +} + +static void +vhost_crypto_create_asym_sess(struct vhost_crypto *vcrypto, + VhostUserCryptoSessionParam *sess_param) +{ + struct rte_cryptodev_asym_session *session = NULL; + struct vhost_crypto_session *vhost_session; + struct rte_crypto_asym_xform xform = {0}; + int ret; + + switch (sess_param->u.asym_sess.algo) { + case VIRTIO_CRYPTO_AKCIPHER_RSA: + ret = transform_rsa_param(&xform, &sess_param->u.asym_sess); + if (unlikely(ret)) { + VC_LOG_ERR("Error transform session msg (%i)", ret); + sess_param->session_id = ret; + return; + } + break; + default: + VC_LOG_ERR("Invalid op algo"); + sess_param->session_id = -VIRTIO_CRYPTO_ERR; + return; + } + + ret = rte_cryptodev_asym_session_create(vcrypto->cid, &xform, + vcrypto->sess_pool, (void *)&session); + if (!session) { + VC_LOG_ERR("Failed to create session"); + sess_param->session_id = -VIRTIO_CRYPTO_ERR; + return; + } + + /* insert session to map */ + vhost_session = rte_malloc(NULL, sizeof(*vhost_session), 0); + if (vhost_session == NULL) { + VC_LOG_ERR("Failed to alloc session memory"); + sess_param->session_id = -VIRTIO_CRYPTO_ERR; + return; + } + + vhost_session->type = RTE_CRYPTO_OP_TYPE_ASYMMETRIC; + vhost_session->asym = session; + if ((rte_hash_add_key_data(vcrypto->session_map, + &vcrypto->last_session_id, vhost_session) < 0)) { + VC_LOG_ERR("Failed to insert session to hash table"); + + if (rte_cryptodev_asym_session_free(vcrypto->cid, session) < 0) + VC_LOG_ERR("Failed to free session"); + sess_param->session_id = -VIRTIO_CRYPTO_ERR; + return; + } + + VC_LOG_INFO("Session %"PRIu64" created for vdev %i.", + vcrypto->last_session_id, vcrypto->dev->vid); + + sess_param->session_id = vcrypto->last_session_id; + vcrypto->last_session_id++; +} + +static void +vhost_crypto_create_sess(struct vhost_crypto *vcrypto, + VhostUserCryptoSessionParam *sess_param) +{ + if (sess_param->op_code == VIRTIO_CRYPTO_AKCIPHER_CREATE_SESSION) + vhost_crypto_create_asym_sess(vcrypto, sess_param); + else + vhost_crypto_create_sym_sess(vcrypto, sess_param); +} + static int vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id) { - struct rte_cryptodev_sym_session *session; + struct rte_cryptodev_asym_session *asym_session = NULL; + struct rte_cryptodev_sym_session *sym_session = NULL; + struct vhost_crypto_session *vhost_session = NULL; uint64_t sess_id = session_id; int ret; ret = rte_hash_lookup_data(vcrypto->session_map, &sess_id, - (void **)&session); - + (void **)&vhost_session); if (unlikely(ret < 0)) { - VC_LOG_ERR("Failed to delete session %"PRIu64".", session_id); + VC_LOG_ERR("Failed to find session for id %"PRIu64".", session_id); + return -VIRTIO_CRYPTO_INVSESS; + } + + if (vhost_session->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + sym_session = vhost_session->sym; + } else if (vhost_session->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + asym_session = vhost_session->asym; + } else { + VC_LOG_ERR("Invalid session for id %"PRIu64".", session_id); return -VIRTIO_CRYPTO_INVSESS; } - if (rte_cryptodev_sym_session_free(vcrypto->cid, session) < 0) { + if (sym_session != NULL && + rte_cryptodev_sym_session_free(vcrypto->cid, sym_session) < 0) { + VC_LOG_DBG("Failed to free session"); + return -VIRTIO_CRYPTO_ERR; + } + + if (asym_session != NULL && + rte_cryptodev_asym_session_free(vcrypto->cid, asym_session) < 0) { VC_LOG_DBG("Failed to free session"); return -VIRTIO_CRYPTO_ERR; } @@ -430,6 +682,7 @@ vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id) VC_LOG_INFO("Session %"PRIu64" deleted for vdev %i.", sess_id, vcrypto->dev->vid); + rte_free(vhost_session); return 0; } @@ -1123,6 +1376,118 @@ prepare_sym_chain_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op, return ret; } +static __rte_always_inline uint8_t +vhost_crypto_check_akcipher_request(struct virtio_crypto_akcipher_data_req *req) +{ + RTE_SET_USED(req); + return VIRTIO_CRYPTO_OK; +} + +static __rte_always_inline uint8_t +prepare_asym_rsa_op(struct vhost_crypto *vcrypto, struct rte_crypto_op *op, + struct vhost_crypto_data_req *vc_req, + struct virtio_crypto_op_data_req *req, + struct vhost_crypto_desc *head, + uint32_t max_n_descs) +{ + uint8_t ret = vhost_crypto_check_akcipher_request(&req->u.akcipher_req); + struct rte_crypto_rsa_op_param *rsa = &op->asym->rsa; + struct vhost_crypto_desc *desc = head; + uint16_t wlen = 0; + + if (unlikely(ret != VIRTIO_CRYPTO_OK)) + goto error_exit; + + /* prepare */ + switch (vcrypto->option) { + case RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE: + vc_req->wb_pool = vcrypto->wb_pool; + if (req->header.opcode == VIRTIO_CRYPTO_AKCIPHER_SIGN) { + rsa->op_type = RTE_CRYPTO_ASYM_OP_SIGN; + rsa->message.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RO); + rsa->message.length = req->u.akcipher_req.para.src_data_len; + rsa->sign.length = req->u.akcipher_req.para.dst_data_len; + wlen = rsa->sign.length; + desc = find_write_desc(head, desc, max_n_descs); + if (unlikely(!desc)) { + VC_LOG_ERR("Cannot find write location"); + ret = VIRTIO_CRYPTO_BADMSG; + goto error_exit; + } + + rsa->sign.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RW); + if (unlikely(rsa->sign.data == NULL)) { + ret = VIRTIO_CRYPTO_ERR; + goto error_exit; + } + + desc += 1; + } else if (req->header.opcode == VIRTIO_CRYPTO_AKCIPHER_VERIFY) { + rsa->op_type = RTE_CRYPTO_ASYM_OP_VERIFY; + rsa->sign.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RO); + rsa->sign.length = req->u.akcipher_req.para.src_data_len; + desc += 1; + rsa->message.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RO); + rsa->message.length = req->u.akcipher_req.para.dst_data_len; + desc += 1; + } else if (req->header.opcode == VIRTIO_CRYPTO_AKCIPHER_ENCRYPT) { + rsa->op_type = RTE_CRYPTO_ASYM_OP_ENCRYPT; + rsa->message.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RO); + rsa->message.length = req->u.akcipher_req.para.src_data_len; + rsa->cipher.length = req->u.akcipher_req.para.dst_data_len; + wlen = rsa->cipher.length; + desc = find_write_desc(head, desc, max_n_descs); + if (unlikely(!desc)) { + VC_LOG_ERR("Cannot find write location"); + ret = VIRTIO_CRYPTO_BADMSG; + goto error_exit; + } + + rsa->cipher.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RW); + if (unlikely(rsa->cipher.data == NULL)) { + ret = VIRTIO_CRYPTO_ERR; + goto error_exit; + } + + desc += 1; + } else if (req->header.opcode == VIRTIO_CRYPTO_AKCIPHER_DECRYPT) { + rsa->op_type = RTE_CRYPTO_ASYM_OP_DECRYPT; + rsa->cipher.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RO); + rsa->cipher.length = req->u.akcipher_req.para.src_data_len; + desc += 1; + rsa->message.data = get_data_ptr(vc_req, desc, VHOST_ACCESS_RO); + rsa->message.length = req->u.akcipher_req.para.dst_data_len; + desc += 1; + } else { + goto error_exit; + } + break; + case RTE_VHOST_CRYPTO_ZERO_COPY_ENABLE: + default: + ret = VIRTIO_CRYPTO_BADMSG; + goto error_exit; + } + + op->type = RTE_CRYPTO_OP_TYPE_ASYMMETRIC; + op->sess_type = RTE_CRYPTO_OP_WITH_SESSION; + + vc_req->inhdr = get_data_ptr(vc_req, desc, VHOST_ACCESS_WO); + if (unlikely(vc_req->inhdr == NULL)) { + ret = VIRTIO_CRYPTO_BADMSG; + goto error_exit; + } + + vc_req->inhdr->status = VIRTIO_CRYPTO_OK; + vc_req->len = wlen + INHDR_LEN; + return 0; +error_exit: + if (vc_req->wb) + free_wb_data(vc_req->wb, vc_req->wb_pool); + + vc_req->len = INHDR_LEN; + return ret; +} + /** * Process on descriptor */ @@ -1133,17 +1498,21 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto, uint16_t desc_idx) __rte_no_thread_safety_analysis /* FIXME: requires iotlb_lock? */ { - struct vhost_crypto_data_req *vc_req = rte_mbuf_to_priv(op->sym->m_src); - struct rte_cryptodev_sym_session *session; + struct vhost_crypto_data_req *vc_req, *vc_req_out; + struct rte_cryptodev_asym_session *asym_session; + struct rte_cryptodev_sym_session *sym_session; + struct vhost_crypto_session *vhost_session; + struct vhost_crypto_desc *desc = descs; + uint32_t nb_descs = 0, max_n_descs, i; + struct vhost_crypto_data_req data_req; struct virtio_crypto_op_data_req req; struct virtio_crypto_inhdr *inhdr; - struct vhost_crypto_desc *desc = descs; struct vring_desc *src_desc; uint64_t session_id; uint64_t dlen; - uint32_t nb_descs = 0, max_n_descs, i; int err; + vc_req = &data_req; vc_req->desc_idx = desc_idx; vc_req->dev = vcrypto->dev; vc_req->vq = vq; @@ -1226,12 +1595,14 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto, switch (req.header.opcode) { case VIRTIO_CRYPTO_CIPHER_ENCRYPT: case VIRTIO_CRYPTO_CIPHER_DECRYPT: + vc_req_out = rte_mbuf_to_priv(op->sym->m_src); + rte_memcpy(vc_req_out, vc_req, sizeof(struct vhost_crypto_data_req)); session_id = req.header.session_id; /* one branch to avoid unnecessary table lookup */ - if (vcrypto->cache_session_id != session_id) { + if (vcrypto->cache_sym_session_id != session_id) { err = rte_hash_lookup_data(vcrypto->session_map, - &session_id, (void **)&session); + &session_id, (void **)&vhost_session); if (unlikely(err < 0)) { err = VIRTIO_CRYPTO_ERR; VC_LOG_ERR("Failed to find session %"PRIu64, @@ -1239,13 +1610,14 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto, goto error_exit; } - vcrypto->cache_session = session; - vcrypto->cache_session_id = session_id; + vcrypto->cache_sym_session = vhost_session->sym; + vcrypto->cache_sym_session_id = session_id; } - session = vcrypto->cache_session; + sym_session = vcrypto->cache_sym_session; + op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; - err = rte_crypto_op_attach_sym_session(op, session); + err = rte_crypto_op_attach_sym_session(op, sym_session); if (unlikely(err < 0)) { err = VIRTIO_CRYPTO_ERR; VC_LOG_ERR("Failed to attach session to op"); @@ -1257,12 +1629,12 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto, err = VIRTIO_CRYPTO_NOTSUPP; break; case VIRTIO_CRYPTO_SYM_OP_CIPHER: - err = prepare_sym_cipher_op(vcrypto, op, vc_req, + err = prepare_sym_cipher_op(vcrypto, op, vc_req_out, &req.u.sym_req.u.cipher, desc, max_n_descs); break; case VIRTIO_CRYPTO_SYM_OP_ALGORITHM_CHAINING: - err = prepare_sym_chain_op(vcrypto, op, vc_req, + err = prepare_sym_chain_op(vcrypto, op, vc_req_out, &req.u.sym_req.u.chain, desc, max_n_descs); break; @@ -1271,6 +1643,53 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto, VC_LOG_ERR("Failed to process sym request"); goto error_exit; } + break; + case VIRTIO_CRYPTO_AKCIPHER_SIGN: + case VIRTIO_CRYPTO_AKCIPHER_VERIFY: + case VIRTIO_CRYPTO_AKCIPHER_ENCRYPT: + case VIRTIO_CRYPTO_AKCIPHER_DECRYPT: + session_id = req.header.session_id; + + /* one branch to avoid unnecessary table lookup */ + if (vcrypto->cache_asym_session_id != session_id) { + err = rte_hash_lookup_data(vcrypto->session_map, + &session_id, (void **)&vhost_session); + if (unlikely(err < 0)) { + err = VIRTIO_CRYPTO_ERR; + VC_LOG_ERR("Failed to find asym session %"PRIu64, + session_id); + goto error_exit; + } + + vcrypto->cache_asym_session = vhost_session->asym; + vcrypto->cache_asym_session_id = session_id; + } + + asym_session = vcrypto->cache_asym_session; + op->type = RTE_CRYPTO_OP_TYPE_ASYMMETRIC; + + err = rte_crypto_op_attach_asym_session(op, asym_session); + if (unlikely(err < 0)) { + err = VIRTIO_CRYPTO_ERR; + VC_LOG_ERR("Failed to attach asym session to op"); + goto error_exit; + } + + vc_req_out = rte_cryptodev_asym_session_get_user_data(asym_session); + rte_memcpy(vc_req_out, vc_req, sizeof(struct vhost_crypto_data_req)); + vc_req_out->wb = NULL; + + switch (req.header.algo) { + case VIRTIO_CRYPTO_AKCIPHER_RSA: + err = prepare_asym_rsa_op(vcrypto, op, vc_req_out, + &req, desc, max_n_descs); + break; + } + if (unlikely(err != 0)) { + VC_LOG_ERR("Failed to process asym request"); + goto error_exit; + } + break; default: err = VIRTIO_CRYPTO_ERR; @@ -1294,12 +1713,22 @@ static __rte_always_inline struct vhost_virtqueue * vhost_crypto_finalize_one_request(struct rte_crypto_op *op, struct vhost_virtqueue *old_vq) { - struct rte_mbuf *m_src = op->sym->m_src; - struct rte_mbuf *m_dst = op->sym->m_dst; - struct vhost_crypto_data_req *vc_req = rte_mbuf_to_priv(m_src); + struct rte_mbuf *m_src = NULL, *m_dst = NULL; + struct vhost_crypto_data_req *vc_req; struct vhost_virtqueue *vq; uint16_t used_idx, desc_idx; + if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + m_src = op->sym->m_src; + m_dst = op->sym->m_dst; + vc_req = rte_mbuf_to_priv(m_src); + } else if (op->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + vc_req = rte_cryptodev_asym_session_get_user_data(op->asym->session); + } else { + VC_LOG_ERR("Invalid crypto op type"); + return NULL; + } + if (unlikely(!vc_req)) { VC_LOG_ERR("Failed to retrieve vc_req"); return NULL; @@ -1321,10 +1750,11 @@ vhost_crypto_finalize_one_request(struct rte_crypto_op *op, vq->used->ring[desc_idx].id = vq->avail->ring[desc_idx]; vq->used->ring[desc_idx].len = vc_req->len; - rte_mempool_put(m_src->pool, (void *)m_src); - - if (m_dst) - rte_mempool_put(m_dst->pool, (void *)m_dst); + if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + rte_mempool_put(m_src->pool, (void *)m_src); + if (m_dst) + rte_mempool_put(m_dst->pool, (void *)m_dst); + } return vc_req->vq; } @@ -1407,8 +1837,9 @@ rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, vcrypto->sess_pool = sess_pool; vcrypto->cid = cryptodev_id; - vcrypto->cache_session_id = UINT64_MAX; - vcrypto->last_session_id = 1; + vcrypto->cache_sym_session_id = UINT64_MAX; + vcrypto->cache_asym_session_id = UINT64_MAX; + vcrypto->last_session_id = 0; vcrypto->dev = dev; vcrypto->option = RTE_VHOST_CRYPTO_ZERO_COPY_DISABLE; @@ -1580,6 +2011,9 @@ rte_vhost_crypto_fetch_requests(int vid, uint32_t qid, vq = dev->virtqueue[qid]; + if (!vq || !vq->avail) + return 0; + avail_idx = *((volatile uint16_t *)&vq->avail->idx); start_idx = vq->last_used_idx; count = avail_idx - start_idx; diff --git a/lib/vhost/vhost_user.h b/lib/vhost/vhost_user.h index edf7adb3c0..3b9e3ce7c2 100644 --- a/lib/vhost/vhost_user.h +++ b/lib/vhost/vhost_user.h @@ -99,11 +99,10 @@ typedef struct VhostUserLog { /* Comply with Cryptodev-Linux */ #define VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH 512 #define VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH 64 +#define VHOST_USER_CRYPTO_MAX_KEY_LENGTH 1024 /* Same structure as vhost-user backend session info */ -typedef struct VhostUserCryptoSessionParam { - int64_t session_id; - uint32_t op_code; +typedef struct VhostUserCryptoSymSessionParam { uint32_t cipher_algo; uint32_t cipher_key_len; uint32_t hash_algo; @@ -114,10 +113,36 @@ typedef struct VhostUserCryptoSessionParam { uint8_t dir; uint8_t hash_mode; uint8_t chaining_dir; - uint8_t *ciphe_key; + uint8_t *cipher_key; uint8_t *auth_key; uint8_t cipher_key_buf[VHOST_USER_CRYPTO_MAX_CIPHER_KEY_LENGTH]; uint8_t auth_key_buf[VHOST_USER_CRYPTO_MAX_HMAC_KEY_LENGTH]; +} VhostUserCryptoSymSessionParam; + + +typedef struct VhostUserCryptoAsymRsaParam { + uint32_t padding_algo; + uint32_t hash_algo; +} VhostUserCryptoAsymRsaParam; + +typedef struct VhostUserCryptoAsymSessionParam { + uint32_t algo; + uint32_t key_type; + uint32_t key_len; + uint8_t *key; + union { + VhostUserCryptoAsymRsaParam rsa; + } u; + uint8_t key_buf[VHOST_USER_CRYPTO_MAX_KEY_LENGTH]; +} VhostUserCryptoAsymSessionParam; + +typedef struct VhostUserCryptoSessionParam { + uint32_t op_code; + union { + VhostUserCryptoSymSessionParam sym_sess; + VhostUserCryptoAsymSessionParam asym_sess; + } u; + uint64_t session_id; } VhostUserCryptoSessionParam; typedef struct VhostUserVringArea { -- 2.25.1