From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id 7FD4910BD for ; Mon, 30 Apr 2018 16:44:31 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id a67so6021880wmf.3 for ; Mon, 30 Apr 2018 07:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oYoiqh4xgbG3k9J7jhliD1rOYYWb/gKRcYttEQldxU0=; b=aJ7O6pEVaA8ZeJB1ckpA3bn7/Ayd9jtq9xjuL8fsdjMaFljsl8yWo5bXENh90jOGzB Hv9ypwg70VFI4p3BHV3FkxKuVu6TtTYI85IKnBSW1k9/P0fvuMJXQwVZuqT69Y4GkKLI olE/USqVJfKi8R+ON4qth6PgsGVAbJTrrqcjctd5H/0h3HbcLCUSF7MqAAdJvqQgpdLJ bJCl8L2JNZH8TNx5FlhW3iwfktylXFlGmGA+ITIQH4ziiC4jYJ67YzNpEQHcEZtvIBv6 DPJzVY3b7SmlEg0FarsSRgKtQxz0fw2qoWR2xGZib7ce7jxXhQymN8STBF8r9JnsQeNU /Swg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oYoiqh4xgbG3k9J7jhliD1rOYYWb/gKRcYttEQldxU0=; b=f67wjjaHcmnVzT5W97bmTvTfOfbxrui8x0lz4KEJ4CdzdSM7QTu32HpNcyw7vW7VEv WSUolcMtP6lb6qP4p4fdGJpaVM6QtJWAesGCEJk3eQrSz2l/2m95kJBZoJL4aaPN2MFs I99KSKaaMFBkCd4CNrNilSxsRgNno+CL2ASEcX+vYEuPvTeaci2qHV6Tsjjhcp5v47I4 a0fwXvaC6nMpV2U91N4BkZM+QsUoLGnB8dojN7jkeALVPKTMIOx6iJNFSetHNd5Lykwf FwgpyVRkyzP3I+pTjshJwAv2EY1s77rHuM+YZOG0B/ju1c83t8kDlWcxX+nx/iDZ701u nPDg== X-Gm-Message-State: ALQs6tCmCZF4+/9Q0W0268DG/qhuxxTX0XvIeSMRZYHAVZshBEEawW0G sqgJP5t9KN/IHtvGGS0A2jM= X-Google-Smtp-Source: AB8JxZqm3KgewiizRWvRmqLoYDVSq3+Qz3DBgQ2pwa9jIhlsGcZvI5SOAjC9Knh5DWV8ecnL+05JVg== X-Received: by 10.28.215.208 with SMTP id o199mr7326139wmg.61.1525099471196; Mon, 30 Apr 2018 07:44:31 -0700 (PDT) Received: from localhost ([2a00:23c5:be9a:5200:ce4c:82c0:d567:ecbb]) by smtp.gmail.com with ESMTPSA id e202sm10747942wma.43.2018.04.30.07.44.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 30 Apr 2018 07:44:30 -0700 (PDT) From: luca.boccassi@gmail.com To: Pablo de Lara Cc: Fiona Trahe , dpdk stable Date: Mon, 30 Apr 2018 15:41:34 +0100 Message-Id: <20180430144223.18657-79-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180430144223.18657-1-luca.boccassi@gmail.com> References: <20180430140606.4615-80-luca.boccassi@gmail.com> <20180430144223.18657-1-luca.boccassi@gmail.com> Subject: [dpdk-stable] patch 'crypto/zuc: batch ops with same transform' has been queued to stable release 18.02.2 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Apr 2018 14:44:31 -0000 Hi, FYI, your patch has been queued to stable release 18.02.2 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 05/02/18. So please shout if anyone has objections. Thanks. Luca Boccassi --- >>From 97b67d592dd3da614ffd065bd5b35c2163bbff5f Mon Sep 17 00:00:00 2001 From: Pablo de Lara Date: Thu, 19 Apr 2018 15:55:41 +0100 Subject: [PATCH] crypto/zuc: batch ops with same transform [ upstream commit 17c36a6746419ae8d1a14b1d1345d4a892511c57 ] The ZUC API to encrypt packets does not require the operations to share the same key. Currently, the operations were being batched only when they shared the same key, but this is not needed. Instead, now operations will be batched based on the transform (cipher only, auth only...). Fixes: cf7685d68f00 ("crypto/zuc: add driver for ZUC library") Signed-off-by: Pablo de Lara Acked-by: Fiona Trahe --- drivers/crypto/zuc/rte_zuc_pmd.c | 105 ++++++++++++++++++++++++--------------- 1 file changed, 64 insertions(+), 41 deletions(-) diff --git a/drivers/crypto/zuc/rte_zuc_pmd.c b/drivers/crypto/zuc/rte_zuc_pmd.c index 2e337c468..8a44331dc 100644 --- a/drivers/crypto/zuc/rte_zuc_pmd.c +++ b/drivers/crypto/zuc/rte_zuc_pmd.c @@ -12,7 +12,7 @@ #include "rte_zuc_pmd_private.h" -#define ZUC_MAX_BURST 8 +#define ZUC_MAX_BURST 4 #define BYTE_LEN 8 static uint8_t cryptodev_driver_id; @@ -168,10 +168,10 @@ zuc_get_session(struct zuc_qp *qp, struct rte_crypto_op *op) return sess; } -/** Encrypt/decrypt mbufs with same cipher key. */ +/** Encrypt/decrypt mbufs. */ static uint8_t process_zuc_cipher_op(struct rte_crypto_op **ops, - struct zuc_session *session, + struct zuc_session **sessions, uint8_t num_ops) { unsigned i; @@ -180,6 +180,7 @@ process_zuc_cipher_op(struct rte_crypto_op **ops, uint8_t *iv[ZUC_MAX_BURST]; uint32_t num_bytes[ZUC_MAX_BURST]; uint8_t *cipher_keys[ZUC_MAX_BURST]; + struct zuc_session *sess; for (i = 0; i < num_ops; i++) { if (((ops[i]->sym->cipher.data.length % BYTE_LEN) != 0) @@ -190,6 +191,8 @@ process_zuc_cipher_op(struct rte_crypto_op **ops, break; } + sess = sessions[i]; + #ifdef RTE_LIBRTE_PMD_ZUC_DEBUG if (!rte_pktmbuf_is_contiguous(ops[i]->sym->m_src) || (ops[i]->sym->m_dst != NULL && @@ -211,10 +214,10 @@ process_zuc_cipher_op(struct rte_crypto_op **ops, rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + (ops[i]->sym->cipher.data.offset >> 3); iv[i] = rte_crypto_op_ctod_offset(ops[i], uint8_t *, - session->cipher_iv_offset); + sess->cipher_iv_offset); num_bytes[i] = ops[i]->sym->cipher.data.length >> 3; - cipher_keys[i] = session->pKey_cipher; + cipher_keys[i] = sess->pKey_cipher; processed_ops++; } @@ -225,10 +228,10 @@ process_zuc_cipher_op(struct rte_crypto_op **ops, return processed_ops; } -/** Generate/verify hash from mbufs with same hash key. */ +/** Generate/verify hash from mbufs. */ static int process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, - struct zuc_session *session, + struct zuc_session **sessions, uint8_t num_ops) { unsigned i; @@ -237,6 +240,7 @@ process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, uint32_t *dst; uint32_t length_in_bits; uint8_t *iv; + struct zuc_session *sess; for (i = 0; i < num_ops; i++) { /* Data must be byte aligned */ @@ -246,17 +250,19 @@ process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, break; } + sess = sessions[i]; + length_in_bits = ops[i]->sym->auth.data.length; src = rte_pktmbuf_mtod(ops[i]->sym->m_src, uint8_t *) + (ops[i]->sym->auth.data.offset >> 3); iv = rte_crypto_op_ctod_offset(ops[i], uint8_t *, - session->auth_iv_offset); + sess->auth_iv_offset); - if (session->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) { + if (sess->auth_op == RTE_CRYPTO_AUTH_OP_VERIFY) { dst = (uint32_t *)qp->temp_digest; - sso_zuc_eia3_1_buffer(session->pKey_hash, + sso_zuc_eia3_1_buffer(sess->pKey_hash, iv, src, length_in_bits, dst); /* Verify digest. */ @@ -266,7 +272,7 @@ process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, } else { dst = (uint32_t *)ops[i]->sym->auth.digest.data; - sso_zuc_eia3_1_buffer(session->pKey_hash, + sso_zuc_eia3_1_buffer(sess->pKey_hash, iv, src, length_in_bits, dst); } @@ -276,33 +282,34 @@ process_zuc_hash_op(struct zuc_qp *qp, struct rte_crypto_op **ops, return processed_ops; } -/** Process a batch of crypto ops which shares the same session. */ +/** Process a batch of crypto ops which shares the same operation type. */ static int -process_ops(struct rte_crypto_op **ops, struct zuc_session *session, +process_ops(struct rte_crypto_op **ops, enum zuc_operation op_type, + struct zuc_session **sessions, struct zuc_qp *qp, uint8_t num_ops, uint16_t *accumulated_enqueued_ops) { unsigned i; unsigned enqueued_ops, processed_ops; - switch (session->op) { + switch (op_type) { case ZUC_OP_ONLY_CIPHER: processed_ops = process_zuc_cipher_op(ops, - session, num_ops); + sessions, num_ops); break; case ZUC_OP_ONLY_AUTH: - processed_ops = process_zuc_hash_op(qp, ops, session, + processed_ops = process_zuc_hash_op(qp, ops, sessions, num_ops); break; case ZUC_OP_CIPHER_AUTH: - processed_ops = process_zuc_cipher_op(ops, session, + processed_ops = process_zuc_cipher_op(ops, sessions, num_ops); - process_zuc_hash_op(qp, ops, session, processed_ops); + process_zuc_hash_op(qp, ops, sessions, processed_ops); break; case ZUC_OP_AUTH_CIPHER: - processed_ops = process_zuc_hash_op(qp, ops, session, + processed_ops = process_zuc_hash_op(qp, ops, sessions, num_ops); - process_zuc_cipher_op(ops, session, processed_ops); + process_zuc_cipher_op(ops, sessions, processed_ops); break; default: /* Operation not supported. */ @@ -318,10 +325,10 @@ process_ops(struct rte_crypto_op **ops, struct zuc_session *session, ops[i]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; /* Free session if a session-less crypto op. */ if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { - memset(session, 0, sizeof(struct zuc_session)); + memset(sessions[i], 0, sizeof(struct zuc_session)); memset(ops[i]->sym->session, 0, rte_cryptodev_get_header_session_size()); - rte_mempool_put(qp->sess_mp, session); + rte_mempool_put(qp->sess_mp, sessions[i]); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; } @@ -342,7 +349,10 @@ zuc_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, struct rte_crypto_op *c_ops[ZUC_MAX_BURST]; struct rte_crypto_op *curr_c_op; - struct zuc_session *prev_sess = NULL, *curr_sess = NULL; + struct zuc_session *curr_sess; + struct zuc_session *sessions[ZUC_MAX_BURST]; + enum zuc_operation prev_zuc_op = ZUC_OP_NOT_SUPPORTED; + enum zuc_operation curr_zuc_op; struct zuc_qp *qp = queue_pair; unsigned i; uint8_t burst_size = 0; @@ -359,50 +369,63 @@ zuc_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, break; } - /* Batch ops that share the same session. */ - if (prev_sess == NULL) { - prev_sess = curr_sess; - c_ops[burst_size++] = curr_c_op; - } else if (curr_sess == prev_sess) { - c_ops[burst_size++] = curr_c_op; + curr_zuc_op = curr_sess->op; + + /* + * Batch ops that share the same operation type + * (cipher only, auth only...). + */ + if (burst_size == 0) { + prev_zuc_op = curr_zuc_op; + c_ops[0] = curr_c_op; + sessions[0] = curr_sess; + burst_size++; + } else if (curr_zuc_op == prev_zuc_op) { + c_ops[burst_size] = curr_c_op; + sessions[burst_size] = curr_sess; + burst_size++; /* * When there are enough ops to process in a batch, * process them, and start a new batch. */ if (burst_size == ZUC_MAX_BURST) { - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); + processed_ops = process_ops(c_ops, curr_zuc_op, + sessions, qp, burst_size, + &enqueued_ops); if (processed_ops < burst_size) { burst_size = 0; break; } burst_size = 0; - prev_sess = NULL; } } else { /* - * Different session, process the ops - * of the previous session. + * Different operation type, process the ops + * of the previous type. */ - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); + processed_ops = process_ops(c_ops, prev_zuc_op, + sessions, qp, burst_size, + &enqueued_ops); if (processed_ops < burst_size) { burst_size = 0; break; } burst_size = 0; - prev_sess = curr_sess; + prev_zuc_op = curr_zuc_op; - c_ops[burst_size++] = curr_c_op; + c_ops[0] = curr_c_op; + sessions[0] = curr_sess; + burst_size++; } } if (burst_size != 0) { - /* Process the crypto ops of the last session. */ - processed_ops = process_ops(c_ops, prev_sess, - qp, burst_size, &enqueued_ops); + /* Process the crypto ops of the last operation type. */ + processed_ops = process_ops(c_ops, prev_zuc_op, + sessions, qp, burst_size, + &enqueued_ops); } qp->qp_stats.enqueue_err_count += nb_ops - enqueued_ops; -- 2.14.2