[dpdk-dev] [PATCH 1/3] crypto/qat: improve out-of-place conditional check

[dpdk-dev] [PATCH 1/3] crypto/qat: improve out-of-place conditional check

[Adam Dybkowski]

From: Fiona Trahe <fiona.trahe@intel.com>

Improve case where application set m_dst to same as m_src
so really an in-place operation, though would have been treated
as out-of-place. No functional change but this path can now benefit
from DMA alignment.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
 drivers/crypto/qat/qat_sym.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index 5c9904cbf..cecced66d 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -62,7 +62,8 @@ qat_bpicipher_preprocess(struct qat_sym_session *ctx,
 		last_block = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_src,
 				uint8_t *, last_block_offset);
 
-		if (unlikely(sym_op->m_dst != NULL))
+		if (unlikely((sym_op->m_dst != NULL)
+				&& (sym_op->m_dst != sym_op->m_src)))
 			/* out-of-place operation (OOP) */
 			dst = (uint8_t *) rte_pktmbuf_mtod_offset(sym_op->m_dst,
 						uint8_t *, last_block_offset);
@@ -437,7 +438,8 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
 	if (unlikely(min_ofs >= rte_pktmbuf_data_len(op->sym->m_src) && do_sgl))
 		min_ofs = 0;
 
-	if (unlikely(op->sym->m_dst != NULL)) {
+	if (unlikely((op->sym->m_dst != NULL) &&
+			(op->sym->m_dst != op->sym->m_src))) {
 		/* Out-of-place operation (OOP)
 		 * Don't align DMA start. DMA the minimum data-set
 		 * so as not to overwrite data in dest buffer
@@ -565,7 +567,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
 			return ret;
 		}
 
-		if (likely(op->sym->m_dst == NULL))
+		if (in_place)
 			qat_req->comn_mid.dest_data_addr =
 				qat_req->comn_mid.src_data_addr =
 				cookie->qat_sgl_src_phys_addr;
-- 
2.17.1

[dpdk-dev] [PATCH 2/3] crypto/qat: optimise check for chained mbufs

[Adam Dybkowski]

From: Fiona Trahe <fiona.trahe@intel.com>

To detect if sgl, use nb_segs > 1, instead of checking for next pointer,
as nb_segs is in first cache-line while next is in second cache-line.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
 drivers/crypto/qat/qat_sym.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
index cecced66d..25b6dd5f4 100644
--- a/drivers/crypto/qat/qat_sym.c
+++ b/drivers/crypto/qat/qat_sym.c
@@ -428,7 +428,8 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg,
 		min_ofs = op->sym->aead.data.offset;
 	}
 
-	if (op->sym->m_src->next || (op->sym->m_dst && op->sym->m_dst->next))
+	if (op->sym->m_src->nb_segs > 1 ||
+			(op->sym->m_dst && op->sym->m_dst->nb_segs > 1))
 		do_sgl = 1;
 
 	/* adjust for chain case */
-- 
2.17.1

Re: [dpdk-dev] [PATCH 2/3] crypto/qat: optimise check for chained mbufs

[Kusztal, ArkadiuszX]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Adam Dybkowski
> Sent: Wednesday, March 11, 2020 1:26 PM
> To: dev@dpdk.org; Trahe, Fiona <fiona.trahe@intel.com>;
> akhil.goyal@nxp.com; Dybkowski, AdamX <adamx.dybkowski@intel.com>
> Subject: [dpdk-dev] [PATCH 2/3] crypto/qat: optimise check for chained
> mbufs
> 
> From: Fiona Trahe <fiona.trahe@intel.com>
> 
> To detect if sgl, use nb_segs > 1, instead of checking for next pointer, as
> nb_segs is in first cache-line while next is in second cache-line.
> 
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
> ---
>  drivers/crypto/qat/qat_sym.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 2.17.1
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

[dpdk-dev] [PATCH 3/3] common/qat: optimise calculation of cookie index

[Adam Dybkowski]

From: Fiona Trahe <fiona.trahe@intel.com>

Avoid costly division, use shift instead.

Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
---
 drivers/common/qat/qat_qp.c | 9 ++++++---
 drivers/common/qat/qat_qp.h | 1 +
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/common/qat/qat_qp.c b/drivers/common/qat/qat_qp.c
index 9958789f0..b0a206434 100644
--- a/drivers/common/qat/qat_qp.c
+++ b/drivers/common/qat/qat_qp.c
@@ -430,6 +430,9 @@ qat_queue_create(struct qat_pci_device *qat_dev, struct qat_queue *queue,
 	queue->tail = 0;
 	queue->msg_size = desc_size;
 
+	/* For fast calculation of cookie index, relies on msg_size being 2^n */
+	queue->trailz = __builtin_ctz(desc_size);
+
 	/*
 	 * Write an unused pattern to the queue memory.
 	 */
@@ -623,7 +626,7 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops)
 
 	while (nb_ops_sent != nb_ops_possible) {
 		ret = tmp_qp->build_request(*ops, base_addr + tail,
-				tmp_qp->op_cookies[tail / queue->msg_size],
+				tmp_qp->op_cookies[tail >> queue->trailz],
 				tmp_qp->qat_dev_gen);
 		if (ret != 0) {
 			tmp_qp->stats.enqueue_err_count++;
@@ -665,12 +668,12 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops)
 			qat_sym_process_response(ops, resp_msg);
 		else if (tmp_qp->service_type == QAT_SERVICE_COMPRESSION)
 			qat_comp_process_response(ops, resp_msg,
-				tmp_qp->op_cookies[head / rx_queue->msg_size],
+				tmp_qp->op_cookies[head >> rx_queue->trailz],
 				&tmp_qp->stats.dequeue_err_count);
 		else if (tmp_qp->service_type == QAT_SERVICE_ASYMMETRIC) {
 #ifdef BUILD_QAT_ASYM
 			qat_asym_process_response(ops, resp_msg,
-				tmp_qp->op_cookies[head / rx_queue->msg_size]);
+				tmp_qp->op_cookies[head >> rx_queue->trailz]);
 #endif
 		}
 
diff --git a/drivers/common/qat/qat_qp.h b/drivers/common/qat/qat_qp.h
index 0b95ea3c9..0d6896d7b 100644
--- a/drivers/common/qat/qat_qp.h
+++ b/drivers/common/qat/qat_qp.h
@@ -54,6 +54,7 @@ struct qat_queue {
 	uint32_t	modulo_mask;
 	uint32_t	msg_size;
 	uint32_t	queue_size;
+	uint8_t		trailz;
 	uint8_t		hw_bundle_number;
 	uint8_t		hw_queue_number;
 	/* HW queue aka ring offset on bundle */
-- 
2.17.1

Re: [dpdk-dev] [PATCH 3/3] common/qat: optimise calculation of cookie index

[Kusztal, ArkadiuszX]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Adam Dybkowski
> Sent: Wednesday, March 11, 2020 1:26 PM
> To: dev@dpdk.org; Trahe, Fiona <fiona.trahe@intel.com>;
> akhil.goyal@nxp.com; Dybkowski, AdamX <adamx.dybkowski@intel.com>
> Subject: [dpdk-dev] [PATCH 3/3] common/qat: optimise calculation of cookie
> index
> 
> From: Fiona Trahe <fiona.trahe@intel.com>
> 
> Avoid costly division, use shift instead.
> 
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
> ---
>  drivers/common/qat/qat_qp.c | 9 ++++++---  drivers/common/qat/qat_qp.h
> | 1 +
>  2 files changed, 7 insertions(+), 3 deletions(-)
> 
> 2.17.1
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

Re: [dpdk-dev] [PATCH 1/3] crypto/qat: improve out-of-place conditional check

[Kusztal, ArkadiuszX]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Adam Dybkowski
> Sent: Wednesday, March 11, 2020 1:26 PM
> To: dev@dpdk.org; Trahe, Fiona <fiona.trahe@intel.com>;
> akhil.goyal@nxp.com; Dybkowski, AdamX <adamx.dybkowski@intel.com>
> Subject: [dpdk-dev] [PATCH 1/3] crypto/qat: improve out-of-place
> conditional check
> 
> From: Fiona Trahe <fiona.trahe@intel.com>
> 
> Improve case where application set m_dst to same as m_src so really an in-
> place operation, though would have been treated as out-of-place. No
> functional change but this path can now benefit from DMA alignment.
> 
> Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
> ---
>  drivers/crypto/qat/qat_sym.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
> index 5c9904cbf..cecced66d 100644
> 2.17.1
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

Re: [dpdk-dev] [PATCH 1/3] crypto/qat: improve out-of-place conditional check

[Akhil Goyal]

> > From: Fiona Trahe <fiona.trahe@intel.com>
> >
> > Improve case where application set m_dst to same as m_src so really an in-
> > place operation, though would have been treated as out-of-place. No
> > functional change but this path can now benefit from DMA alignment.
> >
> > Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
> > ---
> >  drivers/crypto/qat/qat_sym.c | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c
> > index 5c9904cbf..cecced66d 100644
> > 2.17.1
> Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>

Series applied to dpdk-next-crypto

Thanks.