* [dpdk-dev] [PATCH v2 0/2] compress/qat: im buffer too small - split op
@ 2020-04-08 12:50 Adam Dybkowski
2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 1/2] " Adam Dybkowski
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Adam Dybkowski @ 2020-04-08 12:50 UTC (permalink / raw)
To: dev, fiona.trahe, akhil.goyal; +Cc: Adam Dybkowski
This patch implements a special way of buffer handling when internal
QAT IM buffer is too small for Huffman dynamic compression operation.
Instead of falling back to fixed compression, the operation is now
split into multiple smaller dynamic compression requests (possible to
execute on QAT) and their results are then combined and copied into
the output buffer. This is not possible if any checksum calculation
was requested - in such case the code falls back to fixed compression
as before.
v2:
* various post-review small fixes
Adam Dybkowski (2):
compress/qat: im buffer too small - split op
test/compress: im buffer too small - add unit tests
app/test/test_compressdev.c | 1251 ++++++++++++++++++++++--
doc/guides/compressdevs/qat_comp.rst | 3 -
doc/guides/cryptodevs/qat.rst | 7 +-
doc/guides/rel_notes/release_20_05.rst | 10 +
drivers/common/qat/qat_qp.c | 223 ++++-
drivers/common/qat/qat_qp.h | 3 +
drivers/compress/qat/qat_comp.c | 474 ++++++++-
drivers/compress/qat/qat_comp.h | 29 +-
drivers/compress/qat/qat_comp_pmd.c | 27 +-
9 files changed, 1879 insertions(+), 148 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [dpdk-dev] [PATCH v2 1/2] compress/qat: im buffer too small - split op 2020-04-08 12:50 [dpdk-dev] [PATCH v2 0/2] compress/qat: im buffer too small - split op Adam Dybkowski @ 2020-04-08 12:51 ` Adam Dybkowski 2020-04-08 15:43 ` Trahe, Fiona 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2 siblings, 1 reply; 20+ messages in thread From: Adam Dybkowski @ 2020-04-08 12:51 UTC (permalink / raw) To: dev, fiona.trahe, akhil.goyal; +Cc: Adam Dybkowski This patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before. Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> --- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 +++++++++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 +++++++++++++++++++++++-- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 8 files changed, 702 insertions(+), 74 deletions(-) diff --git a/doc/guides/compressdevs/qat_comp.rst b/doc/guides/compressdevs/qat_comp.rst index 757611a30..475c4a9f9 100644 --- a/doc/guides/compressdevs/qat_comp.rst +++ b/doc/guides/compressdevs/qat_comp.rst @@ -42,9 +42,6 @@ Limitations from the RX queue must be done from one thread, but enqueues and dequeues may be done in different threads.) * No BSD support as BSD QAT kernel driver not available. -* When using Deflate dynamic huffman encoding for compression, the input size (op.src.length) - must be < CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE from the config file, - see :ref:`building_qat_config` for more details. * Stateful compression is not supported. diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst index c79e686de..4ea7985a7 100644 --- a/doc/guides/cryptodevs/qat.rst +++ b/doc/guides/cryptodevs/qat.rst @@ -260,8 +260,11 @@ allocated while for GEN1 devices, 12 buffers are allocated, plus 1472 bytes over .. Note:: If the compressed output of a Deflate operation using Dynamic Huffman - Encoding is too big to fit in an intermediate buffer, then the - operation will fall back to fixed compression rather than failing the operation. + Encoding is too big to fit in an intermediate buffer, then the + operation will be split into smaller operations and their results will + be merged afterwards. + This is not possible if any checksum calculation was requested - in such + case the code falls back to fixed compression. To avoid this less performant case, applications should configure the intermediate buffer size to be larger than the expected input data size (compressed output size is usually unknown, so the only option is to make diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 6b1a7c58c..d56d08e57 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -81,6 +81,16 @@ New Features by making use of the event device capabilities. The event mode currently supports only inline IPsec protocol offload. +* **Added QAT intermediate buffer too small handling in QAT compression PMD.** + + Added a special way of buffer handling when internal QAT intermediate buffer + is too small for Huffman dynamic compression operation. Instead of falling + back to fixed compression, the operation is now split into multiple smaller + dynamic compression requests (possible to execute on QAT) and their results + are then combined and copied into the output buffer. This is not possible if + any checksum calculation was requested - in such case the code falls back to + fixed compression as before. + Removed Items ------------- diff --git a/drivers/common/qat/qat_qp.c b/drivers/common/qat/qat_qp.c index eb1da7243..64dfd85c4 100644 --- a/drivers/common/qat/qat_qp.c +++ b/drivers/common/qat/qat_qp.c @@ -650,32 +650,212 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops) return nb_ops_sent; } +/* Use this for compression only - but keep consistent with above common + * function as much as possible. + */ +uint16_t +qat_enqueue_comp_op_burst(void *qp, void **ops, uint16_t nb_ops) +{ + register struct qat_queue *queue; + struct qat_qp *tmp_qp = (struct qat_qp *)qp; + register uint32_t nb_ops_sent = 0; + register int nb_desc_to_build; + uint16_t nb_ops_possible = nb_ops; + register uint8_t *base_addr; + register uint32_t tail; + + int descriptors_built, total_descriptors_built = 0; + int nb_remaining_descriptors; + int overflow = 0; + + if (unlikely(nb_ops == 0)) + return 0; + + /* read params used a lot in main loop into registers */ + queue = &(tmp_qp->tx_q); + base_addr = (uint8_t *)queue->base_addr; + tail = queue->tail; + + /* Find how many can actually fit on the ring */ + { + /* dequeued can only be written by one thread, but it may not + * be this thread. As it's 4-byte aligned it will be read + * atomically here by any Intel CPU. + * enqueued can wrap before dequeued, but cannot + * lap it as var size of enq/deq (uint32_t) > var size of + * max_inflights (uint16_t). In reality inflights is never + * even as big as max uint16_t, as it's <= ADF_MAX_DESC. + * On wrapping, the calculation still returns the correct + * positive value as all three vars are unsigned. + */ + uint32_t inflights = + tmp_qp->enqueued - tmp_qp->dequeued; + + /* Find how many can actually fit on the ring */ + overflow = (inflights + nb_ops) - tmp_qp->max_inflights; + if (overflow > 0) { + nb_ops_possible = nb_ops - overflow; + if (nb_ops_possible == 0) + return 0; + } + + /* QAT has plenty of work queued already, so don't waste cycles + * enqueueing, wait til the application has gathered a bigger + * burst or some completed ops have been dequeued + */ + if (tmp_qp->min_enq_burst_threshold && inflights > + QAT_QP_MIN_INFL_THRESHOLD && nb_ops_possible < + tmp_qp->min_enq_burst_threshold) { + tmp_qp->stats.threshold_hit_count++; + return 0; + } + } + + /* At this point nb_ops_possible is assuming a 1:1 mapping + * between ops and descriptors. + * Fewer may be sent if some ops have to be split. + * nb_ops_possible is <= burst size. + * Find out how many spaces are actually available on the qp in case + * more are needed. + */ + nb_remaining_descriptors = nb_ops_possible + + ((overflow >= 0) ? 0 : overflow * (-1)); + QAT_DP_LOG(DEBUG, "Nb ops requested %d, nb descriptors remaining %d", + nb_ops, nb_remaining_descriptors); + + while (nb_ops_sent != nb_ops_possible && + nb_remaining_descriptors > 0) { + struct qat_comp_op_cookie *cookie = + tmp_qp->op_cookies[tail >> queue->trailz]; + + descriptors_built = 0; + + QAT_DP_LOG(DEBUG, "--- data length: %u", + ((struct rte_comp_op *)*ops)->src.length); + + nb_desc_to_build = qat_comp_build_request(*ops, + base_addr + tail, cookie, tmp_qp->qat_dev_gen); + QAT_DP_LOG(DEBUG, "%d descriptors built, %d remaining, " + "%d ops sent, %d descriptors needed", + total_descriptors_built, nb_remaining_descriptors, + nb_ops_sent, nb_desc_to_build); + + if (unlikely(nb_desc_to_build < 0)) { + /* this message cannot be enqueued */ + tmp_qp->stats.enqueue_err_count++; + if (nb_ops_sent == 0) + return 0; + goto kick_tail; + } else if (unlikely(nb_desc_to_build > 1)) { + /* this op is too big and must be split - get more + * descriptors and retry + */ + + QAT_DP_LOG(DEBUG, "Build %d descriptors for this op", + nb_desc_to_build); + + nb_remaining_descriptors -= nb_desc_to_build; + if (nb_remaining_descriptors >= 0) { + /* There are enough remaining descriptors + * so retry + */ + int ret2 = qat_comp_build_multiple_requests( + *ops, tmp_qp, tail, + nb_desc_to_build); + + if (unlikely(ret2 < 1)) { + QAT_DP_LOG(DEBUG, + "Failed to build (%d) descriptors, status %d", + nb_desc_to_build, ret2); + + qat_comp_free_split_op_memzones(cookie, + nb_desc_to_build - 1); + + tmp_qp->stats.enqueue_err_count++; + + /* This message cannot be enqueued */ + if (nb_ops_sent == 0) + return 0; + goto kick_tail; + } else { + descriptors_built = ret2; + total_descriptors_built += + descriptors_built; + nb_remaining_descriptors -= + descriptors_built; + QAT_DP_LOG(DEBUG, + "Multiple descriptors (%d) built ok", + descriptors_built); + } + } else { + QAT_DP_LOG(ERR, "For the current op, number of requested descriptors (%d) " + "exceeds number of available descriptors (%d)", + nb_desc_to_build, + nb_remaining_descriptors + + nb_desc_to_build); + + qat_comp_free_split_op_memzones(cookie, + nb_desc_to_build - 1); + + /* Not enough extra descriptors */ + if (nb_ops_sent == 0) + return 0; + goto kick_tail; + } + } else { + descriptors_built = 1; + total_descriptors_built++; + nb_remaining_descriptors--; + QAT_DP_LOG(DEBUG, "Single descriptor built ok"); + } + + tail = adf_modulo(tail + (queue->msg_size * descriptors_built), + queue->modulo_mask); + ops++; + nb_ops_sent++; + } + +kick_tail: + queue->tail = tail; + tmp_qp->enqueued += total_descriptors_built; + tmp_qp->stats.enqueued_count += total_descriptors_built; + txq_write_tail(tmp_qp, queue); + return nb_ops_sent; +} + uint16_t qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops) { struct qat_queue *rx_queue; struct qat_qp *tmp_qp = (struct qat_qp *)qp; uint32_t head; - uint32_t resp_counter = 0; + uint32_t op_resp_counter = 0, fw_resp_counter = 0; uint8_t *resp_msg; + int nb_fw_responses = 0; rx_queue = &(tmp_qp->rx_q); head = rx_queue->head; resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head; while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG && - resp_counter != nb_ops) { + op_resp_counter != nb_ops) { - if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC) + nb_fw_responses = 0; + if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC) { qat_sym_process_response(ops, resp_msg); - else if (tmp_qp->service_type == QAT_SERVICE_COMPRESSION) - qat_comp_process_response(ops, resp_msg, + nb_fw_responses = 1; + } else if (tmp_qp->service_type == QAT_SERVICE_COMPRESSION) + + nb_fw_responses = qat_comp_process_response( + ops, resp_msg, tmp_qp->op_cookies[head >> rx_queue->trailz], &tmp_qp->stats.dequeue_err_count); + else if (tmp_qp->service_type == QAT_SERVICE_ASYMMETRIC) { #ifdef BUILD_QAT_ASYM qat_asym_process_response(ops, resp_msg, tmp_qp->op_cookies[head >> rx_queue->trailz]); + nb_fw_responses = 1; #endif } @@ -683,21 +863,38 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops) rx_queue->modulo_mask); resp_msg = (uint8_t *)rx_queue->base_addr + head; - ops++; - resp_counter++; + + if (ops != NULL && nb_fw_responses) { + /* only move on to next op if one was ready to return + * to API + */ + ops++; + op_resp_counter++; + } + + /* A compression op may be broken up into multiple fw requests. + * Only count fw responses as complete once ALL the responses + * associated with an op have been processed, as the cookie + * data from the first response must be available until + * finished with all firmware responses. + */ + fw_resp_counter += nb_fw_responses; } - if (resp_counter > 0) { + + if (fw_resp_counter > 0) { rx_queue->head = head; - tmp_qp->dequeued += resp_counter; - tmp_qp->stats.dequeued_count += resp_counter; - rx_queue->nb_processed_responses += resp_counter; + tmp_qp->dequeued += fw_resp_counter; + tmp_qp->stats.dequeued_count += fw_resp_counter; + rx_queue->nb_processed_responses += fw_resp_counter; if (rx_queue->nb_processed_responses > - QAT_CSR_HEAD_WRITE_THRESH) + QAT_CSR_HEAD_WRITE_THRESH) rxq_free_desc(tmp_qp, rx_queue); } + QAT_DP_LOG(DEBUG, "Dequeue burst return: %u, QAT responses: %u", + op_resp_counter, fw_resp_counter); - return resp_counter; + return op_resp_counter; } /* This is almost same as dequeue_op_burst, without the atomic, without stats diff --git a/drivers/common/qat/qat_qp.h b/drivers/common/qat/qat_qp.h index 88d3c9942..575d69059 100644 --- a/drivers/common/qat/qat_qp.h +++ b/drivers/common/qat/qat_qp.h @@ -89,6 +89,9 @@ extern const struct qat_qp_hw_data qat_gen3_qps[][ADF_MAX_QPS_ON_ANY_SERVICE]; uint16_t qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops); +uint16_t +qat_enqueue_comp_op_burst(void *qp, void **ops, uint16_t nb_ops); + uint16_t qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops); diff --git a/drivers/compress/qat/qat_comp.c b/drivers/compress/qat/qat_comp.c index 533e34f6b..9e1fd2fe9 100644 --- a/drivers/compress/qat/qat_comp.c +++ b/drivers/compress/qat/qat_comp.c @@ -13,11 +13,93 @@ #include <rte_spinlock.h> #include <rte_log.h> #include <rte_malloc.h> +#include <rte_memzone.h> #include "qat_logs.h" #include "qat_comp.h" #include "qat_comp_pmd.h" +static void +qat_comp_fallback_to_fixed(struct icp_qat_fw_comp_req *comp_req) +{ + QAT_DP_LOG(DEBUG, "QAT PMD: fallback to fixed compression!"); + + comp_req->comn_hdr.service_cmd_id = + ICP_QAT_FW_COMP_CMD_STATIC; + + ICP_QAT_FW_COMN_NEXT_ID_SET( + &comp_req->comp_cd_ctrl, + ICP_QAT_FW_SLICE_DRAM_WR); + + ICP_QAT_FW_COMN_NEXT_ID_SET( + &comp_req->u2.xlt_cd_ctrl, + ICP_QAT_FW_SLICE_NULL); + ICP_QAT_FW_COMN_CURR_ID_SET( + &comp_req->u2.xlt_cd_ctrl, + ICP_QAT_FW_SLICE_NULL); +} + +void +qat_comp_free_split_op_memzones(struct qat_comp_op_cookie *cookie, + unsigned int nb_children) +{ + unsigned int i; + + /* free all memzones allocated for child descriptors */ + for (i = 0; i < nb_children; i++) + rte_memzone_free(cookie->dst_memzones[i]); + + /* and free the pointer table */ + rte_free(cookie->dst_memzones); + cookie->dst_memzones = NULL; +} + +static int +qat_comp_allocate_split_op_memzones(struct qat_comp_op_cookie *cookie, + unsigned int nb_descriptors_needed) +{ + struct qat_queue *txq = &(cookie->qp->tx_q); + char dst_memz_name[RTE_MEMZONE_NAMESIZE]; + unsigned int i; + + /* allocate the array of memzone pointers */ + cookie->dst_memzones = rte_zmalloc_socket("qat PMD im buf mz pointers", + (nb_descriptors_needed - 1) * + sizeof(const struct rte_memzone *), + RTE_CACHE_LINE_SIZE, cookie->socket_id); + + if (cookie->dst_memzones == NULL) { + QAT_DP_LOG(ERR, + "QAT PMD: failed to allocate im buf mz pointers"); + return -ENOMEM; + } + + for (i = 0; i < nb_descriptors_needed - 1; i++) { + snprintf(dst_memz_name, + sizeof(dst_memz_name), + "dst_%u_%u_%u_%u_%u", + cookie->qp->qat_dev->qat_dev_id, + txq->hw_bundle_number, txq->hw_queue_number, + cookie->cookie_index, i); + + cookie->dst_memzones[i] = rte_memzone_reserve_aligned( + dst_memz_name, RTE_PMD_QAT_COMP_IM_BUFFER_SIZE, + cookie->socket_id, RTE_MEMZONE_IOVA_CONTIG, + RTE_CACHE_LINE_SIZE); + + if (cookie->dst_memzones[i] == NULL) { + QAT_DP_LOG(ERR, + "QAT PMD: failed to allocate dst buffer memzone"); + + /* let's free all memzones allocated up to now */ + qat_comp_free_split_op_memzones(cookie, i); + + return -ENOMEM; + } + } + + return 0; +} int qat_comp_build_request(void *in_op, uint8_t *out_msg, @@ -57,7 +139,48 @@ qat_comp_build_request(void *in_op, uint8_t *out_msg, rte_mov128(out_msg, tmpl); comp_req->comn_mid.opaque_data = (uint64_t)(uintptr_t)op; - if (op->op_type == RTE_COMP_OP_STATEFUL) { + if (likely(qat_xform->qat_comp_request_type == + QAT_COMP_REQUEST_DYNAMIC_COMP_STATELESS)) { + + if (unlikely(op->src.length > QAT_FALLBACK_THLD)) { + /* the operation must be split into pieces */ + if (qat_xform->checksum_type != + RTE_COMP_CHECKSUM_NONE) { + /* fallback to fixed compression in case any + * checksum calculation was requested + */ + qat_comp_fallback_to_fixed(comp_req); + } else { + /* calculate num. of descriptors for split op */ + unsigned int nb_descriptors_needed = + op->src.length / QAT_FALLBACK_THLD + 1; + /* allocate memzone for output data */ + if (qat_comp_allocate_split_op_memzones( + cookie, nb_descriptors_needed)) { + /* out of memory, fallback to fixed */ + qat_comp_fallback_to_fixed(comp_req); + } else { + QAT_DP_LOG(DEBUG, + "Input data is too big, op must be split into %u descriptors", + nb_descriptors_needed); + return (int) nb_descriptors_needed; + } + } + } + + /* set BFINAL bit according to flush_flag */ + comp_req->comp_pars.req_par_flags = + ICP_QAT_FW_COMP_REQ_PARAM_FLAGS_BUILD( + ICP_QAT_FW_COMP_SOP, + ICP_QAT_FW_COMP_EOP, + op->flush_flag == RTE_COMP_FLUSH_FINAL ? + ICP_QAT_FW_COMP_BFINAL + : ICP_QAT_FW_COMP_NOT_BFINAL, + ICP_QAT_FW_COMP_CNV, + ICP_QAT_FW_COMP_CNV_RECOVERY); + + } else if (op->op_type == RTE_COMP_OP_STATEFUL) { + comp_req->comp_pars.req_par_flags = ICP_QAT_FW_COMP_REQ_PARAM_FLAGS_BUILD( (stream->start_of_packet) ? @@ -72,30 +195,6 @@ qat_comp_build_request(void *in_op, uint8_t *out_msg, ICP_QAT_FW_COMP_NO_CNV_RECOVERY); } - if (likely(qat_xform->qat_comp_request_type == - QAT_COMP_REQUEST_DYNAMIC_COMP_STATELESS)) { - if (unlikely(op->src.length > QAT_FALLBACK_THLD)) { - - /* fallback to fixed compression */ - comp_req->comn_hdr.service_cmd_id = - ICP_QAT_FW_COMP_CMD_STATIC; - - ICP_QAT_FW_COMN_NEXT_ID_SET(&comp_req->comp_cd_ctrl, - ICP_QAT_FW_SLICE_DRAM_WR); - - ICP_QAT_FW_COMN_NEXT_ID_SET(&comp_req->u2.xlt_cd_ctrl, - ICP_QAT_FW_SLICE_NULL); - ICP_QAT_FW_COMN_CURR_ID_SET(&comp_req->u2.xlt_cd_ctrl, - ICP_QAT_FW_SLICE_NULL); - - QAT_DP_LOG(DEBUG, "QAT PMD: fallback to fixed " - "compression! IM buffer size can be too low " - "for produced data.\n Please use input " - "buffer length lower than %d bytes", - QAT_FALLBACK_THLD); - } - } - /* common for sgl and flat buffers */ comp_req->comp_pars.comp_len = op->src.length; comp_req->comp_pars.out_buffer_sz = rte_pktmbuf_pkt_len(op->m_dst) - @@ -233,6 +332,213 @@ qat_comp_build_request(void *in_op, uint8_t *out_msg, return 0; } +static inline uint32_t adf_modulo(uint32_t data, uint32_t modulo_mask) +{ + return data & modulo_mask; +} + +static inline void +qat_comp_mbuf_skip(struct rte_mbuf **mbuf, uint32_t *offset, uint32_t len) +{ + while (*offset + len >= rte_pktmbuf_data_len(*mbuf)) { + len -= (rte_pktmbuf_data_len(*mbuf) - *offset); + *mbuf = (*mbuf)->next; + *offset = 0; + } + *offset = len; +} + +int +qat_comp_build_multiple_requests(void *in_op, struct qat_qp *qp, + uint32_t parent_tail, int nb_descr) +{ + struct rte_comp_op op_backup; + struct rte_mbuf dst_mbuf; + struct rte_comp_op *op = in_op; + struct qat_queue *txq = &(qp->tx_q); + uint8_t *base_addr = (uint8_t *)txq->base_addr; + uint8_t *out_msg = base_addr + parent_tail; + uint32_t tail = parent_tail; + struct icp_qat_fw_comp_req *comp_req = + (struct icp_qat_fw_comp_req *)out_msg; + struct qat_comp_op_cookie *parent_cookie = + (struct qat_comp_op_cookie *) + qp->op_cookies[parent_tail / txq->msg_size]; + struct qat_comp_op_cookie *child_cookie; + uint16_t dst_data_size = + RTE_MIN(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE, 65535); + uint32_t data_to_enqueue = op->src.length - QAT_FALLBACK_THLD; + int num_descriptors_built = 1; + int ret; + + QAT_DP_LOG(DEBUG, "op %p, parent_cookie %p", op, parent_cookie); + + /* copy original op to the local variable for restoring later */ + rte_memcpy(&op_backup, op, sizeof(op_backup)); + + parent_cookie->nb_child_responses = 0; + parent_cookie->nb_children = 0; + parent_cookie->split_op = 1; + parent_cookie->dst_data = op->m_dst; + parent_cookie->dst_data_offset = op->dst.offset; + + op->src.length = QAT_FALLBACK_THLD; + op->flush_flag = RTE_COMP_FLUSH_FULL; + + QAT_DP_LOG(DEBUG, "parent op src len %u dst len %u", + op->src.length, op->m_dst->pkt_len); + + ret = qat_comp_build_request(in_op, out_msg, parent_cookie, + qp->qat_dev_gen); + if (ret != 0) { + /* restore op and clear cookie */ + QAT_DP_LOG(WARNING, "Failed to build parent descriptor"); + op->src.length = op_backup.src.length; + op->flush_flag = op_backup.flush_flag; + parent_cookie->split_op = 0; + return ret; + } + + /* prepare local dst mbuf */ + rte_memcpy(&dst_mbuf, op->m_dst, sizeof(dst_mbuf)); + rte_pktmbuf_reset(&dst_mbuf); + dst_mbuf.buf_len = dst_data_size; + dst_mbuf.data_len = dst_data_size; + dst_mbuf.pkt_len = dst_data_size; + dst_mbuf.data_off = 0; + + /* update op for the child operations */ + op->m_dst = &dst_mbuf; + op->dst.offset = 0; + + while (data_to_enqueue) { + const struct rte_memzone *mz = + parent_cookie->dst_memzones[num_descriptors_built - 1]; + uint32_t src_data_size = RTE_MIN(data_to_enqueue, + QAT_FALLBACK_THLD); + uint32_t cookie_index; + + /* update params for the next op */ + op->src.offset += QAT_FALLBACK_THLD; + op->src.length = src_data_size; + op->flush_flag = (src_data_size == data_to_enqueue) ? + op_backup.flush_flag : RTE_COMP_FLUSH_FULL; + + /* update dst mbuf for the next op (use memzone for dst data) */ + dst_mbuf.buf_addr = mz->addr; + dst_mbuf.buf_iova = mz->iova; + + /* move the tail and calculate next cookie index */ + tail = adf_modulo(tail + txq->msg_size, txq->modulo_mask); + cookie_index = tail / txq->msg_size; + child_cookie = (struct qat_comp_op_cookie *) + qp->op_cookies[cookie_index]; + comp_req = (struct icp_qat_fw_comp_req *)(base_addr + tail); + + /* update child cookie */ + child_cookie->split_op = 1; /* must be set for child as well */ + child_cookie->parent_cookie = parent_cookie; /* same as above */ + child_cookie->nb_children = 0; + child_cookie->dest_buffer = mz->addr; + + QAT_DP_LOG(DEBUG, + "cookie_index %u, child_cookie %p, comp_req %p", + cookie_index, child_cookie, comp_req); + QAT_DP_LOG(DEBUG, + "data_to_enqueue %u, num_descriptors_built %d", + data_to_enqueue, num_descriptors_built); + QAT_DP_LOG(DEBUG, "child op src len %u dst len %u", + op->src.length, op->m_dst->pkt_len); + + /* build the request */ + ret = qat_comp_build_request(op, (uint8_t *)comp_req, + child_cookie, qp->qat_dev_gen); + if (ret < 0) { + QAT_DP_LOG(WARNING, "Failed to build child descriptor"); + /* restore op and clear cookie */ + rte_memcpy(op, &op_backup, sizeof(op_backup)); + parent_cookie->split_op = 0; + parent_cookie->nb_children = 0; + return ret; + } + + data_to_enqueue -= src_data_size; + num_descriptors_built++; + } + + /* restore backed up original op */ + rte_memcpy(op, &op_backup, sizeof(op_backup)); + + if (nb_descr != num_descriptors_built) + QAT_DP_LOG(ERR, "split op. expected %d, built %d", + nb_descr, num_descriptors_built); + + parent_cookie->nb_children = num_descriptors_built - 1; + return num_descriptors_built; +} + +static inline void +qat_comp_response_data_copy(struct qat_comp_op_cookie *cookie, + struct rte_comp_op *rx_op) +{ + struct qat_comp_op_cookie *pc = cookie->parent_cookie; + struct rte_mbuf *sgl_buf = pc->dst_data; + void *op_dst_addr = rte_pktmbuf_mtod_offset(sgl_buf, uint8_t *, + pc->dst_data_offset); + + /* number of bytes left in the current segment */ + uint32_t left_in_current = rte_pktmbuf_data_len(sgl_buf) - + pc->dst_data_offset; + + uint32_t prod, sent; + + if (rx_op->produced <= left_in_current) { + rte_memcpy(op_dst_addr, cookie->dest_buffer, + rx_op->produced); + /* calculate dst mbuf and offset for the next child op */ + if (rx_op->produced == left_in_current) { + pc->dst_data = sgl_buf->next; + pc->dst_data_offset = 0; + } else + pc->dst_data_offset += rx_op->produced; + } else { + rte_memcpy(op_dst_addr, cookie->dest_buffer, + left_in_current); + sgl_buf = sgl_buf->next; + prod = rx_op->produced - left_in_current; + sent = left_in_current; + while (prod > rte_pktmbuf_data_len(sgl_buf)) { + op_dst_addr = rte_pktmbuf_mtod_offset(sgl_buf, + uint8_t *, 0); + + rte_memcpy(op_dst_addr, + ((uint8_t *)cookie->dest_buffer) + + sent, + rte_pktmbuf_data_len(sgl_buf)); + + prod -= rte_pktmbuf_data_len(sgl_buf); + sent += rte_pktmbuf_data_len(sgl_buf); + + sgl_buf = sgl_buf->next; + } + + op_dst_addr = rte_pktmbuf_mtod_offset(sgl_buf, uint8_t *, 0); + + rte_memcpy(op_dst_addr, + ((uint8_t *)cookie->dest_buffer) + sent, + prod); + + /* calculate dst mbuf and offset for the next child op */ + if (prod == rte_pktmbuf_data_len(sgl_buf)) { + pc->dst_data = sgl_buf->next; + pc->dst_data_offset = 0; + } else { + pc->dst_data = sgl_buf; + pc->dst_data_offset = prod; + } + } +} + int qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, uint64_t *dequeue_err_count) @@ -241,6 +547,14 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, (struct icp_qat_fw_comp_resp *)resp; struct qat_comp_op_cookie *cookie = (struct qat_comp_op_cookie *)op_cookie; + + struct icp_qat_fw_resp_comp_pars *comp_resp1 = + (struct icp_qat_fw_resp_comp_pars *)&resp_msg->comp_resp_pars; + + QAT_DP_LOG(DEBUG, "input counter = %u, output counter = %u", + comp_resp1->input_byte_counter, + comp_resp1->output_byte_counter); + struct rte_comp_op *rx_op = (struct rte_comp_op *)(uintptr_t) (resp_msg->opaque_data); struct qat_comp_stream *stream; @@ -275,7 +589,10 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, rx_op->consumed = 0; rx_op->produced = 0; *op = (void *)rx_op; - return 0; + /* also in this case number of returned ops */ + /* must be equal to one, */ + /* appropriate status (error) must be set as well */ + return 1; } if (likely(qat_xform->qat_comp_request_type @@ -288,7 +605,7 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, *op = (void *)rx_op; QAT_DP_LOG(ERR, "QAT has wrong firmware"); ++(*dequeue_err_count); - return 0; + return 1; } } @@ -305,8 +622,9 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, int8_t xlat_err_code = (int8_t)resp_msg->comn_resp.comn_error.xlat_err_code; - /* handle recoverable out-of-buffer condition in stateful */ - /* decompression scenario */ + /* handle recoverable out-of-buffer condition in stateful + * decompression scenario + */ if (cmp_err_code == ERR_CODE_OVERFLOW_ERROR && !xlat_err_code && qat_xform->qat_comp_request_type == QAT_COMP_REQUEST_DECOMPRESS @@ -327,10 +645,12 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, xlat_err_code == ERR_CODE_OVERFLOW_ERROR)){ struct icp_qat_fw_resp_comp_pars *comp_resp = - (struct icp_qat_fw_resp_comp_pars *)&resp_msg->comp_resp_pars; + (struct icp_qat_fw_resp_comp_pars *) + &resp_msg->comp_resp_pars; - /* handle recoverable out-of-buffer condition */ - /* in stateless compression scenario */ + /* handle recoverable out-of-buffer condition + * in stateless compression scenario + */ if (comp_resp->input_byte_counter) { if ((qat_xform->qat_comp_request_type == QAT_COMP_REQUEST_FIXED_COMP_STATELESS) || @@ -375,9 +695,89 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, rx_op->output_chksum = comp_resp->curr_chksum; } } - *op = (void *)rx_op; + QAT_DP_LOG(DEBUG, "About to check for split op :cookies: %p %p, split:%u", + cookie, cookie->parent_cookie, cookie->split_op); + + if (cookie->split_op) { + *op = NULL; + struct qat_comp_op_cookie *pc = cookie->parent_cookie; + + if (cookie->nb_children > 0) { + QAT_DP_LOG(DEBUG, "Parent"); + /* parent - don't return until all children + * responses are collected + */ + cookie->total_consumed = rx_op->consumed; + cookie->total_produced = rx_op->produced; + if (err) { + cookie->error = rx_op->status; + rx_op->status = RTE_COMP_OP_STATUS_SUCCESS; + } else { + /* calculate dst mbuf and offset for child op */ + qat_comp_mbuf_skip(&cookie->dst_data, + &cookie->dst_data_offset, + rx_op->produced); + } + } else { + QAT_DP_LOG(DEBUG, "Child"); + if (pc->error == RTE_COMP_OP_STATUS_SUCCESS) { + if (err) + pc->error = rx_op->status; + if (rx_op->produced) { + /* this covers both SUCCESS and + * OUT_OF_SPACE_RECOVERABLE cases + */ + qat_comp_response_data_copy(cookie, + rx_op); + pc->total_consumed += rx_op->consumed; + pc->total_produced += rx_op->produced; + } + } + rx_op->status = RTE_COMP_OP_STATUS_SUCCESS; + + pc->nb_child_responses++; + + /* (child) cookie fields have to be reset + * to avoid problems with reusability - + * rx and tx queue starting from index zero + */ + cookie->nb_children = 0; + cookie->split_op = 0; + cookie->nb_child_responses = 0; + cookie->dest_buffer = NULL; + + if (pc->nb_child_responses == pc->nb_children) { + uint8_t child_resp; + + /* parent should be included as well */ + child_resp = pc->nb_child_responses + 1; + + rx_op->status = pc->error; + rx_op->consumed = pc->total_consumed; + rx_op->produced = pc->total_produced; + *op = (void *)rx_op; + + /* free memzones used for dst data */ + qat_comp_free_split_op_memzones(pc, + pc->nb_children); + + /* (parent) cookie fields have to be reset + * to avoid problems with reusability - + * rx and tx queue starting from index zero + */ + pc->nb_children = 0; + pc->split_op = 0; + pc->nb_child_responses = 0; + pc->error = RTE_COMP_OP_STATUS_SUCCESS; + + return child_resp; + } + } + return 0; + } - return 0; + *op = (void *)rx_op; + return 1; } unsigned int @@ -443,9 +843,9 @@ static int qat_comp_create_templates(struct qat_comp_xform *qat_xform, comp_level = ICP_QAT_HW_COMPRESSION_DEPTH_1; req_par_flags = ICP_QAT_FW_COMP_REQ_PARAM_FLAGS_BUILD( ICP_QAT_FW_COMP_SOP, ICP_QAT_FW_COMP_EOP, - ICP_QAT_FW_COMP_BFINAL, ICP_QAT_FW_COMP_NO_CNV, - ICP_QAT_FW_COMP_NO_CNV_RECOVERY); - + ICP_QAT_FW_COMP_BFINAL, + ICP_QAT_FW_COMP_CNV, + ICP_QAT_FW_COMP_CNV_RECOVERY); } else { if (xform->compress.level == RTE_COMP_LEVEL_PMD_DEFAULT) comp_level = ICP_QAT_HW_COMPRESSION_DEPTH_8; diff --git a/drivers/compress/qat/qat_comp.h b/drivers/compress/qat/qat_comp.h index 2231451a1..1c07f2233 100644 --- a/drivers/compress/qat/qat_comp.h +++ b/drivers/compress/qat/qat_comp.h @@ -11,6 +11,7 @@ #include <rte_compressdev_pmd.h> #include "qat_common.h" +#include "qat_qp.h" #include "icp_qat_hw.h" #include "icp_qat_fw_comp.h" #include "icp_qat_fw_la.h" @@ -22,7 +23,7 @@ #define ERR_CODE_QAT_COMP_WRONG_FW -99 /* fallback to fixed compression threshold */ -#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.1)) +#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.3)) #define QAT_MIN_OUT_BUF_SIZE 46 @@ -63,6 +64,24 @@ struct qat_comp_op_cookie { uint16_t dst_nb_elems; struct qat_sgl *qat_sgl_src_d; struct qat_sgl *qat_sgl_dst_d; + struct qat_qp *qp; + uint32_t cookie_index; + + /* QAT IM buffer too small handling: */ + uint8_t split_op; + uint8_t nb_children; + + /* used by the parent only */ + uint8_t nb_child_responses; + uint32_t total_consumed; + uint32_t total_produced; + const struct rte_memzone **dst_memzones; + struct rte_mbuf *dst_data; + uint32_t dst_data_offset; + + /* used by the child only */ + struct qat_comp_op_cookie *parent_cookie; + void *dest_buffer; }; struct qat_comp_xform { @@ -86,6 +105,14 @@ int qat_comp_build_request(void *in_op, uint8_t *out_msg, void *op_cookie, enum qat_device_gen qat_dev_gen __rte_unused); +int +qat_comp_build_multiple_requests(void *in_op, struct qat_qp *qp, + uint32_t parent_tail, int nb_descr); + +void +qat_comp_free_split_op_memzones(struct qat_comp_op_cookie *cookie, + unsigned int nb_children); + int qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, uint64_t *dequeue_err_count); diff --git a/drivers/compress/qat/qat_comp_pmd.c b/drivers/compress/qat/qat_comp_pmd.c index 9a7ed19d7..fe62de533 100644 --- a/drivers/compress/qat/qat_comp_pmd.c +++ b/drivers/compress/qat/qat_comp_pmd.c @@ -146,6 +146,9 @@ qat_comp_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, struct qat_comp_op_cookie *cookie = qp->op_cookies[i]; + cookie->qp = qp; + cookie->cookie_index = i; + cookie->qat_sgl_src_d = rte_zmalloc_socket(NULL, sizeof(struct qat_sgl) + sizeof(struct qat_flat_buf) * @@ -560,20 +563,6 @@ qat_comp_dev_info_get(struct rte_compressdev *dev, } } -static uint16_t -qat_comp_pmd_enqueue_op_burst(void *qp, struct rte_comp_op **ops, - uint16_t nb_ops) -{ - return qat_enqueue_op_burst(qp, (void **)ops, nb_ops); -} - -static uint16_t -qat_comp_pmd_dequeue_op_burst(void *qp, struct rte_comp_op **ops, - uint16_t nb_ops) -{ - return qat_dequeue_op_burst(qp, (void **)ops, nb_ops); -} - static uint16_t qat_comp_pmd_enq_deq_dummy_op_burst(void *qp __rte_unused, struct rte_comp_op **ops __rte_unused, @@ -603,7 +592,7 @@ static struct rte_compressdev_ops compress_qat_dummy_ops = { }; static uint16_t -qat_comp_pmd_dequeue_frst_op_burst(void *qp, struct rte_comp_op **ops, +qat_comp_pmd_dequeue_first_op_burst(void *qp, struct rte_comp_op **ops, uint16_t nb_ops) { uint16_t ret = qat_dequeue_op_burst(qp, (void **)ops, nb_ops); @@ -623,7 +612,8 @@ qat_comp_pmd_dequeue_frst_op_burst(void *qp, struct rte_comp_op **ops, } else { tmp_qp->qat_dev->comp_dev->compressdev->dequeue_burst = - qat_comp_pmd_dequeue_op_burst; + (compressdev_dequeue_pkt_burst_t) + qat_dequeue_op_burst; } } return ret; @@ -698,8 +688,9 @@ qat_comp_dev_create(struct qat_pci_device *qat_pci_dev, compressdev->dev_ops = &compress_qat_ops; - compressdev->enqueue_burst = qat_comp_pmd_enqueue_op_burst; - compressdev->dequeue_burst = qat_comp_pmd_dequeue_frst_op_burst; + compressdev->enqueue_burst = (compressdev_enqueue_pkt_burst_t) + qat_enqueue_comp_op_burst; + compressdev->dequeue_burst = qat_comp_pmd_dequeue_first_op_burst; compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED; -- 2.17.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/2] compress/qat: im buffer too small - split op 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 1/2] " Adam Dybkowski @ 2020-04-08 15:43 ` Trahe, Fiona 0 siblings, 0 replies; 20+ messages in thread From: Trahe, Fiona @ 2020-04-08 15:43 UTC (permalink / raw) To: Dybkowski, AdamX, dev, akhil.goyal; +Cc: Trahe, Fiona > -----Original Message----- > From: Dybkowski, AdamX <adamx.dybkowski@intel.com> > Sent: Wednesday, April 8, 2020 1:51 PM > To: dev@dpdk.org; Trahe, Fiona <fiona.trahe@intel.com>; akhil.goyal@nxp.com > Cc: Dybkowski, AdamX <adamx.dybkowski@intel.com> > Subject: [PATCH v2 1/2] compress/qat: im buffer too small - split op > > This patch implements a special way of buffer handling when internal > QAT IM buffer is too small for Huffman dynamic compression operation. > Instead of falling back to fixed compression, the operation is now > split into multiple smaller dynamic compression requests (possible to > execute on QAT) and their results are then combined and copied into > the output buffer. This is not possible if any checksum calculation > was requested - in such case the code falls back to fixed compression > as before. > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> Acked-by: Fiona Trahe <fiona.trahe@intel.com> ^ permalink raw reply [flat|nested] 20+ messages in thread
* [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-08 12:50 [dpdk-dev] [PATCH v2 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 1/2] " Adam Dybkowski @ 2020-04-08 12:51 ` Adam Dybkowski 2020-04-08 15:44 ` Trahe, Fiona 2020-04-15 18:35 ` Akhil Goyal 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2 siblings, 2 replies; 20+ messages in thread From: Adam Dybkowski @ 2020-04-08 12:51 UTC (permalink / raw) To: dev, fiona.trahe, akhil.goyal; +Cc: Adam Dybkowski This patch adds a new test suite for verification of the "internal QAT IM buffer too small" case handling. These unit tests are specific to the QAT PMD only - that's why they are contained in a separate test suite. Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> --- app/test/test_compressdev.c | 1251 ++++++++++++++++++++++++++++++++--- 1 file changed, 1177 insertions(+), 74 deletions(-) diff --git a/app/test/test_compressdev.c b/app/test/test_compressdev.c index 7549135c2..40eb74dd3 100644 --- a/app/test/test_compressdev.c +++ b/app/test/test_compressdev.c @@ -1,10 +1,11 @@ /* SPDX-License-Identifier: BSD-3-Clause * Copyright(c) 2018 - 2019 Intel Corporation */ -#include <string.h> -#include <zlib.h> #include <math.h> +#include <stdio.h> #include <stdlib.h> +#include <string.h> +#include <zlib.h> #include <unistd.h> #include <rte_cycles.h> @@ -18,6 +19,8 @@ #include "test.h" #define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) != 0)) +#define ENDS_WITH(s, suffix) (strlen(s) >= strlen(suffix) && \ + !strcmp(suffix, s + strlen(s) - strlen(suffix))) #define DEFAULT_WINDOW_SIZE 15 #define DEFAULT_MEM_LEVEL 8 @@ -30,6 +33,7 @@ * due to the compress block headers */ #define COMPRESS_BUF_SIZE_RATIO 1.3 +#define COMPRESS_BUF_SIZE_RATIO_DISABLED 1.0 #define COMPRESS_BUF_SIZE_RATIO_OVERFLOW 0.2 #define NUM_LARGE_MBUFS 16 #define SMALL_SEG_SIZE 256 @@ -52,6 +56,25 @@ #define NUM_BIG_MBUFS 4 #define BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * NUM_BIG_MBUFS / 2) +/* constants for "im buffer" tests start here */ +#define IM_NUM_BIG_MBUFS (512 + 1) +#define IM_BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * 2) +/* number of mbufs lower than number of inflight ops */ +#define IM_BUF_NUM_MBUFS 3 +/* above threshold (QAT_FALLBACK_THLD) and below max mbuf size */ +#define IM_BUF_DATA_TEST_SIZE_LB 59600 +/* data size smaller than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_SGL (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS) +/* number of mbufs bigger than number of inflight ops */ +#define IM_BUF_NUM_MBUFS_OVER (NUM_MAX_INFLIGHT_OPS + 1) +/* data size bigger than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_OVER (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_OVER) +/* number of mid-size mbufs */ +#define IM_BUF_NUM_MBUFS_MID ((NUM_MAX_INFLIGHT_OPS / 3) + 1) +/* capacity of mid-size mbufs */ +#define IM_BUF_DATA_TEST_SIZE_MID (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_MID) + + const char * huffman_type_strings[] = { [RTE_COMP_HUFFMAN_DEFAULT] = "PMD default", @@ -78,6 +101,11 @@ enum overflow_test { OVERFLOW_ENABLED }; +enum ratio_switch { + RATIO_DISABLED, + RATIO_ENABLED +}; + enum operation_type { OPERATION_COMPRESSION, OPERATION_DECOMPRESSION @@ -88,6 +116,7 @@ struct priv_op_data { }; struct comp_testsuite_params { + const char *suite_name; struct rte_mempool *large_mbuf_pool; struct rte_mempool *small_mbuf_pool; struct rte_mempool *big_mbuf_pool; @@ -123,6 +152,7 @@ struct test_data_params { const struct rte_memzone *uncompbuf_memzone; /* overflow test activation */ enum overflow_test overflow; + enum ratio_switch ratio; }; struct test_private_arrays { @@ -141,6 +171,26 @@ struct test_private_arrays { static struct comp_testsuite_params testsuite_params = { 0 }; + +static uint8_t +is_qat_specific_testsuite(void) +{ + return strstr(testsuite_params.suite_name, "QAT") != NULL; +} + +static unsigned int +get_num_big_mbufs(void) +{ + return is_qat_specific_testsuite() ? IM_NUM_BIG_MBUFS : NUM_BIG_MBUFS; +} + +static unsigned int +get_big_data_test_size(void) +{ + return is_qat_specific_testsuite() ? + IM_BIG_DATA_TEST_SIZE : BIG_DATA_TEST_SIZE; +} + static void testsuite_teardown(void) { @@ -212,7 +262,7 @@ testsuite_setup(void) /* Create mempool with big buffers for SGL testing */ ts_params->big_mbuf_pool = rte_pktmbuf_pool_create("big_mbuf_pool", - NUM_BIG_MBUFS + 1, + get_num_big_mbufs() + 1, CACHE_SIZE, 0, MAX_MBUF_SEGMENT_SIZE, rte_socket_id()); @@ -316,6 +366,8 @@ test_compressdev_invalid_configuration(void) }; struct rte_compressdev_info dev_info; + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + /* Invalid configuration with 0 queue pairs */ memcpy(&invalid_config, &valid_config, sizeof(struct rte_compressdev_config)); @@ -691,7 +743,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf, if (data_ptr != NULL) { /* Copy characters without NULL terminator */ - strncpy(buf_ptr, data_ptr, data_size); + memcpy(buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size; @@ -731,7 +783,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf, } if (data_ptr != NULL) { /* Copy characters without NULL terminator */ - strncpy(buf_ptr, data_ptr, data_size); + memcpy(buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size; @@ -760,17 +812,20 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, { uint16_t num_enqd, num_deqd, num_total_deqd; unsigned int deqd_retries = 0; + int res = 0; /* Enqueue and dequeue all operations */ num_enqd = rte_compressdev_enqueue_burst(0, 0, ops, num_bufs); if (num_enqd < num_bufs) { RTE_LOG(ERR, USER1, "Some operations could not be enqueued\n"); - return -1; + res = -1; } + /* dequeue ops even on error (same number of ops as was enqueued) */ + num_total_deqd = 0; - do { + while (num_total_deqd < num_enqd) { /* * If retrying a dequeue call, wait for 10 ms to allow * enough time to the driver to process the operations @@ -783,7 +838,8 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, if (deqd_retries == MAX_DEQD_RETRIES) { RTE_LOG(ERR, USER1, "Not all operations could be dequeued\n"); - return -1; + res = -1; + break; } usleep(DEQUEUE_WAIT_TIME); } @@ -792,9 +848,9 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, num_total_deqd += num_deqd; deqd_retries++; - } while (num_total_deqd < num_enqd); + } - return 0; + return res; } /** @@ -956,7 +1012,9 @@ test_mbufs_calculate_data_size( /* local variables: */ uint32_t data_size; struct priv_op_data *priv_data; - float ratio; + float ratio_val; + enum ratio_switch ratio = test_data->ratio; + uint8_t not_zlib_compr; /* true if zlib isn't current compression dev */ enum overflow_test overflow = test_data->overflow; @@ -973,13 +1031,16 @@ test_mbufs_calculate_data_size( not_zlib_compr = (test_data->zlib_dir == ZLIB_DECOMPRESS || test_data->zlib_dir == ZLIB_NONE); - ratio = (not_zlib_compr && + ratio_val = (ratio == RATIO_ENABLED) ? + COMPRESS_BUF_SIZE_RATIO : + COMPRESS_BUF_SIZE_RATIO_DISABLED; + + ratio_val = (not_zlib_compr && (overflow == OVERFLOW_ENABLED)) ? COMPRESS_BUF_SIZE_RATIO_OVERFLOW : - COMPRESS_BUF_SIZE_RATIO; - - data_size = strlen(test_bufs[i]) * ratio; + ratio_val; + data_size = strlen(test_bufs[i]) * ratio_val; } else { priv_data = (struct priv_op_data *) (ops_processed[i] + 1); @@ -1085,6 +1146,9 @@ test_setup_output_bufs( } else { for (i = 0; i < num_bufs; i++) { + enum rte_comp_huffman comp_huffman = + ts_params->def_comp_xform->compress.deflate.huffman; + /* data size calculation */ data_size = test_mbufs_calculate_data_size( op_type, @@ -1094,6 +1158,11 @@ test_setup_output_bufs( test_data, i); + if (comp_huffman != RTE_COMP_HUFFMAN_DYNAMIC) { + if (op_type == OPERATION_DECOMPRESSION) + data_size *= COMPRESS_BUF_SIZE_RATIO; + } + /* data allocation */ if (buff_type == SGL_BOTH || buff_type == LB_TO_SGL) { ret = prepare_sgl_bufs(NULL, current_bufs[i], @@ -1192,6 +1261,11 @@ test_deflate_comp_run(const struct interim_data_params *int_data, ops[i]->src.length = rte_pktmbuf_pkt_len(uncomp_bufs[i]); ops[i]->dst.offset = 0; + RTE_LOG(DEBUG, USER1, + "Uncompressed buffer length = %u compressed buffer length = %u", + rte_pktmbuf_pkt_len(uncomp_bufs[i]), + rte_pktmbuf_pkt_len(comp_bufs[i])); + if (operation_type == RTE_COMP_OP_STATELESS) { ops[i]->flush_flag = RTE_COMP_FLUSH_FINAL; } else { @@ -1313,6 +1387,7 @@ test_deflate_comp_run(const struct interim_data_params *int_data, if (ret_status < 0) for (i = 0; i < num_bufs; i++) { rte_comp_op_free(ops[i]); + ops[i] = NULL; ops_processed[i] = NULL; } @@ -1431,7 +1506,7 @@ test_deflate_comp_finalize(const struct interim_data_params *int_data, } RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Comp: Some operations were not successful\n"); return -1; } priv_data = (struct priv_op_data *)(ops_processed[i] + 1); @@ -1490,6 +1565,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data, /* from test_priv_data: */ struct rte_mbuf **uncomp_bufs = test_priv_data->uncomp_bufs; + struct rte_mbuf **comp_bufs = test_priv_data->comp_bufs; struct rte_comp_op **ops = test_priv_data->ops; struct rte_comp_op **ops_processed = test_priv_data->ops_processed; void **priv_xforms = test_priv_data->priv_xforms; @@ -1510,7 +1586,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data, /* Source buffer is the compressed data from the previous operations */ for (i = 0; i < num_bufs; i++) { - ops[i]->m_src = ops_processed[i]->m_dst; + ops[i]->m_src = comp_bufs[i]; ops[i]->m_dst = uncomp_bufs[i]; ops[i]->src.offset = 0; /* @@ -1740,6 +1816,10 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE || ops_processed[i]->status == RTE_COMP_OP_STATUS_SUCCESS)) { + + RTE_LOG(DEBUG, USER1, + ".............RECOVERABLE\n"); + /* collect the output into all_decomp_data */ const void *ptr = rte_pktmbuf_read( ops_processed[i]->m_dst, @@ -1777,7 +1857,6 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, ops[i]->src.length -= ops_processed[i]->consumed; /* repeat the operation */ - //goto next_step; return 2; } else { /* Compare the original stream with the */ @@ -1808,7 +1887,8 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, } else if (ops_processed[i]->status != RTE_COMP_OP_STATUS_SUCCESS) { RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Decomp: Some operations were not successful, status = %u\n", + ops_processed[i]->status); return -1; } priv_data = (struct priv_op_data *)(ops_processed[i] + 1); @@ -1986,7 +2066,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, "Compress device does not support DEFLATE\n"); return -1; } - //test_objects_init(&test_priv_data, num_bufs); /* Prepare the source mbufs with the data */ ret = test_setup_com_bufs(int_data, test_data, &test_priv_data); @@ -1995,6 +2074,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, goto exit; } + RTE_LOG(DEBUG, USER1, "<<< COMPRESSION >>>\n"); + /* COMPRESSION */ /* Prepare output (destination) mbufs for compressed data */ @@ -2031,6 +2112,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, /* DECOMPRESSION */ + RTE_LOG(DEBUG, USER1, "<<< DECOMPRESSION >>>\n"); + /* Prepare output (destination) mbufs for decompressed data */ ret = test_setup_output_bufs( OPERATION_DECOMPRESSION, @@ -2096,7 +2179,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, priv_xforms[i] = NULL; } } - for (i = 0; i < num_bufs; i++) { rte_pktmbuf_free(uncomp_bufs[i]); rte_pktmbuf_free(comp_bufs[i]); @@ -2152,7 +2234,8 @@ test_compressdev_deflate_stateless_fixed(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2223,7 +2306,8 @@ test_compressdev_deflate_stateless_dynamic(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2278,7 +2362,8 @@ test_compressdev_deflate_stateless_multi_op(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with compressdev, decompress with Zlib */ @@ -2332,7 +2417,8 @@ test_compressdev_deflate_stateless_multi_level(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2422,7 +2508,8 @@ test_compressdev_deflate_stateless_multi_xform(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with compressdev, decompress with Zlib */ @@ -2471,7 +2558,8 @@ test_compressdev_deflate_stateless_sgl(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2582,7 +2670,8 @@ test_compressdev_deflate_stateless_checksum(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Check if driver supports crc32 checksum and test */ @@ -2700,7 +2789,8 @@ test_compressdev_out_of_space_buffer(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 1, /* run out-of-space test */ .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir = ZLIB_DECOMPRESS; @@ -2742,7 +2832,7 @@ test_compressdev_deflate_stateless_dynamic_big(void) struct comp_testsuite_params *ts_params = &testsuite_params; uint16_t i = 0; int ret; - int j; + unsigned int j; const struct rte_compressdev_capabilities *capab; char *test_buffer = NULL; @@ -2755,7 +2845,7 @@ test_compressdev_deflate_stateless_dynamic_big(void) if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) return -ENOTSUP; - test_buffer = rte_malloc(NULL, BIG_DATA_TEST_SIZE, 0); + test_buffer = rte_malloc(NULL, get_big_data_test_size(), 0); if (test_buffer == NULL) { RTE_LOG(ERR, USER1, "Can't allocate buffer for big-data\n"); @@ -2778,7 +2868,8 @@ test_compressdev_deflate_stateless_dynamic_big(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 1, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED }; ts_params->def_comp_xform->compress.deflate.huffman = @@ -2786,9 +2877,9 @@ test_compressdev_deflate_stateless_dynamic_big(void) /* fill the buffer with data based on rand. data */ srand(BIG_DATA_TEST_SIZE); - for (j = 0; j < BIG_DATA_TEST_SIZE - 1; ++j) + for (j = 0; j < get_big_data_test_size() - 1; ++j) test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; - test_buffer[BIG_DATA_TEST_SIZE-1] = 0; + test_buffer[get_big_data_test_size() - 1] = 0; /* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir = ZLIB_DECOMPRESS; @@ -2843,7 +2934,8 @@ test_compressdev_deflate_stateful_decomp(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with Zlib, decompress with compressdev */ @@ -2926,7 +3018,8 @@ test_compressdev_deflate_stateful_decomp_checksum(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Check if driver supports crc32 checksum and test */ @@ -3139,7 +3232,8 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_ENABLED + .overflow = OVERFLOW_ENABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -3176,47 +3270,1056 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void) return ret; } -static struct unit_test_suite compressdev_testsuite = { - .suite_name = "compressdev unit test suite", - .setup = testsuite_setup, - .teardown = testsuite_teardown, - .unit_test_cases = { - TEST_CASE_ST(NULL, NULL, - test_compressdev_invalid_configuration), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic_big), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_op), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_level), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_xform), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_sgl), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_checksum), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_out_of_space_buffer), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp_checksum), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_external_mbufs), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed_oos_recoverable), - TEST_CASES_END() /**< NULL terminate unit test array */ +static int +test_compressdev_deflate_im_buffers_LB_1op(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; } -}; + + struct interim_data_params int_data = { + (const char * const *)&test_buffer, + 1, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + /* must be LB to SGL, + * input LB buffer reaches its maximum, + * if ratio 1.3 than another mbuf must be + * created and attached + */ + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} static int -test_compressdev(void) +test_compressdev_deflate_im_buffers_LB_2ops_first(void) { - return unit_test_suite_runner(&compressdev_testsuite); + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = test_buffer; + test_buffers[1] = compress_test_bufs[0]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_2ops_second(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_3ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[3]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 3, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_4ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[4]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + test_buffers[3] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 4, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_1op(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + struct interim_data_params int_data = { + (const char * const *)&test_buffer, + 1, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_first(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = test_buffer; + test_buffers[1] = compress_test_bufs[0]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_second(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_3ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[3]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 3, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_4ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[4]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + test_buffers[3] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 4, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_1op(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_OVER, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + struct interim_data_params int_data = { + (const char * const *)&test_buffer, + 1, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_OVER); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_OVER - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_SUCCESS; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + + return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_first(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_OVER, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = test_buffer; + test_buffers[1] = compress_test_bufs[0]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_OVER); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_OVER - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_SUCCESS; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_second(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_OVER, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_OVER); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_OVER - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_SUCCESS; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static struct unit_test_suite compressdev_testsuite = { + .suite_name = "compressdev unit test suite", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(NULL, NULL, + test_compressdev_invalid_configuration), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic_big), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_op), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_level), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_xform), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_sgl), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_checksum), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_out_of_space_buffer), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp_checksum), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_external_mbufs), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed_oos_recoverable), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +/**********************************/ + +static struct unit_test_suite compressdev_testsuite_qat = { + .suite_name = "compressdev unit test suite for QAT PMD", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { + /* Positive test cases for IM buffer handling verification */ + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_1op), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_first), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_second), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_3ops), + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_4ops), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_1op), + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_first), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_second), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_3ops), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_4ops), + + /* Negative test cases for IM buffer handling verification */ + + /* For this test huge mempool is necessary. + * It tests one case: + * only one op containing big amount of data, so that + * number of requested descriptors higher than number + * of available descriptors (128) + */ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_1op), + + /* For this test huge mempool is necessary. + * 2 ops. First op contains big amount of data: + * number of requested descriptors higher than number + * of available descriptors (128), the second op is + * relatively small. In this case both ops are rejected + */ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_first), + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_second), + + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static int +test_compressdev(void) +{ + testsuite_params.suite_name = compressdev_testsuite.suite_name; + return unit_test_suite_runner(&compressdev_testsuite); +} + +static int +test_compressdev_qat_specific(void) +{ + /* All tests are run on device 0. Check if it's QAT PMD. */ + if (rte_compressdev_count() == 0 || + !ENDS_WITH(rte_compressdev_name_get(0), "qat_comp")) { + RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check if " + "CONFIG_RTE_LIBRTE_PMD_QAT is enabled " + "in config file to run this testsuite.\n"); + return TEST_FAILED; + } + + testsuite_params.suite_name = compressdev_testsuite_qat.suite_name; + return unit_test_suite_runner(&compressdev_testsuite_qat); } REGISTER_TEST_COMMAND(compressdev_autotest, test_compressdev); +REGISTER_TEST_COMMAND(compressdev_qat_specific, test_compressdev_qat_specific); -- 2.17.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski @ 2020-04-08 15:44 ` Trahe, Fiona 2020-04-15 18:35 ` Akhil Goyal 1 sibling, 0 replies; 20+ messages in thread From: Trahe, Fiona @ 2020-04-08 15:44 UTC (permalink / raw) To: Dybkowski, AdamX, dev, akhil.goyal; +Cc: Trahe, Fiona > -----Original Message----- > From: Dybkowski, AdamX <adamx.dybkowski@intel.com> > Sent: Wednesday, April 8, 2020 1:51 PM > To: dev@dpdk.org; Trahe, Fiona <fiona.trahe@intel.com>; akhil.goyal@nxp.com > Cc: Dybkowski, AdamX <adamx.dybkowski@intel.com> > Subject: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests > > This patch adds a new test suite for verification of the "internal > QAT IM buffer too small" case handling. These unit tests are > specific to the QAT PMD only - that's why they are contained in > a separate test suite. > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> Acked-by: Fiona Trahe <fiona.trahe@intel.com> ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski 2020-04-08 15:44 ` Trahe, Fiona @ 2020-04-15 18:35 ` Akhil Goyal 2020-04-16 10:02 ` Trahe, Fiona 1 sibling, 1 reply; 20+ messages in thread From: Akhil Goyal @ 2020-04-15 18:35 UTC (permalink / raw) To: Adam Dybkowski, dev, fiona.trahe; +Cc: Shally Verma Hi Fiona/Adam, > This patch adds a new test suite for verification of the "internal > QAT IM buffer too small" case handling. These unit tests are > specific to the QAT PMD only - that's why they are contained in > a separate test suite. > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > --- Why do we need to have separate testsuite for QAT? Can't we have a single one and based on capability of the driver, Determine which tests need to be skipped in case they are not supported. This would create a mess in the longer run just like cryptodev. Please fix this, we cannot take this patch as is. > + > +static struct unit_test_suite compressdev_testsuite = { > + .suite_name = "compressdev unit test suite", > + .setup = testsuite_setup, > + .teardown = testsuite_teardown, > + .unit_test_cases = { > + TEST_CASE_ST(NULL, NULL, > + test_compressdev_invalid_configuration), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_fixed), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_dynamic), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_dynamic_big), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_multi_op), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_multi_level), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_multi_xform), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_sgl), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_checksum), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_out_of_space_buffer), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateful_decomp), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + > test_compressdev_deflate_stateful_decomp_checksum), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_external_mbufs), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_stateless_fixed_oos_recoverable), > + TEST_CASES_END() /**< NULL terminate unit test array */ > + } > +}; > + > +/**********************************/ > + > +static struct unit_test_suite compressdev_testsuite_qat = { > + .suite_name = "compressdev unit test suite for QAT PMD", > + .setup = testsuite_setup, > + .teardown = testsuite_teardown, > + .unit_test_cases = { > + /* Positive test cases for IM buffer handling verification */ > + > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_LB_1op), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_LB_2ops_first), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + > test_compressdev_deflate_im_buffers_LB_2ops_second), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_LB_3ops), > + > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_LB_4ops), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_SGL_1op), > + > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_SGL_2ops_first), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + > test_compressdev_deflate_im_buffers_SGL_2ops_second), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_SGL_3ops), > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_SGL_4ops), > + > + /* Negative test cases for IM buffer handling verification */ > + > + /* For this test huge mempool is necessary. > + * It tests one case: > + * only one op containing big amount of data, so that > + * number of requested descriptors higher than number > + * of available descriptors (128) > + */ > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + test_compressdev_deflate_im_buffers_SGL_over_1op), > + > + /* For this test huge mempool is necessary. > + * 2 ops. First op contains big amount of data: > + * number of requested descriptors higher than number > + * of available descriptors (128), the second op is > + * relatively small. In this case both ops are rejected > + */ > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + > test_compressdev_deflate_im_buffers_SGL_over_2ops_first), > + > + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, > + > test_compressdev_deflate_im_buffers_SGL_over_2ops_second), > + > + TEST_CASES_END() /**< NULL terminate unit test array */ > + } > +}; > + > +static int > +test_compressdev(void) > +{ > + testsuite_params.suite_name = compressdev_testsuite.suite_name; > + return unit_test_suite_runner(&compressdev_testsuite); > +} > + > +static int > +test_compressdev_qat_specific(void) > +{ > + /* All tests are run on device 0. Check if it's QAT PMD. */ > + if (rte_compressdev_count() == 0 || > + !ENDS_WITH(rte_compressdev_name_get(0), > "qat_comp")) { > + RTE_LOG(ERR, USER1, "QAT PMD must be loaded. Check if " > + "CONFIG_RTE_LIBRTE_PMD_QAT is enabled " > + "in config file to run this testsuite.\n"); > + return TEST_FAILED; > + } > + > + testsuite_params.suite_name = compressdev_testsuite_qat.suite_name; > + return unit_test_suite_runner(&compressdev_testsuite_qat); > } > > REGISTER_TEST_COMMAND(compressdev_autotest, test_compressdev); > +REGISTER_TEST_COMMAND(compressdev_qat_specific, > test_compressdev_qat_specific); > -- > 2.17.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-15 18:35 ` Akhil Goyal @ 2020-04-16 10:02 ` Trahe, Fiona 2020-04-16 10:25 ` Akhil Goyal 0 siblings, 1 reply; 20+ messages in thread From: Trahe, Fiona @ 2020-04-16 10:02 UTC (permalink / raw) To: Akhil Goyal, Dybkowski, AdamX, dev; +Cc: Shally Verma, Trahe, Fiona Hi Akhil, > -----Original Message----- > From: Akhil Goyal <akhil.goyal@nxp.com> > Sent: Wednesday, April 15, 2020 7:36 PM > To: Dybkowski, AdamX <adamx.dybkowski@intel.com>; dev@dpdk.org; Trahe, Fiona > <fiona.trahe@intel.com> > Cc: Shally Verma <shallyv@marvell.com> > Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests > > Hi Fiona/Adam, > > > This patch adds a new test suite for verification of the "internal > > QAT IM buffer too small" case handling. These unit tests are > > specific to the QAT PMD only - that's why they are contained in > > a separate test suite. > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > --- > > Why do we need to have separate testsuite for QAT? > Can't we have a single one and based on capability of the driver, > Determine which tests need to be skipped in case they are not supported. > This would create a mess in the longer run just like cryptodev. > > Please fix this, we cannot take this patch as is. [Fiona] Yes, I understand your concern and we considered including in the main suite. However these tests are not based on something that can be checked in capabilities. They are tests to hone in on a specific corner case based on a QAT limitation in its intermediate buffer size. So some of the tests are to validate that the recent changes we made in the PMD correctly work around that limitation, but other tests are negative and expected to fail as provoking a corner-case that still exists. Other devices would probably not fail the same tests. This was why we felt it better to have a qat-specific suite. We would expect any future tests to go into the main suite, so don't think this should cause a mess in the future. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 10:02 ` Trahe, Fiona @ 2020-04-16 10:25 ` Akhil Goyal 2020-04-16 11:26 ` Trahe, Fiona 0 siblings, 1 reply; 20+ messages in thread From: Akhil Goyal @ 2020-04-16 10:25 UTC (permalink / raw) To: Trahe, Fiona, Dybkowski, AdamX, dev; +Cc: Shally Verma Hi Fiona, > > Hi Akhil, > > > > > Hi Fiona/Adam, > > > > > This patch adds a new test suite for verification of the "internal > > > QAT IM buffer too small" case handling. These unit tests are > > > specific to the QAT PMD only - that's why they are contained in > > > a separate test suite. > > > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > > --- > > > > Why do we need to have separate testsuite for QAT? > > Can't we have a single one and based on capability of the driver, > > Determine which tests need to be skipped in case they are not supported. > > This would create a mess in the longer run just like cryptodev. > > > > Please fix this, we cannot take this patch as is. > > [Fiona] Yes, I understand your concern and we considered including in the main > suite. > However these tests are not based on something that can be > checked in capabilities. They are tests to hone in on a specific corner case > based on a QAT limitation in its intermediate buffer size. So some of the > tests are to validate that the recent changes we made in the PMD correctly > work around that limitation, but other tests are negative and expected to fail > as provoking a corner-case that still exists. Other devices would probably not fail > the same tests. Does that mean that all PMDs will pass with the newly added testcase which is for A corner case in QAT. If that is the case what is the issue in adding that in the main Test suite. It will get passed in all PMDs, isn't it? Am I missing something? I believe we should not have PMD specific test suites, rather it should be based on Capabilities to identify the cases which should be run for that particular PMD. > This was why we felt it better to have a qat-specific suite. > We would expect any future tests to go into the main suite, so don't think this > should cause a > mess in the future. > > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 10:25 ` Akhil Goyal @ 2020-04-16 11:26 ` Trahe, Fiona 2020-04-16 14:31 ` Bruce Richardson 2020-04-16 14:37 ` Akhil Goyal 0 siblings, 2 replies; 20+ messages in thread From: Trahe, Fiona @ 2020-04-16 11:26 UTC (permalink / raw) To: Akhil Goyal, Dybkowski, AdamX, dev; +Cc: Shally Verma, Trahe, Fiona > -----Original Message----- > From: Akhil Goyal <akhil.goyal@nxp.com> > Sent: Thursday, April 16, 2020 11:25 AM > To: Trahe, Fiona <fiona.trahe@intel.com>; Dybkowski, AdamX <adamx.dybkowski@intel.com>; > dev@dpdk.org > Cc: Shally Verma <shallyv@marvell.com> > Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests > > Hi Fiona, > > > > Hi Akhil, > > > > > > > > Hi Fiona/Adam, > > > > > > > This patch adds a new test suite for verification of the "internal > > > > QAT IM buffer too small" case handling. These unit tests are > > > > specific to the QAT PMD only - that's why they are contained in > > > > a separate test suite. > > > > > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > > > --- > > > > > > Why do we need to have separate testsuite for QAT? > > > Can't we have a single one and based on capability of the driver, > > > Determine which tests need to be skipped in case they are not supported. > > > This would create a mess in the longer run just like cryptodev. > > > > > > Please fix this, we cannot take this patch as is. > > > > [Fiona] Yes, I understand your concern and we considered including in the main > > suite. > > However these tests are not based on something that can be > > checked in capabilities. They are tests to hone in on a specific corner case > > based on a QAT limitation in its intermediate buffer size. So some of the > > tests are to validate that the recent changes we made in the PMD correctly > > work around that limitation, but other tests are negative and expected to fail > > as provoking a corner-case that still exists. Other devices would probably not fail > > the same tests. > > Does that mean that all PMDs will pass with the newly added testcase which is for > A corner case in QAT. If that is the case what is the issue in adding that in the main > Test suite. It will get passed in all PMDs, isn't it? Am I missing something? > > I believe we should not have PMD specific test suites, rather it should be based on > Capabilities to identify the cases which should be run for that particular PMD. [Fiona] yes, several of the cases should pass on all PMDs. So we could move those into the main suite. But what to do about the negative tests? Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get split in the PMD into many smaller requests to the hardware. However if the number of requests is bigger than can fit on the qp then this will never succeed. The test validates that the PMD behaves appropriately in this expected error case. That same case would probably not have an error on another device. Maybe we should just leave out such negative tests, but I find them useful as they validate the known behaviour. The buffer size used in the test is based on the known size QAT can handle and the corner case in which QAT will return an error. I see 4 options to handle this: 1. Leave out those tests 2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data. 3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type) 4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices. My preference is for 2. But up to you. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 11:26 ` Trahe, Fiona @ 2020-04-16 14:31 ` Bruce Richardson 2020-04-16 14:55 ` Trahe, Fiona 2020-04-16 14:37 ` Akhil Goyal 1 sibling, 1 reply; 20+ messages in thread From: Bruce Richardson @ 2020-04-16 14:31 UTC (permalink / raw) To: Trahe, Fiona; +Cc: Akhil Goyal, Dybkowski, AdamX, dev, Shally Verma On Thu, Apr 16, 2020 at 11:26:46AM +0000, Trahe, Fiona wrote: > > > > -----Original Message----- > > From: Akhil Goyal <akhil.goyal@nxp.com> > > Sent: Thursday, April 16, 2020 11:25 AM > > To: Trahe, Fiona <fiona.trahe@intel.com>; Dybkowski, AdamX <adamx.dybkowski@intel.com>; > > dev@dpdk.org > > Cc: Shally Verma <shallyv@marvell.com> > > Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests > > > > Hi Fiona, > > > > > > Hi Akhil, > > > > > > > > > > > Hi Fiona/Adam, > > > > > > > > > This patch adds a new test suite for verification of the "internal > > > > > QAT IM buffer too small" case handling. These unit tests are > > > > > specific to the QAT PMD only - that's why they are contained in > > > > > a separate test suite. > > > > > > > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > > > > --- > > > > > > > > Why do we need to have separate testsuite for QAT? > > > > Can't we have a single one and based on capability of the driver, > > > > Determine which tests need to be skipped in case they are not supported. > > > > This would create a mess in the longer run just like cryptodev. > > > > > > > > Please fix this, we cannot take this patch as is. > > > > > > [Fiona] Yes, I understand your concern and we considered including in the main > > > suite. > > > However these tests are not based on something that can be > > > checked in capabilities. They are tests to hone in on a specific corner case > > > based on a QAT limitation in its intermediate buffer size. So some of the > > > tests are to validate that the recent changes we made in the PMD correctly > > > work around that limitation, but other tests are negative and expected to fail > > > as provoking a corner-case that still exists. Other devices would probably not fail > > > the same tests. > > > > Does that mean that all PMDs will pass with the newly added testcase which is for > > A corner case in QAT. If that is the case what is the issue in adding that in the main > > Test suite. It will get passed in all PMDs, isn't it? Am I missing something? > > > > I believe we should not have PMD specific test suites, rather it should be based on > > Capabilities to identify the cases which should be run for that particular PMD. > [Fiona] yes, several of the cases should pass on all PMDs. > So we could move those into the main suite. > But what to do about the negative tests? > Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get > split in the PMD into many smaller requests to the hardware. However if the number > of requests is bigger than can fit on the qp then this will never succeed. The test > validates that the PMD behaves appropriately in this expected error case. That same > case would probably not have an error on another device. Maybe we should just leave out > such negative tests, but I find them useful as they validate the known behaviour. > The buffer size used in the test is based on the known size QAT can handle and the > corner case in which QAT will return an error. > > I see 4 options to handle this: > 1. Leave out those tests > 2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data. > 3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type) > 4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices. > > My preference is for 2. > But up to you. > While not something for this release, perhaps in future cryptodev could implement a "selftest()" callback API like rawdev does [1], which allows drivers to implement their own specific test cases too. [1] http://doc.dpdk.org/api-20.02/rte__rawdev_8h.html#a776edaa7060fc6a9d66e00f84132e140 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 14:31 ` Bruce Richardson @ 2020-04-16 14:55 ` Trahe, Fiona 0 siblings, 0 replies; 20+ messages in thread From: Trahe, Fiona @ 2020-04-16 14:55 UTC (permalink / raw) To: Richardson, Bruce Cc: Akhil Goyal, Dybkowski, AdamX, dev, Shally Verma, Trahe, Fiona Hi Bruce, > While not something for this release, perhaps in future cryptodev could > implement a "selftest()" callback API like rawdev does [1], which allows > drivers to implement their own specific test cases too. > > [1] http://doc.dpdk.org/api-20.02/rte__rawdev_8h.html#a776edaa7060fc6a9d66e00f84132e140 [Fiona] Thanks for the suggestion. Yes, that's probably a good way to handle useful, but device-specific test cases, and would make sense for this particular case. We'll consider it the next time such a case arises. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 11:26 ` Trahe, Fiona 2020-04-16 14:31 ` Bruce Richardson @ 2020-04-16 14:37 ` Akhil Goyal 2020-04-16 14:52 ` Trahe, Fiona 1 sibling, 1 reply; 20+ messages in thread From: Akhil Goyal @ 2020-04-16 14:37 UTC (permalink / raw) To: Trahe, Fiona, Dybkowski, AdamX, dev; +Cc: Shally Verma > > > > > > Hi Akhil, > > > > > > > > > > > Hi Fiona/Adam, > > > > > > > > > This patch adds a new test suite for verification of the "internal > > > > > QAT IM buffer too small" case handling. These unit tests are > > > > > specific to the QAT PMD only - that's why they are contained in > > > > > a separate test suite. > > > > > > > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > > > > --- > > > > > > > > Why do we need to have separate testsuite for QAT? > > > > Can't we have a single one and based on capability of the driver, > > > > Determine which tests need to be skipped in case they are not supported. > > > > This would create a mess in the longer run just like cryptodev. > > > > > > > > Please fix this, we cannot take this patch as is. > > > > > > [Fiona] Yes, I understand your concern and we considered including in the > main > > > suite. > > > However these tests are not based on something that can be > > > checked in capabilities. They are tests to hone in on a specific corner case > > > based on a QAT limitation in its intermediate buffer size. So some of the > > > tests are to validate that the recent changes we made in the PMD correctly > > > work around that limitation, but other tests are negative and expected to fail > > > as provoking a corner-case that still exists. Other devices would probably not > fail > > > the same tests. > > > > Does that mean that all PMDs will pass with the newly added testcase which is > for > > A corner case in QAT. If that is the case what is the issue in adding that in the > main > > Test suite. It will get passed in all PMDs, isn't it? Am I missing something? > > > > I believe we should not have PMD specific test suites, rather it should be based > on > > Capabilities to identify the cases which should be run for that particular PMD. > [Fiona] yes, several of the cases should pass on all PMDs. > So we could move those into the main suite. > But what to do about the negative tests? > Example: If a very large data buffer is passed to QAT to compress with dyn > compression, it will get > split in the PMD into many smaller requests to the hardware. However if the > number > of requests is bigger than can fit on the qp then this will never succeed. The test > validates that the PMD behaves appropriately in this expected error case. That > same > case would probably not have an error on another device. Maybe we should just > leave out > such negative tests, but I find them useful as they validate the known behaviour. > The buffer size used in the test is based on the known size QAT can handle and > the > corner case in which QAT will return an error. > > I see 4 options to handle this: > 1. Leave out those tests > 2. Use a qat-specific test suite only for negative cases which are constructed > based on specific qat internal meta-data. > 3. Include the negative tests in the main suite, but only run them on QAT (by > checking driver type) > 4. include the negative tests in the main suite, run them on all, expecting a FAIL > from QAT and a PASS from other devices. > > My preference is for 2. > But up to you. > I would say 4 is better. And why do you say negative cases will fail on QAT and pass on all other. The test cases are to test the library APIs which are same for all the PMDs and the behavior should Be same. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 14:37 ` Akhil Goyal @ 2020-04-16 14:52 ` Trahe, Fiona 2020-04-17 15:39 ` Akhil Goyal 0 siblings, 1 reply; 20+ messages in thread From: Trahe, Fiona @ 2020-04-16 14:52 UTC (permalink / raw) To: Akhil Goyal, Dybkowski, AdamX, dev; +Cc: Shally Verma, Trahe, Fiona Hi Akhil, > -----Original Message----- > From: Akhil Goyal <akhil.goyal@nxp.com> > Sent: Thursday, April 16, 2020 3:38 PM > To: Trahe, Fiona <fiona.trahe@intel.com>; Dybkowski, AdamX <adamx.dybkowski@intel.com>; > dev@dpdk.org > Cc: Shally Verma <shallyv@marvell.com> > Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests > > > > > > > > > Hi Akhil, > > > > > > > > > > > > > > Hi Fiona/Adam, > > > > > > > > > > > This patch adds a new test suite for verification of the "internal > > > > > > QAT IM buffer too small" case handling. These unit tests are > > > > > > specific to the QAT PMD only - that's why they are contained in > > > > > > a separate test suite. > > > > > > > > > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > > > > > --- > > > > > > > > > > Why do we need to have separate testsuite for QAT? > > > > > Can't we have a single one and based on capability of the driver, > > > > > Determine which tests need to be skipped in case they are not supported. > > > > > This would create a mess in the longer run just like cryptodev. > > > > > > > > > > Please fix this, we cannot take this patch as is. > > > > > > > > [Fiona] Yes, I understand your concern and we considered including in the > > main > > > > suite. > > > > However these tests are not based on something that can be > > > > checked in capabilities. They are tests to hone in on a specific corner case > > > > based on a QAT limitation in its intermediate buffer size. So some of the > > > > tests are to validate that the recent changes we made in the PMD correctly > > > > work around that limitation, but other tests are negative and expected to fail > > > > as provoking a corner-case that still exists. Other devices would probably not > > fail > > > > the same tests. > > > > > > Does that mean that all PMDs will pass with the newly added testcase which is > > for > > > A corner case in QAT. If that is the case what is the issue in adding that in the > > main > > > Test suite. It will get passed in all PMDs, isn't it? Am I missing something? > > > > > > I believe we should not have PMD specific test suites, rather it should be based > > on > > > Capabilities to identify the cases which should be run for that particular PMD. > > [Fiona] yes, several of the cases should pass on all PMDs. > > So we could move those into the main suite. > > But what to do about the negative tests? > > Example: If a very large data buffer is passed to QAT to compress with dyn > > compression, it will get > > split in the PMD into many smaller requests to the hardware. However if the > > number > > of requests is bigger than can fit on the qp then this will never succeed. The test > > validates that the PMD behaves appropriately in this expected error case. That > > same > > case would probably not have an error on another device. Maybe we should just > > leave out > > such negative tests, but I find them useful as they validate the known behaviour. > > The buffer size used in the test is based on the known size QAT can handle and > > the > > corner case in which QAT will return an error. > > > > I see 4 options to handle this: > > 1. Leave out those tests > > 2. Use a qat-specific test suite only for negative cases which are constructed > > based on specific qat internal meta-data. > > 3. Include the negative tests in the main suite, but only run them on QAT (by > > checking driver type) > > 4. include the negative tests in the main suite, run them on all, expecting a FAIL > > from QAT and a PASS from other devices. > > > > My preference is for 2. > > But up to you. > > > I would say 4 is better. And why do you say negative cases will fail on QAT and pass on all other. > The test cases are to test the library APIs which are same for all the PMDs and the behavior should > Be same. [Fiona] I've explained above why QAT fails, sorry if it isn't clear. Any device can have errors - it's not an API or capability issue, it's a device limitation in a very unlikely corner case. So 4 is ok? i.e. if there is conditional code in the UT expecting different result depending on PMD type? If not, we'll revert to 1 and leave out those tests. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-16 14:52 ` Trahe, Fiona @ 2020-04-17 15:39 ` Akhil Goyal 2020-04-17 15:56 ` Trahe, Fiona 0 siblings, 1 reply; 20+ messages in thread From: Akhil Goyal @ 2020-04-17 15:39 UTC (permalink / raw) To: Trahe, Fiona, Dybkowski, AdamX, dev; +Cc: Shally Verma > > > > > > Hi Fiona/Adam, > > > > > > > > > > > > > This patch adds a new test suite for verification of the "internal > > > > > > > QAT IM buffer too small" case handling. These unit tests are > > > > > > > specific to the QAT PMD only - that's why they are contained in > > > > > > > a separate test suite. > > > > > > > > > > > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > > > > > > > --- > > > > > > > > > > > > Why do we need to have separate testsuite for QAT? > > > > > > Can't we have a single one and based on capability of the driver, > > > > > > Determine which tests need to be skipped in case they are not > supported. > > > > > > This would create a mess in the longer run just like cryptodev. > > > > > > > > > > > > Please fix this, we cannot take this patch as is. > > > > > > > > > > [Fiona] Yes, I understand your concern and we considered including in the > > > main > > > > > suite. > > > > > However these tests are not based on something that can be > > > > > checked in capabilities. They are tests to hone in on a specific corner case > > > > > based on a QAT limitation in its intermediate buffer size. So some of the > > > > > tests are to validate that the recent changes we made in the PMD > correctly > > > > > work around that limitation, but other tests are negative and expected to > fail > > > > > as provoking a corner-case that still exists. Other devices would probably > not > > > fail > > > > > the same tests. > > > > > > > > Does that mean that all PMDs will pass with the newly added testcase > which is > > > for > > > > A corner case in QAT. If that is the case what is the issue in adding that in > the > > > main > > > > Test suite. It will get passed in all PMDs, isn't it? Am I missing something? > > > > > > > > I believe we should not have PMD specific test suites, rather it should be > based > > > on > > > > Capabilities to identify the cases which should be run for that particular > PMD. > > > [Fiona] yes, several of the cases should pass on all PMDs. > > > So we could move those into the main suite. > > > But what to do about the negative tests? > > > Example: If a very large data buffer is passed to QAT to compress with dyn > > > compression, it will get > > > split in the PMD into many smaller requests to the hardware. However if the > > > number > > > of requests is bigger than can fit on the qp then this will never succeed. The > test > > > validates that the PMD behaves appropriately in this expected error case. > That > > > same > > > case would probably not have an error on another device. Maybe we should > just > > > leave out > > > such negative tests, but I find them useful as they validate the known > behaviour. > > > The buffer size used in the test is based on the known size QAT can handle > and > > > the > > > corner case in which QAT will return an error. > > > > > > I see 4 options to handle this: > > > 1. Leave out those tests > > > 2. Use a qat-specific test suite only for negative cases which are constructed > > > based on specific qat internal meta-data. > > > 3. Include the negative tests in the main suite, but only run them on QAT (by > > > checking driver type) > > > 4. include the negative tests in the main suite, run them on all, expecting a > FAIL > > > from QAT and a PASS from other devices. > > > > > > My preference is for 2. > > > But up to you. > > > > > I would say 4 is better. And why do you say negative cases will fail on QAT and > pass on all other. > > The test cases are to test the library APIs which are same for all the PMDs and > the behavior should > > Be same. > [Fiona] I've explained above why QAT fails, sorry if it isn't clear. > Any device can have errors - it's not an API or capability issue, it's a device > limitation in a very unlikely corner case. > So 4 is ok? i.e. if there is conditional code in the UT expecting different result > depending on PMD type? > If not, we'll revert to 1 and leave out those tests. I am still not convinced how different PMDs will behave differently for a particular case. Even if QAT/any PMD has a corner case, the test case will fail in that case. You mean you want to make that case pass if the corner case has hit because you have A known issue reported for that case and you don't want to highlight that in the test summary? I am not sure if that is a good thing to do. If the case is failing, then it should report as failed even if you have a defined known issue for that. We don't need to add any checks for PMD types. Regards, Akhil ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests 2020-04-17 15:39 ` Akhil Goyal @ 2020-04-17 15:56 ` Trahe, Fiona 0 siblings, 0 replies; 20+ messages in thread From: Trahe, Fiona @ 2020-04-17 15:56 UTC (permalink / raw) To: Akhil Goyal, Dybkowski, AdamX, dev; +Cc: Shally Verma Hi Akhil, > -----Original Message----- > From: Akhil Goyal <akhil.goyal@nxp.com> > Sent: Friday, April 17, 2020 4:39 PM > To: Trahe, Fiona <fiona.trahe@intel.com>; Dybkowski, AdamX <adamx.dybkowski@intel.com>; > dev@dpdk.org > Cc: Shally Verma <shallyv@marvell.com> > Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests > > > > > > > > Hi Fiona/Adam, > > > > > > > > > I am still not convinced how different PMDs will behave differently for a particular case. > Even if QAT/any PMD has a corner case, the test case will fail in that case. > You mean you want to make that case pass if the corner case has hit because you have > A known issue reported for that case and you don't want to highlight that in the test summary? > I am not sure if that is a good thing to do. > If the case is failing, then it should report as failed even if you have a defined known issue for that. > > We don't need to add any checks for PMD types. [Fiona] ok, we've reworked the tests so all are common. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [dpdk-dev] [PATCH v3 0/2] compress/qat: im buffer too small - split op 2020-04-08 12:50 [dpdk-dev] [PATCH v2 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 1/2] " Adam Dybkowski 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski @ 2020-04-17 15:44 ` Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 1/2] " Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski 2 siblings, 2 replies; 20+ messages in thread From: Adam Dybkowski @ 2020-04-17 15:44 UTC (permalink / raw) To: dev, fiona.trahe, akhil.goyal; +Cc: Adam Dybkowski This patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before. v2: * various post-review small fixes v3: * refactor and simplify unit tests, removing the need of adding a new test suite separate for QAT-specific tests Adam Dybkowski (2): compress/qat: im buffer too small - split op test/compress: im buffer too small - add unit tests app/test/test_compressdev.c | 1185 ++++++++++++++++++++++-- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 ++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 +++++++++- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 9 files changed, 1820 insertions(+), 141 deletions(-) -- 2.17.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [dpdk-dev] [PATCH v3 1/2] compress/qat: im buffer too small - split op 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 0/2] compress/qat: im buffer too small - split op Adam Dybkowski @ 2020-04-17 15:44 ` Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski 1 sibling, 0 replies; 20+ messages in thread From: Adam Dybkowski @ 2020-04-17 15:44 UTC (permalink / raw) To: dev, fiona.trahe, akhil.goyal; +Cc: Adam Dybkowski This patch implements a special way of buffer handling when internal QAT IM buffer is too small for Huffman dynamic compression operation. Instead of falling back to fixed compression, the operation is now split into multiple smaller dynamic compression requests (possible to execute on QAT) and their results are then combined and copied into the output buffer. This is not possible if any checksum calculation was requested - in such case the code falls back to fixed compression as before. Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> Acked-by: Fiona Trahe <fiona.trahe@intel.com> --- doc/guides/compressdevs/qat_comp.rst | 3 - doc/guides/cryptodevs/qat.rst | 7 +- doc/guides/rel_notes/release_20_05.rst | 10 + drivers/common/qat/qat_qp.c | 223 +++++++++++- drivers/common/qat/qat_qp.h | 3 + drivers/compress/qat/qat_comp.c | 474 +++++++++++++++++++++++-- drivers/compress/qat/qat_comp.h | 29 +- drivers/compress/qat/qat_comp_pmd.c | 27 +- 8 files changed, 702 insertions(+), 74 deletions(-) diff --git a/doc/guides/compressdevs/qat_comp.rst b/doc/guides/compressdevs/qat_comp.rst index 757611a30..475c4a9f9 100644 --- a/doc/guides/compressdevs/qat_comp.rst +++ b/doc/guides/compressdevs/qat_comp.rst @@ -42,9 +42,6 @@ Limitations from the RX queue must be done from one thread, but enqueues and dequeues may be done in different threads.) * No BSD support as BSD QAT kernel driver not available. -* When using Deflate dynamic huffman encoding for compression, the input size (op.src.length) - must be < CONFIG_RTE_PMD_QAT_COMP_IM_BUFFER_SIZE from the config file, - see :ref:`building_qat_config` for more details. * Stateful compression is not supported. diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst index c79e686de..4ea7985a7 100644 --- a/doc/guides/cryptodevs/qat.rst +++ b/doc/guides/cryptodevs/qat.rst @@ -260,8 +260,11 @@ allocated while for GEN1 devices, 12 buffers are allocated, plus 1472 bytes over .. Note:: If the compressed output of a Deflate operation using Dynamic Huffman - Encoding is too big to fit in an intermediate buffer, then the - operation will fall back to fixed compression rather than failing the operation. + Encoding is too big to fit in an intermediate buffer, then the + operation will be split into smaller operations and their results will + be merged afterwards. + This is not possible if any checksum calculation was requested - in such + case the code falls back to fixed compression. To avoid this less performant case, applications should configure the intermediate buffer size to be larger than the expected input data size (compressed output size is usually unknown, so the only option is to make diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 184967844..78ec5fec3 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -81,6 +81,16 @@ New Features by making use of the event device capabilities. The event mode currently supports only inline IPsec protocol offload. +* **Added QAT intermediate buffer too small handling in QAT compression PMD.** + + Added a special way of buffer handling when internal QAT intermediate buffer + is too small for Huffman dynamic compression operation. Instead of falling + back to fixed compression, the operation is now split into multiple smaller + dynamic compression requests (possible to execute on QAT) and their results + are then combined and copied into the output buffer. This is not possible if + any checksum calculation was requested - in such case the code falls back to + fixed compression as before. + Removed Items ------------- diff --git a/drivers/common/qat/qat_qp.c b/drivers/common/qat/qat_qp.c index eb1da7243..64dfd85c4 100644 --- a/drivers/common/qat/qat_qp.c +++ b/drivers/common/qat/qat_qp.c @@ -650,32 +650,212 @@ qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops) return nb_ops_sent; } +/* Use this for compression only - but keep consistent with above common + * function as much as possible. + */ +uint16_t +qat_enqueue_comp_op_burst(void *qp, void **ops, uint16_t nb_ops) +{ + register struct qat_queue *queue; + struct qat_qp *tmp_qp = (struct qat_qp *)qp; + register uint32_t nb_ops_sent = 0; + register int nb_desc_to_build; + uint16_t nb_ops_possible = nb_ops; + register uint8_t *base_addr; + register uint32_t tail; + + int descriptors_built, total_descriptors_built = 0; + int nb_remaining_descriptors; + int overflow = 0; + + if (unlikely(nb_ops == 0)) + return 0; + + /* read params used a lot in main loop into registers */ + queue = &(tmp_qp->tx_q); + base_addr = (uint8_t *)queue->base_addr; + tail = queue->tail; + + /* Find how many can actually fit on the ring */ + { + /* dequeued can only be written by one thread, but it may not + * be this thread. As it's 4-byte aligned it will be read + * atomically here by any Intel CPU. + * enqueued can wrap before dequeued, but cannot + * lap it as var size of enq/deq (uint32_t) > var size of + * max_inflights (uint16_t). In reality inflights is never + * even as big as max uint16_t, as it's <= ADF_MAX_DESC. + * On wrapping, the calculation still returns the correct + * positive value as all three vars are unsigned. + */ + uint32_t inflights = + tmp_qp->enqueued - tmp_qp->dequeued; + + /* Find how many can actually fit on the ring */ + overflow = (inflights + nb_ops) - tmp_qp->max_inflights; + if (overflow > 0) { + nb_ops_possible = nb_ops - overflow; + if (nb_ops_possible == 0) + return 0; + } + + /* QAT has plenty of work queued already, so don't waste cycles + * enqueueing, wait til the application has gathered a bigger + * burst or some completed ops have been dequeued + */ + if (tmp_qp->min_enq_burst_threshold && inflights > + QAT_QP_MIN_INFL_THRESHOLD && nb_ops_possible < + tmp_qp->min_enq_burst_threshold) { + tmp_qp->stats.threshold_hit_count++; + return 0; + } + } + + /* At this point nb_ops_possible is assuming a 1:1 mapping + * between ops and descriptors. + * Fewer may be sent if some ops have to be split. + * nb_ops_possible is <= burst size. + * Find out how many spaces are actually available on the qp in case + * more are needed. + */ + nb_remaining_descriptors = nb_ops_possible + + ((overflow >= 0) ? 0 : overflow * (-1)); + QAT_DP_LOG(DEBUG, "Nb ops requested %d, nb descriptors remaining %d", + nb_ops, nb_remaining_descriptors); + + while (nb_ops_sent != nb_ops_possible && + nb_remaining_descriptors > 0) { + struct qat_comp_op_cookie *cookie = + tmp_qp->op_cookies[tail >> queue->trailz]; + + descriptors_built = 0; + + QAT_DP_LOG(DEBUG, "--- data length: %u", + ((struct rte_comp_op *)*ops)->src.length); + + nb_desc_to_build = qat_comp_build_request(*ops, + base_addr + tail, cookie, tmp_qp->qat_dev_gen); + QAT_DP_LOG(DEBUG, "%d descriptors built, %d remaining, " + "%d ops sent, %d descriptors needed", + total_descriptors_built, nb_remaining_descriptors, + nb_ops_sent, nb_desc_to_build); + + if (unlikely(nb_desc_to_build < 0)) { + /* this message cannot be enqueued */ + tmp_qp->stats.enqueue_err_count++; + if (nb_ops_sent == 0) + return 0; + goto kick_tail; + } else if (unlikely(nb_desc_to_build > 1)) { + /* this op is too big and must be split - get more + * descriptors and retry + */ + + QAT_DP_LOG(DEBUG, "Build %d descriptors for this op", + nb_desc_to_build); + + nb_remaining_descriptors -= nb_desc_to_build; + if (nb_remaining_descriptors >= 0) { + /* There are enough remaining descriptors + * so retry + */ + int ret2 = qat_comp_build_multiple_requests( + *ops, tmp_qp, tail, + nb_desc_to_build); + + if (unlikely(ret2 < 1)) { + QAT_DP_LOG(DEBUG, + "Failed to build (%d) descriptors, status %d", + nb_desc_to_build, ret2); + + qat_comp_free_split_op_memzones(cookie, + nb_desc_to_build - 1); + + tmp_qp->stats.enqueue_err_count++; + + /* This message cannot be enqueued */ + if (nb_ops_sent == 0) + return 0; + goto kick_tail; + } else { + descriptors_built = ret2; + total_descriptors_built += + descriptors_built; + nb_remaining_descriptors -= + descriptors_built; + QAT_DP_LOG(DEBUG, + "Multiple descriptors (%d) built ok", + descriptors_built); + } + } else { + QAT_DP_LOG(ERR, "For the current op, number of requested descriptors (%d) " + "exceeds number of available descriptors (%d)", + nb_desc_to_build, + nb_remaining_descriptors + + nb_desc_to_build); + + qat_comp_free_split_op_memzones(cookie, + nb_desc_to_build - 1); + + /* Not enough extra descriptors */ + if (nb_ops_sent == 0) + return 0; + goto kick_tail; + } + } else { + descriptors_built = 1; + total_descriptors_built++; + nb_remaining_descriptors--; + QAT_DP_LOG(DEBUG, "Single descriptor built ok"); + } + + tail = adf_modulo(tail + (queue->msg_size * descriptors_built), + queue->modulo_mask); + ops++; + nb_ops_sent++; + } + +kick_tail: + queue->tail = tail; + tmp_qp->enqueued += total_descriptors_built; + tmp_qp->stats.enqueued_count += total_descriptors_built; + txq_write_tail(tmp_qp, queue); + return nb_ops_sent; +} + uint16_t qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops) { struct qat_queue *rx_queue; struct qat_qp *tmp_qp = (struct qat_qp *)qp; uint32_t head; - uint32_t resp_counter = 0; + uint32_t op_resp_counter = 0, fw_resp_counter = 0; uint8_t *resp_msg; + int nb_fw_responses = 0; rx_queue = &(tmp_qp->rx_q); head = rx_queue->head; resp_msg = (uint8_t *)rx_queue->base_addr + rx_queue->head; while (*(uint32_t *)resp_msg != ADF_RING_EMPTY_SIG && - resp_counter != nb_ops) { + op_resp_counter != nb_ops) { - if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC) + nb_fw_responses = 0; + if (tmp_qp->service_type == QAT_SERVICE_SYMMETRIC) { qat_sym_process_response(ops, resp_msg); - else if (tmp_qp->service_type == QAT_SERVICE_COMPRESSION) - qat_comp_process_response(ops, resp_msg, + nb_fw_responses = 1; + } else if (tmp_qp->service_type == QAT_SERVICE_COMPRESSION) + + nb_fw_responses = qat_comp_process_response( + ops, resp_msg, tmp_qp->op_cookies[head >> rx_queue->trailz], &tmp_qp->stats.dequeue_err_count); + else if (tmp_qp->service_type == QAT_SERVICE_ASYMMETRIC) { #ifdef BUILD_QAT_ASYM qat_asym_process_response(ops, resp_msg, tmp_qp->op_cookies[head >> rx_queue->trailz]); + nb_fw_responses = 1; #endif } @@ -683,21 +863,38 @@ qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops) rx_queue->modulo_mask); resp_msg = (uint8_t *)rx_queue->base_addr + head; - ops++; - resp_counter++; + + if (ops != NULL && nb_fw_responses) { + /* only move on to next op if one was ready to return + * to API + */ + ops++; + op_resp_counter++; + } + + /* A compression op may be broken up into multiple fw requests. + * Only count fw responses as complete once ALL the responses + * associated with an op have been processed, as the cookie + * data from the first response must be available until + * finished with all firmware responses. + */ + fw_resp_counter += nb_fw_responses; } - if (resp_counter > 0) { + + if (fw_resp_counter > 0) { rx_queue->head = head; - tmp_qp->dequeued += resp_counter; - tmp_qp->stats.dequeued_count += resp_counter; - rx_queue->nb_processed_responses += resp_counter; + tmp_qp->dequeued += fw_resp_counter; + tmp_qp->stats.dequeued_count += fw_resp_counter; + rx_queue->nb_processed_responses += fw_resp_counter; if (rx_queue->nb_processed_responses > - QAT_CSR_HEAD_WRITE_THRESH) + QAT_CSR_HEAD_WRITE_THRESH) rxq_free_desc(tmp_qp, rx_queue); } + QAT_DP_LOG(DEBUG, "Dequeue burst return: %u, QAT responses: %u", + op_resp_counter, fw_resp_counter); - return resp_counter; + return op_resp_counter; } /* This is almost same as dequeue_op_burst, without the atomic, without stats diff --git a/drivers/common/qat/qat_qp.h b/drivers/common/qat/qat_qp.h index 88d3c9942..575d69059 100644 --- a/drivers/common/qat/qat_qp.h +++ b/drivers/common/qat/qat_qp.h @@ -89,6 +89,9 @@ extern const struct qat_qp_hw_data qat_gen3_qps[][ADF_MAX_QPS_ON_ANY_SERVICE]; uint16_t qat_enqueue_op_burst(void *qp, void **ops, uint16_t nb_ops); +uint16_t +qat_enqueue_comp_op_burst(void *qp, void **ops, uint16_t nb_ops); + uint16_t qat_dequeue_op_burst(void *qp, void **ops, uint16_t nb_ops); diff --git a/drivers/compress/qat/qat_comp.c b/drivers/compress/qat/qat_comp.c index 533e34f6b..9e1fd2fe9 100644 --- a/drivers/compress/qat/qat_comp.c +++ b/drivers/compress/qat/qat_comp.c @@ -13,11 +13,93 @@ #include <rte_spinlock.h> #include <rte_log.h> #include <rte_malloc.h> +#include <rte_memzone.h> #include "qat_logs.h" #include "qat_comp.h" #include "qat_comp_pmd.h" +static void +qat_comp_fallback_to_fixed(struct icp_qat_fw_comp_req *comp_req) +{ + QAT_DP_LOG(DEBUG, "QAT PMD: fallback to fixed compression!"); + + comp_req->comn_hdr.service_cmd_id = + ICP_QAT_FW_COMP_CMD_STATIC; + + ICP_QAT_FW_COMN_NEXT_ID_SET( + &comp_req->comp_cd_ctrl, + ICP_QAT_FW_SLICE_DRAM_WR); + + ICP_QAT_FW_COMN_NEXT_ID_SET( + &comp_req->u2.xlt_cd_ctrl, + ICP_QAT_FW_SLICE_NULL); + ICP_QAT_FW_COMN_CURR_ID_SET( + &comp_req->u2.xlt_cd_ctrl, + ICP_QAT_FW_SLICE_NULL); +} + +void +qat_comp_free_split_op_memzones(struct qat_comp_op_cookie *cookie, + unsigned int nb_children) +{ + unsigned int i; + + /* free all memzones allocated for child descriptors */ + for (i = 0; i < nb_children; i++) + rte_memzone_free(cookie->dst_memzones[i]); + + /* and free the pointer table */ + rte_free(cookie->dst_memzones); + cookie->dst_memzones = NULL; +} + +static int +qat_comp_allocate_split_op_memzones(struct qat_comp_op_cookie *cookie, + unsigned int nb_descriptors_needed) +{ + struct qat_queue *txq = &(cookie->qp->tx_q); + char dst_memz_name[RTE_MEMZONE_NAMESIZE]; + unsigned int i; + + /* allocate the array of memzone pointers */ + cookie->dst_memzones = rte_zmalloc_socket("qat PMD im buf mz pointers", + (nb_descriptors_needed - 1) * + sizeof(const struct rte_memzone *), + RTE_CACHE_LINE_SIZE, cookie->socket_id); + + if (cookie->dst_memzones == NULL) { + QAT_DP_LOG(ERR, + "QAT PMD: failed to allocate im buf mz pointers"); + return -ENOMEM; + } + + for (i = 0; i < nb_descriptors_needed - 1; i++) { + snprintf(dst_memz_name, + sizeof(dst_memz_name), + "dst_%u_%u_%u_%u_%u", + cookie->qp->qat_dev->qat_dev_id, + txq->hw_bundle_number, txq->hw_queue_number, + cookie->cookie_index, i); + + cookie->dst_memzones[i] = rte_memzone_reserve_aligned( + dst_memz_name, RTE_PMD_QAT_COMP_IM_BUFFER_SIZE, + cookie->socket_id, RTE_MEMZONE_IOVA_CONTIG, + RTE_CACHE_LINE_SIZE); + + if (cookie->dst_memzones[i] == NULL) { + QAT_DP_LOG(ERR, + "QAT PMD: failed to allocate dst buffer memzone"); + + /* let's free all memzones allocated up to now */ + qat_comp_free_split_op_memzones(cookie, i); + + return -ENOMEM; + } + } + + return 0; +} int qat_comp_build_request(void *in_op, uint8_t *out_msg, @@ -57,7 +139,48 @@ qat_comp_build_request(void *in_op, uint8_t *out_msg, rte_mov128(out_msg, tmpl); comp_req->comn_mid.opaque_data = (uint64_t)(uintptr_t)op; - if (op->op_type == RTE_COMP_OP_STATEFUL) { + if (likely(qat_xform->qat_comp_request_type == + QAT_COMP_REQUEST_DYNAMIC_COMP_STATELESS)) { + + if (unlikely(op->src.length > QAT_FALLBACK_THLD)) { + /* the operation must be split into pieces */ + if (qat_xform->checksum_type != + RTE_COMP_CHECKSUM_NONE) { + /* fallback to fixed compression in case any + * checksum calculation was requested + */ + qat_comp_fallback_to_fixed(comp_req); + } else { + /* calculate num. of descriptors for split op */ + unsigned int nb_descriptors_needed = + op->src.length / QAT_FALLBACK_THLD + 1; + /* allocate memzone for output data */ + if (qat_comp_allocate_split_op_memzones( + cookie, nb_descriptors_needed)) { + /* out of memory, fallback to fixed */ + qat_comp_fallback_to_fixed(comp_req); + } else { + QAT_DP_LOG(DEBUG, + "Input data is too big, op must be split into %u descriptors", + nb_descriptors_needed); + return (int) nb_descriptors_needed; + } + } + } + + /* set BFINAL bit according to flush_flag */ + comp_req->comp_pars.req_par_flags = + ICP_QAT_FW_COMP_REQ_PARAM_FLAGS_BUILD( + ICP_QAT_FW_COMP_SOP, + ICP_QAT_FW_COMP_EOP, + op->flush_flag == RTE_COMP_FLUSH_FINAL ? + ICP_QAT_FW_COMP_BFINAL + : ICP_QAT_FW_COMP_NOT_BFINAL, + ICP_QAT_FW_COMP_CNV, + ICP_QAT_FW_COMP_CNV_RECOVERY); + + } else if (op->op_type == RTE_COMP_OP_STATEFUL) { + comp_req->comp_pars.req_par_flags = ICP_QAT_FW_COMP_REQ_PARAM_FLAGS_BUILD( (stream->start_of_packet) ? @@ -72,30 +195,6 @@ qat_comp_build_request(void *in_op, uint8_t *out_msg, ICP_QAT_FW_COMP_NO_CNV_RECOVERY); } - if (likely(qat_xform->qat_comp_request_type == - QAT_COMP_REQUEST_DYNAMIC_COMP_STATELESS)) { - if (unlikely(op->src.length > QAT_FALLBACK_THLD)) { - - /* fallback to fixed compression */ - comp_req->comn_hdr.service_cmd_id = - ICP_QAT_FW_COMP_CMD_STATIC; - - ICP_QAT_FW_COMN_NEXT_ID_SET(&comp_req->comp_cd_ctrl, - ICP_QAT_FW_SLICE_DRAM_WR); - - ICP_QAT_FW_COMN_NEXT_ID_SET(&comp_req->u2.xlt_cd_ctrl, - ICP_QAT_FW_SLICE_NULL); - ICP_QAT_FW_COMN_CURR_ID_SET(&comp_req->u2.xlt_cd_ctrl, - ICP_QAT_FW_SLICE_NULL); - - QAT_DP_LOG(DEBUG, "QAT PMD: fallback to fixed " - "compression! IM buffer size can be too low " - "for produced data.\n Please use input " - "buffer length lower than %d bytes", - QAT_FALLBACK_THLD); - } - } - /* common for sgl and flat buffers */ comp_req->comp_pars.comp_len = op->src.length; comp_req->comp_pars.out_buffer_sz = rte_pktmbuf_pkt_len(op->m_dst) - @@ -233,6 +332,213 @@ qat_comp_build_request(void *in_op, uint8_t *out_msg, return 0; } +static inline uint32_t adf_modulo(uint32_t data, uint32_t modulo_mask) +{ + return data & modulo_mask; +} + +static inline void +qat_comp_mbuf_skip(struct rte_mbuf **mbuf, uint32_t *offset, uint32_t len) +{ + while (*offset + len >= rte_pktmbuf_data_len(*mbuf)) { + len -= (rte_pktmbuf_data_len(*mbuf) - *offset); + *mbuf = (*mbuf)->next; + *offset = 0; + } + *offset = len; +} + +int +qat_comp_build_multiple_requests(void *in_op, struct qat_qp *qp, + uint32_t parent_tail, int nb_descr) +{ + struct rte_comp_op op_backup; + struct rte_mbuf dst_mbuf; + struct rte_comp_op *op = in_op; + struct qat_queue *txq = &(qp->tx_q); + uint8_t *base_addr = (uint8_t *)txq->base_addr; + uint8_t *out_msg = base_addr + parent_tail; + uint32_t tail = parent_tail; + struct icp_qat_fw_comp_req *comp_req = + (struct icp_qat_fw_comp_req *)out_msg; + struct qat_comp_op_cookie *parent_cookie = + (struct qat_comp_op_cookie *) + qp->op_cookies[parent_tail / txq->msg_size]; + struct qat_comp_op_cookie *child_cookie; + uint16_t dst_data_size = + RTE_MIN(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE, 65535); + uint32_t data_to_enqueue = op->src.length - QAT_FALLBACK_THLD; + int num_descriptors_built = 1; + int ret; + + QAT_DP_LOG(DEBUG, "op %p, parent_cookie %p", op, parent_cookie); + + /* copy original op to the local variable for restoring later */ + rte_memcpy(&op_backup, op, sizeof(op_backup)); + + parent_cookie->nb_child_responses = 0; + parent_cookie->nb_children = 0; + parent_cookie->split_op = 1; + parent_cookie->dst_data = op->m_dst; + parent_cookie->dst_data_offset = op->dst.offset; + + op->src.length = QAT_FALLBACK_THLD; + op->flush_flag = RTE_COMP_FLUSH_FULL; + + QAT_DP_LOG(DEBUG, "parent op src len %u dst len %u", + op->src.length, op->m_dst->pkt_len); + + ret = qat_comp_build_request(in_op, out_msg, parent_cookie, + qp->qat_dev_gen); + if (ret != 0) { + /* restore op and clear cookie */ + QAT_DP_LOG(WARNING, "Failed to build parent descriptor"); + op->src.length = op_backup.src.length; + op->flush_flag = op_backup.flush_flag; + parent_cookie->split_op = 0; + return ret; + } + + /* prepare local dst mbuf */ + rte_memcpy(&dst_mbuf, op->m_dst, sizeof(dst_mbuf)); + rte_pktmbuf_reset(&dst_mbuf); + dst_mbuf.buf_len = dst_data_size; + dst_mbuf.data_len = dst_data_size; + dst_mbuf.pkt_len = dst_data_size; + dst_mbuf.data_off = 0; + + /* update op for the child operations */ + op->m_dst = &dst_mbuf; + op->dst.offset = 0; + + while (data_to_enqueue) { + const struct rte_memzone *mz = + parent_cookie->dst_memzones[num_descriptors_built - 1]; + uint32_t src_data_size = RTE_MIN(data_to_enqueue, + QAT_FALLBACK_THLD); + uint32_t cookie_index; + + /* update params for the next op */ + op->src.offset += QAT_FALLBACK_THLD; + op->src.length = src_data_size; + op->flush_flag = (src_data_size == data_to_enqueue) ? + op_backup.flush_flag : RTE_COMP_FLUSH_FULL; + + /* update dst mbuf for the next op (use memzone for dst data) */ + dst_mbuf.buf_addr = mz->addr; + dst_mbuf.buf_iova = mz->iova; + + /* move the tail and calculate next cookie index */ + tail = adf_modulo(tail + txq->msg_size, txq->modulo_mask); + cookie_index = tail / txq->msg_size; + child_cookie = (struct qat_comp_op_cookie *) + qp->op_cookies[cookie_index]; + comp_req = (struct icp_qat_fw_comp_req *)(base_addr + tail); + + /* update child cookie */ + child_cookie->split_op = 1; /* must be set for child as well */ + child_cookie->parent_cookie = parent_cookie; /* same as above */ + child_cookie->nb_children = 0; + child_cookie->dest_buffer = mz->addr; + + QAT_DP_LOG(DEBUG, + "cookie_index %u, child_cookie %p, comp_req %p", + cookie_index, child_cookie, comp_req); + QAT_DP_LOG(DEBUG, + "data_to_enqueue %u, num_descriptors_built %d", + data_to_enqueue, num_descriptors_built); + QAT_DP_LOG(DEBUG, "child op src len %u dst len %u", + op->src.length, op->m_dst->pkt_len); + + /* build the request */ + ret = qat_comp_build_request(op, (uint8_t *)comp_req, + child_cookie, qp->qat_dev_gen); + if (ret < 0) { + QAT_DP_LOG(WARNING, "Failed to build child descriptor"); + /* restore op and clear cookie */ + rte_memcpy(op, &op_backup, sizeof(op_backup)); + parent_cookie->split_op = 0; + parent_cookie->nb_children = 0; + return ret; + } + + data_to_enqueue -= src_data_size; + num_descriptors_built++; + } + + /* restore backed up original op */ + rte_memcpy(op, &op_backup, sizeof(op_backup)); + + if (nb_descr != num_descriptors_built) + QAT_DP_LOG(ERR, "split op. expected %d, built %d", + nb_descr, num_descriptors_built); + + parent_cookie->nb_children = num_descriptors_built - 1; + return num_descriptors_built; +} + +static inline void +qat_comp_response_data_copy(struct qat_comp_op_cookie *cookie, + struct rte_comp_op *rx_op) +{ + struct qat_comp_op_cookie *pc = cookie->parent_cookie; + struct rte_mbuf *sgl_buf = pc->dst_data; + void *op_dst_addr = rte_pktmbuf_mtod_offset(sgl_buf, uint8_t *, + pc->dst_data_offset); + + /* number of bytes left in the current segment */ + uint32_t left_in_current = rte_pktmbuf_data_len(sgl_buf) - + pc->dst_data_offset; + + uint32_t prod, sent; + + if (rx_op->produced <= left_in_current) { + rte_memcpy(op_dst_addr, cookie->dest_buffer, + rx_op->produced); + /* calculate dst mbuf and offset for the next child op */ + if (rx_op->produced == left_in_current) { + pc->dst_data = sgl_buf->next; + pc->dst_data_offset = 0; + } else + pc->dst_data_offset += rx_op->produced; + } else { + rte_memcpy(op_dst_addr, cookie->dest_buffer, + left_in_current); + sgl_buf = sgl_buf->next; + prod = rx_op->produced - left_in_current; + sent = left_in_current; + while (prod > rte_pktmbuf_data_len(sgl_buf)) { + op_dst_addr = rte_pktmbuf_mtod_offset(sgl_buf, + uint8_t *, 0); + + rte_memcpy(op_dst_addr, + ((uint8_t *)cookie->dest_buffer) + + sent, + rte_pktmbuf_data_len(sgl_buf)); + + prod -= rte_pktmbuf_data_len(sgl_buf); + sent += rte_pktmbuf_data_len(sgl_buf); + + sgl_buf = sgl_buf->next; + } + + op_dst_addr = rte_pktmbuf_mtod_offset(sgl_buf, uint8_t *, 0); + + rte_memcpy(op_dst_addr, + ((uint8_t *)cookie->dest_buffer) + sent, + prod); + + /* calculate dst mbuf and offset for the next child op */ + if (prod == rte_pktmbuf_data_len(sgl_buf)) { + pc->dst_data = sgl_buf->next; + pc->dst_data_offset = 0; + } else { + pc->dst_data = sgl_buf; + pc->dst_data_offset = prod; + } + } +} + int qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, uint64_t *dequeue_err_count) @@ -241,6 +547,14 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, (struct icp_qat_fw_comp_resp *)resp; struct qat_comp_op_cookie *cookie = (struct qat_comp_op_cookie *)op_cookie; + + struct icp_qat_fw_resp_comp_pars *comp_resp1 = + (struct icp_qat_fw_resp_comp_pars *)&resp_msg->comp_resp_pars; + + QAT_DP_LOG(DEBUG, "input counter = %u, output counter = %u", + comp_resp1->input_byte_counter, + comp_resp1->output_byte_counter); + struct rte_comp_op *rx_op = (struct rte_comp_op *)(uintptr_t) (resp_msg->opaque_data); struct qat_comp_stream *stream; @@ -275,7 +589,10 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, rx_op->consumed = 0; rx_op->produced = 0; *op = (void *)rx_op; - return 0; + /* also in this case number of returned ops */ + /* must be equal to one, */ + /* appropriate status (error) must be set as well */ + return 1; } if (likely(qat_xform->qat_comp_request_type @@ -288,7 +605,7 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, *op = (void *)rx_op; QAT_DP_LOG(ERR, "QAT has wrong firmware"); ++(*dequeue_err_count); - return 0; + return 1; } } @@ -305,8 +622,9 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, int8_t xlat_err_code = (int8_t)resp_msg->comn_resp.comn_error.xlat_err_code; - /* handle recoverable out-of-buffer condition in stateful */ - /* decompression scenario */ + /* handle recoverable out-of-buffer condition in stateful + * decompression scenario + */ if (cmp_err_code == ERR_CODE_OVERFLOW_ERROR && !xlat_err_code && qat_xform->qat_comp_request_type == QAT_COMP_REQUEST_DECOMPRESS @@ -327,10 +645,12 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, xlat_err_code == ERR_CODE_OVERFLOW_ERROR)){ struct icp_qat_fw_resp_comp_pars *comp_resp = - (struct icp_qat_fw_resp_comp_pars *)&resp_msg->comp_resp_pars; + (struct icp_qat_fw_resp_comp_pars *) + &resp_msg->comp_resp_pars; - /* handle recoverable out-of-buffer condition */ - /* in stateless compression scenario */ + /* handle recoverable out-of-buffer condition + * in stateless compression scenario + */ if (comp_resp->input_byte_counter) { if ((qat_xform->qat_comp_request_type == QAT_COMP_REQUEST_FIXED_COMP_STATELESS) || @@ -375,9 +695,89 @@ qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, rx_op->output_chksum = comp_resp->curr_chksum; } } - *op = (void *)rx_op; + QAT_DP_LOG(DEBUG, "About to check for split op :cookies: %p %p, split:%u", + cookie, cookie->parent_cookie, cookie->split_op); + + if (cookie->split_op) { + *op = NULL; + struct qat_comp_op_cookie *pc = cookie->parent_cookie; + + if (cookie->nb_children > 0) { + QAT_DP_LOG(DEBUG, "Parent"); + /* parent - don't return until all children + * responses are collected + */ + cookie->total_consumed = rx_op->consumed; + cookie->total_produced = rx_op->produced; + if (err) { + cookie->error = rx_op->status; + rx_op->status = RTE_COMP_OP_STATUS_SUCCESS; + } else { + /* calculate dst mbuf and offset for child op */ + qat_comp_mbuf_skip(&cookie->dst_data, + &cookie->dst_data_offset, + rx_op->produced); + } + } else { + QAT_DP_LOG(DEBUG, "Child"); + if (pc->error == RTE_COMP_OP_STATUS_SUCCESS) { + if (err) + pc->error = rx_op->status; + if (rx_op->produced) { + /* this covers both SUCCESS and + * OUT_OF_SPACE_RECOVERABLE cases + */ + qat_comp_response_data_copy(cookie, + rx_op); + pc->total_consumed += rx_op->consumed; + pc->total_produced += rx_op->produced; + } + } + rx_op->status = RTE_COMP_OP_STATUS_SUCCESS; + + pc->nb_child_responses++; + + /* (child) cookie fields have to be reset + * to avoid problems with reusability - + * rx and tx queue starting from index zero + */ + cookie->nb_children = 0; + cookie->split_op = 0; + cookie->nb_child_responses = 0; + cookie->dest_buffer = NULL; + + if (pc->nb_child_responses == pc->nb_children) { + uint8_t child_resp; + + /* parent should be included as well */ + child_resp = pc->nb_child_responses + 1; + + rx_op->status = pc->error; + rx_op->consumed = pc->total_consumed; + rx_op->produced = pc->total_produced; + *op = (void *)rx_op; + + /* free memzones used for dst data */ + qat_comp_free_split_op_memzones(pc, + pc->nb_children); + + /* (parent) cookie fields have to be reset + * to avoid problems with reusability - + * rx and tx queue starting from index zero + */ + pc->nb_children = 0; + pc->split_op = 0; + pc->nb_child_responses = 0; + pc->error = RTE_COMP_OP_STATUS_SUCCESS; + + return child_resp; + } + } + return 0; + } - return 0; + *op = (void *)rx_op; + return 1; } unsigned int @@ -443,9 +843,9 @@ static int qat_comp_create_templates(struct qat_comp_xform *qat_xform, comp_level = ICP_QAT_HW_COMPRESSION_DEPTH_1; req_par_flags = ICP_QAT_FW_COMP_REQ_PARAM_FLAGS_BUILD( ICP_QAT_FW_COMP_SOP, ICP_QAT_FW_COMP_EOP, - ICP_QAT_FW_COMP_BFINAL, ICP_QAT_FW_COMP_NO_CNV, - ICP_QAT_FW_COMP_NO_CNV_RECOVERY); - + ICP_QAT_FW_COMP_BFINAL, + ICP_QAT_FW_COMP_CNV, + ICP_QAT_FW_COMP_CNV_RECOVERY); } else { if (xform->compress.level == RTE_COMP_LEVEL_PMD_DEFAULT) comp_level = ICP_QAT_HW_COMPRESSION_DEPTH_8; diff --git a/drivers/compress/qat/qat_comp.h b/drivers/compress/qat/qat_comp.h index 2231451a1..1c07f2233 100644 --- a/drivers/compress/qat/qat_comp.h +++ b/drivers/compress/qat/qat_comp.h @@ -11,6 +11,7 @@ #include <rte_compressdev_pmd.h> #include "qat_common.h" +#include "qat_qp.h" #include "icp_qat_hw.h" #include "icp_qat_fw_comp.h" #include "icp_qat_fw_la.h" @@ -22,7 +23,7 @@ #define ERR_CODE_QAT_COMP_WRONG_FW -99 /* fallback to fixed compression threshold */ -#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.1)) +#define QAT_FALLBACK_THLD ((uint32_t)(RTE_PMD_QAT_COMP_IM_BUFFER_SIZE / 1.3)) #define QAT_MIN_OUT_BUF_SIZE 46 @@ -63,6 +64,24 @@ struct qat_comp_op_cookie { uint16_t dst_nb_elems; struct qat_sgl *qat_sgl_src_d; struct qat_sgl *qat_sgl_dst_d; + struct qat_qp *qp; + uint32_t cookie_index; + + /* QAT IM buffer too small handling: */ + uint8_t split_op; + uint8_t nb_children; + + /* used by the parent only */ + uint8_t nb_child_responses; + uint32_t total_consumed; + uint32_t total_produced; + const struct rte_memzone **dst_memzones; + struct rte_mbuf *dst_data; + uint32_t dst_data_offset; + + /* used by the child only */ + struct qat_comp_op_cookie *parent_cookie; + void *dest_buffer; }; struct qat_comp_xform { @@ -86,6 +105,14 @@ int qat_comp_build_request(void *in_op, uint8_t *out_msg, void *op_cookie, enum qat_device_gen qat_dev_gen __rte_unused); +int +qat_comp_build_multiple_requests(void *in_op, struct qat_qp *qp, + uint32_t parent_tail, int nb_descr); + +void +qat_comp_free_split_op_memzones(struct qat_comp_op_cookie *cookie, + unsigned int nb_children); + int qat_comp_process_response(void **op, uint8_t *resp, void *op_cookie, uint64_t *dequeue_err_count); diff --git a/drivers/compress/qat/qat_comp_pmd.c b/drivers/compress/qat/qat_comp_pmd.c index 9a7ed19d7..fe62de533 100644 --- a/drivers/compress/qat/qat_comp_pmd.c +++ b/drivers/compress/qat/qat_comp_pmd.c @@ -146,6 +146,9 @@ qat_comp_qp_setup(struct rte_compressdev *dev, uint16_t qp_id, struct qat_comp_op_cookie *cookie = qp->op_cookies[i]; + cookie->qp = qp; + cookie->cookie_index = i; + cookie->qat_sgl_src_d = rte_zmalloc_socket(NULL, sizeof(struct qat_sgl) + sizeof(struct qat_flat_buf) * @@ -560,20 +563,6 @@ qat_comp_dev_info_get(struct rte_compressdev *dev, } } -static uint16_t -qat_comp_pmd_enqueue_op_burst(void *qp, struct rte_comp_op **ops, - uint16_t nb_ops) -{ - return qat_enqueue_op_burst(qp, (void **)ops, nb_ops); -} - -static uint16_t -qat_comp_pmd_dequeue_op_burst(void *qp, struct rte_comp_op **ops, - uint16_t nb_ops) -{ - return qat_dequeue_op_burst(qp, (void **)ops, nb_ops); -} - static uint16_t qat_comp_pmd_enq_deq_dummy_op_burst(void *qp __rte_unused, struct rte_comp_op **ops __rte_unused, @@ -603,7 +592,7 @@ static struct rte_compressdev_ops compress_qat_dummy_ops = { }; static uint16_t -qat_comp_pmd_dequeue_frst_op_burst(void *qp, struct rte_comp_op **ops, +qat_comp_pmd_dequeue_first_op_burst(void *qp, struct rte_comp_op **ops, uint16_t nb_ops) { uint16_t ret = qat_dequeue_op_burst(qp, (void **)ops, nb_ops); @@ -623,7 +612,8 @@ qat_comp_pmd_dequeue_frst_op_burst(void *qp, struct rte_comp_op **ops, } else { tmp_qp->qat_dev->comp_dev->compressdev->dequeue_burst = - qat_comp_pmd_dequeue_op_burst; + (compressdev_dequeue_pkt_burst_t) + qat_dequeue_op_burst; } } return ret; @@ -698,8 +688,9 @@ qat_comp_dev_create(struct qat_pci_device *qat_pci_dev, compressdev->dev_ops = &compress_qat_ops; - compressdev->enqueue_burst = qat_comp_pmd_enqueue_op_burst; - compressdev->dequeue_burst = qat_comp_pmd_dequeue_frst_op_burst; + compressdev->enqueue_burst = (compressdev_enqueue_pkt_burst_t) + qat_enqueue_comp_op_burst; + compressdev->dequeue_burst = qat_comp_pmd_dequeue_first_op_burst; compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED; -- 2.17.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 1/2] " Adam Dybkowski @ 2020-04-17 15:44 ` Adam Dybkowski 2020-04-17 15:58 ` Trahe, Fiona 1 sibling, 1 reply; 20+ messages in thread From: Adam Dybkowski @ 2020-04-17 15:44 UTC (permalink / raw) To: dev, fiona.trahe, akhil.goyal; +Cc: Adam Dybkowski This patch adds new tests for verification of the "internal QAT IM buffer too small" case handling. These unit tests aren't specific to the QAT PMD only - they pass or skip on other PMDs like ISAL and ZLIB (depending on particular PMD capabilities). Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> --- app/test/test_compressdev.c | 1185 +++++++++++++++++++++++++++++++++-- 1 file changed, 1118 insertions(+), 67 deletions(-) diff --git a/app/test/test_compressdev.c b/app/test/test_compressdev.c index 7549135c2..0571c17ec 100644 --- a/app/test/test_compressdev.c +++ b/app/test/test_compressdev.c @@ -6,6 +6,7 @@ #include <math.h> #include <stdlib.h> #include <unistd.h> +#include <stdio.h> #include <rte_cycles.h> #include <rte_malloc.h> @@ -30,6 +31,7 @@ * due to the compress block headers */ #define COMPRESS_BUF_SIZE_RATIO 1.3 +#define COMPRESS_BUF_SIZE_RATIO_DISABLED 1.0 #define COMPRESS_BUF_SIZE_RATIO_OVERFLOW 0.2 #define NUM_LARGE_MBUFS 16 #define SMALL_SEG_SIZE 256 @@ -49,8 +51,26 @@ #define MAX_MBUF_SEGMENT_SIZE 65535 #define MAX_DATA_MBUF_SIZE (MAX_MBUF_SEGMENT_SIZE - RTE_PKTMBUF_HEADROOM) -#define NUM_BIG_MBUFS 4 -#define BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * NUM_BIG_MBUFS / 2) +#define NUM_BIG_MBUFS (512 + 1) +#define BIG_DATA_TEST_SIZE (MAX_DATA_MBUF_SIZE * 2) + +/* constants for "im buffer" tests start here */ + +/* number of mbufs lower than number of inflight ops */ +#define IM_BUF_NUM_MBUFS 3 +/* above threshold (QAT_FALLBACK_THLD) and below max mbuf size */ +#define IM_BUF_DATA_TEST_SIZE_LB 59600 +/* data size smaller than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_SGL (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS) +/* number of mbufs bigger than number of inflight ops */ +#define IM_BUF_NUM_MBUFS_OVER (NUM_MAX_INFLIGHT_OPS + 1) +/* data size bigger than the queue capacity */ +#define IM_BUF_DATA_TEST_SIZE_OVER (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_OVER) +/* number of mid-size mbufs */ +#define IM_BUF_NUM_MBUFS_MID ((NUM_MAX_INFLIGHT_OPS / 3) + 1) +/* capacity of mid-size mbufs */ +#define IM_BUF_DATA_TEST_SIZE_MID (MAX_DATA_MBUF_SIZE * IM_BUF_NUM_MBUFS_MID) + const char * huffman_type_strings[] = { @@ -78,6 +98,11 @@ enum overflow_test { OVERFLOW_ENABLED }; +enum ratio_switch { + RATIO_DISABLED, + RATIO_ENABLED +}; + enum operation_type { OPERATION_COMPRESSION, OPERATION_DECOMPRESSION @@ -123,6 +148,7 @@ struct test_data_params { const struct rte_memzone *uncompbuf_memzone; /* overflow test activation */ enum overflow_test overflow; + enum ratio_switch ratio; }; struct test_private_arrays { @@ -141,6 +167,7 @@ struct test_private_arrays { static struct comp_testsuite_params testsuite_params = { 0 }; + static void testsuite_teardown(void) { @@ -316,6 +343,8 @@ test_compressdev_invalid_configuration(void) }; struct rte_compressdev_info dev_info; + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + /* Invalid configuration with 0 queue pairs */ memcpy(&invalid_config, &valid_config, sizeof(struct rte_compressdev_config)); @@ -691,7 +720,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf, if (data_ptr != NULL) { /* Copy characters without NULL terminator */ - strncpy(buf_ptr, data_ptr, data_size); + memcpy(buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size; @@ -731,7 +760,7 @@ prepare_sgl_bufs(const char *test_buf, struct rte_mbuf *head_buf, } if (data_ptr != NULL) { /* Copy characters without NULL terminator */ - strncpy(buf_ptr, data_ptr, data_size); + memcpy(buf_ptr, data_ptr, data_size); data_ptr += data_size; } remaining_data -= data_size; @@ -760,17 +789,20 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, { uint16_t num_enqd, num_deqd, num_total_deqd; unsigned int deqd_retries = 0; + int res = 0; /* Enqueue and dequeue all operations */ num_enqd = rte_compressdev_enqueue_burst(0, 0, ops, num_bufs); if (num_enqd < num_bufs) { RTE_LOG(ERR, USER1, "Some operations could not be enqueued\n"); - return -1; + res = -1; } + /* dequeue ops even on error (same number of ops as was enqueued) */ + num_total_deqd = 0; - do { + while (num_total_deqd < num_enqd) { /* * If retrying a dequeue call, wait for 10 ms to allow * enough time to the driver to process the operations @@ -783,7 +815,8 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, if (deqd_retries == MAX_DEQD_RETRIES) { RTE_LOG(ERR, USER1, "Not all operations could be dequeued\n"); - return -1; + res = -1; + break; } usleep(DEQUEUE_WAIT_TIME); } @@ -792,9 +825,9 @@ test_run_enqueue_dequeue(struct rte_comp_op **ops, num_total_deqd += num_deqd; deqd_retries++; - } while (num_total_deqd < num_enqd); + } - return 0; + return res; } /** @@ -956,7 +989,9 @@ test_mbufs_calculate_data_size( /* local variables: */ uint32_t data_size; struct priv_op_data *priv_data; - float ratio; + float ratio_val; + enum ratio_switch ratio = test_data->ratio; + uint8_t not_zlib_compr; /* true if zlib isn't current compression dev */ enum overflow_test overflow = test_data->overflow; @@ -973,13 +1008,16 @@ test_mbufs_calculate_data_size( not_zlib_compr = (test_data->zlib_dir == ZLIB_DECOMPRESS || test_data->zlib_dir == ZLIB_NONE); - ratio = (not_zlib_compr && + ratio_val = (ratio == RATIO_ENABLED) ? + COMPRESS_BUF_SIZE_RATIO : + COMPRESS_BUF_SIZE_RATIO_DISABLED; + + ratio_val = (not_zlib_compr && (overflow == OVERFLOW_ENABLED)) ? COMPRESS_BUF_SIZE_RATIO_OVERFLOW : - COMPRESS_BUF_SIZE_RATIO; - - data_size = strlen(test_bufs[i]) * ratio; + ratio_val; + data_size = strlen(test_bufs[i]) * ratio_val; } else { priv_data = (struct priv_op_data *) (ops_processed[i] + 1); @@ -1085,6 +1123,9 @@ test_setup_output_bufs( } else { for (i = 0; i < num_bufs; i++) { + enum rte_comp_huffman comp_huffman = + ts_params->def_comp_xform->compress.deflate.huffman; + /* data size calculation */ data_size = test_mbufs_calculate_data_size( op_type, @@ -1094,6 +1135,11 @@ test_setup_output_bufs( test_data, i); + if (comp_huffman != RTE_COMP_HUFFMAN_DYNAMIC) { + if (op_type == OPERATION_DECOMPRESSION) + data_size *= COMPRESS_BUF_SIZE_RATIO; + } + /* data allocation */ if (buff_type == SGL_BOTH || buff_type == LB_TO_SGL) { ret = prepare_sgl_bufs(NULL, current_bufs[i], @@ -1192,6 +1238,11 @@ test_deflate_comp_run(const struct interim_data_params *int_data, ops[i]->src.length = rte_pktmbuf_pkt_len(uncomp_bufs[i]); ops[i]->dst.offset = 0; + RTE_LOG(DEBUG, USER1, + "Uncompressed buffer length = %u compressed buffer length = %u", + rte_pktmbuf_pkt_len(uncomp_bufs[i]), + rte_pktmbuf_pkt_len(comp_bufs[i])); + if (operation_type == RTE_COMP_OP_STATELESS) { ops[i]->flush_flag = RTE_COMP_FLUSH_FINAL; } else { @@ -1313,6 +1364,7 @@ test_deflate_comp_run(const struct interim_data_params *int_data, if (ret_status < 0) for (i = 0; i < num_bufs; i++) { rte_comp_op_free(ops[i]); + ops[i] = NULL; ops_processed[i] = NULL; } @@ -1431,7 +1483,7 @@ test_deflate_comp_finalize(const struct interim_data_params *int_data, } RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Comp: Some operations were not successful\n"); return -1; } priv_data = (struct priv_op_data *)(ops_processed[i] + 1); @@ -1490,6 +1542,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data, /* from test_priv_data: */ struct rte_mbuf **uncomp_bufs = test_priv_data->uncomp_bufs; + struct rte_mbuf **comp_bufs = test_priv_data->comp_bufs; struct rte_comp_op **ops = test_priv_data->ops; struct rte_comp_op **ops_processed = test_priv_data->ops_processed; void **priv_xforms = test_priv_data->priv_xforms; @@ -1510,7 +1563,7 @@ test_deflate_decomp_run(const struct interim_data_params *int_data, /* Source buffer is the compressed data from the previous operations */ for (i = 0; i < num_bufs; i++) { - ops[i]->m_src = ops_processed[i]->m_dst; + ops[i]->m_src = comp_bufs[i]; ops[i]->m_dst = uncomp_bufs[i]; ops[i]->src.offset = 0; /* @@ -1740,6 +1793,10 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE || ops_processed[i]->status == RTE_COMP_OP_STATUS_SUCCESS)) { + + RTE_LOG(DEBUG, USER1, + ".............RECOVERABLE\n"); + /* collect the output into all_decomp_data */ const void *ptr = rte_pktmbuf_read( ops_processed[i]->m_dst, @@ -1777,7 +1834,6 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, ops[i]->src.length -= ops_processed[i]->consumed; /* repeat the operation */ - //goto next_step; return 2; } else { /* Compare the original stream with the */ @@ -1808,7 +1864,8 @@ test_deflate_decomp_finalize(const struct interim_data_params *int_data, } else if (ops_processed[i]->status != RTE_COMP_OP_STATUS_SUCCESS) { RTE_LOG(ERR, USER1, - "Some operations were not successful\n"); + "Decomp: Some operations were not successful, status = %u\n", + ops_processed[i]->status); return -1; } priv_data = (struct priv_op_data *)(ops_processed[i] + 1); @@ -1986,7 +2043,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, "Compress device does not support DEFLATE\n"); return -1; } - //test_objects_init(&test_priv_data, num_bufs); /* Prepare the source mbufs with the data */ ret = test_setup_com_bufs(int_data, test_data, &test_priv_data); @@ -1995,6 +2051,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, goto exit; } + RTE_LOG(DEBUG, USER1, "<<< COMPRESSION >>>\n"); + /* COMPRESSION */ /* Prepare output (destination) mbufs for compressed data */ @@ -2031,6 +2089,8 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, /* DECOMPRESSION */ + RTE_LOG(DEBUG, USER1, "<<< DECOMPRESSION >>>\n"); + /* Prepare output (destination) mbufs for decompressed data */ ret = test_setup_output_bufs( OPERATION_DECOMPRESSION, @@ -2096,7 +2156,6 @@ test_deflate_comp_decomp(const struct interim_data_params *int_data, priv_xforms[i] = NULL; } } - for (i = 0; i < num_bufs; i++) { rte_pktmbuf_free(uncomp_bufs[i]); rte_pktmbuf_free(comp_bufs[i]); @@ -2152,7 +2211,8 @@ test_compressdev_deflate_stateless_fixed(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2223,7 +2283,8 @@ test_compressdev_deflate_stateless_dynamic(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2278,7 +2339,8 @@ test_compressdev_deflate_stateless_multi_op(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with compressdev, decompress with Zlib */ @@ -2332,7 +2394,8 @@ test_compressdev_deflate_stateless_multi_level(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2422,7 +2485,8 @@ test_compressdev_deflate_stateless_multi_xform(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with compressdev, decompress with Zlib */ @@ -2471,7 +2535,8 @@ test_compressdev_deflate_stateless_sgl(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -2582,7 +2647,8 @@ test_compressdev_deflate_stateless_checksum(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Check if driver supports crc32 checksum and test */ @@ -2700,7 +2766,8 @@ test_compressdev_out_of_space_buffer(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 1, /* run out-of-space test */ .big_data = 0, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir = ZLIB_DECOMPRESS; @@ -2742,7 +2809,7 @@ test_compressdev_deflate_stateless_dynamic_big(void) struct comp_testsuite_params *ts_params = &testsuite_params; uint16_t i = 0; int ret; - int j; + unsigned int j; const struct rte_compressdev_capabilities *capab; char *test_buffer = NULL; @@ -2778,7 +2845,8 @@ test_compressdev_deflate_stateless_dynamic_big(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 1, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED }; ts_params->def_comp_xform->compress.deflate.huffman = @@ -2788,7 +2856,7 @@ test_compressdev_deflate_stateless_dynamic_big(void) srand(BIG_DATA_TEST_SIZE); for (j = 0; j < BIG_DATA_TEST_SIZE - 1; ++j) test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; - test_buffer[BIG_DATA_TEST_SIZE-1] = 0; + test_buffer[BIG_DATA_TEST_SIZE - 1] = 0; /* Compress with compressdev, decompress with Zlib */ test_data.zlib_dir = ZLIB_DECOMPRESS; @@ -2843,7 +2911,8 @@ test_compressdev_deflate_stateful_decomp(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Compress with Zlib, decompress with compressdev */ @@ -2926,7 +2995,8 @@ test_compressdev_deflate_stateful_decomp_checksum(void) .big_data = 0, .decompress_output_block_size = 2000, .decompress_steps_max = 4, - .overflow = OVERFLOW_DISABLED + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_ENABLED }; /* Check if driver supports crc32 checksum and test */ @@ -3139,7 +3209,8 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void) .zlib_dir = ZLIB_DECOMPRESS, .out_of_space = 0, .big_data = 0, - .overflow = OVERFLOW_ENABLED + .overflow = OVERFLOW_ENABLED, + .ratio = RATIO_ENABLED }; for (i = 0; i < RTE_DIM(compress_test_bufs); i++) { @@ -3176,39 +3247,1019 @@ test_compressdev_deflate_stateless_fixed_oos_recoverable(void) return ret; } -static struct unit_test_suite compressdev_testsuite = { - .suite_name = "compressdev unit test suite", - .setup = testsuite_setup, - .teardown = testsuite_teardown, - .unit_test_cases = { - TEST_CASE_ST(NULL, NULL, - test_compressdev_invalid_configuration), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_dynamic_big), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_op), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_level), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_multi_xform), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_sgl), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_checksum), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_out_of_space_buffer), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateful_decomp_checksum), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_external_mbufs), - TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, - test_compressdev_deflate_stateless_fixed_oos_recoverable), +static int +test_compressdev_deflate_im_buffers_LB_1op(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + struct interim_data_params int_data = { + (const char * const *)&test_buffer, + 1, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + /* must be LB to SGL, + * input LB buffer reaches its maximum, + * if ratio 1.3 than another mbuf must be + * created and attached + */ + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_2ops_first(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = test_buffer; + test_buffers[1] = compress_test_bufs[0]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_2ops_second(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_3ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[3]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 3, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_LB_4ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[4]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_LB, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for 'im buffer' test\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + test_buffers[3] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 4, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = LB_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_LB); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_LB - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_1op(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + struct interim_data_params int_data = { + (const char * const *)&test_buffer, + 1, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_first(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = test_buffer; + test_buffers[1] = compress_test_bufs[0]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_2ops_second(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_3ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[3]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 3, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_4ops(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[4]; + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_SGL, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + test_buffers[2] = compress_test_bufs[1]; + test_buffers[3] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 4, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_SGL); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_SGL - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_FAILED; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_1op(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_OVER, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + struct interim_data_params int_data = { + (const char * const *)&test_buffer, + 1, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_OVER); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_OVER - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_SUCCESS; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + + return ret; +} + + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_first(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_OVER, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = test_buffer; + test_buffers[1] = compress_test_bufs[0]; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_OVER); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_OVER - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_SUCCESS; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static int +test_compressdev_deflate_im_buffers_SGL_over_2ops_second(void) +{ + struct comp_testsuite_params *ts_params = &testsuite_params; + uint16_t i = 0; + int ret = TEST_SUCCESS; + int j; + const struct rte_compressdev_capabilities *capab; + char *test_buffer = NULL; + const char *test_buffers[2]; + + RTE_LOG(INFO, USER1, "This is a negative test, errors are expected\n"); + + capab = rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE); + TEST_ASSERT(capab != NULL, "Failed to retrieve device capabilities"); + + if ((capab->comp_feature_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) + return -ENOTSUP; + + if ((capab->comp_feature_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) == 0) + return -ENOTSUP; + + test_buffer = rte_malloc(NULL, IM_BUF_DATA_TEST_SIZE_OVER, 0); + if (test_buffer == NULL) { + RTE_LOG(ERR, USER1, + "Can't allocate buffer for big-data\n"); + return TEST_FAILED; + } + + test_buffers[0] = compress_test_bufs[0]; + test_buffers[1] = test_buffer; + + struct interim_data_params int_data = { + (const char * const *)test_buffers, + 2, + &i, + &ts_params->def_comp_xform, + &ts_params->def_decomp_xform, + 1 + }; + + struct test_data_params test_data = { + .compress_state = RTE_COMP_OP_STATELESS, + .decompress_state = RTE_COMP_OP_STATELESS, + .buff_type = SGL_BOTH, + .zlib_dir = ZLIB_NONE, + .out_of_space = 0, + .big_data = 1, + .overflow = OVERFLOW_DISABLED, + .ratio = RATIO_DISABLED + }; + + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DYNAMIC; + + /* fill the buffer with data based on rand. data */ + srand(IM_BUF_DATA_TEST_SIZE_OVER); + for (j = 0; j < IM_BUF_DATA_TEST_SIZE_OVER - 1; ++j) + test_buffer[j] = (uint8_t)(rand() % ((uint8_t)-1)) | 1; + + /* Compress with compressdev, decompress with compressdev */ + if (test_deflate_comp_decomp(&int_data, &test_data) < 0) { + ret = TEST_SUCCESS; + goto end; + } + +end: + ts_params->def_comp_xform->compress.deflate.huffman = + RTE_COMP_HUFFMAN_DEFAULT; + rte_free(test_buffer); + return ret; +} + +static struct unit_test_suite compressdev_testsuite = { + .suite_name = "compressdev unit test suite", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(NULL, NULL, + test_compressdev_invalid_configuration), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_dynamic_big), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_op), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_level), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_multi_xform), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_sgl), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_checksum), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_out_of_space_buffer), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateful_decomp_checksum), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_external_mbufs), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_stateless_fixed_oos_recoverable), + + /* Positive test cases for IM buffer handling verification */ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_1op), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_first), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_2ops_second), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_3ops), + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_LB_4ops), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_1op), + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_first), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_2ops_second), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_3ops), + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_4ops), + + /* Negative test cases for IM buffer handling verification */ + + /* For this test huge mempool is necessary. + * It tests one case: + * only one op containing big amount of data, so that + * number of requested descriptors higher than number + * of available descriptors (128) + */ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_1op), + + /* For this test huge mempool is necessary. + * 2 ops. First op contains big amount of data: + * number of requested descriptors higher than number + * of available descriptors (128), the second op is + * relatively small. In this case both ops are rejected + */ + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_first), + + TEST_CASE_ST(generic_ut_setup, generic_ut_teardown, + test_compressdev_deflate_im_buffers_SGL_over_2ops_second), + TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.17.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski @ 2020-04-17 15:58 ` Trahe, Fiona 2020-04-17 21:50 ` Akhil Goyal 0 siblings, 1 reply; 20+ messages in thread From: Trahe, Fiona @ 2020-04-17 15:58 UTC (permalink / raw) To: Dybkowski, AdamX, dev, akhil.goyal > -----Original Message----- > From: Dybkowski, AdamX <adamx.dybkowski@intel.com> > Sent: Friday, April 17, 2020 4:44 PM > To: dev@dpdk.org; Trahe, Fiona <fiona.trahe@intel.com>; akhil.goyal@nxp.com > Cc: Dybkowski, AdamX <adamx.dybkowski@intel.com> > Subject: [PATCH v3 2/2] test/compress: im buffer too small - add unit tests > > This patch adds new tests for verification of the "internal > QAT IM buffer too small" case handling. These unit tests aren't > specific to the QAT PMD only - they pass or skip on other PMDs like > ISAL and ZLIB (depending on particular PMD capabilities). > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> Acked-by: Fiona Trahe <fiona.trahe@intel.com> ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests 2020-04-17 15:58 ` Trahe, Fiona @ 2020-04-17 21:50 ` Akhil Goyal 0 siblings, 0 replies; 20+ messages in thread From: Akhil Goyal @ 2020-04-17 21:50 UTC (permalink / raw) To: Trahe, Fiona, Dybkowski, AdamX, dev > > This patch adds new tests for verification of the "internal > > QAT IM buffer too small" case handling. These unit tests aren't > > specific to the QAT PMD only - they pass or skip on other PMDs like > > ISAL and ZLIB (depending on particular PMD capabilities). > > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com> > Acked-by: Fiona Trahe <fiona.trahe@intel.com> Updated patch title compress/qat: support IM buffer too small operation test/compress: add cases for IM buffer too small Series applied to dpdk-next-crypto Thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2020-04-17 21:50 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-04-08 12:50 [dpdk-dev] [PATCH v2 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 1/2] " Adam Dybkowski 2020-04-08 15:43 ` Trahe, Fiona 2020-04-08 12:51 ` [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski 2020-04-08 15:44 ` Trahe, Fiona 2020-04-15 18:35 ` Akhil Goyal 2020-04-16 10:02 ` Trahe, Fiona 2020-04-16 10:25 ` Akhil Goyal 2020-04-16 11:26 ` Trahe, Fiona 2020-04-16 14:31 ` Bruce Richardson 2020-04-16 14:55 ` Trahe, Fiona 2020-04-16 14:37 ` Akhil Goyal 2020-04-16 14:52 ` Trahe, Fiona 2020-04-17 15:39 ` Akhil Goyal 2020-04-17 15:56 ` Trahe, Fiona 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 0/2] compress/qat: im buffer too small - split op Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 1/2] " Adam Dybkowski 2020-04-17 15:44 ` [dpdk-dev] [PATCH v3 2/2] test/compress: im buffer too small - add unit tests Adam Dybkowski 2020-04-17 15:58 ` Trahe, Fiona 2020-04-17 21:50 ` Akhil Goyal
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).