From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50E1FA04A2; Mon, 4 Nov 2019 11:24:46 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9CF7F37AF; Mon, 4 Nov 2019 11:24:45 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 8C432378B for ; Mon, 4 Nov 2019 11:24:43 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 04 Nov 2019 02:24:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,266,1569308400"; d="scan'208";a="205139375" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga006.jf.intel.com with ESMTP; 04 Nov 2019 02:24:42 -0800 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 4 Nov 2019 02:24:41 -0800 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 4 Nov 2019 02:24:40 -0800 Received: from HASMSX110.ger.corp.intel.com (10.184.198.28) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Mon, 4 Nov 2019 02:24:40 -0800 Received: from hasmsx107.ger.corp.intel.com ([169.254.2.187]) by HASMSX110.ger.corp.intel.com ([169.254.6.106]) with mapi id 14.03.0439.000; Mon, 4 Nov 2019 12:24:37 +0200 From: "Trybula, ArturX" To: Shally Verma , "dev@dpdk.org" , "Trahe, Fiona" , "Dybkowski, AdamX" , "akhil.goyal@nxp.com" Thread-Topic: [EXT] [PATCH v3 1/1] test/compress: unit tests refactoring Thread-Index: AQHVikwL8Flm67vmt0eCZaJdlv6k+Kd0+lkAgAXXd3A= Date: Mon, 4 Nov 2019 10:24:36 +0000 Message-ID: <5B6D1C77E9D7034C93E97BD83D1D9F572F231F97@hasmsx107.ger.corp.intel.com> References: <20190912143431.26917-1-arturx.trybula@intel.com> <20191024091616.31224-1-arturx.trybula@intel.com> <20191024091616.31224-2-arturx.trybula@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNzIyNTJkZTMtYWI2Zi00MzYxLThkNTgtMTQ5ZDZhYWQ4YzBlIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiNk4yOHRRc2wxaFZqTllwRlhRekdxYjlkZldRYnFTdkF6UTRqTm4zQXlMczhhc2l3Y0NTNkQ1b3I5bTNkWHkzYyJ9 x-originating-ip: [10.184.70.11] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [EXT] [PATCH v3 1/1] test/compress: unit tests refactoring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Shally, Please find my comments below. =20 > ---------------------------------------------------------------------- > Core engine refactoring (test_deflate_comp_decomp function). > Smaller specialized functions created. >=20 > Signed-off-by: Artur Trybula > --- > app/test/test_compressdev.c | 1118 +++++++++++++++++------- > doc/guides/rel_notes/release_19_11.rst | 5 + > 2 files changed, 826 insertions(+), 297 deletions(-) >=20 > diff --git a/app/test/test_compressdev.c b/app/test/test_compressdev.c > index 283c64971..4932bec76 100644 > --- a/app/test/test_compressdev.c > +++ b/app/test/test_compressdev.c > @@ -120,6 +120,20 @@ struct test_data_params { > enum overflow_test overflow; > }; >=20 > +struct test_private_arrays { > + struct rte_mbuf **uncomp_bufs; > + struct rte_mbuf **comp_bufs; > + struct rte_comp_op **ops; > + struct rte_comp_op **ops_processed; > + void **priv_xforms; > + uint64_t *compress_checksum; > + uint32_t *compressed_data_size; > + void **stream; > + char **all_decomp_data; > + unsigned int *decomp_produced_data_size; > + uint16_t num_priv_xforms; > +}; > + > static struct comp_testsuite_params testsuite_params =3D { 0 }; >=20 > static void > @@ -662,6 +676,7 @@ prepare_sgl_bufs(const char *test_buf, struct > rte_mbuf *head_buf, > data_size =3D remaining_data; > else > data_size =3D seg_size; > + > buf_ptr =3D rte_pktmbuf_append(head_buf, data_size); > if (buf_ptr =3D=3D NULL) { > RTE_LOG(ERR, USER1, > @@ -734,8 +749,9 @@ extbuf_free_callback(void *addr __rte_unused, void > *opaque __rte_unused) } >=20 > static int > -test_run_enqueue_dequeue(struct rte_comp_op **ops, unsigned int > num_bufs, > - struct rte_comp_op **ops_processed) > +test_run_enqueue_dequeue(struct rte_comp_op **ops, > + struct rte_comp_op **ops_processed, > + unsigned int num_bufs) > { > uint16_t num_enqd, num_deqd, num_total_deqd; > unsigned int deqd_retries =3D 0; > @@ -776,81 +792,77 @@ test_run_enqueue_dequeue(struct rte_comp_op > **ops, unsigned int num_bufs, > return 0; > } >=20 > -/* > - * Compresses and decompresses buffer with compressdev API and Zlib API > +/** > + * Initialize all the arrays used in comp/decomp to NULL. > + * > + * If the length is greater than the length of the last segment, the > + * function will fail and return -1 without modifying the mbuf. > + * > + * @param prv > + * A container used for aggregation all the arrays. > + * @param num_bufs > + * The number of elements in each array. > + */ > +static void > +test_objects_init(struct test_private_arrays *prv, > + unsigned int num_bufs) > +{ > + /* Initialize all arrays to NULL */ > + memset(prv->uncomp_bufs, 0, sizeof(struct rte_mbuf *) * > num_bufs); > + memset(prv->comp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs); > + memset(prv->ops, 0, sizeof(struct rte_comp_op *) * num_bufs); > + memset(prv->ops_processed, 0, sizeof(struct rte_comp_op *) * > num_bufs); > + memset(prv->priv_xforms, 0, sizeof(void *) * num_bufs); > + memset(prv->compressed_data_size, 0, sizeof(uint32_t) * > num_bufs); } > + Does it really matter what pointer is pointing to? Sizeof(any pointer) * nu= m_bufs .. should be enough. [Artur] You are absolutely right that sizeof() of any pointer gives the sam= e result. But to be honest I don't understand your idea. Would you like to = have an extra variable to initialise all the pointer arrays? Eg: uint32_t array_size =3D sizeof(void *) * num_bufs; ... memset(prv->uncomp_bufs, 0, array_size); memset(prv->comp_bufs, 0, array_size); ... > +/** > + * Source buffers preparation (for compression). > + * > + * Memory allocation for each buffer from mempool > + * -1 returned if function fail, without modifying the mbuf. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > */ > static int > -test_deflate_comp_decomp(const struct interim_data_params *int_data, > - const struct test_data_params *test_data) > +test_mbufs_source_preparation(const struct interim_data_params > *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) > { > + /* local variables: */ > + unsigned int i; > + uint32_t data_size; > + char *buf_ptr; > + int ret; > + char **all_decomp_data =3D priv_arrays->all_decomp_data; > + > struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from int_data: */ > const char * const *test_bufs =3D int_data->test_bufs; > unsigned int num_bufs =3D int_data->num_bufs; > - uint16_t *buf_idx =3D int_data->buf_idx; > - struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > - struct rte_comp_xform **decompress_xforms =3D int_data- > >decompress_xforms; > - unsigned int num_xforms =3D int_data->num_xforms; > - enum rte_comp_op_type compress_state =3D test_data- > >compress_state; > - enum rte_comp_op_type decompress_state =3D test_data- > >decompress_state; > + > + /* from test_data: */ > unsigned int buff_type =3D test_data->buff_type; > - unsigned int out_of_space =3D test_data->out_of_space; > unsigned int big_data =3D test_data->big_data; > - enum zlib_direction zlib_dir =3D test_data->zlib_dir; > - enum overflow_test overflow_tst =3D test_data->overflow; > - int ret_status =3D TEST_FAILED; > - struct rte_mbuf_ext_shared_info inbuf_info; > - struct rte_mbuf_ext_shared_info compbuf_info; > - struct rte_mbuf_ext_shared_info decompbuf_info; > - int ret; > - struct rte_mbuf *uncomp_bufs[num_bufs]; > - struct rte_mbuf *comp_bufs[num_bufs]; > - struct rte_comp_op *ops[num_bufs]; > - struct rte_comp_op *ops_processed[num_bufs]; > - void *priv_xforms[num_bufs]; > - uint16_t num_enqd, num_deqd, num_total_deqd; > - uint16_t num_priv_xforms =3D 0; > - unsigned int deqd_retries =3D 0; > - struct priv_op_data *priv_data; > - char *buf_ptr; > - unsigned int i; > - struct rte_mempool *buf_pool; > - uint32_t data_size; > - /* Compressing with CompressDev */ > - unsigned int oos_zlib_decompress =3D > - (zlib_dir =3D=3D ZLIB_NONE || zlib_dir =3D=3D > ZLIB_DECOMPRESS); > - /* Decompressing with CompressDev */ > - unsigned int oos_zlib_compress =3D > - (zlib_dir =3D=3D ZLIB_NONE || zlib_dir =3D=3D > ZLIB_COMPRESS); > - const struct rte_compressdev_capabilities *capa =3D > - rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); > - char *contig_buf =3D NULL; > - uint64_t compress_checksum[num_bufs]; > - uint32_t compressed_data_size[num_bufs]; > - void *stream =3D NULL; > - char *all_decomp_data =3D NULL; > - unsigned int decomp_produced_data_size =3D 0; > - unsigned int step =3D 0; >=20 > - TEST_ASSERT(decompress_state =3D=3D RTE_COMP_OP_STATELESS || > num_bufs =3D=3D 1, > - "Number of stateful operations in a step should be 1"); > + /* from priv_arrays: */ > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_mempool *buf_pool; >=20 > - if (capa =3D=3D NULL) { > - RTE_LOG(ERR, USER1, > - "Compress device does not support DEFLATE\n"); > - return -ENOTSUP; > - } > + static struct rte_mbuf_ext_shared_info inbuf_info; >=20 > - /* Initialize all arrays to NULL */ > - memset(uncomp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs); > - memset(comp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs); > - memset(ops, 0, sizeof(struct rte_comp_op *) * num_bufs); > - memset(ops_processed, 0, sizeof(struct rte_comp_op *) * > num_bufs); > - memset(priv_xforms, 0, sizeof(void *) * num_bufs); > - memset(compressed_data_size, 0, sizeof(uint32_t) * num_bufs); > - > - if (decompress_state =3D=3D RTE_COMP_OP_STATEFUL) { > + if (test_data->decompress_state =3D=3D RTE_COMP_OP_STATEFUL) { > data_size =3D strlen(test_bufs[0]) + 1; > - all_decomp_data =3D rte_malloc(NULL, data_size, > + *all_decomp_data =3D rte_malloc(NULL, data_size, > RTE_CACHE_LINE_SIZE); > } >=20 > @@ -861,14 +873,15 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > else > buf_pool =3D ts_params->large_mbuf_pool; >=20 > - /* Prepare the source mbufs with the data */ > + /* for compression uncomp_bufs is used as a source buffer */ > + /* allocation from buf_pool (mempool type) */ > ret =3D rte_pktmbuf_alloc_bulk(buf_pool, > uncomp_bufs, num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Source mbufs could not be allocated " > "from the mempool\n"); > - goto exit; > + return -1; > } >=20 > if (test_data->use_external_mbufs) { > @@ -881,8 +894,13 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > test_data->inbuf_memzone->iova, > test_data->inbuf_data_size, > &inbuf_info); > - rte_pktmbuf_append(uncomp_bufs[i], > + buf_ptr =3D rte_pktmbuf_append(uncomp_bufs[i], > test_data->inbuf_data_size); > + if (buf_ptr =3D=3D NULL) { > + RTE_LOG(ERR, USER1, > + "Append extra bytes to the source > mbuf failed\n"); > + return -1; > + } > } > } else if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D SGL_TO_LB) { > for (i =3D 0; i < num_bufs; i++) { > @@ -893,106 +911,296 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > big_data ? buf_pool : ts_params- > >large_mbuf_pool, > big_data ? 0 : MAX_SEGS, > big_data ? MAX_DATA_MBUF_SIZE : > SMALL_SEG_SIZE) < 0) > - goto exit; > + return -1; > } > } else { > for (i =3D 0; i < num_bufs; i++) { > data_size =3D strlen(test_bufs[i]) + 1; > + > buf_ptr =3D rte_pktmbuf_append(uncomp_bufs[i], > data_size); > if (buf_ptr =3D=3D NULL) { > RTE_LOG(ERR, USER1, > "Append extra bytes to the source > mbuf failed\n"); > - goto exit; > + return -1; > } > strlcpy(buf_ptr, test_bufs[i], data_size); > } > } >=20 > - /* Prepare the destination mbufs */ > - ret =3D rte_pktmbuf_alloc_bulk(buf_pool, comp_bufs, num_bufs); > + return 0; > +} > + > +/** > + * Data size calculation (for both compression and decompression). > + * > + * Developer is requested to provide input params > + * according to the following rule: > + * if ops_processed =3D=3D NULL -> compression > + * if ops_processed !=3D NULL -> decompression > + * Values bigger than 0 have to be returned to avoid problems > + * with memory allocation > + * > + * @param ops_processed > + * Operations created as a result of compression phase. Should be > + * set to NULL for compression > + * @param out_of_space_and_zlib > + * Boolean value to switch into "out of space" buffer if set. > + * To test "out-of-space" data size, zlib_decompress must be set as we= ll. > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param i > + * current buffer index > + * @return > + * - values bigger than 0 > + */ > +static inline uint32_t > +test_mbufs_calculate_data_size( > + struct rte_comp_op *ops_processed[], /*can be equal to > NULL*/ > + unsigned int out_of_space_and_zlib, > + const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + unsigned int i) > +{ > + /* local variables: */ > + uint32_t data_size =3D 0; > + struct priv_op_data *priv_data; > + float ratio; > + uint8_t not_zlib_compr; /* true if zlib isn't current compression dev > */ > + enum overflow_test overflow =3D test_data->overflow; > + > + /* from int_data: */ > + const char * const *test_bufs =3D int_data->test_bufs; > + > + if (out_of_space_and_zlib) > + data_size =3D OUT_OF_SPACE_BUF; > + else { > + if (ops_processed =3D=3D NULL) { > + > + not_zlib_compr =3D (test_data->zlib_dir =3D=3D > ZLIB_DECOMPRESS > + || test_data->zlib_dir =3D=3D ZLIB_NONE); > + > + ratio =3D (not_zlib_compr && > + (overflow =3D=3D OVERFLOW_ENABLED)) ? > + COMPRESS_BUF_SIZE_RATIO_OVERFLOW : > + COMPRESS_BUF_SIZE_RATIO; > + > + data_size =3D strlen(test_bufs[i]) * ratio; > + > + } else { > + priv_data =3D (struct priv_op_data *) > + (ops_processed[i] + 1); > + data_size =3D strlen(test_bufs[priv_data->orig_idx]) + > 1; > + } > + } > + > + return data_size; > +} > + > + > +/** > + * Memory buffers preparation (for both compression and decompression). > + * > + * Memory allocation for comp/decomp buffers from mempool, depending > on > + * ops_processed value. Developer is requested to provide input params > + * according to the following rule: > + * if ops_processed =3D=3D NULL -> current_bufs =3D comp_bufs[] > + * if ops_processed !=3D NULL -> current_bufs =3D decomp_bufs[] > + * -1 returned if function fail, without modifying the mbuf. > + * > + * @param ops_processed > + * Operations created as a result of compression phase. Should be > + * set to NULL for compression > + * @param current_bufs > + * mbufs being prepared in the function. > + * @param out_of_space_and_zlib > + * Boolean value to switch into "out of space" buffer if set. > + * To test "out-of-space" data size, zlib_decompress must be set as we= ll. > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param current_extbuf_info, > + * The structure containing all the information related to external mb= ufs > + * @param current_memzone > + * memzone used by external buffers > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_mbufs_destination_preparation( > + struct rte_comp_op *ops_processed[], /*can be equal to > NULL*/ > + struct rte_mbuf *current_bufs[], > + unsigned int out_of_space_and_zlib, > + const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + struct rte_mbuf_ext_shared_info *current_extbuf_info, > + const struct rte_memzone *current_memzone) { > + /* local variables: */ > + unsigned int i; > + uint32_t data_size; > + int ret; > + char *buf_ptr; > + > + struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from int_data: */ > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from test_data: */ > + unsigned int buff_type =3D test_data->buff_type; > + unsigned int big_data =3D test_data->big_data; > + > + struct rte_mempool *buf_pool; > + > + if (big_data) > + buf_pool =3D ts_params->big_mbuf_pool; > + else if (buff_type =3D=3D SGL_BOTH) > + buf_pool =3D ts_params->small_mbuf_pool; > + else > + buf_pool =3D ts_params->large_mbuf_pool; > + > + /* the mbufs allocation*/ > + ret =3D rte_pktmbuf_alloc_bulk(buf_pool, current_bufs, num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Destination mbufs could not be allocated " > "from the mempool\n"); > - goto exit; > + return -1; > } >=20 > if (test_data->use_external_mbufs) { > - compbuf_info.free_cb =3D extbuf_free_callback; > - compbuf_info.fcb_opaque =3D NULL; > - rte_mbuf_ext_refcnt_set(&compbuf_info, 1); > + current_extbuf_info->free_cb =3D extbuf_free_callback; > + current_extbuf_info->fcb_opaque =3D NULL; > + rte_mbuf_ext_refcnt_set(current_extbuf_info, 1); > for (i =3D 0; i < num_bufs; i++) { > - rte_pktmbuf_attach_extbuf(comp_bufs[i], > - test_data->compbuf_memzone- > >addr, > - test_data->compbuf_memzone- > >iova, > - test_data->compbuf_memzone- > >len, > - &compbuf_info); > - rte_pktmbuf_append(comp_bufs[i], > - test_data->compbuf_memzone- > >len); > + rte_pktmbuf_attach_extbuf(current_bufs[i], > + current_memzone->addr, > + current_memzone->iova, > + current_memzone->len, > + current_extbuf_info); > + rte_pktmbuf_append(current_bufs[i], > + current_memzone->len); > } > - } else if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D LB_TO_SGL) { > - for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space =3D=3D 1 && oos_zlib_decompress) > - data_size =3D OUT_OF_SPACE_BUF; > - else > - (data_size =3D strlen(test_bufs[i]) * > - COMPRESS_BUF_SIZE_RATIO); > - > - if (prepare_sgl_bufs(NULL, comp_bufs[i], > - data_size, > - big_data ? buf_pool : ts_params- > >small_mbuf_pool, > - big_data ? buf_pool : ts_params- > >large_mbuf_pool, > - big_data ? 0 : MAX_SEGS, > - big_data ? MAX_DATA_MBUF_SIZE : > SMALL_SEG_SIZE) > - < 0) > - goto exit; > - } > - > } else { > for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space =3D=3D 1 && oos_zlib_decompress) > - data_size =3D OUT_OF_SPACE_BUF; > - else { > - float ratio =3D > - ((test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS > || > - test_data->zlib_dir =3D=3D ZLIB_NONE) && > - overflow_tst =3D=3D OVERFLOW_ENABLED) ? > - > COMPRESS_BUF_SIZE_RATIO_OVERFLOW : > - COMPRESS_BUF_SIZE_RATIO; > - > - data_size =3D strlen(test_bufs[i]) * ratio; > - } > - buf_ptr =3D rte_pktmbuf_append(comp_bufs[i], > data_size); > - if (buf_ptr =3D=3D NULL) { > - RTE_LOG(ERR, USER1, > - "Append extra bytes to the > destination mbuf failed\n"); > - goto exit; > + > + /* data size calculation */ > + data_size =3D test_mbufs_calculate_data_size( > + ops_processed, > + out_of_space_and_zlib, > + int_data, > + test_data, > + i); > + > + /* data allocation */ > + if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D > LB_TO_SGL) { > + ret =3D prepare_sgl_bufs(NULL, > current_bufs[i], > + data_size, > + big_data ? buf_pool : > + ts_params- > >small_mbuf_pool, > + big_data ? buf_pool : > + ts_params- > >large_mbuf_pool, > + big_data ? 0 : MAX_SEGS, > + big_data ? MAX_DATA_MBUF_SIZE : > + SMALL_SEG_SIZE); > + if (ret < 0) > + return -1; > + } else { > + buf_ptr =3D > rte_pktmbuf_append(current_bufs[i], > + data_size); > + if (buf_ptr =3D=3D NULL) { > + RTE_LOG(ERR, USER1, > + "Append extra bytes to the > destination mbuf failed\n"); > + return -1; > + } > } > } > } >=20 > + return 0; > +} > + > +/** > + * The main compression function. > + * > + * Operation(s) configuration, depending on CLI parameters. > + * Operation(s) processing. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_comp_run(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + struct priv_op_data *priv_data; > + unsigned int i; > + uint16_t num_priv_xforms =3D 0; > + int ret; > + int ret_status =3D 0; > + char *buf_ptr; > + > + struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from test_data: */ > + enum rte_comp_op_type operation_type =3D test_data- > >compress_state; > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + > + /* from int_data: */ > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + unsigned int num_xforms =3D int_data->num_xforms; > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from priv_arrays: */ > + struct rte_mbuf **comp_bufs =3D priv_arrays->comp_bufs; > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + void **priv_xforms =3D priv_arrays->priv_xforms; > + > + const struct rte_compressdev_capabilities *capa =3D > + rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); > + > /* Build the compression operations */ > ret =3D rte_comp_op_bulk_alloc(ts_params->op_pool, ops, > num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Compress operations could not be allocated " > "from the mempool\n"); > + ret_status =3D -1; > goto exit; > } >=20 > - > for (i =3D 0; i < num_bufs; i++) { > ops[i]->m_src =3D uncomp_bufs[i]; > ops[i]->m_dst =3D comp_bufs[i]; > ops[i]->src.offset =3D 0; > ops[i]->src.length =3D rte_pktmbuf_pkt_len(uncomp_bufs[i]); > ops[i]->dst.offset =3D 0; > - if (compress_state =3D=3D RTE_COMP_OP_STATELESS) > + > + if (operation_type =3D=3D RTE_COMP_OP_STATELESS) { > ops[i]->flush_flag =3D RTE_COMP_FLUSH_FINAL; > - else { > + } else { > RTE_LOG(ERR, USER1, > - "Stateful operations are not supported " > - "in these tests yet\n"); > + "Compression: stateful operations are not " > + "supported in these tests yet\n"); > + ret_status =3D -1; > goto exit; > } > ops[i]->input_chksum =3D 0; > @@ -1007,14 +1215,16 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > } >=20 > /* Compress data (either with Zlib API or compressdev API */ > - if (zlib_dir =3D=3D ZLIB_COMPRESS || zlib_dir =3D=3D ZLIB_ALL) { > + if (zlib_compress) { > for (i =3D 0; i < num_bufs; i++) { > const struct rte_comp_xform *compress_xform =3D > compress_xforms[i % num_xforms]; > ret =3D compress_zlib(ops[i], compress_xform, > DEFAULT_MEM_LEVEL); > - if (ret < 0) > + if (ret < 0) { > + ret_status =3D -1; > goto exit; > + } >=20 > ops_processed[i] =3D ops[i]; > } > @@ -1022,24 +1232,26 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > /* Create compress private xform data */ > for (i =3D 0; i < num_xforms; i++) { > ret =3D rte_compressdev_private_xform_create(0, > - (const struct rte_comp_xform > *)compress_xforms[i], > + (const struct rte_comp_xform *) > + compress_xforms[i], > &priv_xforms[i]); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Compression private xform " > "could not be created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > } > - > if (capa->comp_feature_flags & > RTE_COMP_FF_SHAREABLE_PRIV_XFORM) { > /* Attach shareable private xform data to ops */ > for (i =3D 0; i < num_bufs; i++) > - ops[i]->private_xform =3D priv_xforms[i % > num_xforms]; > + ops[i]->private_xform =3D > + priv_xforms[i % > num_xforms]; > } else { > - /* Create rest of the private xforms for the other ops > */ > + /* Create rest of the private xforms for the other ops */ > for (i =3D num_xforms; i < num_bufs; i++) { > ret =3D > rte_compressdev_private_xform_create(0, > compress_xforms[i % num_xforms], > @@ -1048,26 +1260,28 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > RTE_LOG(ERR, USER1, > "Compression private xform " > "could not be created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > } > - > /* Attach non shareable private xform data to ops */ > for (i =3D 0; i < num_bufs; i++) > ops[i]->private_xform =3D priv_xforms[i]; > } >=20 > recovery_lb: > - ret =3D test_run_enqueue_dequeue(ops, num_bufs, > ops_processed); > + ret =3D test_run_enqueue_dequeue(ops, ops_processed, > num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > - "Enqueue/dequeue operation failed\n"); > + "Compression: enqueue/dequeue operation > failed\n"); > + ret_status =3D -1; > goto exit; > } >=20 > for (i =3D 0; i < num_bufs; i++) { > - compressed_data_size[i] +=3D ops_processed[i]- > >produced; > + priv_arrays->compressed_data_size[i] +=3D > + ops_processed[i]->produced; >=20 > if (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE) { @@ - > 1088,21 +1302,83 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > if (buf_ptr =3D=3D NULL) { > RTE_LOG(ERR, USER1, > "Data recovery: append extra > bytes to the current mbuf failed\n"); > + ret_status =3D -1; > goto exit; > } > goto recovery_lb; > } > } > - deqd_retries =3D 0; > + } > + > +exit: > + /* Free resources */ > + if (ret_status < 0) > + for (i =3D 0; i < num_bufs; i++) { > + rte_comp_op_free(ops[i]); > + ops_processed[i] =3D NULL; > + } >=20 > - /* Free compress private xforms */ > - for (i =3D 0; i < num_priv_xforms; i++) { > + /* Free compress private xforms */ > + for (i =3D 0; i < num_priv_xforms; i++) { > + if (priv_xforms[i] !=3D NULL) { > rte_compressdev_private_xform_free(0, > priv_xforms[i]); > priv_xforms[i] =3D NULL; > } > - num_priv_xforms =3D 0; > } >=20 > + return ret_status; > +} > + > +/** > + * Prints out the test report. Memory freeing. > + * > + * Called after successful compression. > + * Operation(s) status validation and decompression buffers freeing. > + > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 2: Some operation is not supported > + * - 1: Decompression should be skipped > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_comp_finalize(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + unsigned int i; > + struct priv_op_data *priv_data; > + > + /* from int_data: */ > + unsigned int num_xforms =3D int_data->num_xforms; > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + uint16_t *buf_idx =3D int_data->buf_idx; > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from priv_arrays: */ > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + uint64_t *compress_checksum =3D priv_arrays->compress_checksum; > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + > + /* from test_data: */ > + unsigned int out_of_space =3D test_data->out_of_space; > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); > + > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > uint16_t xform_idx =3D priv_data->orig_idx % num_xforms; > @@ -1111,7 +1387,7 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > enum rte_comp_huffman huffman_type =3D > compress_xform->deflate.huffman; > char engine[] =3D "zlib (directly, not PMD)"; > - if (zlib_dir !=3D ZLIB_COMPRESS && zlib_dir !=3D ZLIB_ALL) > + if (zlib_decompress) > strlcpy(engine, "PMD", sizeof(engine)); >=20 > RTE_LOG(DEBUG, USER1, "Buffer %u compressed by %s from > %u to" > @@ -1134,116 +1410,103 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > * compress operation information is needed for the decompression > stage) > */ > for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space && oos_zlib_decompress) { > + if (out_of_space && !zlib_compress) { > if (ops_processed[i]->status !=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > - ret_status =3D TEST_FAILED; > RTE_LOG(ERR, USER1, > "Operation without expected out of " > "space status error\n"); > - goto exit; > + return -1; > } else > continue; > } >=20 > if (ops_processed[i]->status !=3D > RTE_COMP_OP_STATUS_SUCCESS) { > - if (overflow_tst =3D=3D OVERFLOW_ENABLED) { > + if (test_data->overflow =3D=3D OVERFLOW_ENABLED) { > if (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > - ret_status =3D 1; > RTE_LOG(INFO, USER1, > "Out-of-space-recoverable > functionality" > " is not supported on this device\n"); > - goto exit; > + return 2; > } > } > + > RTE_LOG(ERR, USER1, > "Some operations were not successful\n"); > - goto exit; > + return -1; > } > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > rte_pktmbuf_free(uncomp_bufs[priv_data->orig_idx]); > uncomp_bufs[priv_data->orig_idx] =3D NULL; > } >=20 > - if (out_of_space && oos_zlib_decompress) { > - ret_status =3D TEST_SUCCESS; > - goto exit; > - } > + if (out_of_space && !zlib_compress) > + return 1; >=20 > - /* Allocate buffers for decompressed data */ > - ret =3D rte_pktmbuf_alloc_bulk(buf_pool, uncomp_bufs, num_bufs); > - if (ret < 0) { > - RTE_LOG(ERR, USER1, > - "Destination mbufs could not be allocated " > - "from the mempool\n"); > - goto exit; > - } > + return 0; > +} >=20 > - if (test_data->use_external_mbufs) { > - decompbuf_info.free_cb =3D extbuf_free_callback; > - decompbuf_info.fcb_opaque =3D NULL; > - rte_mbuf_ext_refcnt_set(&decompbuf_info, 1); > - for (i =3D 0; i < num_bufs; i++) { > - rte_pktmbuf_attach_extbuf(uncomp_bufs[i], > - test_data->uncompbuf_memzone- > >addr, > - test_data->uncompbuf_memzone- > >iova, > - test_data->uncompbuf_memzone- > >len, > - &decompbuf_info); > - rte_pktmbuf_append(uncomp_bufs[i], > - test_data->uncompbuf_memzone- > >len); > - } > - } else if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D LB_TO_SGL) { > - for (i =3D 0; i < num_bufs; i++) { > - priv_data =3D (struct priv_op_data *) > - (ops_processed[i] + 1); > - if (out_of_space =3D=3D 1 && oos_zlib_compress) > - data_size =3D OUT_OF_SPACE_BUF; > - else if (test_data->decompress_output_block_size > !=3D 0) > - data_size =3D > - test_data- > >decompress_output_block_size; > - else > - data_size =3D > - strlen(test_bufs[priv_data->orig_idx]) + 1; > - > - if (prepare_sgl_bufs(NULL, uncomp_bufs[i], > - data_size, > - big_data ? buf_pool : ts_params- > >small_mbuf_pool, > - big_data ? buf_pool : ts_params- > >large_mbuf_pool, > - big_data ? 0 : MAX_SEGS, > - big_data ? MAX_DATA_MBUF_SIZE : > SMALL_SEG_SIZE) > - < 0) > - goto exit; > - } > +/** > + * The main decompression function. > + * > + * Operation(s) configuration, depending on CLI parameters. > + * Operation(s) processing. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_decomp_run(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + struct test_private_arrays *priv_arrays) { >=20 > - } else { > - for (i =3D 0; i < num_bufs; i++) { > - priv_data =3D (struct priv_op_data *) > - (ops_processed[i] + 1); > - if (out_of_space =3D=3D 1 && oos_zlib_compress) > - data_size =3D OUT_OF_SPACE_BUF; > - else if (test_data->decompress_output_block_size > !=3D 0) > - data_size =3D > - test_data- > >decompress_output_block_size; > - else > - data_size =3D > - strlen(test_bufs[priv_data->orig_idx]) + 1; > + /* local variables: */ > + struct priv_op_data *priv_data; > + unsigned int i; > + uint16_t num_priv_xforms =3D 0; > + int ret; > + int ret_status =3D 0; >=20 > - buf_ptr =3D rte_pktmbuf_append(uncomp_bufs[i], > data_size); > - if (buf_ptr =3D=3D NULL) { > - RTE_LOG(ERR, USER1, > - "Append extra bytes to the > decompressed mbuf failed\n"); > - goto exit; > - } > - } > - } > + struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from test_data: */ > + enum rte_comp_op_type operation_type =3D test_data- > >decompress_state; > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); > + > + /* from int_data: */ > + struct rte_comp_xform **decompress_xforms =3D int_data- > >decompress_xforms; > + unsigned int num_xforms =3D int_data->num_xforms; > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from priv_arrays: */ > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + void **priv_xforms =3D priv_arrays->priv_xforms; > + uint32_t *compressed_data_size =3D priv_arrays- > >compressed_data_size; > + void **stream =3D priv_arrays->stream; > + > + const struct rte_compressdev_capabilities *capa =3D > + rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); >=20 > - /* Build the decompression operations */ > ret =3D rte_comp_op_bulk_alloc(ts_params->op_pool, ops, > num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Decompress operations could not be allocated " > "from the mempool\n"); > + ret_status =3D -1; > goto exit; > } >=20 > @@ -1256,22 +1519,25 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > * Set the length of the compressed data to the > * number of bytes that were produced in the previous stage > */ > + > if (compressed_data_size[i]) > ops[i]->src.length =3D compressed_data_size[i]; > else > ops[i]->src.length =3D ops_processed[i]->produced; >=20 > ops[i]->dst.offset =3D 0; > - if (decompress_state =3D=3D RTE_COMP_OP_STATELESS) { > + > + if (operation_type =3D=3D RTE_COMP_OP_STATELESS) { > ops[i]->flush_flag =3D RTE_COMP_FLUSH_FINAL; > ops[i]->op_type =3D RTE_COMP_OP_STATELESS; > - } else if (zlib_dir =3D=3D ZLIB_COMPRESS || zlib_dir =3D=3D > ZLIB_NONE) { > + } else if (!zlib_decompress) { > ops[i]->flush_flag =3D RTE_COMP_FLUSH_SYNC; > ops[i]->op_type =3D RTE_COMP_OP_STATEFUL; > } else { > RTE_LOG(ERR, USER1, > - "Stateful operations are not supported " > - "in these tests yet\n"); > + "Decompression: stateful operations are" > + " not supported in these tests yet\n"); > + ret_status =3D -1; > goto exit; > } > ops[i]->input_chksum =3D 0; > @@ -1290,7 +1556,7 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > rte_comp_op_bulk_free(ops_processed, num_bufs); >=20 > /* Decompress data (either with Zlib API or compressdev API */ > - if (zlib_dir =3D=3D ZLIB_DECOMPRESS || zlib_dir =3D=3D ZLIB_ALL) { > + if (zlib_decompress) { > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops[i] + 1); > uint16_t xform_idx =3D priv_data->orig_idx % > num_xforms; @@ -1298,13 +1564,15 @@ test_deflate_comp_decomp(const > struct interim_data_params *int_data, > decompress_xforms[xform_idx]; >=20 > ret =3D decompress_zlib(ops[i], decompress_xform); > - if (ret < 0) > + if (ret < 0) { > + ret_status =3D -1; > goto exit; > + } >=20 > ops_processed[i] =3D ops[i]; > } > } else { > - if (decompress_state =3D=3D RTE_COMP_OP_STATELESS) { > + if (operation_type =3D=3D RTE_COMP_OP_STATELESS) { > /* Create decompress private xform data */ > for (i =3D 0; i < num_xforms; i++) { > ret =3D > rte_compressdev_private_xform_create(0, > @@ -1315,6 +1583,7 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > RTE_LOG(ERR, USER1, > "Decompression private > xform " > "could not be created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > @@ -1341,7 +1610,9 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > &priv_xforms[i]); > if (ret < 0) { > RTE_LOG(ERR, USER1, > - "Decompression > private xform could not be created\n"); > + "Decompression > private xform" > + " could not be > created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > @@ -1361,59 +1632,86 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > } else { > /* Create a stream object for stateful decompression > */ > ret =3D rte_compressdev_stream_create(0, > - decompress_xforms[0], &stream); > + decompress_xforms[0], stream); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Decompression stream could not be > created, error %d\n", > ret); > + ret_status =3D -1; > goto exit; > } > /* Attach stream to ops */ > for (i =3D 0; i < num_bufs; i++) > - ops[i]->stream =3D stream; > + ops[i]->stream =3D *stream; > } >=20 > -next_step: > - /* Enqueue and dequeue all operations */ > - num_enqd =3D rte_compressdev_enqueue_burst(0, 0, ops, > num_bufs); > - if (num_enqd < num_bufs) { > - RTE_LOG(ERR, USER1, > - "The operations could not be enqueued\n"); > - goto exit; > - } > + priv_arrays->num_priv_xforms =3D num_priv_xforms; > + } >=20 > - num_total_deqd =3D 0; > - do { > - /* > - * If retrying a dequeue call, wait for 10 ms to allow > - * enough time to the driver to process the > operations > - */ > - if (deqd_retries !=3D 0) { > - /* > - * Avoid infinite loop if not all the > - * operations get out of the device > - */ > - if (deqd_retries =3D=3D MAX_DEQD_RETRIES) { > - RTE_LOG(ERR, USER1, > - "Not all operations could be " > - "dequeued\n"); > - goto exit; > - } > - usleep(DEQUEUE_WAIT_TIME); > - } > - num_deqd =3D rte_compressdev_dequeue_burst(0, 0, > - &ops_processed[num_total_deqd], > num_bufs); > - num_total_deqd +=3D num_deqd; > - deqd_retries++; > - } while (num_total_deqd < num_enqd); > +exit: > + return ret_status; > +} >=20 > - deqd_retries =3D 0; > - } > +/** > + * Prints out the test report. Memory freeing. > + * > + * Called after successful decompression. > + * Operation(s) status validation and compression buffers freeing. > + > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 2: Next step must be executed by the caller (stateful decompressi= on > only) > + * - 1: On success (caller should stop and exit) > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_decomp_finalize(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + unsigned int i; > + struct priv_op_data *priv_data; > + static unsigned int step; > + > + /* from int_data: */ > + uint16_t *buf_idx =3D int_data->buf_idx; > + unsigned int num_bufs =3D int_data->num_bufs; > + const char * const *test_bufs =3D int_data->test_bufs; > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + > + /* from priv_arrays: */ > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + struct rte_mbuf **comp_bufs =3D priv_arrays->comp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + uint64_t *compress_checksum =3D priv_arrays->compress_checksum; > + unsigned int *decomp_produced_data_size =3D > + priv_arrays->decomp_produced_data_size; > + char **all_decomp_data =3D priv_arrays->all_decomp_data; > + > + /* from test_data: */ > + unsigned int out_of_space =3D test_data->out_of_space; > + enum rte_comp_op_type operation_type =3D test_data- > >decompress_state; > + > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); >=20 > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > char engine[] =3D "zlib, (directly, no PMD)"; > - if (zlib_dir !=3D ZLIB_DECOMPRESS && zlib_dir !=3D ZLIB_ALL) > + if (zlib_compress) > strlcpy(engine, "pmd", sizeof(engine)); > RTE_LOG(DEBUG, USER1, > "Buffer %u decompressed by %s from %u to %u > bytes\n", @@ -1427,19 +1725,19 @@ test_deflate_comp_decomp(const > struct interim_data_params *int_data, > * compress operation information is still needed) > */ > for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space && oos_zlib_compress) { > + if (out_of_space && !zlib_decompress) { > if (ops_processed[i]->status !=3D > - > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > - ret_status =3D TEST_FAILED; > + > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > + > RTE_LOG(ERR, USER1, > "Operation without expected out of " > "space status error\n"); > - goto exit; > + return -1; > } else > continue; > } >=20 > - if (decompress_state =3D=3D RTE_COMP_OP_STATEFUL > + if (operation_type =3D=3D RTE_COMP_OP_STATEFUL > && (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE > || ops_processed[i]->status =3D=3D > @@ -1449,28 +1747,29 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > ops_processed[i]->m_dst, > ops_processed[i]->dst.offset, > ops_processed[i]->produced, > - all_decomp_data + > - > decomp_produced_data_size); > - if (ptr !=3D all_decomp_data + > decomp_produced_data_size) > - rte_memcpy(all_decomp_data + > - decomp_produced_data_size, > + *all_decomp_data + > + > *decomp_produced_data_size); > + if (ptr !=3D *all_decomp_data + > + *decomp_produced_data_size) > + rte_memcpy(*all_decomp_data + > + *decomp_produced_data_size, > ptr, ops_processed[i]->produced); > - decomp_produced_data_size +=3D ops_processed[i]- > >produced; > + > + *decomp_produced_data_size +=3D > + ops_processed[i]->produced; > if (ops_processed[i]->src.length > > ops_processed[i]->consumed) { > if (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_SUCCESS) { > - ret_status =3D -1; > RTE_LOG(ERR, USER1, > "Operation finished too early\n"); > - goto exit; > + return -1; > } > step++; > if (step >=3D test_data- > >decompress_steps_max) { > - ret_status =3D -1; > RTE_LOG(ERR, USER1, > "Operation exceeded maximum > steps\n"); > - goto exit; > + return -1; > } > ops[i] =3D ops_processed[i]; > ops[i]->status =3D > @@ -1479,7 +1778,9 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > ops_processed[i]- > >consumed; > ops[i]->src.length -=3D > ops_processed[i]- > >consumed; > - goto next_step; > + /* repeat the operation */ > + //goto next_step; > + return 2; > } else { > /* Compare the original stream with the */ > /* decompressed stream (in size and the > data) */ @@ -1487,22 +1788,22 @@ test_deflate_comp_decomp(const > struct interim_data_params *int_data, > (ops_processed[i] + 1); > const char *buf1 =3D > test_bufs[priv_data- > >orig_idx]; > - const char *buf2 =3D all_decomp_data; > + const char *buf2 =3D *all_decomp_data; >=20 > if (compare_buffers(buf1, strlen(buf1) + 1, > - buf2, decomp_produced_data_size) > < 0) > - goto exit; > + buf2, > *decomp_produced_data_size) < 0) > + return -1; > /* Test checksums */ > if (compress_xforms[0]->compress.chksum > !=3D > RTE_COMP_CHECKSUM_NONE) { > if (ops_processed[i]- > >output_chksum > !=3D compress_checksum[i]) > { > RTE_LOG(ERR, USER1, > - "The checksums > differ\n" > - "Compression Checksum: %" PRIu64 > "\tDecompression " > - "Checksum: %" PRIu64 "\n", > compress_checksum[i], > + "The checksums differ\n" > + "Compression Checksum: %" PRIu64 > "\tDecompression " > + "Checksum: %" PRIu64 "\n", compress_checksum[i], > ops_processed[i]- > >output_chksum); > - goto exit; > + return -1; > } > } > } > @@ -1510,18 +1811,58 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > RTE_COMP_OP_STATUS_SUCCESS) { > RTE_LOG(ERR, USER1, > "Some operations were not successful\n"); > - goto exit; > + return -1; > } > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > rte_pktmbuf_free(comp_bufs[priv_data->orig_idx]); > comp_bufs[priv_data->orig_idx] =3D NULL; > } >=20 > - if ((out_of_space && oos_zlib_compress) > - || (decompress_state =3D=3D > RTE_COMP_OP_STATEFUL)) { > - ret_status =3D TEST_SUCCESS; > - goto exit; > - } > + if (out_of_space && !zlib_decompress) > + return 1; > + > + return 0; > +} > + > +/** > + * Validation of the output (compression/decompression) data. > + * > + * The function compares the source stream with the output stream, > + * after decompression, to check if compression/decompression > + * was correct. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_results_validation(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + unsigned int i; > + struct priv_op_data *priv_data; > + const char *buf1; > + const char *buf2; > + char *contig_buf =3D NULL; > + uint32_t data_size; > + > + /* from int_data: */ > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + unsigned int num_bufs =3D int_data->num_bufs; > + const char * const *test_bufs =3D int_data->test_bufs; > + > + /* from priv_arrays: */ > + uint64_t *compress_checksum =3D priv_arrays->compress_checksum; > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; >=20 > /* > * Compare the original stream with the decompressed stream @@ - > 1529,13 +1870,13 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > */ > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > - const char *buf1 =3D test_data->use_external_mbufs ? > + buf1 =3D test_data->use_external_mbufs ? > test_data->inbuf_memzone->addr : > test_bufs[priv_data->orig_idx]; > - const char *buf2; > data_size =3D test_data->use_external_mbufs ? > test_data->inbuf_data_size : > strlen(buf1) + 1; > + > contig_buf =3D rte_malloc(NULL, ops_processed[i]->produced, > 0); > if (contig_buf =3D=3D NULL) { > RTE_LOG(ERR, USER1, "Contiguous buffer could not " > @@ -1565,24 +1906,207 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > rte_free(contig_buf); > contig_buf =3D NULL; > } > + return 0; > + > +exit: > + rte_free(contig_buf); > + return -1; > +} > + > +/** > + * Compresses and decompresses input stream with compressdev API and > +Zlib API > + * > + * Basic test function. Common for all the functional tests. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @return > + * - 1: Some operation not supported > + * - 0: On success. > + * - -1: On error. > + */ > + > +static int > +test_deflate_comp_decomp(const struct interim_data_params *int_data, > + const struct test_data_params *test_data) { > + unsigned int num_bufs =3D int_data->num_bufs; > + unsigned int out_of_space =3D test_data->out_of_space; > + > + void *stream =3D NULL; > + char *all_decomp_data =3D NULL; > + unsigned int decomp_produced_data_size =3D 0; > + > + int ret_status =3D -1; > + int ret; > + struct rte_mbuf *uncomp_bufs[num_bufs]; > + struct rte_mbuf *comp_bufs[num_bufs]; > + struct rte_comp_op *ops[num_bufs]; > + struct rte_comp_op *ops_processed[num_bufs]; > + void *priv_xforms[num_bufs]; > + unsigned int i; > + > + uint64_t compress_checksum[num_bufs]; > + uint32_t compressed_data_size[num_bufs]; > + char *contig_buf =3D NULL; > + > + struct rte_mbuf_ext_shared_info compbuf_info; > + struct rte_mbuf_ext_shared_info decompbuf_info; > + > + const struct rte_compressdev_capabilities *capa; > + > + /* Compressing with CompressDev */ > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); > + > + struct test_private_arrays priv_arrays; > + > + priv_arrays.uncomp_bufs =3D uncomp_bufs; > + priv_arrays.comp_bufs =3D comp_bufs; > + priv_arrays.ops =3D ops; > + priv_arrays.ops_processed =3D ops_processed; > + priv_arrays.priv_xforms =3D priv_xforms; > + priv_arrays.compress_checksum =3D compress_checksum; > + priv_arrays.compressed_data_size =3D compressed_data_size; > + > + priv_arrays.stream =3D &stream; > + priv_arrays.all_decomp_data =3D &all_decomp_data; > + priv_arrays.decomp_produced_data_size =3D > &decomp_produced_data_size; > + > + priv_arrays.num_priv_xforms =3D 0; /* it's used for deompression only > */ > + > + capa =3D rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); > + if (capa =3D=3D NULL) { > + RTE_LOG(ERR, USER1, > + "Compress device does not support DEFLATE\n"); > + return -1; > + } > + test_objects_init(&priv_arrays, num_bufs); Do we need this? Why we just cant call source_prep API? That will also setu= p uncompressed_bufs [Artur] This is my approach. Do you prefer to have arrays initialisation mo= ved to test_mbufs_source_preparation?=20 If "yes", test_objects_init will be removed from the code. > + > + /* Prepare the source mbufs with the data */ > + ret =3D test_mbufs_source_preparation(int_data, test_data, > &priv_arrays); For code readability sake, it could test_setup_uncom_bufs()=20 [Artur] Good idea. Would it be ok if I change: 1. test_mbufs_destination_preparation into test_setup_uncom_bufs 2. test_mbufs_source_preparation into test_setup_com_bufs > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > +/* COMPRESSION */ > + > + /* Prepare the destination mbufs */ > + ret =3D test_mbufs_destination_preparation( Why we cant keep this prototype same as source_preparation() API for better= readability? I mean we can pass priv_arry here too... [Artur] To get better readability it has been implemented in this way. That= was only my point of view but I don't mind to modify the code according to= your suggestion. It's also ok. > + NULL, /*can be equal to NULL*/ > + comp_bufs, > + out_of_space =3D=3D 1 && !zlib_compress, > + int_data, > + test_data, > + &compbuf_info, > + test_data->compbuf_memzone); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + /* Run compression */ > + ret =3D test_deflate_comp_run(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + ret =3D test_deflate_comp_finalize(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } else if (ret =3D=3D 1) { > + ret_status =3D 0; > + goto exit; > + } else if (ret =3D=3D 2) { > + ret_status =3D 1; /* some operation not supported */ > + goto exit; > + } > + > +/* DECOMPRESSION */ > + > + /* Allocate buffers for decompressed data */ > + ret =3D test_mbufs_destination_preparation( > + ops_processed, /*can be equal to NULL*/ > + uncomp_bufs, > + out_of_space =3D=3D 1 && !zlib_decompress, > + int_data, > + test_data, > + &decompbuf_info, > + test_data->uncompbuf_memzone); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + /* Run decompression */ > + ret =3D test_deflate_decomp_run(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + if (!zlib_decompress) { > +next_step: /* next step for stateful decompression only */ > + ret =3D test_run_enqueue_dequeue(ops, ops_processed, > num_bufs); > + if (ret < 0) { > + ret_status =3D -1; > + RTE_LOG(ERR, USER1, > + "Decompression: enqueue/dequeue > operation failed\n"); > + } > + } >=20 > - ret_status =3D TEST_SUCCESS; > + ret =3D test_deflate_decomp_finalize(int_data, test_data, > &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } else if (ret =3D=3D 1) { > + ret_status =3D 0; > + goto exit; > + } else if (ret =3D=3D 2) { > + goto next_step; > + } > + > +/* FINAL PROCESSING */ > + > + ret =3D test_results_validation(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + ret_status =3D 0; >=20 > exit: > /* Free resources */ > + > + if (stream !=3D NULL) > + rte_compressdev_stream_free(0, stream); > + if (all_decomp_data !=3D NULL) > + rte_free(all_decomp_data); > + > + /* Free compress private xforms */ > + for (i =3D 0; i < priv_arrays.num_priv_xforms; i++) { > + if (priv_xforms[i] !=3D NULL) { > + rte_compressdev_private_xform_free(0, > priv_xforms[i]); > + priv_xforms[i] =3D NULL; > + } > + } > + > for (i =3D 0; i < num_bufs; i++) { > rte_pktmbuf_free(uncomp_bufs[i]); > rte_pktmbuf_free(comp_bufs[i]); > rte_comp_op_free(ops[i]); > rte_comp_op_free(ops_processed[i]); > } > - for (i =3D 0; i < num_priv_xforms; i++) > - if (priv_xforms[i] !=3D NULL) > - rte_compressdev_private_xform_free(0, > priv_xforms[i]); > - if (stream !=3D NULL) > - rte_compressdev_stream_free(0, stream); > - if (all_decomp_data !=3D NULL) > - rte_free(all_decomp_data); > rte_free(contig_buf); >=20 > return ret_status; > diff --git a/doc/guides/rel_notes/release_19_11.rst > b/doc/guides/rel_notes/release_19_11.rst > index 066bb3c19..5906f8b94 100644 > --- a/doc/guides/rel_notes/release_19_11.rst > +++ b/doc/guides/rel_notes/release_19_11.rst > @@ -152,6 +152,11 @@ New Features > Added eBPF JIT support for arm64 architecture to improve the eBPF > program > performance. >=20 > +* **Refactored compression unit tests.** > + > + Refactored core function to get clear code structure, better for > maintenance. > + Created smaller specialized functions. > + >=20 > Removed Items > ------------- > -- > 2.17.1