From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FAC0A00D7; Thu, 31 Oct 2019 19:23:13 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F28121D16E; Thu, 31 Oct 2019 19:23:12 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 9E1971D14B for ; Thu, 31 Oct 2019 19:23:10 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x9VIJliv011173; Thu, 31 Oct 2019 11:23:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=D/A31M0NQoWQw0tvPFYeIT3Wo2mlHAfmIDCrD21qwVM=; b=fg0vtdZGRIRqFENtJlyToiZG/TSXuqqJ67srOMRVf1RKDMvauKSebltVGvEMNAvt1El+ orxCqPHuShAk0dZXT8mET0lsKUejGl922iXTJnVs6Tp0x7mmXz+DvqfJHaNzINLwN444 mNUkQntCjwD4vklJe1x2TRUrA8UuKTHyQ4l+41u8pPCKwBXFlTeg9b2/r+XEQEBIZCZJ CkU9fyjFNUFVbA/XexzeP0B6rXVBiN4sGwNrT1hrcHEokVl96Q5/0MifzQJJIm4qPhCw OhQwF9GhMPqaJPab88b1k1ob6elQ1gygoXV2i+2wlLkxbkk/YvU+TTqSqzMNNtxMH0BW ug== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2vxwjm8ma8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 31 Oct 2019 11:23:09 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 31 Oct 2019 11:23:08 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.56) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Thu, 31 Oct 2019 11:23:07 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fCCzVM87SH4VvNgsSbC4MSuLjwcbyHcZA7cybnna2P+n5iCC5/JvNH7H4N37Yn+VTm0XPu9BjNB8VZFJ+TRsuMDgKYACdTzUPFEsiCtD928VG6kiXua2+7UYOl10pyV4c9vEAn00eaHUqnGnnRAaVkjTfPgCR+gnEwVLvbeu7Ohl/FoJUmCga/UV9JQMx6A6i9yj3Omotg3ErOkk6RnMhhSeb3ST60VAl2Q7nA0pcqqGozR2sQqGSrOapjiqYdbpOPazmvH6wLeUeTBEnDgBJCHUJ84ysxsDMc6gTY+enobORqR8qScym9k88rML3/rvt7MgSVkoiTkTlDJDOog1kA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D/A31M0NQoWQw0tvPFYeIT3Wo2mlHAfmIDCrD21qwVM=; b=jZ2Bx2mG4hcrJSnAooZjgw59DRcAjmoRs0gsnX01R9vbjGohgr9D+XEjte5nUvk5FrFyjEnh90A+ulMA5ZLapQzfxAh1ZTH+/aCDDHfRjwJVePHWXcFTTJIcdR3VdxKlu/ws4ihQ3fhJsm3pSV+gV7gxvIxk/5uACwBTS+2AOKbUC5tVXYCkHlWeb+cmCCWfeW4IucVjcMhIGWIOo05z74Aqzs9eJFAGQfb3pCv6McC3O8tX4+bHb9b7hljybwqf1QSae1hzB55sDu7g31qDiMsbD7JAtYXBJxxS9pKJ+JhyPR7/CxO5DRKNK/0eWCfpdQMfey/vM+wZrqRJMJ3Jng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D/A31M0NQoWQw0tvPFYeIT3Wo2mlHAfmIDCrD21qwVM=; b=sWXf0qzxlaKZ3/tY9mJGpcPfRv45RjA0gQACnGkFwraOs6s38lcVGs3TFpej3yAve7cfvs9TA/FvHQFbjRqqQj2xR98f0sUx1VR54b+fo1eRS+Or438rJyaHGG0qOup0a0fZ8kniTLnfF+N34OW8086e2wOaNxpDo4Dg6jzfpFo= Received: from BN8PR18MB2867.namprd18.prod.outlook.com (20.179.72.89) by BN8PR18MB2483.namprd18.prod.outlook.com (20.179.65.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2408.17; Thu, 31 Oct 2019 18:23:05 +0000 Received: from BN8PR18MB2867.namprd18.prod.outlook.com ([fe80::cc7c:df90:83dd:ad6c]) by BN8PR18MB2867.namprd18.prod.outlook.com ([fe80::cc7c:df90:83dd:ad6c%4]) with mapi id 15.20.2387.027; Thu, 31 Oct 2019 18:23:05 +0000 From: Shally Verma To: Artur Trybula , "dev@dpdk.org" , "fiona.trahe@intel.com" , "adamx.dybkowski@intel.com" , "akhil.goyal@nxp.com" Thread-Topic: [EXT] [PATCH v3 1/1] test/compress: unit tests refactoring Thread-Index: AQHVikwCFNfkVivdu0aZlwUMoGELQ6d1EEPA Date: Thu, 31 Oct 2019 18:23:04 +0000 Message-ID: References: <20190912143431.26917-1-arturx.trybula@intel.com> <20191024091616.31224-1-arturx.trybula@intel.com> <20191024091616.31224-2-arturx.trybula@intel.com> In-Reply-To: <20191024091616.31224-2-arturx.trybula@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [122.175.27.214] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 52e1ba27-c715-4a17-ad8e-08d75e2f5f3d x-ms-traffictypediagnostic: BN8PR18MB2483: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2582; x-forefront-prvs: 02070414A1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(39850400004)(376002)(346002)(366004)(136003)(13464003)(52084003)(189003)(199004)(476003)(8936002)(256004)(52536014)(9686003)(5660300002)(74316002)(99286004)(30864003)(55016002)(305945005)(5024004)(14444005)(8676002)(81166006)(2501003)(66066001)(6246003)(7736002)(81156014)(76176011)(71200400001)(6116002)(110136005)(7696005)(2906002)(11346002)(53546011)(316002)(102836004)(33656002)(6506007)(25786009)(6436002)(229853002)(66556008)(64756008)(2201001)(446003)(66446008)(66476007)(478600001)(26005)(486006)(66946007)(71190400001)(186003)(14454004)(3846002)(86362001)(76116006)(569006); DIR:OUT; SFP:1101; SCL:1; SRVR:BN8PR18MB2483; H:BN8PR18MB2867.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 8iZtX1M3XNTaRPYnMwezsIKhu18Jb6rSPaze3QBUwyPYHcVLOL9PhtZb+kvvlMUeHvTzPhbh1Ty38L0f41r3HcEWxiMYyWjioTkmDBTZlCuKGBHP2+dFSeVLmxFXH4lTOVnVd52rKrrGw9Azq5jkS4PeQJExE4da+8CUMCE07m8kCiYIz/iFUMUk971LSYVAOTyhXgSHYHcsofy5ROtvtjTdna1ULb1Ef2pnbo1RWj5TqS09cXIbHTqRLRcBBjhv5pyZ92YS3pg7BVsmwwn3T0G32aa5DdNsH6asA317ToMqbG564ZIslE0TzC4iHaypryRoqoJ7qtFb4mm13bA7J+BmuknF7znGVba3nhv3WWiJy7OXeVp4hnyXXzk3JWukWizoSv7ysQk1lF1OCDIjdCXfP01dr8KpreAefqc71mHlk0Eq2cJTkEZw8FKnj2lg x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 52e1ba27-c715-4a17-ad8e-08d75e2f5f3d X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Oct 2019 18:23:04.9076 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: e9VttSFOEky4scj4f9SGg26d6nFcvbJcQlQqfWWM5mM99n6pnJFTL35XlVIP1OIBT1eO0GZh/8+ZtU8sP29DWA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR18MB2483 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8 definitions=2019-10-31_07:2019-10-30,2019-10-31 signatures=0 Subject: Re: [dpdk-dev] [EXT] [PATCH v3 1/1] test/compress: unit tests refactoring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Artur Trybula > Sent: Thursday, October 24, 2019 2:46 PM > To: dev@dpdk.org; fiona.trahe@intel.com; Shally Verma > ; adamx.dybkowski@intel.com; > arturx.trybula@intel.com; akhil.goyal@nxp.com > Subject: [EXT] [PATCH v3 1/1] test/compress: unit tests refactoring >=20 > External Email >=20 > ---------------------------------------------------------------------- > Core engine refactoring (test_deflate_comp_decomp function). > Smaller specialized functions created. >=20 > Signed-off-by: Artur Trybula > --- > app/test/test_compressdev.c | 1118 +++++++++++++++++------- > doc/guides/rel_notes/release_19_11.rst | 5 + > 2 files changed, 826 insertions(+), 297 deletions(-) >=20 > diff --git a/app/test/test_compressdev.c b/app/test/test_compressdev.c > index 283c64971..4932bec76 100644 > --- a/app/test/test_compressdev.c > +++ b/app/test/test_compressdev.c > @@ -120,6 +120,20 @@ struct test_data_params { > enum overflow_test overflow; > }; >=20 > +struct test_private_arrays { > + struct rte_mbuf **uncomp_bufs; > + struct rte_mbuf **comp_bufs; > + struct rte_comp_op **ops; > + struct rte_comp_op **ops_processed; > + void **priv_xforms; > + uint64_t *compress_checksum; > + uint32_t *compressed_data_size; > + void **stream; > + char **all_decomp_data; > + unsigned int *decomp_produced_data_size; > + uint16_t num_priv_xforms; > +}; > + > static struct comp_testsuite_params testsuite_params =3D { 0 }; >=20 > static void > @@ -662,6 +676,7 @@ prepare_sgl_bufs(const char *test_buf, struct > rte_mbuf *head_buf, > data_size =3D remaining_data; > else > data_size =3D seg_size; > + > buf_ptr =3D rte_pktmbuf_append(head_buf, data_size); > if (buf_ptr =3D=3D NULL) { > RTE_LOG(ERR, USER1, > @@ -734,8 +749,9 @@ extbuf_free_callback(void *addr __rte_unused, void > *opaque __rte_unused) } >=20 > static int > -test_run_enqueue_dequeue(struct rte_comp_op **ops, unsigned int > num_bufs, > - struct rte_comp_op **ops_processed) > +test_run_enqueue_dequeue(struct rte_comp_op **ops, > + struct rte_comp_op **ops_processed, > + unsigned int num_bufs) > { > uint16_t num_enqd, num_deqd, num_total_deqd; > unsigned int deqd_retries =3D 0; > @@ -776,81 +792,77 @@ test_run_enqueue_dequeue(struct rte_comp_op > **ops, unsigned int num_bufs, > return 0; > } >=20 > -/* > - * Compresses and decompresses buffer with compressdev API and Zlib API > +/** > + * Initialize all the arrays used in comp/decomp to NULL. > + * > + * If the length is greater than the length of the last segment, the > + * function will fail and return -1 without modifying the mbuf. > + * > + * @param prv > + * A container used for aggregation all the arrays. > + * @param num_bufs > + * The number of elements in each array. > + */ > +static void > +test_objects_init(struct test_private_arrays *prv, > + unsigned int num_bufs) > +{ > + /* Initialize all arrays to NULL */ > + memset(prv->uncomp_bufs, 0, sizeof(struct rte_mbuf *) * > num_bufs); > + memset(prv->comp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs); > + memset(prv->ops, 0, sizeof(struct rte_comp_op *) * num_bufs); > + memset(prv->ops_processed, 0, sizeof(struct rte_comp_op *) * > num_bufs); > + memset(prv->priv_xforms, 0, sizeof(void *) * num_bufs); > + memset(prv->compressed_data_size, 0, sizeof(uint32_t) * > num_bufs); } > + Does it really matter what pointer is pointing to? Sizeof(any pointer) * nu= m_bufs .. should be enough. > +/** > + * Source buffers preparation (for compression). > + * > + * Memory allocation for each buffer from mempool > + * -1 returned if function fail, without modifying the mbuf. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > */ > static int > -test_deflate_comp_decomp(const struct interim_data_params *int_data, > - const struct test_data_params *test_data) > +test_mbufs_source_preparation(const struct interim_data_params > *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) > { > + /* local variables: */ > + unsigned int i; > + uint32_t data_size; > + char *buf_ptr; > + int ret; > + char **all_decomp_data =3D priv_arrays->all_decomp_data; > + > struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from int_data: */ > const char * const *test_bufs =3D int_data->test_bufs; > unsigned int num_bufs =3D int_data->num_bufs; > - uint16_t *buf_idx =3D int_data->buf_idx; > - struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > - struct rte_comp_xform **decompress_xforms =3D int_data- > >decompress_xforms; > - unsigned int num_xforms =3D int_data->num_xforms; > - enum rte_comp_op_type compress_state =3D test_data- > >compress_state; > - enum rte_comp_op_type decompress_state =3D test_data- > >decompress_state; > + > + /* from test_data: */ > unsigned int buff_type =3D test_data->buff_type; > - unsigned int out_of_space =3D test_data->out_of_space; > unsigned int big_data =3D test_data->big_data; > - enum zlib_direction zlib_dir =3D test_data->zlib_dir; > - enum overflow_test overflow_tst =3D test_data->overflow; > - int ret_status =3D TEST_FAILED; > - struct rte_mbuf_ext_shared_info inbuf_info; > - struct rte_mbuf_ext_shared_info compbuf_info; > - struct rte_mbuf_ext_shared_info decompbuf_info; > - int ret; > - struct rte_mbuf *uncomp_bufs[num_bufs]; > - struct rte_mbuf *comp_bufs[num_bufs]; > - struct rte_comp_op *ops[num_bufs]; > - struct rte_comp_op *ops_processed[num_bufs]; > - void *priv_xforms[num_bufs]; > - uint16_t num_enqd, num_deqd, num_total_deqd; > - uint16_t num_priv_xforms =3D 0; > - unsigned int deqd_retries =3D 0; > - struct priv_op_data *priv_data; > - char *buf_ptr; > - unsigned int i; > - struct rte_mempool *buf_pool; > - uint32_t data_size; > - /* Compressing with CompressDev */ > - unsigned int oos_zlib_decompress =3D > - (zlib_dir =3D=3D ZLIB_NONE || zlib_dir =3D=3D > ZLIB_DECOMPRESS); > - /* Decompressing with CompressDev */ > - unsigned int oos_zlib_compress =3D > - (zlib_dir =3D=3D ZLIB_NONE || zlib_dir =3D=3D > ZLIB_COMPRESS); > - const struct rte_compressdev_capabilities *capa =3D > - rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); > - char *contig_buf =3D NULL; > - uint64_t compress_checksum[num_bufs]; > - uint32_t compressed_data_size[num_bufs]; > - void *stream =3D NULL; > - char *all_decomp_data =3D NULL; > - unsigned int decomp_produced_data_size =3D 0; > - unsigned int step =3D 0; >=20 > - TEST_ASSERT(decompress_state =3D=3D RTE_COMP_OP_STATELESS || > num_bufs =3D=3D 1, > - "Number of stateful operations in a step should be 1"); > + /* from priv_arrays: */ > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_mempool *buf_pool; >=20 > - if (capa =3D=3D NULL) { > - RTE_LOG(ERR, USER1, > - "Compress device does not support DEFLATE\n"); > - return -ENOTSUP; > - } > + static struct rte_mbuf_ext_shared_info inbuf_info; >=20 > - /* Initialize all arrays to NULL */ > - memset(uncomp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs); > - memset(comp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs); > - memset(ops, 0, sizeof(struct rte_comp_op *) * num_bufs); > - memset(ops_processed, 0, sizeof(struct rte_comp_op *) * > num_bufs); > - memset(priv_xforms, 0, sizeof(void *) * num_bufs); > - memset(compressed_data_size, 0, sizeof(uint32_t) * num_bufs); > - > - if (decompress_state =3D=3D RTE_COMP_OP_STATEFUL) { > + if (test_data->decompress_state =3D=3D RTE_COMP_OP_STATEFUL) { > data_size =3D strlen(test_bufs[0]) + 1; > - all_decomp_data =3D rte_malloc(NULL, data_size, > + *all_decomp_data =3D rte_malloc(NULL, data_size, > RTE_CACHE_LINE_SIZE); > } >=20 > @@ -861,14 +873,15 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > else > buf_pool =3D ts_params->large_mbuf_pool; >=20 > - /* Prepare the source mbufs with the data */ > + /* for compression uncomp_bufs is used as a source buffer */ > + /* allocation from buf_pool (mempool type) */ > ret =3D rte_pktmbuf_alloc_bulk(buf_pool, > uncomp_bufs, num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Source mbufs could not be allocated " > "from the mempool\n"); > - goto exit; > + return -1; > } >=20 > if (test_data->use_external_mbufs) { > @@ -881,8 +894,13 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > test_data->inbuf_memzone->iova, > test_data->inbuf_data_size, > &inbuf_info); > - rte_pktmbuf_append(uncomp_bufs[i], > + buf_ptr =3D rte_pktmbuf_append(uncomp_bufs[i], > test_data->inbuf_data_size); > + if (buf_ptr =3D=3D NULL) { > + RTE_LOG(ERR, USER1, > + "Append extra bytes to the source > mbuf failed\n"); > + return -1; > + } > } > } else if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D SGL_TO_LB) { > for (i =3D 0; i < num_bufs; i++) { > @@ -893,106 +911,296 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > big_data ? buf_pool : ts_params- > >large_mbuf_pool, > big_data ? 0 : MAX_SEGS, > big_data ? MAX_DATA_MBUF_SIZE : > SMALL_SEG_SIZE) < 0) > - goto exit; > + return -1; > } > } else { > for (i =3D 0; i < num_bufs; i++) { > data_size =3D strlen(test_bufs[i]) + 1; > + > buf_ptr =3D rte_pktmbuf_append(uncomp_bufs[i], > data_size); > if (buf_ptr =3D=3D NULL) { > RTE_LOG(ERR, USER1, > "Append extra bytes to the source > mbuf failed\n"); > - goto exit; > + return -1; > } > strlcpy(buf_ptr, test_bufs[i], data_size); > } > } >=20 > - /* Prepare the destination mbufs */ > - ret =3D rte_pktmbuf_alloc_bulk(buf_pool, comp_bufs, num_bufs); > + return 0; > +} > + > +/** > + * Data size calculation (for both compression and decompression). > + * > + * Developer is requested to provide input params > + * according to the following rule: > + * if ops_processed =3D=3D NULL -> compression > + * if ops_processed !=3D NULL -> decompression > + * Values bigger than 0 have to be returned to avoid problems > + * with memory allocation > + * > + * @param ops_processed > + * Operations created as a result of compression phase. Should be > + * set to NULL for compression > + * @param out_of_space_and_zlib > + * Boolean value to switch into "out of space" buffer if set. > + * To test "out-of-space" data size, zlib_decompress must be set as we= ll. > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param i > + * current buffer index > + * @return > + * - values bigger than 0 > + */ > +static inline uint32_t > +test_mbufs_calculate_data_size( > + struct rte_comp_op *ops_processed[], /*can be equal to > NULL*/ > + unsigned int out_of_space_and_zlib, > + const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + unsigned int i) > +{ > + /* local variables: */ > + uint32_t data_size =3D 0; > + struct priv_op_data *priv_data; > + float ratio; > + uint8_t not_zlib_compr; /* true if zlib isn't current compression dev > */ > + enum overflow_test overflow =3D test_data->overflow; > + > + /* from int_data: */ > + const char * const *test_bufs =3D int_data->test_bufs; > + > + if (out_of_space_and_zlib) > + data_size =3D OUT_OF_SPACE_BUF; > + else { > + if (ops_processed =3D=3D NULL) { > + > + not_zlib_compr =3D (test_data->zlib_dir =3D=3D > ZLIB_DECOMPRESS > + || test_data->zlib_dir =3D=3D ZLIB_NONE); > + > + ratio =3D (not_zlib_compr && > + (overflow =3D=3D OVERFLOW_ENABLED)) ? > + COMPRESS_BUF_SIZE_RATIO_OVERFLOW : > + COMPRESS_BUF_SIZE_RATIO; > + > + data_size =3D strlen(test_bufs[i]) * ratio; > + > + } else { > + priv_data =3D (struct priv_op_data *) > + (ops_processed[i] + 1); > + data_size =3D strlen(test_bufs[priv_data->orig_idx]) + > 1; > + } > + } > + > + return data_size; > +} > + > + > +/** > + * Memory buffers preparation (for both compression and decompression). > + * > + * Memory allocation for comp/decomp buffers from mempool, depending > on > + * ops_processed value. Developer is requested to provide input params > + * according to the following rule: > + * if ops_processed =3D=3D NULL -> current_bufs =3D comp_bufs[] > + * if ops_processed !=3D NULL -> current_bufs =3D decomp_bufs[] > + * -1 returned if function fail, without modifying the mbuf. > + * > + * @param ops_processed > + * Operations created as a result of compression phase. Should be > + * set to NULL for compression > + * @param current_bufs > + * mbufs being prepared in the function. > + * @param out_of_space_and_zlib > + * Boolean value to switch into "out of space" buffer if set. > + * To test "out-of-space" data size, zlib_decompress must be set as we= ll. > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param current_extbuf_info, > + * The structure containing all the information related to external mb= ufs > + * @param current_memzone > + * memzone used by external buffers > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_mbufs_destination_preparation( > + struct rte_comp_op *ops_processed[], /*can be equal to > NULL*/ > + struct rte_mbuf *current_bufs[], > + unsigned int out_of_space_and_zlib, > + const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + struct rte_mbuf_ext_shared_info *current_extbuf_info, > + const struct rte_memzone *current_memzone) { > + /* local variables: */ > + unsigned int i; > + uint32_t data_size; > + int ret; > + char *buf_ptr; > + > + struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from int_data: */ > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from test_data: */ > + unsigned int buff_type =3D test_data->buff_type; > + unsigned int big_data =3D test_data->big_data; > + > + struct rte_mempool *buf_pool; > + > + if (big_data) > + buf_pool =3D ts_params->big_mbuf_pool; > + else if (buff_type =3D=3D SGL_BOTH) > + buf_pool =3D ts_params->small_mbuf_pool; > + else > + buf_pool =3D ts_params->large_mbuf_pool; > + > + /* the mbufs allocation*/ > + ret =3D rte_pktmbuf_alloc_bulk(buf_pool, current_bufs, num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Destination mbufs could not be allocated " > "from the mempool\n"); > - goto exit; > + return -1; > } >=20 > if (test_data->use_external_mbufs) { > - compbuf_info.free_cb =3D extbuf_free_callback; > - compbuf_info.fcb_opaque =3D NULL; > - rte_mbuf_ext_refcnt_set(&compbuf_info, 1); > + current_extbuf_info->free_cb =3D extbuf_free_callback; > + current_extbuf_info->fcb_opaque =3D NULL; > + rte_mbuf_ext_refcnt_set(current_extbuf_info, 1); > for (i =3D 0; i < num_bufs; i++) { > - rte_pktmbuf_attach_extbuf(comp_bufs[i], > - test_data->compbuf_memzone- > >addr, > - test_data->compbuf_memzone- > >iova, > - test_data->compbuf_memzone- > >len, > - &compbuf_info); > - rte_pktmbuf_append(comp_bufs[i], > - test_data->compbuf_memzone- > >len); > + rte_pktmbuf_attach_extbuf(current_bufs[i], > + current_memzone->addr, > + current_memzone->iova, > + current_memzone->len, > + current_extbuf_info); > + rte_pktmbuf_append(current_bufs[i], > + current_memzone->len); > } > - } else if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D LB_TO_SGL) { > - for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space =3D=3D 1 && oos_zlib_decompress) > - data_size =3D OUT_OF_SPACE_BUF; > - else > - (data_size =3D strlen(test_bufs[i]) * > - COMPRESS_BUF_SIZE_RATIO); > - > - if (prepare_sgl_bufs(NULL, comp_bufs[i], > - data_size, > - big_data ? buf_pool : ts_params- > >small_mbuf_pool, > - big_data ? buf_pool : ts_params- > >large_mbuf_pool, > - big_data ? 0 : MAX_SEGS, > - big_data ? MAX_DATA_MBUF_SIZE : > SMALL_SEG_SIZE) > - < 0) > - goto exit; > - } > - > } else { > for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space =3D=3D 1 && oos_zlib_decompress) > - data_size =3D OUT_OF_SPACE_BUF; > - else { > - float ratio =3D > - ((test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS > || > - test_data->zlib_dir =3D=3D ZLIB_NONE) && > - overflow_tst =3D=3D OVERFLOW_ENABLED) ? > - > COMPRESS_BUF_SIZE_RATIO_OVERFLOW : > - COMPRESS_BUF_SIZE_RATIO; > - > - data_size =3D strlen(test_bufs[i]) * ratio; > - } > - buf_ptr =3D rte_pktmbuf_append(comp_bufs[i], > data_size); > - if (buf_ptr =3D=3D NULL) { > - RTE_LOG(ERR, USER1, > - "Append extra bytes to the > destination mbuf failed\n"); > - goto exit; > + > + /* data size calculation */ > + data_size =3D test_mbufs_calculate_data_size( > + ops_processed, > + out_of_space_and_zlib, > + int_data, > + test_data, > + i); > + > + /* data allocation */ > + if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D > LB_TO_SGL) { > + ret =3D prepare_sgl_bufs(NULL, > current_bufs[i], > + data_size, > + big_data ? buf_pool : > + ts_params- > >small_mbuf_pool, > + big_data ? buf_pool : > + ts_params- > >large_mbuf_pool, > + big_data ? 0 : MAX_SEGS, > + big_data ? MAX_DATA_MBUF_SIZE : > + SMALL_SEG_SIZE); > + if (ret < 0) > + return -1; > + } else { > + buf_ptr =3D > rte_pktmbuf_append(current_bufs[i], > + data_size); > + if (buf_ptr =3D=3D NULL) { > + RTE_LOG(ERR, USER1, > + "Append extra bytes to the > destination mbuf failed\n"); > + return -1; > + } > } > } > } >=20 > + return 0; > +} > + > +/** > + * The main compression function. > + * > + * Operation(s) configuration, depending on CLI parameters. > + * Operation(s) processing. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_comp_run(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + struct priv_op_data *priv_data; > + unsigned int i; > + uint16_t num_priv_xforms =3D 0; > + int ret; > + int ret_status =3D 0; > + char *buf_ptr; > + > + struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from test_data: */ > + enum rte_comp_op_type operation_type =3D test_data- > >compress_state; > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + > + /* from int_data: */ > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + unsigned int num_xforms =3D int_data->num_xforms; > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from priv_arrays: */ > + struct rte_mbuf **comp_bufs =3D priv_arrays->comp_bufs; > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + void **priv_xforms =3D priv_arrays->priv_xforms; > + > + const struct rte_compressdev_capabilities *capa =3D > + rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); > + > /* Build the compression operations */ > ret =3D rte_comp_op_bulk_alloc(ts_params->op_pool, ops, > num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Compress operations could not be allocated " > "from the mempool\n"); > + ret_status =3D -1; > goto exit; > } >=20 > - > for (i =3D 0; i < num_bufs; i++) { > ops[i]->m_src =3D uncomp_bufs[i]; > ops[i]->m_dst =3D comp_bufs[i]; > ops[i]->src.offset =3D 0; > ops[i]->src.length =3D rte_pktmbuf_pkt_len(uncomp_bufs[i]); > ops[i]->dst.offset =3D 0; > - if (compress_state =3D=3D RTE_COMP_OP_STATELESS) > + > + if (operation_type =3D=3D RTE_COMP_OP_STATELESS) { > ops[i]->flush_flag =3D RTE_COMP_FLUSH_FINAL; > - else { > + } else { > RTE_LOG(ERR, USER1, > - "Stateful operations are not supported " > - "in these tests yet\n"); > + "Compression: stateful operations are not " > + "supported in these tests yet\n"); > + ret_status =3D -1; > goto exit; > } > ops[i]->input_chksum =3D 0; > @@ -1007,14 +1215,16 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > } >=20 > /* Compress data (either with Zlib API or compressdev API */ > - if (zlib_dir =3D=3D ZLIB_COMPRESS || zlib_dir =3D=3D ZLIB_ALL) { > + if (zlib_compress) { > for (i =3D 0; i < num_bufs; i++) { > const struct rte_comp_xform *compress_xform =3D > compress_xforms[i % num_xforms]; > ret =3D compress_zlib(ops[i], compress_xform, > DEFAULT_MEM_LEVEL); > - if (ret < 0) > + if (ret < 0) { > + ret_status =3D -1; > goto exit; > + } >=20 > ops_processed[i] =3D ops[i]; > } > @@ -1022,24 +1232,26 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > /* Create compress private xform data */ > for (i =3D 0; i < num_xforms; i++) { > ret =3D rte_compressdev_private_xform_create(0, > - (const struct rte_comp_xform > *)compress_xforms[i], > + (const struct rte_comp_xform *) > + compress_xforms[i], > &priv_xforms[i]); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Compression private xform " > "could not be created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > } > - > if (capa->comp_feature_flags & > RTE_COMP_FF_SHAREABLE_PRIV_XFORM) { > /* Attach shareable private xform data to ops */ > for (i =3D 0; i < num_bufs; i++) > - ops[i]->private_xform =3D priv_xforms[i % > num_xforms]; > + ops[i]->private_xform =3D > + priv_xforms[i % > num_xforms]; > } else { > - /* Create rest of the private xforms for the other ops > */ > + /* Create rest of the private xforms for the other ops */ > for (i =3D num_xforms; i < num_bufs; i++) { > ret =3D > rte_compressdev_private_xform_create(0, > compress_xforms[i % num_xforms], > @@ -1048,26 +1260,28 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > RTE_LOG(ERR, USER1, > "Compression private xform " > "could not be created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > } > - > /* Attach non shareable private xform data to ops */ > for (i =3D 0; i < num_bufs; i++) > ops[i]->private_xform =3D priv_xforms[i]; > } >=20 > recovery_lb: > - ret =3D test_run_enqueue_dequeue(ops, num_bufs, > ops_processed); > + ret =3D test_run_enqueue_dequeue(ops, ops_processed, > num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > - "Enqueue/dequeue operation failed\n"); > + "Compression: enqueue/dequeue operation > failed\n"); > + ret_status =3D -1; > goto exit; > } >=20 > for (i =3D 0; i < num_bufs; i++) { > - compressed_data_size[i] +=3D ops_processed[i]- > >produced; > + priv_arrays->compressed_data_size[i] +=3D > + ops_processed[i]->produced; >=20 > if (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE) { @@ - > 1088,21 +1302,83 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > if (buf_ptr =3D=3D NULL) { > RTE_LOG(ERR, USER1, > "Data recovery: append extra > bytes to the current mbuf failed\n"); > + ret_status =3D -1; > goto exit; > } > goto recovery_lb; > } > } > - deqd_retries =3D 0; > + } > + > +exit: > + /* Free resources */ > + if (ret_status < 0) > + for (i =3D 0; i < num_bufs; i++) { > + rte_comp_op_free(ops[i]); > + ops_processed[i] =3D NULL; > + } >=20 > - /* Free compress private xforms */ > - for (i =3D 0; i < num_priv_xforms; i++) { > + /* Free compress private xforms */ > + for (i =3D 0; i < num_priv_xforms; i++) { > + if (priv_xforms[i] !=3D NULL) { > rte_compressdev_private_xform_free(0, > priv_xforms[i]); > priv_xforms[i] =3D NULL; > } > - num_priv_xforms =3D 0; > } >=20 > + return ret_status; > +} > + > +/** > + * Prints out the test report. Memory freeing. > + * > + * Called after successful compression. > + * Operation(s) status validation and decompression buffers freeing. > + > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 2: Some operation is not supported > + * - 1: Decompression should be skipped > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_comp_finalize(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + unsigned int i; > + struct priv_op_data *priv_data; > + > + /* from int_data: */ > + unsigned int num_xforms =3D int_data->num_xforms; > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + uint16_t *buf_idx =3D int_data->buf_idx; > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from priv_arrays: */ > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + uint64_t *compress_checksum =3D priv_arrays->compress_checksum; > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + > + /* from test_data: */ > + unsigned int out_of_space =3D test_data->out_of_space; > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); > + > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > uint16_t xform_idx =3D priv_data->orig_idx % num_xforms; > @@ -1111,7 +1387,7 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > enum rte_comp_huffman huffman_type =3D > compress_xform->deflate.huffman; > char engine[] =3D "zlib (directly, not PMD)"; > - if (zlib_dir !=3D ZLIB_COMPRESS && zlib_dir !=3D ZLIB_ALL) > + if (zlib_decompress) > strlcpy(engine, "PMD", sizeof(engine)); >=20 > RTE_LOG(DEBUG, USER1, "Buffer %u compressed by %s from > %u to" > @@ -1134,116 +1410,103 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > * compress operation information is needed for the decompression > stage) > */ > for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space && oos_zlib_decompress) { > + if (out_of_space && !zlib_compress) { > if (ops_processed[i]->status !=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > - ret_status =3D TEST_FAILED; > RTE_LOG(ERR, USER1, > "Operation without expected out of " > "space status error\n"); > - goto exit; > + return -1; > } else > continue; > } >=20 > if (ops_processed[i]->status !=3D > RTE_COMP_OP_STATUS_SUCCESS) { > - if (overflow_tst =3D=3D OVERFLOW_ENABLED) { > + if (test_data->overflow =3D=3D OVERFLOW_ENABLED) { > if (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > - ret_status =3D 1; > RTE_LOG(INFO, USER1, > "Out-of-space-recoverable > functionality" > " is not supported on this device\n"); > - goto exit; > + return 2; > } > } > + > RTE_LOG(ERR, USER1, > "Some operations were not successful\n"); > - goto exit; > + return -1; > } > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > rte_pktmbuf_free(uncomp_bufs[priv_data->orig_idx]); > uncomp_bufs[priv_data->orig_idx] =3D NULL; > } >=20 > - if (out_of_space && oos_zlib_decompress) { > - ret_status =3D TEST_SUCCESS; > - goto exit; > - } > + if (out_of_space && !zlib_compress) > + return 1; >=20 > - /* Allocate buffers for decompressed data */ > - ret =3D rte_pktmbuf_alloc_bulk(buf_pool, uncomp_bufs, num_bufs); > - if (ret < 0) { > - RTE_LOG(ERR, USER1, > - "Destination mbufs could not be allocated " > - "from the mempool\n"); > - goto exit; > - } > + return 0; > +} >=20 > - if (test_data->use_external_mbufs) { > - decompbuf_info.free_cb =3D extbuf_free_callback; > - decompbuf_info.fcb_opaque =3D NULL; > - rte_mbuf_ext_refcnt_set(&decompbuf_info, 1); > - for (i =3D 0; i < num_bufs; i++) { > - rte_pktmbuf_attach_extbuf(uncomp_bufs[i], > - test_data->uncompbuf_memzone- > >addr, > - test_data->uncompbuf_memzone- > >iova, > - test_data->uncompbuf_memzone- > >len, > - &decompbuf_info); > - rte_pktmbuf_append(uncomp_bufs[i], > - test_data->uncompbuf_memzone- > >len); > - } > - } else if (buff_type =3D=3D SGL_BOTH || buff_type =3D=3D LB_TO_SGL) { > - for (i =3D 0; i < num_bufs; i++) { > - priv_data =3D (struct priv_op_data *) > - (ops_processed[i] + 1); > - if (out_of_space =3D=3D 1 && oos_zlib_compress) > - data_size =3D OUT_OF_SPACE_BUF; > - else if (test_data->decompress_output_block_size > !=3D 0) > - data_size =3D > - test_data- > >decompress_output_block_size; > - else > - data_size =3D > - strlen(test_bufs[priv_data->orig_idx]) + 1; > - > - if (prepare_sgl_bufs(NULL, uncomp_bufs[i], > - data_size, > - big_data ? buf_pool : ts_params- > >small_mbuf_pool, > - big_data ? buf_pool : ts_params- > >large_mbuf_pool, > - big_data ? 0 : MAX_SEGS, > - big_data ? MAX_DATA_MBUF_SIZE : > SMALL_SEG_SIZE) > - < 0) > - goto exit; > - } > +/** > + * The main decompression function. > + * > + * Operation(s) configuration, depending on CLI parameters. > + * Operation(s) processing. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_decomp_run(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + struct test_private_arrays *priv_arrays) { >=20 > - } else { > - for (i =3D 0; i < num_bufs; i++) { > - priv_data =3D (struct priv_op_data *) > - (ops_processed[i] + 1); > - if (out_of_space =3D=3D 1 && oos_zlib_compress) > - data_size =3D OUT_OF_SPACE_BUF; > - else if (test_data->decompress_output_block_size > !=3D 0) > - data_size =3D > - test_data- > >decompress_output_block_size; > - else > - data_size =3D > - strlen(test_bufs[priv_data->orig_idx]) + 1; > + /* local variables: */ > + struct priv_op_data *priv_data; > + unsigned int i; > + uint16_t num_priv_xforms =3D 0; > + int ret; > + int ret_status =3D 0; >=20 > - buf_ptr =3D rte_pktmbuf_append(uncomp_bufs[i], > data_size); > - if (buf_ptr =3D=3D NULL) { > - RTE_LOG(ERR, USER1, > - "Append extra bytes to the > decompressed mbuf failed\n"); > - goto exit; > - } > - } > - } > + struct comp_testsuite_params *ts_params =3D &testsuite_params; > + > + /* from test_data: */ > + enum rte_comp_op_type operation_type =3D test_data- > >decompress_state; > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); > + > + /* from int_data: */ > + struct rte_comp_xform **decompress_xforms =3D int_data- > >decompress_xforms; > + unsigned int num_xforms =3D int_data->num_xforms; > + unsigned int num_bufs =3D int_data->num_bufs; > + > + /* from priv_arrays: */ > + struct rte_mbuf **uncomp_bufs =3D priv_arrays->uncomp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + void **priv_xforms =3D priv_arrays->priv_xforms; > + uint32_t *compressed_data_size =3D priv_arrays- > >compressed_data_size; > + void **stream =3D priv_arrays->stream; > + > + const struct rte_compressdev_capabilities *capa =3D > + rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); >=20 > - /* Build the decompression operations */ > ret =3D rte_comp_op_bulk_alloc(ts_params->op_pool, ops, > num_bufs); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Decompress operations could not be allocated " > "from the mempool\n"); > + ret_status =3D -1; > goto exit; > } >=20 > @@ -1256,22 +1519,25 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > * Set the length of the compressed data to the > * number of bytes that were produced in the previous stage > */ > + > if (compressed_data_size[i]) > ops[i]->src.length =3D compressed_data_size[i]; > else > ops[i]->src.length =3D ops_processed[i]->produced; >=20 > ops[i]->dst.offset =3D 0; > - if (decompress_state =3D=3D RTE_COMP_OP_STATELESS) { > + > + if (operation_type =3D=3D RTE_COMP_OP_STATELESS) { > ops[i]->flush_flag =3D RTE_COMP_FLUSH_FINAL; > ops[i]->op_type =3D RTE_COMP_OP_STATELESS; > - } else if (zlib_dir =3D=3D ZLIB_COMPRESS || zlib_dir =3D=3D > ZLIB_NONE) { > + } else if (!zlib_decompress) { > ops[i]->flush_flag =3D RTE_COMP_FLUSH_SYNC; > ops[i]->op_type =3D RTE_COMP_OP_STATEFUL; > } else { > RTE_LOG(ERR, USER1, > - "Stateful operations are not supported " > - "in these tests yet\n"); > + "Decompression: stateful operations are" > + " not supported in these tests yet\n"); > + ret_status =3D -1; > goto exit; > } > ops[i]->input_chksum =3D 0; > @@ -1290,7 +1556,7 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > rte_comp_op_bulk_free(ops_processed, num_bufs); >=20 > /* Decompress data (either with Zlib API or compressdev API */ > - if (zlib_dir =3D=3D ZLIB_DECOMPRESS || zlib_dir =3D=3D ZLIB_ALL) { > + if (zlib_decompress) { > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops[i] + 1); > uint16_t xform_idx =3D priv_data->orig_idx % > num_xforms; @@ -1298,13 +1564,15 @@ test_deflate_comp_decomp(const > struct interim_data_params *int_data, > decompress_xforms[xform_idx]; >=20 > ret =3D decompress_zlib(ops[i], decompress_xform); > - if (ret < 0) > + if (ret < 0) { > + ret_status =3D -1; > goto exit; > + } >=20 > ops_processed[i] =3D ops[i]; > } > } else { > - if (decompress_state =3D=3D RTE_COMP_OP_STATELESS) { > + if (operation_type =3D=3D RTE_COMP_OP_STATELESS) { > /* Create decompress private xform data */ > for (i =3D 0; i < num_xforms; i++) { > ret =3D > rte_compressdev_private_xform_create(0, > @@ -1315,6 +1583,7 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > RTE_LOG(ERR, USER1, > "Decompression private > xform " > "could not be created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > @@ -1341,7 +1610,9 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > &priv_xforms[i]); > if (ret < 0) { > RTE_LOG(ERR, USER1, > - "Decompression > private xform could not be created\n"); > + "Decompression > private xform" > + " could not be > created\n"); > + ret_status =3D -1; > goto exit; > } > num_priv_xforms++; > @@ -1361,59 +1632,86 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > } else { > /* Create a stream object for stateful decompression > */ > ret =3D rte_compressdev_stream_create(0, > - decompress_xforms[0], &stream); > + decompress_xforms[0], stream); > if (ret < 0) { > RTE_LOG(ERR, USER1, > "Decompression stream could not be > created, error %d\n", > ret); > + ret_status =3D -1; > goto exit; > } > /* Attach stream to ops */ > for (i =3D 0; i < num_bufs; i++) > - ops[i]->stream =3D stream; > + ops[i]->stream =3D *stream; > } >=20 > -next_step: > - /* Enqueue and dequeue all operations */ > - num_enqd =3D rte_compressdev_enqueue_burst(0, 0, ops, > num_bufs); > - if (num_enqd < num_bufs) { > - RTE_LOG(ERR, USER1, > - "The operations could not be enqueued\n"); > - goto exit; > - } > + priv_arrays->num_priv_xforms =3D num_priv_xforms; > + } >=20 > - num_total_deqd =3D 0; > - do { > - /* > - * If retrying a dequeue call, wait for 10 ms to allow > - * enough time to the driver to process the > operations > - */ > - if (deqd_retries !=3D 0) { > - /* > - * Avoid infinite loop if not all the > - * operations get out of the device > - */ > - if (deqd_retries =3D=3D MAX_DEQD_RETRIES) { > - RTE_LOG(ERR, USER1, > - "Not all operations could be " > - "dequeued\n"); > - goto exit; > - } > - usleep(DEQUEUE_WAIT_TIME); > - } > - num_deqd =3D rte_compressdev_dequeue_burst(0, 0, > - &ops_processed[num_total_deqd], > num_bufs); > - num_total_deqd +=3D num_deqd; > - deqd_retries++; > - } while (num_total_deqd < num_enqd); > +exit: > + return ret_status; > +} >=20 > - deqd_retries =3D 0; > - } > +/** > + * Prints out the test report. Memory freeing. > + * > + * Called after successful decompression. > + * Operation(s) status validation and compression buffers freeing. > + > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 2: Next step must be executed by the caller (stateful decompressi= on > only) > + * - 1: On success (caller should stop and exit) > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_deflate_decomp_finalize(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + unsigned int i; > + struct priv_op_data *priv_data; > + static unsigned int step; > + > + /* from int_data: */ > + uint16_t *buf_idx =3D int_data->buf_idx; > + unsigned int num_bufs =3D int_data->num_bufs; > + const char * const *test_bufs =3D int_data->test_bufs; > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + > + /* from priv_arrays: */ > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; > + struct rte_mbuf **comp_bufs =3D priv_arrays->comp_bufs; > + struct rte_comp_op **ops =3D priv_arrays->ops; > + uint64_t *compress_checksum =3D priv_arrays->compress_checksum; > + unsigned int *decomp_produced_data_size =3D > + priv_arrays->decomp_produced_data_size; > + char **all_decomp_data =3D priv_arrays->all_decomp_data; > + > + /* from test_data: */ > + unsigned int out_of_space =3D test_data->out_of_space; > + enum rte_comp_op_type operation_type =3D test_data- > >decompress_state; > + > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); >=20 > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > char engine[] =3D "zlib, (directly, no PMD)"; > - if (zlib_dir !=3D ZLIB_DECOMPRESS && zlib_dir !=3D ZLIB_ALL) > + if (zlib_compress) > strlcpy(engine, "pmd", sizeof(engine)); > RTE_LOG(DEBUG, USER1, > "Buffer %u decompressed by %s from %u to %u > bytes\n", @@ -1427,19 +1725,19 @@ test_deflate_comp_decomp(const > struct interim_data_params *int_data, > * compress operation information is still needed) > */ > for (i =3D 0; i < num_bufs; i++) { > - if (out_of_space && oos_zlib_compress) { > + if (out_of_space && !zlib_decompress) { > if (ops_processed[i]->status !=3D > - > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > - ret_status =3D TEST_FAILED; > + > RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED) { > + > RTE_LOG(ERR, USER1, > "Operation without expected out of " > "space status error\n"); > - goto exit; > + return -1; > } else > continue; > } >=20 > - if (decompress_state =3D=3D RTE_COMP_OP_STATEFUL > + if (operation_type =3D=3D RTE_COMP_OP_STATEFUL > && (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE > || ops_processed[i]->status =3D=3D > @@ -1449,28 +1747,29 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > ops_processed[i]->m_dst, > ops_processed[i]->dst.offset, > ops_processed[i]->produced, > - all_decomp_data + > - > decomp_produced_data_size); > - if (ptr !=3D all_decomp_data + > decomp_produced_data_size) > - rte_memcpy(all_decomp_data + > - decomp_produced_data_size, > + *all_decomp_data + > + > *decomp_produced_data_size); > + if (ptr !=3D *all_decomp_data + > + *decomp_produced_data_size) > + rte_memcpy(*all_decomp_data + > + *decomp_produced_data_size, > ptr, ops_processed[i]->produced); > - decomp_produced_data_size +=3D ops_processed[i]- > >produced; > + > + *decomp_produced_data_size +=3D > + ops_processed[i]->produced; > if (ops_processed[i]->src.length > > ops_processed[i]->consumed) { > if (ops_processed[i]->status =3D=3D >=20 > RTE_COMP_OP_STATUS_SUCCESS) { > - ret_status =3D -1; > RTE_LOG(ERR, USER1, > "Operation finished too early\n"); > - goto exit; > + return -1; > } > step++; > if (step >=3D test_data- > >decompress_steps_max) { > - ret_status =3D -1; > RTE_LOG(ERR, USER1, > "Operation exceeded maximum > steps\n"); > - goto exit; > + return -1; > } > ops[i] =3D ops_processed[i]; > ops[i]->status =3D > @@ -1479,7 +1778,9 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > ops_processed[i]- > >consumed; > ops[i]->src.length -=3D > ops_processed[i]- > >consumed; > - goto next_step; > + /* repeat the operation */ > + //goto next_step; > + return 2; > } else { > /* Compare the original stream with the */ > /* decompressed stream (in size and the > data) */ @@ -1487,22 +1788,22 @@ test_deflate_comp_decomp(const > struct interim_data_params *int_data, > (ops_processed[i] + 1); > const char *buf1 =3D > test_bufs[priv_data- > >orig_idx]; > - const char *buf2 =3D all_decomp_data; > + const char *buf2 =3D *all_decomp_data; >=20 > if (compare_buffers(buf1, strlen(buf1) + 1, > - buf2, decomp_produced_data_size) > < 0) > - goto exit; > + buf2, > *decomp_produced_data_size) < 0) > + return -1; > /* Test checksums */ > if (compress_xforms[0]->compress.chksum > !=3D > RTE_COMP_CHECKSUM_NONE) { > if (ops_processed[i]- > >output_chksum > !=3D compress_checksum[i]) > { > RTE_LOG(ERR, USER1, > - "The checksums > differ\n" > - "Compression Checksum: %" PRIu64 > "\tDecompression " > - "Checksum: %" PRIu64 "\n", > compress_checksum[i], > + "The checksums differ\n" > + "Compression Checksum: %" PRIu64 > "\tDecompression " > + "Checksum: %" PRIu64 "\n", compress_checksum[i], > ops_processed[i]- > >output_chksum); > - goto exit; > + return -1; > } > } > } > @@ -1510,18 +1811,58 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > RTE_COMP_OP_STATUS_SUCCESS) { > RTE_LOG(ERR, USER1, > "Some operations were not successful\n"); > - goto exit; > + return -1; > } > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > rte_pktmbuf_free(comp_bufs[priv_data->orig_idx]); > comp_bufs[priv_data->orig_idx] =3D NULL; > } >=20 > - if ((out_of_space && oos_zlib_compress) > - || (decompress_state =3D=3D > RTE_COMP_OP_STATEFUL)) { > - ret_status =3D TEST_SUCCESS; > - goto exit; > - } > + if (out_of_space && !zlib_decompress) > + return 1; > + > + return 0; > +} > + > +/** > + * Validation of the output (compression/decompression) data. > + * > + * The function compares the source stream with the output stream, > + * after decompression, to check if compression/decompression > + * was correct. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @param priv_arrays > + * A container used for aggregation all the private test arrays. > + * @return > + * - 0: On success. > + * - -1: On error. > + */ > +static int > +test_results_validation(const struct interim_data_params *int_data, > + const struct test_data_params *test_data, > + const struct test_private_arrays *priv_arrays) { > + /* local variables: */ > + unsigned int i; > + struct priv_op_data *priv_data; > + const char *buf1; > + const char *buf2; > + char *contig_buf =3D NULL; > + uint32_t data_size; > + > + /* from int_data: */ > + struct rte_comp_xform **compress_xforms =3D int_data- > >compress_xforms; > + unsigned int num_bufs =3D int_data->num_bufs; > + const char * const *test_bufs =3D int_data->test_bufs; > + > + /* from priv_arrays: */ > + uint64_t *compress_checksum =3D priv_arrays->compress_checksum; > + struct rte_comp_op **ops_processed =3D priv_arrays- > >ops_processed; >=20 > /* > * Compare the original stream with the decompressed stream @@ - > 1529,13 +1870,13 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > */ > for (i =3D 0; i < num_bufs; i++) { > priv_data =3D (struct priv_op_data *)(ops_processed[i] + 1); > - const char *buf1 =3D test_data->use_external_mbufs ? > + buf1 =3D test_data->use_external_mbufs ? > test_data->inbuf_memzone->addr : > test_bufs[priv_data->orig_idx]; > - const char *buf2; > data_size =3D test_data->use_external_mbufs ? > test_data->inbuf_data_size : > strlen(buf1) + 1; > + > contig_buf =3D rte_malloc(NULL, ops_processed[i]->produced, > 0); > if (contig_buf =3D=3D NULL) { > RTE_LOG(ERR, USER1, "Contiguous buffer could not " > @@ -1565,24 +1906,207 @@ test_deflate_comp_decomp(const struct > interim_data_params *int_data, > rte_free(contig_buf); > contig_buf =3D NULL; > } > + return 0; > + > +exit: > + rte_free(contig_buf); > + return -1; > +} > + > +/** > + * Compresses and decompresses input stream with compressdev API and > +Zlib API > + * > + * Basic test function. Common for all the functional tests. > + * -1 returned if function fail. > + * > + * @param int_data > + * Interim data containing session/transformation objects. > + * @param test_data > + * The test parameters set by users (command line parameters). > + * @return > + * - 1: Some operation not supported > + * - 0: On success. > + * - -1: On error. > + */ > + > +static int > +test_deflate_comp_decomp(const struct interim_data_params *int_data, > + const struct test_data_params *test_data) { > + unsigned int num_bufs =3D int_data->num_bufs; > + unsigned int out_of_space =3D test_data->out_of_space; > + > + void *stream =3D NULL; > + char *all_decomp_data =3D NULL; > + unsigned int decomp_produced_data_size =3D 0; > + > + int ret_status =3D -1; > + int ret; > + struct rte_mbuf *uncomp_bufs[num_bufs]; > + struct rte_mbuf *comp_bufs[num_bufs]; > + struct rte_comp_op *ops[num_bufs]; > + struct rte_comp_op *ops_processed[num_bufs]; > + void *priv_xforms[num_bufs]; > + unsigned int i; > + > + uint64_t compress_checksum[num_bufs]; > + uint32_t compressed_data_size[num_bufs]; > + char *contig_buf =3D NULL; > + > + struct rte_mbuf_ext_shared_info compbuf_info; > + struct rte_mbuf_ext_shared_info decompbuf_info; > + > + const struct rte_compressdev_capabilities *capa; > + > + /* Compressing with CompressDev */ > + unsigned int zlib_compress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_COMPRESS); > + unsigned int zlib_decompress =3D > + (test_data->zlib_dir =3D=3D ZLIB_ALL || > + test_data->zlib_dir =3D=3D ZLIB_DECOMPRESS); > + > + struct test_private_arrays priv_arrays; > + > + priv_arrays.uncomp_bufs =3D uncomp_bufs; > + priv_arrays.comp_bufs =3D comp_bufs; > + priv_arrays.ops =3D ops; > + priv_arrays.ops_processed =3D ops_processed; > + priv_arrays.priv_xforms =3D priv_xforms; > + priv_arrays.compress_checksum =3D compress_checksum; > + priv_arrays.compressed_data_size =3D compressed_data_size; > + > + priv_arrays.stream =3D &stream; > + priv_arrays.all_decomp_data =3D &all_decomp_data; > + priv_arrays.decomp_produced_data_size =3D > &decomp_produced_data_size; > + > + priv_arrays.num_priv_xforms =3D 0; /* it's used for deompression only > */ > + > + capa =3D rte_compressdev_capability_get(0, > RTE_COMP_ALGO_DEFLATE); > + if (capa =3D=3D NULL) { > + RTE_LOG(ERR, USER1, > + "Compress device does not support DEFLATE\n"); > + return -1; > + } > + test_objects_init(&priv_arrays, num_bufs); Do we need this? Why we just cant call source_prep API? That will also setu= p uncompressed_bufs > + > + /* Prepare the source mbufs with the data */ > + ret =3D test_mbufs_source_preparation(int_data, test_data, > &priv_arrays); For code readability sake, it could test_setup_uncom_bufs()=20 > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > +/* COMPRESSION */ > + > + /* Prepare the destination mbufs */ > + ret =3D test_mbufs_destination_preparation( Why we cant keep this prototype same as source_preparation() API for better= readability? I mean we can pass priv_arry here too... > + NULL, /*can be equal to NULL*/ > + comp_bufs, > + out_of_space =3D=3D 1 && !zlib_compress, > + int_data, > + test_data, > + &compbuf_info, > + test_data->compbuf_memzone); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + /* Run compression */ > + ret =3D test_deflate_comp_run(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + ret =3D test_deflate_comp_finalize(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } else if (ret =3D=3D 1) { > + ret_status =3D 0; > + goto exit; > + } else if (ret =3D=3D 2) { > + ret_status =3D 1; /* some operation not supported */ > + goto exit; > + } > + > +/* DECOMPRESSION */ > + > + /* Allocate buffers for decompressed data */ > + ret =3D test_mbufs_destination_preparation( > + ops_processed, /*can be equal to NULL*/ > + uncomp_bufs, > + out_of_space =3D=3D 1 && !zlib_decompress, > + int_data, > + test_data, > + &decompbuf_info, > + test_data->uncompbuf_memzone); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + /* Run decompression */ > + ret =3D test_deflate_decomp_run(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + > + if (!zlib_decompress) { > +next_step: /* next step for stateful decompression only */ > + ret =3D test_run_enqueue_dequeue(ops, ops_processed, > num_bufs); > + if (ret < 0) { > + ret_status =3D -1; > + RTE_LOG(ERR, USER1, > + "Decompression: enqueue/dequeue > operation failed\n"); > + } > + } >=20 > - ret_status =3D TEST_SUCCESS; > + ret =3D test_deflate_decomp_finalize(int_data, test_data, > &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } else if (ret =3D=3D 1) { > + ret_status =3D 0; > + goto exit; > + } else if (ret =3D=3D 2) { > + goto next_step; > + } > + > +/* FINAL PROCESSING */ > + > + ret =3D test_results_validation(int_data, test_data, &priv_arrays); > + if (ret < 0) { > + ret_status =3D -1; > + goto exit; > + } > + ret_status =3D 0; >=20 > exit: > /* Free resources */ > + > + if (stream !=3D NULL) > + rte_compressdev_stream_free(0, stream); > + if (all_decomp_data !=3D NULL) > + rte_free(all_decomp_data); > + > + /* Free compress private xforms */ > + for (i =3D 0; i < priv_arrays.num_priv_xforms; i++) { > + if (priv_xforms[i] !=3D NULL) { > + rte_compressdev_private_xform_free(0, > priv_xforms[i]); > + priv_xforms[i] =3D NULL; > + } > + } > + > for (i =3D 0; i < num_bufs; i++) { > rte_pktmbuf_free(uncomp_bufs[i]); > rte_pktmbuf_free(comp_bufs[i]); > rte_comp_op_free(ops[i]); > rte_comp_op_free(ops_processed[i]); > } > - for (i =3D 0; i < num_priv_xforms; i++) > - if (priv_xforms[i] !=3D NULL) > - rte_compressdev_private_xform_free(0, > priv_xforms[i]); > - if (stream !=3D NULL) > - rte_compressdev_stream_free(0, stream); > - if (all_decomp_data !=3D NULL) > - rte_free(all_decomp_data); > rte_free(contig_buf); >=20 > return ret_status; > diff --git a/doc/guides/rel_notes/release_19_11.rst > b/doc/guides/rel_notes/release_19_11.rst > index 066bb3c19..5906f8b94 100644 > --- a/doc/guides/rel_notes/release_19_11.rst > +++ b/doc/guides/rel_notes/release_19_11.rst > @@ -152,6 +152,11 @@ New Features > Added eBPF JIT support for arm64 architecture to improve the eBPF > program > performance. >=20 > +* **Refactored compression unit tests.** > + > + Refactored core function to get clear code structure, better for > maintenance. > + Created smaller specialized functions. > + >=20 > Removed Items > ------------- > -- > 2.17.1