From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM05-BY2-obe.outbound.protection.outlook.com (mail-eopbgr710064.outbound.protection.outlook.com [40.107.71.64]) by dpdk.org (Postfix) with ESMTP id 1AB4158F6 for ; Mon, 5 Nov 2018 09:56:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s6XOut/eIrI2wz8/wuexVMmLH1Hx0ENs3aFpBxYjdJs=; b=MD1ADCFOOY6GkjzFTKzBrQkVoQYvWdH/r2ym11k7VMoQ3OyXay0GqbZZ6KEhGrTWUgiGZGfWSDJx3/qx3vkS9kfz8GKWi4YL6/Sj6mbfU2BXcfzYKgURAtQSdyy8oRjjmB5pIaXE4LkaEKxvxNxUdVLJoMFxpwCSmj6IEKGE6nY= Received: from SN6PR07MB5152.namprd07.prod.outlook.com (52.135.101.33) by SN6PR07MB5470.namprd07.prod.outlook.com (20.177.249.80) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.20; Mon, 5 Nov 2018 08:56:33 +0000 Received: from SN6PR07MB5152.namprd07.prod.outlook.com ([fe80::49cb:b2a:974:2211]) by SN6PR07MB5152.namprd07.prod.outlook.com ([fe80::49cb:b2a:974:2211%4]) with mapi id 15.20.1294.028; Mon, 5 Nov 2018 08:56:33 +0000 From: "Verma, Shally" To: Tomasz Jozwiak , "dev@dpdk.org" , "fiona.trahe@intel.com" , "akhil.goyal@nxp.com" Thread-Topic: [PATCH v2 2/3] app/compress-perf: add performance measurement Thread-Index: AQHUcpCcS5ESYz6wkEqWlqsL+AItVqVA5DbQ Date: Mon, 5 Nov 2018 08:56:33 +0000 Message-ID: References: <1538400427-20164-1-git-send-email-tomaszx.jozwiak@intel.com> <1541151842-8746-1-git-send-email-tomaszx.jozwiak@intel.com> <1541151842-8746-3-git-send-email-tomaszx.jozwiak@intel.com> In-Reply-To: <1541151842-8746-3-git-send-email-tomaszx.jozwiak@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [115.113.156.3] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; SN6PR07MB5470; 6:Fy32onzHBBLn+3KHKNoeB9PSKqvFKgaXBuxy7ONSSC2O/q8RkGsEmg2tApUgIwgFBvpvXKWUjG8hJ653XPFDF/fdtiAA6WnIulrFElP9y10NZclIHhqYvtWy+70eJ5y8t7/CvJQVLmIrk/GUiCDMja42gxdHPx4l0kKaY8yp2D7now6emRYUdTLtGMjIpXkhInLWbnuFaDX4wEVJmMNfV5dn2ku5NJPlmBLAs9H1B0/fvJO5AckZC0PDdcjkXQhOF3s/TKcQYOVqNi4kHNJ4/oRQ1LxX16IIhBQbUntnO0oSJ0/cIIBGaD9dHfN6IfUMeRTcV1zdvVHi9d0Gb65gI289a9gsNjzEsXudPguk1kWqxRULZFWfvyty8XIeOd6a1K+TFIWq6yRJDiej6jBFhYJkm9j78pf6jBU5glutQaGXDpo3+AtlewmpiE+f9PpfxUr8ljh4NHcnvsh7iiHl5Q==; 5:Ezk7uaBxJ6dV/v+y0IbMTMx9asdhHPErla+c5USZdZV7TdrPQNvM/NyBzg2ixUOBp+E+/gqOumDo9P9oYKxvInh48sTniZV8wkC3gVewEWzpCctxN3KAk4rtig3d3IiQLxeZbGEWVmHVgCUjOec67tdwMXOnsoCXRS/OF8uTxBg=; 7:ASA4ohlFqxlLIH1nUy0DWB4TUdDHDb2H4DwxnfyGflpOzQscDAXao0GN++2XOXmoU0g9SJMZXt+grKQXIrNd+XeNm0kBAntfus78unSK454/jUjV+HlIJg+YL3nhLZoDDBenG94Z3uIa3SlODvqQcg== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 082aa637-5b41-41a1-4577-08d642fc95d6 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:SN6PR07MB5470; x-ms-traffictypediagnostic: SN6PR07MB5470: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(228905959029699)(185117386973197); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231382)(944501410)(52105095)(3002001)(10201501046)(93006095)(93001095)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:SN6PR07MB5470; BCL:0; PCL:0; RULEID:; SRVR:SN6PR07MB5470; x-forefront-prvs: 08476BC6EF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(376002)(346002)(396003)(366004)(39860400002)(199004)(189003)(13464003)(478600001)(256004)(97736004)(14454004)(6246003)(4744004)(486006)(74316002)(14444005)(476003)(446003)(105586002)(11346002)(106356001)(71200400001)(71190400001)(26005)(6436002)(7736002)(305945005)(2906002)(68736007)(55016002)(53936002)(53946003)(33656002)(9686003)(72206003)(3846002)(6116002)(66066001)(5024004)(6506007)(99286004)(8936002)(102836004)(55236004)(76176011)(7696005)(25786009)(2501003)(110136005)(316002)(86362001)(229853002)(2201001)(186003)(2900100001)(5660300001)(8676002)(81166006)(81156014)(559001)(569006); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR07MB5470; H:SN6PR07MB5152.namprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: oZyp7eJKRtnemXRROqVZ7uSReoxN+9odWJwMdvHYLrbo6gKwhlM/bP4cbn3748Mn/knLgP+a1Bt/s9lV9kxD1jGCNoLo9jgCu+xovI07BlaZ3MAmWKHTuDXyDRbDqJxne9Joj1lG/i/imIP09l0KR0MTelr3yhYF6R+e5M9mVAPJVpksSQcxKxSbZJnkeggkWnh/2ND67tM5YUESHLRhO3qqjI01eA6+n0fpfUD27Yf1LzGWs3HGRi3ovGbRFr/1yG0dYbZnYLfiNePx1ucM75GI3xxmWU5Z/Ho/R3qHJqQPvQbunEbO7j5eWQbSK/S2c8Qy0H+q5BzCHqQzDXx9QVNyXItGHfRfyq/qQCnCHgQ= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 082aa637-5b41-41a1-4577-08d642fc95d6 X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Nov 2018 08:56:33.1102 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR07MB5470 Subject: Re: [dpdk-dev] [PATCH v2 2/3] app/compress-perf: add performance measurement X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 08:56:39 -0000 >-----Original Message----- >From: Tomasz Jozwiak >Sent: 02 November 2018 15:14 >To: dev@dpdk.org; fiona.trahe@intel.com; tomaszx.jozwiak@intel.com; Verma,= Shally ; >akhil.goyal@nxp.com >Subject: [PATCH v2 2/3] app/compress-perf: add performance measurement > >External Email > >Added performance measurement part into compression perf. test. > >Signed-off-by: De Lara Guarch, Pablo >Signed-off-by: Tomasz Jozwiak >--- > app/test-compress-perf/comp_perf_options_parse.c | 8 +- > app/test-compress-perf/main.c | 886 ++++++++++++++++++= ++++- > 2 files changed, 883 insertions(+), 11 deletions(-) > >diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-c= ompress-perf/comp_perf_options_parse.c >index bef4d2f..e5da3ad 100644 >--- a/app/test-compress-perf/comp_perf_options_parse.c >+++ b/app/test-compress-perf/comp_perf_options_parse.c >@@ -431,10 +431,6 @@ parse_huffman_enc(struct comp_test_data *test_data, c= onst char *arg) > { > "fixed", > RTE_COMP_HUFFMAN_FIXED >- }, >- { >- "dynamic", >- RTE_COMP_HUFFMAN_DYNAMIC > } > }; > >@@ -569,9 +565,9 @@ comp_perf_options_default(struct comp_test_data *test_= data) > test_data->seg_sz =3D 2048; > test_data->burst_sz =3D 32; > test_data->pool_sz =3D 8192; >- test_data->max_sgl_segs =3D UINT16_MAX; >+ test_data->max_sgl_segs =3D 16; > test_data->num_iter =3D 10000; >- test_data->huffman_enc =3D RTE_COMP_HUFFMAN_DYNAMIC; >+ test_data->huffman_enc =3D RTE_COMP_HUFFMAN_FIXED; > test_data->test_op =3D COMPRESS_DECOMPRESS; > test_data->window_sz =3D -1; > test_data->level.min =3D 1; >diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c >index f52b98d..e3f4bf6 100644 >--- a/app/test-compress-perf/main.c >+++ b/app/test-compress-perf/main.c >@@ -5,14 +5,728 @@ > #include > #include > #include >+#include > #include > > #include "comp_perf_options.h" > >+#define NUM_MAX_XFORMS 16 >+#define NUM_MAX_INFLIGHT_OPS 512 >+#define EXPANSE_RATIO 1.05 >+#define MIN_ISAL_SIZE 8 Can we avoid ISAL specific naming ? >+ >+#define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) !=3D 0)) >+ >+/* Cleanup state machine */ >+static enum cleanup_st { >+ ST_CLEAR =3D 0, >+ ST_TEST_DATA, >+ ST_COMPDEV, >+ ST_INPUT_DATA, >+ ST_MEMORY_ALLOC, >+ ST_PREPARE_BUF, >+ ST_DURING_TEST >+} cleanup =3D ST_CLEAR; >+ >+static int >+param_range_check(uint16_t size, const struct rte_param_log2_range *range= ) >+{ >+ unsigned int next_size; >+ >+ /* Check lower/upper bounds */ >+ if (size < range->min) >+ return -1; >+ >+ if (size > range->max) >+ return -1; >+ >+ /* If range is actually only one value, size is correct */ >+ if (range->increment =3D=3D 0) >+ return 0; >+ >+ /* Check if value is one of the supported sizes */ >+ for (next_size =3D range->min; next_size <=3D range->max; >+ next_size +=3D range->increment) >+ if (size =3D=3D next_size) >+ return 0; >+ >+ return -1; >+} >+ >+static int >+comp_perf_check_capabilities(struct comp_test_data *test_data) >+{ >+ const struct rte_compressdev_capabilities *cap; >+ >+ cap =3D rte_compressdev_capability_get(test_data->cdev_id, >+ RTE_COMP_ALGO_DEFLATE); >+ >+ if (cap =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, >+ "Compress device does not support DEFLATE\n"); >+ return -1; >+ } >+ >+ uint64_t comp_flags =3D cap->comp_feature_flags; >+ >+ /* Huffman enconding */ >+ if (test_data->huffman_enc =3D=3D RTE_COMP_HUFFMAN_FIXED && >+ (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) =3D=3D 0)= { >+ RTE_LOG(ERR, USER1, >+ "Compress device does not supported Fixed Huffman\= n"); >+ return -1; >+ } >+ >+ if (test_data->huffman_enc =3D=3D RTE_COMP_HUFFMAN_DYNAMIC && >+ (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) =3D=3D = 0) { >+ RTE_LOG(ERR, USER1, >+ "Compress device does not supported Dynamic Huffma= n\n"); >+ return -1; >+ } >+ >+ /* Window size */ >+ if (test_data->window_sz !=3D -1) { >+ if (param_range_check(test_data->window_sz, &cap->window_s= ize) >+ < 0) { >+ RTE_LOG(ERR, USER1, >+ "Compress device does not support " >+ "this window size\n"); >+ return -1; >+ } >+ } else >+ /* Set window size to PMD maximum if none was specified */ >+ test_data->window_sz =3D cap->window_size.max; >+ >+ /* Check if chained mbufs is supported */ >+ if (test_data->max_sgl_segs > 1 && >+ (comp_flags & RTE_COMP_FF_OOP_SGL_IN_SGL_OUT) =3D= =3D 0) { >+ RTE_LOG(INFO, USER1, "Compress device does not support " >+ "chained mbufs. Max SGL segments set to 1\= n"); >+ test_data->max_sgl_segs =3D 1; >+ } >+ >+ /* Level 0 support */ >+ if (test_data->level.min =3D=3D 0 && >+ (comp_flags & RTE_COMP_FF_NONCOMPRESSED_BLOCKS) = =3D=3D 0) { >+ RTE_LOG(ERR, USER1, "Compress device does not support " >+ "level 0 (no compression)\n"); >+ return -1; >+ } >+ >+ return 0; >+} >+ >+static int >+comp_perf_allocate_memory(struct comp_test_data *test_data) >+{ >+ /* Number of segments for input and output >+ * (compression and decompression) >+ */ >+ uint32_t total_segs =3D DIV_CEIL(test_data->input_data_sz, >+ test_data->seg_sz); >+ test_data->comp_buf_pool =3D rte_pktmbuf_pool_create("comp_buf_poo= l", >+ total_segs, >+ 0, 0, test_data->seg_sz + RTE_PKTMBUF_HEAD= ROOM, >+ rte_socket_id()); >+ if (test_data->comp_buf_pool =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n")= ; >+ return -1; >+ } >+ >+ cleanup =3D ST_MEMORY_ALLOC; >+ test_data->decomp_buf_pool =3D rte_pktmbuf_pool_create("decomp_buf= _pool", >+ total_segs, >+ 0, 0, test_data->seg_sz + RTE_PKTMBUF_HEAD= ROOM, >+ rte_socket_id()); >+ if (test_data->decomp_buf_pool =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Mbuf mempool could not be created\n")= ; >+ return -1; >+ } Unless am missing to see it, you need to free pre-allocated memories here b= efore return call for all failed cases. >+ >+ test_data->total_bufs =3D DIV_CEIL(total_segs, test_data->max_sgl_= segs); >+ >+ test_data->op_pool =3D rte_comp_op_pool_create("op_pool", >+ test_data->total_bufs, >+ 0, 0, rte_socket_id()); >+ if (test_data->op_pool =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Comp op mempool could not be created\= n"); >+ return -1; >+ } >+ >+ /* >+ * Compressed data might be a bit larger than input data, >+ * if data cannot be compressed >+ */ >+ test_data->compressed_data =3D rte_zmalloc_socket(NULL, >+ test_data->input_data_sz * EXPANSE_RATIO >+ + MIN_ISAL_SIZE, 0= , MIN_ISAL_SIZE looks specific to ISAL driver. if so, then is this perf app s= pecific to that PMD? or Can we make it somewhat generic? >+ rte_socket_id()); >+ if (test_data->compressed_data =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Memory to hold the data from the inpu= t " >+ "file could not be allocated\n"); >+ return -1; >+ } >+ >+ test_data->decompressed_data =3D rte_zmalloc_socket(NULL, >+ test_data->input_data_sz, 0, >+ rte_socket_id()); >+ if (test_data->decompressed_data =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Memory to hold the data from the inpu= t " >+ "file could not be allocated\n"); >+ return -1; >+ } >+ >+ test_data->comp_bufs =3D rte_zmalloc_socket(NULL, >+ test_data->total_bufs * sizeof(struct rte_mbuf *), >+ 0, rte_socket_id()); >+ if (test_data->comp_bufs =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Memory to hold the compression mbufs" >+ " could not be allocated\n"); >+ return -1; >+ } >+ >+ test_data->decomp_bufs =3D rte_zmalloc_socket(NULL, >+ test_data->total_bufs * sizeof(struct rte_mbuf *), >+ 0, rte_socket_id()); >+ if (test_data->decomp_bufs =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Memory to hold the decompression mbuf= s" >+ " could not be allocated\n"); >+ return -1; >+ } >+ return 0; >+} >+ >+static int >+comp_perf_dump_input_data(struct comp_test_data *test_data) >+{ >+ FILE *f =3D fopen(test_data->input_file, "r"); >+ int ret =3D -1; >+ >+ if (f =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Input file could not be opened\n"); >+ return -1; >+ } >+ >+ if (fseek(f, 0, SEEK_END) !=3D 0) { >+ RTE_LOG(ERR, USER1, "Size of input could not be calculated= \n"); >+ goto end; >+ } >+ size_t actual_file_sz =3D ftell(f); >+ /* If extended input data size has not been set, >+ * input data size =3D file size >+ */ >+ >+ if (test_data->input_data_sz =3D=3D 0) >+ test_data->input_data_sz =3D actual_file_sz; >+ >+ if (fseek(f, 0, SEEK_SET) !=3D 0) { >+ RTE_LOG(ERR, USER1, "Size of input could not be calculated= \n"); >+ goto end; >+ } >+ >+ test_data->input_data =3D rte_zmalloc_socket(NULL, >+ test_data->input_data_sz, 0, rte_socket_id= ()); >+ >+ if (test_data->input_data =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Memory to hold the data from the inpu= t " >+ "file could not be allocated\n"); >+ goto end; >+ } >+ >+ size_t remaining_data =3D test_data->input_data_sz; >+ uint8_t *data =3D test_data->input_data; >+ >+ while (remaining_data > 0) { >+ size_t data_to_read =3D RTE_MIN(remaining_data, actual_fil= e_sz); >+ >+ if (fread(data, data_to_read, 1, f) !=3D 1) { >+ RTE_LOG(ERR, USER1, "Input file could not be read\= n"); >+ goto end; >+ } >+ if (fseek(f, 0, SEEK_SET) !=3D 0) { >+ RTE_LOG(ERR, USER1, >+ "Size of input could not be calculated\n")= ; >+ goto end; >+ } >+ remaining_data -=3D data_to_read; >+ data +=3D data_to_read; >+ } >+ >+ if (test_data->input_data_sz > actual_file_sz) >+ RTE_LOG(INFO, USER1, >+ "%zu bytes read from file %s, extending the file %.2f ti= mes\n", >+ test_data->input_data_sz, test_data->input_file, >+ (double)test_data->input_data_sz/actual_file_sz); >+ else >+ RTE_LOG(INFO, USER1, >+ "%zu bytes read from file %s\n", >+ test_data->input_data_sz, test_data->input_file); >+ >+ ret =3D 0; >+ >+end: >+ fclose(f); >+ return ret; >+} >+ >+static int >+comp_perf_initialize_compressdev(struct comp_test_data *test_data) >+{ >+ uint8_t enabled_cdev_count; >+ uint8_t enabled_cdevs[RTE_COMPRESS_MAX_DEVS]; >+ >+ enabled_cdev_count =3D rte_compressdev_devices_get(test_data->driv= er_name, >+ enabled_cdevs, RTE_COMPRESS_MAX_DEVS); >+ if (enabled_cdev_count =3D=3D 0) { >+ RTE_LOG(ERR, USER1, "No compress devices type %s available= \n", >+ test_data->driver_name); >+ return -EINVAL; >+ } >+ >+ if (enabled_cdev_count > 1) >+ RTE_LOG(INFO, USER1, >+ "Only the first compress device will be used\n"); >+ >+ test_data->cdev_id =3D enabled_cdevs[0]; >+ >+ if (comp_perf_check_capabilities(test_data) < 0) >+ return -1; >+ >+ /* Configure compressdev (one device, one queue pair) */ >+ struct rte_compressdev_config config =3D { >+ .socket_id =3D rte_socket_id(), >+ .nb_queue_pairs =3D 1, >+ .max_nb_priv_xforms =3D NUM_MAX_XFORMS, >+ .max_nb_streams =3D 0 >+ }; >+ >+ if (rte_compressdev_configure(test_data->cdev_id, &config) < 0) { >+ RTE_LOG(ERR, USER1, "Device configuration failed\n"); >+ return -1; >+ } >+ >+ if (rte_compressdev_queue_pair_setup(test_data->cdev_id, 0, >+ NUM_MAX_INFLIGHT_OPS, rte_socket_id()) < 0) { >+ RTE_LOG(ERR, USER1, "Queue pair setup failed\n"); >+ return -1; >+ } >+ >+ if (rte_compressdev_start(test_data->cdev_id) < 0) { >+ RTE_LOG(ERR, USER1, "Device could not be started\n"); >+ return -1; >+ } >+ >+ return 0; >+} >+ >+static int >+prepare_bufs(struct comp_test_data *test_data) >+{ >+ uint32_t remaining_data =3D test_data->input_data_sz; >+ uint8_t *input_data_ptr =3D test_data->input_data; >+ size_t data_sz; >+ uint8_t *data_addr; >+ uint32_t i, j; >+ >+ for (i =3D 0; i < test_data->total_bufs; i++) { >+ /* Allocate data in input mbuf and copy data from input fi= le */ >+ test_data->decomp_bufs[i] =3D >+ rte_pktmbuf_alloc(test_data->decomp_buf_pool); >+ if (test_data->decomp_bufs[i] =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); >+ return -1; >+ } >+ >+ cleanup =3D ST_PREPARE_BUF; >+ data_sz =3D RTE_MIN(remaining_data, test_data->seg_sz); >+ data_addr =3D (uint8_t *) rte_pktmbuf_append( >+ test_data->decomp_bufs[i], data_sz= ); >+ if (data_addr =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Could not append data\n"); So is here ..free of allocated buffer before return from failed cases Thanks Shally >+ return -1; >+ } >+ rte_memcpy(data_addr, input_data_ptr, data_sz); >+ >+ input_data_ptr +=3D data_sz; >+ remaining_data -=3D data_sz; >+ >+ /* Already one segment in the mbuf */ >+ uint16_t segs_per_mbuf =3D 1; >+ >+ /* Chain mbufs if needed for input mbufs */ >+ while (segs_per_mbuf < test_data->max_sgl_segs >+ && remaining_data > 0) { >+ struct rte_mbuf *next_seg =3D >+ rte_pktmbuf_alloc(test_data->decomp_buf_po= ol); >+ >+ if (next_seg =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, >+ "Could not allocate mbuf\n"); >+ return -1; >+ } >+ >+ data_sz =3D RTE_MIN(remaining_data, test_data->seg= _sz); >+ data_addr =3D (uint8_t *)rte_pktmbuf_append(next_s= eg, >+ data_sz); >+ >+ if (data_addr =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Could not append data= \n"); >+ return -1; >+ } >+ >+ rte_memcpy(data_addr, input_data_ptr, data_sz); >+ input_data_ptr +=3D data_sz; >+ remaining_data -=3D data_sz; >+ >+ if (rte_pktmbuf_chain(test_data->decomp_bufs[i], >+ next_seg) < 0) { >+ RTE_LOG(ERR, USER1, "Could not chain mbufs= \n"); >+ return -1; >+ } >+ segs_per_mbuf++; >+ } >+ >+ /* Allocate data in output mbuf */ >+ test_data->comp_bufs[i] =3D >+ rte_pktmbuf_alloc(test_data->comp_buf_pool); >+ if (test_data->comp_bufs[i] =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"); >+ return -1; >+ } >+ data_addr =3D (uint8_t *) rte_pktmbuf_append( >+ test_data->comp_bufs[i], >+ test_data->seg_sz); >+ if (data_addr =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Could not append data\n"); >+ return -1; >+ } >+ >+ /* Chain mbufs if needed for output mbufs */ >+ for (j =3D 1; j < segs_per_mbuf; j++) { >+ struct rte_mbuf *next_seg =3D >+ rte_pktmbuf_alloc(test_data->comp_buf_pool= ); >+ >+ if (next_seg =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, >+ "Could not allocate mbuf\n"); >+ return -1; >+ } >+ >+ data_addr =3D (uint8_t *)rte_pktmbuf_append(next_s= eg, >+ test_data->seg_sz); >+ >+ if (data_addr =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, "Could not append data= \n"); >+ return -1; >+ } >+ >+ if (rte_pktmbuf_chain(test_data->comp_bufs[i], >+ next_seg) < 0) { >+ RTE_LOG(ERR, USER1, "Could not chain mbufs= \n"); >+ return -1; >+ } >+ } >+ } >+ >+ return 0; >+} >+ >+static void >+free_bufs(struct comp_test_data *test_data) >+{ >+ uint32_t i; >+ >+ for (i =3D 0; i < test_data->total_bufs; i++) { >+ rte_pktmbuf_free(test_data->comp_bufs[i]); >+ rte_pktmbuf_free(test_data->decomp_bufs[i]); >+ } >+} >+ >+static int >+main_loop(struct comp_test_data *test_data, uint8_t level, >+ enum rte_comp_xform_type type, >+ uint8_t *output_data_ptr, >+ size_t *output_data_sz, >+ unsigned int benchmarking) >+{ >+ uint8_t dev_id =3D test_data->cdev_id; >+ uint32_t i, iter, num_iter; >+ struct rte_comp_op **ops, **deq_ops; >+ void *priv_xform =3D NULL; >+ struct rte_comp_xform xform; >+ size_t output_size =3D 0; >+ struct rte_mbuf **input_bufs, **output_bufs; >+ int res =3D 0; >+ int allocated =3D 0; >+ >+ if (test_data =3D=3D NULL || !test_data->burst_sz) { >+ RTE_LOG(ERR, USER1, >+ "Unknown burst size\n"); >+ return -1; >+ } >+ >+ ops =3D rte_zmalloc_socket(NULL, >+ 2 * test_data->total_bufs * sizeof(struct rte_comp_op *), >+ 0, rte_socket_id()); >+ >+ if (ops =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, >+ "Can't allocate memory for ops strucures\n"); >+ return -1; >+ } >+ >+ deq_ops =3D &ops[test_data->total_bufs]; >+ >+ if (type =3D=3D RTE_COMP_COMPRESS) { >+ xform =3D (struct rte_comp_xform) { >+ .type =3D RTE_COMP_COMPRESS, >+ .compress =3D { >+ .algo =3D RTE_COMP_ALGO_DEFLATE, >+ .deflate.huffman =3D test_data->huffman_en= c, >+ .level =3D level, >+ .window_size =3D test_data->window_sz, >+ .chksum =3D RTE_COMP_CHECKSUM_NONE, >+ .hash_algo =3D RTE_COMP_HASH_ALGO_NONE >+ } >+ }; >+ input_bufs =3D test_data->decomp_bufs; >+ output_bufs =3D test_data->comp_bufs; >+ } else { >+ xform =3D (struct rte_comp_xform) { >+ .type =3D RTE_COMP_DECOMPRESS, >+ .decompress =3D { >+ .algo =3D RTE_COMP_ALGO_DEFLATE, >+ .chksum =3D RTE_COMP_CHECKSUM_NONE, >+ .window_size =3D test_data->window_sz, >+ .hash_algo =3D RTE_COMP_HASH_ALGO_NONE >+ } >+ }; >+ input_bufs =3D test_data->comp_bufs; >+ output_bufs =3D test_data->decomp_bufs; >+ } >+ >+ /* Create private xform */ >+ if (rte_compressdev_private_xform_create(dev_id, &xform, >+ &priv_xform) < 0) { >+ RTE_LOG(ERR, USER1, "Private xform could not be created\n"= ); >+ res =3D -1; >+ goto end; >+ } >+ >+ uint64_t tsc_start, tsc_end, tsc_duration; >+ >+ tsc_start =3D tsc_end =3D tsc_duration =3D 0; >+ if (benchmarking) { >+ tsc_start =3D rte_rdtsc(); >+ num_iter =3D test_data->num_iter; >+ } else >+ num_iter =3D 1; >+ >+ for (iter =3D 0; iter < num_iter; iter++) { >+ uint32_t total_ops =3D test_data->total_bufs; >+ uint32_t remaining_ops =3D test_data->total_bufs; >+ uint32_t total_deq_ops =3D 0; >+ uint32_t total_enq_ops =3D 0; >+ uint16_t ops_unused =3D 0; >+ uint16_t num_enq =3D 0; >+ uint16_t num_deq =3D 0; >+ >+ output_size =3D 0; >+ >+ while (remaining_ops > 0) { >+ uint16_t num_ops =3D RTE_MIN(remaining_ops, >+ test_data->burst_sz); >+ uint16_t ops_needed =3D num_ops - ops_unused; >+ >+ /* >+ * Move the unused operations from the previous >+ * enqueue_burst call to the front, to maintain or= der >+ */ >+ if ((ops_unused > 0) && (num_enq > 0)) { >+ size_t nb_b_to_mov =3D >+ ops_unused * sizeof(struct rte_comp_= op *); >+ >+ memmove(ops, &ops[num_enq], nb_b_to_mov); >+ } >+ >+ /* Allocate compression operations */ >+ if (ops_needed && !rte_comp_op_bulk_alloc( >+ test_data->op_pool, >+ &ops[ops_unused], >+ ops_needed)) { >+ RTE_LOG(ERR, USER1, >+ "Could not allocate enough operation= s\n"); >+ res =3D -1; >+ goto end; >+ } >+ allocated +=3D ops_needed; >+ >+ for (i =3D 0; i < ops_needed; i++) { >+ /* >+ * Calculate next buffer to attach to oper= ation >+ */ >+ uint32_t buf_id =3D total_enq_ops + i + >+ ops_unused; >+ uint16_t op_id =3D ops_unused + i; >+ /* Reset all data in output buffers */ >+ struct rte_mbuf *m =3D output_bufs[buf_id]= ; >+ >+ m->pkt_len =3D test_data->seg_sz * m->nb_s= egs; >+ while (m) { >+ m->data_len =3D m->buf_len - m->da= ta_off; >+ m =3D m->next; >+ } >+ ops[op_id]->m_src =3D input_bufs[buf_id]; >+ ops[op_id]->m_dst =3D output_bufs[buf_id]; >+ ops[op_id]->src.offset =3D 0; >+ ops[op_id]->src.length =3D >+ rte_pktmbuf_pkt_len(input_bufs[buf= _id]); >+ ops[op_id]->dst.offset =3D 0; >+ ops[op_id]->flush_flag =3D RTE_COMP_FLUSH_= FINAL; >+ ops[op_id]->input_chksum =3D buf_id; >+ ops[op_id]->private_xform =3D priv_xform; >+ } >+ >+ num_enq =3D rte_compressdev_enqueue_burst(dev_id, = 0, ops, >+ num_ops); >+ ops_unused =3D num_ops - num_enq; >+ remaining_ops -=3D num_enq; >+ total_enq_ops +=3D num_enq; >+ >+ num_deq =3D rte_compressdev_dequeue_burst(dev_id, = 0, >+ deq_ops, >+ test_data->burs= t_sz); >+ total_deq_ops +=3D num_deq; >+ if (benchmarking =3D=3D 0) { >+ for (i =3D 0; i < num_deq; i++) { >+ struct rte_comp_op *op =3D deq_ops= [i]; >+ const void *read_data_addr =3D >+ rte_pktmbuf_read(op->m_dst= , 0, >+ op->produced, output_data_= ptr); >+ if (read_data_addr =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, >+ "Could not copy buffer in destinatio= n\n"); >+ res =3D -1; >+ goto end; >+ } >+ >+ if (read_data_addr !=3D output_dat= a_ptr) >+ rte_memcpy(output_data_ptr= , >+ rte_pktmbuf_mtod( >+ op->m_dst, uint8= _t *), >+ op->produced); >+ output_data_ptr +=3D op->produced; >+ output_size +=3D op->produced; >+ >+ } >+ } >+ >+ if (iter =3D=3D num_iter - 1) { >+ for (i =3D 0; i < num_deq; i++) { >+ struct rte_comp_op *op =3D deq_ops= [i]; >+ struct rte_mbuf *m =3D op->m_dst; >+ >+ m->pkt_len =3D op->produced; >+ uint32_t remaining_data =3D op->pr= oduced; >+ uint16_t data_to_append; >+ >+ while (remaining_data > 0) { >+ data_to_append =3D >+ RTE_MIN(remaining_= data, >+ test_data->se= g_sz); >+ m->data_len =3D data_to_ap= pend; >+ remaining_data -=3D >+ data_to_ap= pend; >+ m =3D m->next; >+ } >+ } >+ } >+ rte_mempool_put_bulk(test_data->op_pool, >+ (void **)deq_ops, num_deq); >+ allocated -=3D num_deq; >+ } >+ >+ /* Dequeue the last operations */ >+ while (total_deq_ops < total_ops) { >+ num_deq =3D rte_compressdev_dequeue_burst(dev_id, = 0, >+ deq_ops, test_data->burst_= sz); >+ total_deq_ops +=3D num_deq; >+ if (benchmarking =3D=3D 0) { >+ for (i =3D 0; i < num_deq; i++) { >+ struct rte_comp_op *op =3D deq_ops= [i]; >+ const void *read_data_addr =3D >+ rte_pktmbuf_read(op->m_dst= , 0, >+ op->produced, output_data_= ptr); >+ if (read_data_addr =3D=3D NULL) { >+ RTE_LOG(ERR, USER1, >+ "Could not copy buffer in destinatio= n\n"); >+ res =3D -1; >+ goto end; >+ } >+ >+ if (read_data_addr !=3D output_dat= a_ptr) >+ rte_memcpy(output_data_ptr= , >+ rte_pktmbuf_mtod( >+ op->m_dst, uint8_t= *), >+ op->produced); >+ output_data_ptr +=3D op->produced; >+ output_size +=3D op->produced; >+ >+ } >+ } >+ >+ if (iter =3D=3D num_iter - 1) { >+ for (i =3D 0; i < num_deq; i++) { >+ struct rte_comp_op *op =3D deq_ops= [i]; >+ struct rte_mbuf *m =3D op->m_dst; >+ >+ m->pkt_len =3D op->produced; >+ uint32_t remaining_data =3D op->pr= oduced; >+ uint16_t data_to_append; >+ >+ while (remaining_data > 0) { >+ data_to_append =3D >+ RTE_MIN(remaining_data, >+ test_data->seg_sz)= ; >+ m->data_len =3D data_to_ap= pend; >+ remaining_data -=3D >+ data_to_ap= pend; >+ m =3D m->next; >+ } >+ } >+ } >+ rte_mempool_put_bulk(test_data->op_pool, >+ (void **)deq_ops, num_deq); >+ allocated -=3D num_deq; >+ } >+ } >+ >+ if (benchmarking) { >+ tsc_end =3D rte_rdtsc(); >+ tsc_duration =3D tsc_end - tsc_start; >+ >+ if (type =3D=3D RTE_COMP_COMPRESS) >+ test_data->comp_tsc_duration[level] =3D >+ tsc_duration / num_iter; >+ else >+ test_data->decomp_tsc_duration[level] =3D >+ tsc_duration / num_iter; >+ } >+ >+ if (benchmarking =3D=3D 0 && output_data_sz) >+ *output_data_sz =3D output_size; >+end: >+ rte_mempool_put_bulk(test_data->op_pool, (void **)ops, allocated); >+ rte_compressdev_private_xform_free(dev_id, priv_xform); >+ rte_free(ops); >+ return res; >+} >+ > int > main(int argc, char **argv) > { >- int ret; >+ uint8_t level, level_idx =3D 0; >+ int ret, i; > struct comp_test_data *test_data; > > /* Initialise DPDK EAL */ >@@ -29,24 +743,186 @@ main(int argc, char **argv) > rte_exit(EXIT_FAILURE, "Cannot reserve memory in socket %d= \n", > rte_socket_id()); > >+ cleanup =3D ST_TEST_DATA; > comp_perf_options_default(test_data); > > if (comp_perf_options_parse(test_data, argc, argv) < 0) { > RTE_LOG(ERR, USER1, > "Parsing one or more user options failed\n"); > ret =3D EXIT_FAILURE; >- goto err; >+ goto end; > } > > if (comp_perf_options_check(test_data) < 0) { > ret =3D EXIT_FAILURE; >- goto err; >+ goto end; >+ } >+ >+ if (comp_perf_initialize_compressdev(test_data) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ >+ cleanup =3D ST_COMPDEV; >+ if (comp_perf_dump_input_data(test_data) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ >+ cleanup =3D ST_INPUT_DATA; >+ if (comp_perf_allocate_memory(test_data) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ >+ if (prepare_bufs(test_data) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ >+ if (test_data->level.inc !=3D 0) >+ level =3D test_data->level.min; >+ else >+ level =3D test_data->level.list[0]; >+ >+ size_t comp_data_sz; >+ size_t decomp_data_sz; >+ >+ printf("Burst size =3D %u\n", test_data->burst_sz); >+ printf("File size =3D %zu\n", test_data->input_data_sz); >+ >+ printf("%6s%12s%17s%19s%21s%15s%21s%23s%16s\n", >+ "Level", "Comp size", "Comp ratio [%]", >+ "Comp [Cycles/it]", "Comp [Cycles/Byte]", "Comp [Gbps]", >+ "Decomp [Cycles/it]", "Decomp [Cycles/Byte]", "Decomp [Gbp= s]"); >+ >+ cleanup =3D ST_DURING_TEST; >+ while (level <=3D test_data->level.max) { >+ /* >+ * Run a first iteration, to verify compression and >+ * get the compression ratio for the level >+ */ >+ if (main_loop(test_data, level, RTE_COMP_COMPRESS, >+ test_data->compressed_data, >+ &comp_data_sz, 0) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ >+ if (main_loop(test_data, level, RTE_COMP_DECOMPRESS, >+ test_data->decompressed_data, >+ &decomp_data_sz, 0) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ >+ if (decomp_data_sz !=3D test_data->input_data_sz) { >+ RTE_LOG(ERR, USER1, >+ "Decompressed data length not equal to input data lengt= h\n"); >+ RTE_LOG(ERR, USER1, >+ "Decompressed size =3D %zu, expected =3D %= zu\n", >+ decomp_data_sz, test_data->input_data_sz); >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } else { >+ if (memcmp(test_data->decompressed_data, >+ test_data->input_data, >+ test_data->input_data_sz) !=3D 0) = { >+ RTE_LOG(ERR, USER1, >+ "Decompressed data is not the same as file dat= a\n"); >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ } >+ >+ double ratio =3D (double) comp_data_sz / >+ test_data->input_data_sz *= 100; >+ >+ /* >+ * Run the tests twice, discarding the first performance >+ * results, before the cache is warmed up >+ */ >+ for (i =3D 0; i < 2; i++) { >+ if (main_loop(test_data, level, RTE_COMP_COMPRESS, >+ NULL, NULL, 1) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ } >+ >+ for (i =3D 0; i < 2; i++) { >+ if (main_loop(test_data, level, RTE_COMP_DECOMPRES= S, >+ NULL, NULL, 1) < 0) { >+ ret =3D EXIT_FAILURE; >+ goto end; >+ } >+ } >+ >+ uint64_t comp_tsc_duration =3D >+ test_data->comp_tsc_duration[level]; >+ double comp_tsc_byte =3D (double)comp_tsc_duration / >+ test_data->input_data_sz; >+ double comp_gbps =3D rte_get_tsc_hz() / comp_tsc_byte * 8 = / >+ 1000000000; >+ uint64_t decomp_tsc_duration =3D >+ test_data->decomp_tsc_duration[level]; >+ double decomp_tsc_byte =3D (double)decomp_tsc_duration / >+ test_data->input_data_sz; >+ double decomp_gbps =3D rte_get_tsc_hz() / decomp_tsc_byte = * 8 / >+ 1000000000; >+ >+ printf("%6u%12zu%17.2f%19"PRIu64"%21.2f" >+ "%15.2f%21"PRIu64"%23.2f%16.2f\n", >+ level, comp_data_sz, ratio, comp_tsc_duration, >+ comp_tsc_byte, comp_gbps, decomp_tsc_duration, >+ decomp_tsc_byte, decomp_gbps); >+ >+ if (test_data->level.inc !=3D 0) >+ level +=3D test_data->level.inc; >+ else { >+ if (++level_idx =3D=3D test_data->level.count) >+ break; >+ level =3D test_data->level.list[level_idx]; >+ } > } > > ret =3D EXIT_SUCCESS; > >-err: >- rte_free(test_data); >+end: >+ switch (cleanup) { > >+ case ST_DURING_TEST: >+ case ST_PREPARE_BUF: >+ free_bufs(test_data); >+ /* fallthrough */ >+ case ST_MEMORY_ALLOC: >+ rte_free(test_data->decomp_bufs); >+ rte_free(test_data->comp_bufs); >+ rte_free(test_data->decompressed_data); >+ rte_free(test_data->compressed_data); >+ rte_mempool_free(test_data->op_pool); >+ rte_mempool_free(test_data->decomp_buf_pool); >+ rte_mempool_free(test_data->comp_buf_pool); >+ /* fallthrough */ >+ case ST_INPUT_DATA: >+ rte_free(test_data->input_data); >+ /* fallthrough */ >+ case ST_COMPDEV: >+ if (test_data->cdev_id !=3D -1) >+ rte_compressdev_stop(test_data->cdev_id); >+ /* fallthrough */ >+ case ST_TEST_DATA: >+ rte_free(test_data); >+ /* fallthrough */ >+ case ST_CLEAR: >+ default: >+ i =3D rte_eal_cleanup(); >+ if (i) { >+ RTE_LOG(ERR, USER1, >+ "Error from rte_eal_cleanup(), %d\n", i); >+ ret =3D i; >+ } >+ break; >+ } > return ret; > } >-- >2.7.4