From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM05-CO1-obe.outbound.protection.outlook.com (mail-eopbgr720083.outbound.protection.outlook.com [40.107.72.83]) by dpdk.org (Postfix) with ESMTP id 3063F5689 for ; Mon, 5 Nov 2018 09:34:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dLuOwa1eOPU+KVSVn2N6qtsWxtILR7YCjRc2sj3bTzo=; b=C+Z7ehQl1osYTGN76RPRHM5QDW7j8gevKSHZcC7vxbrfeXJHm/YrZlWbX1ztEjxkmMPDCVokC4H1e3s6ftEhkm9uDdmNPyOyh80LanW+ZbnM0RCAmwwp8wY7yaxxC2gIpz6/uzuAjE5IGbhj+MzbZPzrmtd28cDMWpugj1BBTvY= Received: from SN6PR07MB5152.namprd07.prod.outlook.com (52.135.101.33) by SN6PR07MB5344.namprd07.prod.outlook.com (52.135.105.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.26; Mon, 5 Nov 2018 08:34:14 +0000 Received: from SN6PR07MB5152.namprd07.prod.outlook.com ([fe80::49cb:b2a:974:2211]) by SN6PR07MB5152.namprd07.prod.outlook.com ([fe80::49cb:b2a:974:2211%4]) with mapi id 15.20.1294.028; Mon, 5 Nov 2018 08:34:14 +0000 From: "Verma, Shally" To: "Jozwiak, TomaszX" , "dev@dpdk.org" , "Trahe, Fiona" , "akhil.goyal@nxp.com" , "De Lara Guarch, Pablo" CC: "De@dpdk.org" , "Lara@dpdk.org" , "Guarch@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance measurement Thread-Index: AQHUWYqjhLJt3XAtpkeXnW8WfRqeNaUVmp7AgCbXsACABJ3+AA== Date: Mon, 5 Nov 2018 08:34:14 +0000 Message-ID: References: <1538400427-20164-1-git-send-email-tomaszx.jozwiak@intel.com> <1538400427-20164-3-git-send-email-tomaszx.jozwiak@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [115.113.156.3] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; SN6PR07MB5344; 6:e2iFf39+oNNWRyo6AR2eUlDPjMH4E1gA9XqaDtwaToXnS6eQdKtVAnUmIHBIJ3s/GdqwfZKvzcHE2qTmnf2iu9myHVNp75XmKZmoC9eU1zAMzpogjiAmdfRgrOsWScdBLtsezVROpyMSd71DkDf15DpWfdVz8fFpa04IvzRg2c3OT1ZNhZtG3E0vHfKrubJaxKBAfFSoLY5eMcMpMd6EjSfjT8hMMUkHC7WazOrkl61/HLlFLQx7CLr+z17ryGEEnjObGGGYybfaCiGatQi6HiW0PkZj7Xy85O0PGaer/MpUkLuTnU0ukb47yf7IaA8DhWt6ePL1g4GKYhBHC3y4AwsSovysslLFizJhJlOwtLDfFEMbR/V0yjtQ2xuZ/gPARYxAmfX+kaNBqMwUf5WzxH99ONnQgNHs7qD7MRONse08D+wJrc6zBGnXuoELIdNE/HCxUl7CXhBZqbsKeMstEw==; 5:w+SQRrmOofulIHcLo+s16N8YOkYMs5pTJSWN+Rwf0Wg/v0mmOAn6m3zb0EPDyXCJ071dCXKVx0J/1QLdVOhSLHY7IzzJqQutHHw8WYE7kv+7abTV185ImylGnJBP48N/vLiZ6Zr+QEFPzyaho5XJK3PcMJFwbaxiP59Aj0VmlTA=; 7:cPvic6e4jCeHzFujJgpDYO0dPdMZx8c7Hv11XpYkuabsp50xF6EgXu3x/Xz9RtMftVxaTS7AVVfx3XHyekXFT4LgbidB1aJdbz8N19H5Kye/BEgK7+Tj28ChO3EPleOOUKg1JBLJ1SRMEyXMDz6OWA== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 6f5bb221-979e-44d9-5dea-08d642f977ed x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:SN6PR07MB5344; x-ms-traffictypediagnostic: SN6PR07MB5344: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(185117386973197)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(3231382)(944501410)(52105095)(3002001)(93006095)(93001095)(148016)(149066)(150057)(6041310)(20161123560045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:SN6PR07MB5344; BCL:0; PCL:0; RULEID:; SRVR:SN6PR07MB5344; x-forefront-prvs: 08476BC6EF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(136003)(396003)(346002)(39860400002)(13464003)(189003)(199004)(3846002)(71200400001)(6116002)(71190400001)(4744004)(305945005)(25786009)(74316002)(8936002)(66066001)(7736002)(68736007)(478600001)(4326008)(86362001)(2906002)(81166006)(81156014)(105586002)(106356001)(229853002)(2501003)(97736004)(55236004)(102836004)(316002)(76176011)(486006)(5660300001)(476003)(93886005)(11346002)(26005)(446003)(186003)(54906003)(110136005)(256004)(6246003)(53546011)(9686003)(53936002)(2900100001)(72206003)(99286004)(7696005)(6436002)(6506007)(14454004)(55016002)(33656002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR07MB5344; H:SN6PR07MB5152.namprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: U9bFVlLPEiyJux52cVYg+imTaD4iv4g4AV75XOAaG+MC8vmAsir3uu0k3kSQLFAX59GinS3FQ34wZtC0ugPY9cuQca/Ysek1vDmGYd44Issfv2D1DLAYnkXp3w4RnTbZkeFzoXlywSYwBE3x5scuzoCTFpRb0V2rDwv3iRJDWBlY+1YVw/DnOEBdHVWw4yq+wx497hR1AhIDFjXnIRQVRNqOotPy/T72VjxGu7FHjN1J6lyzO//4r6fY4b+FaJGPnfVBSnc/pN4WwSJLnCIAa0wIPYF4Lw3P2TU/04JQtZuvc99pPO9Wd5mqTL3d6fz+VUk4NrMXfMV22ZsMs2v23HkXudHYouFqpSs/jNra72g= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6f5bb221-979e-44d9-5dea-08d642f977ed X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Nov 2018 08:34:14.4597 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR07MB5344 Subject: Re: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance measurement X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 08:34:17 -0000 >-----Original Message----- >From: Jozwiak, TomaszX >Sent: 02 November 2018 15:29 >To: dev@dpdk.org; Trahe, Fiona ; akhil.goyal@nxp.co= m; Verma, Shally ; De Lara >Guarch, Pablo >Cc: De@dpdk.org; Lara@dpdk.org; Guarch@dpdk.org >Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance mea= surement > >External Email > >Hi Shally, > >Sorry for delay - I was on sick leave. >We had some issues with dynamic compression test so I block this test in V= 2. May be there's too late to add this into this release but >we've decided to send this V2 to DPDK. > >My comment inline (not all have answer so far, still working on that) > >> -----Original Message----- >> From: Verma, Shally [mailto:Shally.Verma@cavium.com] >> Sent: Friday, October 12, 2018 12:16 PM >> To: Jozwiak, TomaszX ; dev@dpdk.org; Trahe, >> Fiona ; akhil.goyal@nxp.com; De Lara Guarch, Pabl= o >> >> Cc: De@dpdk.org; Lara@dpdk.org; Guarch@dpdk.org >> Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance >> measurement >> >> HI TomaszX >> >> Sorry for delay in response. Comments inline. >> >> >-----Original Message----- >> >From: dev On Behalf Of Tomasz Jozwiak >> >Sent: 01 October 2018 18:57 >> >To: dev@dpdk.org; fiona.trahe@intel.com; tomaszx.jozwiak@intel.com; >> >akhil.goyal@nxp.com; pablo.de.lara.guarch@intel.com >> >Cc: De@dpdk.org; Lara@dpdk.org; Guarch@dpdk.org >> >Subject: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance >> >measurement >> > >> >External Email >> > >> >Added performance measurement part into compression perf. test. >> > >> >Signed-off-by: De Lara Guarch, Pablo >> >Signed-off-by: Tomasz Jozwiak >> >--- >> > app/test-compress-perf/main.c | 844 >> >++++++++++++++++++++++++++++++++++++++++++ >> > 1 file changed, 844 insertions(+) >> > >> >diff --git a/app/test-compress-perf/main.c >> >b/app/test-compress-perf/main.c index f52b98d..093dfaf 100644 >> >--- a/app/test-compress-perf/main.c >> >+++ b/app/test-compress-perf/main.c >> >@@ -5,13 +5,721 @@ >> > #include >> > #include >> > #include >> >+#include >> > #include >> > >> > #include "comp_perf_options.h" >> > >> >+#define NUM_MAX_XFORMS 16 >> >+#define NUM_MAX_INFLIGHT_OPS 512 >> >+#define EXPANSE_RATIO 1.05 >> >+#define MIN_ISAL_SIZE 8 >> >+ >> >+#define DIV_CEIL(a, b) ((a) / (b) + ((a) % (b) !=3D 0)) >> >+ >> >+static int >> >+param_range_check(uint16_t size, const struct rte_param_log2_range >> >+*range) { >> >+ unsigned int next_size; >> >+ >> >+ /* Check lower/upper bounds */ >> >+ if (size < range->min) >> >+ return -1; >> >+ >> >+ if (size > range->max) >> >+ return -1; >> >+ >> >+ /* If range is actually only one value, size is correct */ >> >+ if (range->increment =3D=3D 0) >> >+ return 0; >> >+ >> >+ /* Check if value is one of the supported sizes */ >> >+ for (next_size =3D range->min; next_size <=3D range->max; >> >+ next_size +=3D range->increment) >> >+ if (size =3D=3D next_size) >> >+ return 0; >> >+ >> >+ return -1; >> >+} >> >+ >> >+static int >> >+comp_perf_check_capabilities(struct comp_test_data *test_data) { >> >+ const struct rte_compressdev_capabilities *cap; >> >+ >> >+ cap =3D rte_compressdev_capability_get(test_data->cdev_id, >> >+ RTE_COMP_ALGO_DEFLATE); >> >+ >> >+ if (cap =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Compress device does not support DEFLATE\n"); >> >+ return -1; >> >+ } >> >+ >> >+ uint64_t comp_flags =3D cap->comp_feature_flags; >> >+ >> >+ /* Huffman enconding */ >> >+ if (test_data->huffman_enc =3D=3D RTE_COMP_HUFFMAN_FIXED && >> >+ (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) =3D=3D= 0) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Compress device does not supported Fixed Huffm= an\n"); >> >+ return -1; >> >+ } >> >+ >> >+ if (test_data->huffman_enc =3D=3D RTE_COMP_HUFFMAN_DYNAMIC && >> >+ (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) =3D= =3D 0) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Compress device does not supported Dynamic Huf= fman\n"); >> >+ return -1; >> >+ } >> >+ >> >+ /* Window size */ >> >+ if (test_data->window_sz !=3D -1) { >> >+ if (param_range_check(test_data->window_sz, >> >+ &cap->window_size) >> What if cap->window_size is 0 i.e. implementation default? > >TJ: You probably mean cap->window_size.increment =3D 0 (because cap->windo= w_size is a structure). In that case we check if >test_data->window_sz >=3Dmin and test_data->window_sz <=3D max only, becau= se increment =3D 0 means (base on compression API) we >have only one value of windows_size (no range is supported). But PMD can set min and max too 0 for such case. > > > .... >> >+ >> >+ if (fread(data, data_to_read, 1, f) !=3D 1) { >> >+ RTE_LOG(ERR, USER1, "Input file could not be re= ad\n"); >> >+ goto err; >> >+ } >> >+ if (fseek(f, 0, SEEK_SET) !=3D 0) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Size of input could not be calculated\= n"); >> >+ goto err; >> >+ } >> >+ remaining_data -=3D data_to_read; >> >+ data +=3D data_to_read; >> It looks like it will run 2nd time only if input file size < input data = size in which >> case it will just keep filling input buffer with repeated data. >> Is that the intention here? > >TJ: Yes exactly. If test_data->input_data_sz is bigger than actual_file_s= z then we fill the buffer with repeated data from file to fill >whole buffer. I mentioned in one of the earlier reply, wont that then influence the compr= ession behaviour and o/p? my suggestion was to work on actual user provided= input to take perf to get actual perf for given content. > >> ... >> >+ if (data_addr =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, "Could not append >> >+ data\n"); >> Since a new buffer per segment is allocated, so is it possible for appen= d to >> fail? think, this check is redundant here. > >TJ: Yes, you're right, it should never fail. But I think it's good coding = practice to add the check just in case. > Unless it is called in data path which might cost perf a bit. Thanks Shally >> >+ return -1; >> >+ } >> >+ >> >+ rte_memcpy(data_addr, input_data_ptr, data_sz); >> >+ input_data_ptr +=3D data_sz; >> >+ remaining_data -=3D data_sz; >> >+ >> >+ if (rte_pktmbuf_chain(test_data->decomp_bufs[i]= , >> >+ next_seg) < 0) { >> >+ RTE_LOG(ERR, USER1, "Could not chain mb= ufs\n"); >> >+ return -1; >> >+ } >> >+ segs_per_mbuf++; >> >+ } >> >+ >> >+ /* Allocate data in output mbuf */ >> >+ test_data->comp_bufs[i] =3D >> >+ rte_pktmbuf_alloc(test_data->comp_buf_pool); >> >+ if (test_data->comp_bufs[i] =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, "Could not allocate mbuf\n"= ); >> >+ return -1; >> >+ } >> >+ data_addr =3D (uint8_t *) rte_pktmbuf_append( >> >+ test_data->comp_bufs[i], >> >+ test_data->seg_sz); >> >+ if (data_addr =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, "Could not append data\n"); >> >+ return -1; >> >+ } >> >+ >> >+ /* Chain mbufs if needed for output mbufs */ >> >+ for (j =3D 1; j < segs_per_mbuf; j++) { >> >+ struct rte_mbuf *next_seg =3D >> >+ >> >+ rte_pktmbuf_alloc(test_data->comp_buf_pool); >> >+ >> >+ if (next_seg =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Could not allocate mbuf\n"); >> >+ return -1; >> >+ } >> >+ >> >+ data_addr =3D (uint8_t *)rte_pktmbuf_append(nex= t_seg, >> >+ test_data->seg_sz); >> >+ >> >+ if (data_addr =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, "Could not append d= ata\n"); >> >+ return -1; >> >+ } >> >+ >> >+ if (rte_pktmbuf_chain(test_data->comp_bufs[i], >> >+ next_seg) < 0) { >> >+ RTE_LOG(ERR, USER1, "Could not chain mb= ufs\n"); >> >+ return -1; >> >+ } >> >+ } >> >+ } >> >+ >> >+ return 0; >> >+} >> >+ >> >+static void >> >+free_bufs(struct comp_test_data *test_data) { >> >+ uint32_t i; >> >+ >> >+ for (i =3D 0; i < test_data->total_bufs; i++) { >> >+ rte_pktmbuf_free(test_data->comp_bufs[i]); >> >+ rte_pktmbuf_free(test_data->decomp_bufs[i]); >> >+ } >> >+ rte_free(test_data->comp_bufs); >> >+ rte_free(test_data->decomp_bufs); } >> >+ >> >+static int >> >+main_loop(struct comp_test_data *test_data, uint8_t level, >> >+ enum rte_comp_xform_type type, >> >+ uint8_t *output_data_ptr, >> >+ size_t *output_data_sz, >> >+ unsigned int benchmarking) { >> >+ uint8_t dev_id =3D test_data->cdev_id; >> >+ uint32_t i, iter, num_iter; >> >+ struct rte_comp_op **ops, **deq_ops; >> >+ void *priv_xform =3D NULL; >> >+ struct rte_comp_xform xform; >> >+ size_t output_size =3D 0; >> >+ struct rte_mbuf **input_bufs, **output_bufs; >> >+ int res =3D 0; >> >+ int allocated =3D 0; >> >+ >> >+ if (test_data =3D=3D NULL || !test_data->burst_sz) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Unknow burst size\n"); >> >+ return -1; >> >+ } >> >+ >> >+ ops =3D rte_zmalloc_socket(NULL, >> >+ 2 * test_data->total_bufs * sizeof(struct rte_comp_op *= ), >> >+ 0, rte_socket_id()); >> >+ >> >+ if (ops =3D=3D NULL) { >> >+ RTE_LOG(ERR, USER1, >> >+ "Can't allocate memory for ops strucures\n"); >> >+ return -1; >> >+ } >> >+ >> >+ deq_ops =3D &ops[test_data->total_bufs]; >> >+ >> >+ if (type =3D=3D RTE_COMP_COMPRESS) { >> >+ xform =3D (struct rte_comp_xform) { >> >+ .type =3D RTE_COMP_COMPRESS, >> >+ .compress =3D { >> >+ .algo =3D RTE_COMP_ALGO_DEFLATE, >> >+ .deflate.huffman =3D test_data->huffman= _enc, >> >+ .level =3D level, >> >+ .window_size =3D test_data->window_sz, >> >+ .chksum =3D RTE_COMP_CHECKSUM_NONE, >> >+ .hash_algo =3D RTE_COMP_HASH_ALGO_NONE >> >+ } >> >+ }; >> >+ input_bufs =3D test_data->decomp_bufs; >> >+ output_bufs =3D test_data->comp_bufs; >> >+ } else { >> >+ xform =3D (struct rte_comp_xform) { >> >+ .type =3D RTE_COMP_DECOMPRESS, >> >+ .decompress =3D { >> >+ .algo =3D RTE_COMP_ALGO_DEFLATE, >> >+ .chksum =3D RTE_COMP_CHECKSUM_NONE, >> >+ .window_size =3D test_data->window_sz, >> >+ .hash_algo =3D RTE_COMP_HASH_ALGO_NONE >> >+ } >> >+ }; >> >+ input_bufs =3D test_data->comp_bufs; >> >+ output_bufs =3D test_data->decomp_bufs; >> >+ } >> >+ >> >+ /* Create private xform */ >> >+ if (rte_compressdev_private_xform_create(dev_id, &xform, >> >+ &priv_xform) < 0) { >> >+ RTE_LOG(ERR, USER1, "Private xform could not be created= \n"); >> >+ res =3D -1; >> >+ goto end; >> >+ } >> >+ >> >+ uint64_t tsc_start, tsc_end, tsc_duration; >> >+ >> >+ tsc_start =3D tsc_end =3D tsc_duration =3D 0; >> >+ if (benchmarking) { >> >+ tsc_start =3D rte_rdtsc(); >> >+ num_iter =3D test_data->num_iter; >> >+ } else >> >+ num_iter =3D 1; >> Looks like in same code we're doing benchmarking and functional validati= on. >> It can be reorganised to keep validation test separately like done in >> crypto_perf. > >TJ: Ok, makes sense. However in the interests of getting this into the 18.= 11 release I'd like to >defer this refactoring and the remainder of your comments below to the nex= t release. > > >Next comments - WIP > > >Br, Tomek