From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0086.outbound.protection.outlook.com [104.47.37.86]) by dpdk.org (Postfix) with ESMTP id 12E4B2C4F for ; Thu, 15 Feb 2018 06:53:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=mBnng9csOfhO9V53y/OY3sVriz5vyxAkhjpbz2DR1mM=; b=TKbsEdf33QFKchBFNPxWZKK/lNnJjbZZ6D4arjxfmzvvrXnYBuYTz/p5coBwfuqnp+raZQGyJW84W7nbG/W/uGUrN2xcqD1uzgfcoK2gxo9UuvTxZoBBoBoLwTcmH+kFiKC8fHzfyHk4DYBgEhd1mEoqpONhnWhgtA4F94Gxv1c= Received: from CY4PR0701MB3634.namprd07.prod.outlook.com (52.132.101.164) by CY4PR0701MB3667.namprd07.prod.outlook.com (52.132.102.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.485.10; Thu, 15 Feb 2018 05:53:31 +0000 Received: from CY4PR0701MB3634.namprd07.prod.outlook.com ([fe80::a90e:9fcd:9ebd:8cad]) by CY4PR0701MB3634.namprd07.prod.outlook.com ([fe80::a90e:9fcd:9ebd:8cad%13]) with mapi id 15.20.0485.019; Thu, 15 Feb 2018 05:53:29 +0000 From: "Verma, Shally" To: Ahmed Mansour , "Trahe, Fiona" , "dev@dpdk.org" CC: "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "De Lara Guarch, Pablo" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , Roy Pledge , Youri Querry Thread-Topic: [RFC v2] doc compression API for DPDK Thread-Index: AdOFUW8Wdt99b3u6RKydGSrxJwvtHggzj4/g Date: Thu, 15 Feb 2018 05:53:29 +0000 Message-ID: References: <348A99DA5F5B7549AA880327E580B435892F589D@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B43589315232@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B43589315AF3@IRSMSX101.ger.corp.intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [117.98.159.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY4PR0701MB3667; 7:wnDLrSoKvhzaexp/oZ/nWWiwwiZ+uIMEHWp2+jGDm1JkNalNFWD2MobKYskoSNp3OzyBItFF4rIuLAUD2Acn8m9HtsmHjvRC6ohOki/lY4T24C51ev01SXWbyVLRQWdVJVYI1mvWnf4OJnQPZzzMbaQaPksq++ojyR7h79Vbupb7Gq6fC62WGhPpbOatR3wo33IoSvmg1SunFxF7hc9uSt8G8EgRnTOUkDutOLTaQticxVrJBKJhlOta0WX1c/jn x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(396003)(366004)(39380400002)(376002)(39860400002)(346002)(199004)(189003)(57704003)(13464003)(53936002)(305945005)(7736002)(74316002)(7696005)(110136005)(54906003)(3280700002)(6436002)(3660700001)(86362001)(14454004)(99286004)(55016002)(316002)(9686003)(8656006)(186003)(97736004)(26005)(106356001)(25786009)(105586002)(478600001)(66066001)(33656002)(4326008)(6246003)(76176011)(8676002)(6506007)(59450400001)(102836004)(53546011)(72206003)(5250100002)(2501003)(68736007)(2906002)(5660300001)(6116002)(3846002)(2900100001)(229853002)(93886005)(2950100002)(81166006)(8936002)(81156014); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR0701MB3667; H:CY4PR0701MB3634.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; x-ms-office365-filtering-correlation-id: 321a0e2e-6f20-48ee-f437-08d57438706b x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060)(7193020); SRVR:CY4PR0701MB3667; x-ms-traffictypediagnostic: CY4PR0701MB3667: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(278428928389397)(185117386973197)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231101)(944501161)(6041288)(20161123558120)(20161123564045)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:CY4PR0701MB3667; BCL:0; PCL:0; RULEID:; SRVR:CY4PR0701MB3667; x-forefront-prvs: 058441C12A received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 2w0lTlA+PZmfRrSGP0I+MvPrMZUnDxhwKmJ7lHEv0o/BXWboZPb1wQxfVrVSwglseI0LEnromUhxeLKCw/32xg== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 321a0e2e-6f20-48ee-f437-08d57438706b X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Feb 2018 05:53:29.4188 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR0701MB3667 Subject: Re: [dpdk-dev] [RFC v2] doc compression API for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Feb 2018 05:53:33 -0000 >-----Original Message----- >From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com] >Sent: 14 February 2018 22:25 >To: Verma, Shally ; Trahe, Fiona ; dev@dpdk.org >Cc: Athreya, Narayana Prasad ; Gupta, A= shish ; Sahu, Sunila >; De Lara Guarch, Pablo ; Challa, Mahipal >; Jain, Deepak K ; Hem= ant Agrawal ; Roy >Pledge ; Youri Querry >Subject: Re: [RFC v2] doc compression API for DPDK > >On 2/14/2018 12:41 AM, Verma, Shally wrote: >> Hi Ahmed >> >>> -----Original Message----- >>> From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com] >>> Sent: 02 February 2018 02:20 >>> To: Trahe, Fiona ; Verma, Shally ; dev@dpdk.org >>> Cc: Athreya, Narayana Prasad ; Gupta= , Ashish ; Sahu, Sunila >>> ; De Lara Guarch, Pablo ; Challa, Mahipal >>> ; Jain, Deepak K ; = Hemant Agrawal ; Roy >>> Pledge ; Youri Querry >>> Subject: Re: [RFC v2] doc compression API for DPDK >>> >>>>>> [Fiona] I propose if BFINAL bit is detected before end of input >>>>>> the decompression should stop. In this case consumed will be < src.l= ength. >>>>>> produced will be < dst buffer size. Do we need an extra STATUS respo= nse? >>>>>> STATUS_BFINAL_DETECTED ? >>>>> [Shally] @fiona, I assume you mean here decompressor stop after proce= ssing Final block right? >>>> [Fiona] Yes. >>>> >>>> And if yes, >>>>> and if it can process that final block successfully/unsuccessfully, t= hen status could simply be >>>>> SUCCESS/FAILED. >>>>> I don't see need of specific return code for this use case. Just to s= hare, in past, we have practically run into >>>>> such cases with boost lib, and decompressor has simply worked this wa= y. >>>> [Fiona] I'm ok with this. >>>> >>>>>> Only thing I don't like this is it can impact on performance, as nor= mally >>>>>> we can just look for STATUS =3D=3D SUCCESS. Anything else should be = an exception. >>>>>> Now the application would have to check for SUCCESS || BFINAL_DETECT= ED every time. >>>>>> Do you have a suggestion on how we should handle this? >>>>>> >>> [Ahmed] This makes sense. So in all cases the PMD should assume that it >>> should stop as soon as a BFINAL is observed. >>> >>> A question. What happens ins stateful vs stateless modes when >>> decompressing an op that encompasses multiple BFINALs. I assume the >>> caller in that case will use the consumed=3Dx bytes to find out how far= in >>> to the input is the end of the first stream and start from the next >>> byte. Is this correct? >> [Shally] As per my understanding, each op can be tied up to only one st= ream as we have only one stream pointer per op and one >stream can have only one BFINAL (as stream is one complete compressed data= ) but looks like you're suggesting a case where one op >can carry multiple independent streams? and thus multiple BFINAL?! , such = as, below here is op pointing to more than one streams >> >> -------------------------------------------- >> op --> |stream1|stream2| |stream3| >> -------------------------------------------- >> >> Could you confirm if I understand your question correct? >[Ahmed] Correct. We found that in some storage applications the user >does not know where exactly the BFINAL is. They rely on zlib software >today. zlib.net software halts at the first BFINAL. Users put multiple >streams in one op and rely on zlib to stop and inform them of the end >location of the first stream. [Shally] Then this is practically case possible on decompressor and decompr= essor doesn't regard flush flag. So in that case, I expect PMD to internall= y reset themselves (say in case of zlib going through cycle of deflateEnd a= nd deflateInit or deflateReset) and return with status =3D SUCCESS with upd= ated produced and consumed. Now in such case, if previous stream also has s= ome footer followed by start of next stream, then I am not sure how PMD / l= ib can support that case. Have you had practically run of such use-case on = zlib? If yes, how then such application handle it in your experience?=20 I can imagine for such input zlib would return with Z_FLUSH_END after 1st B= FINAL is processed to the user. Then application doing deflateReset() or In= it-End() cycle before starting with next. But if it starts with input that = doesn't have valid zlib header, then likely it will throw an error. >> >> Thanks >> Shally >>