From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 89FDB1B29F for ; Thu, 15 Feb 2018 18:20:07 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Feb 2018 09:20:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,517,1511856000"; d="scan'208";a="20399503" Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159]) by fmsmga002.fm.intel.com with ESMTP; 15 Feb 2018 09:20:03 -0800 Received: from irsmsx111.ger.corp.intel.com (10.108.20.4) by IRSMSX104.ger.corp.intel.com (163.33.3.159) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 15 Feb 2018 17:20:02 +0000 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.188]) by irsmsx111.ger.corp.intel.com ([169.254.2.246]) with mapi id 14.03.0319.002; Thu, 15 Feb 2018 17:20:02 +0000 From: "Trahe, Fiona" To: "Verma, Shally" , Ahmed Mansour , "dev@dpdk.org" CC: "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "De Lara Guarch, Pablo" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , Roy Pledge , Youri Querry , "Trahe, Fiona" Thread-Topic: [RFC v2] doc compression API for DPDK Thread-Index: AdOFUW8Wdt99b3u6RKydGSrxJwvtHggzj4/gABZEu6A= Date: Thu, 15 Feb 2018 17:20:02 +0000 Message-ID: <348A99DA5F5B7549AA880327E580B4358931F4E3@IRSMSX101.ger.corp.intel.com> References: <348A99DA5F5B7549AA880327E580B435892F589D@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B43589315232@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B43589315AF3@IRSMSX101.ger.corp.intel.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZjNiMjYyNGEtYTE1NC00ZjEzLTkwOTMtYTMzN2IzMzY5N2YyIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IlpXTDBtYjJLUG5nSzlUSWNRYWVuRkdYcmprUmJDd0ZWRGNqYlwvXC95c0l1dz0ifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC v2] doc compression API for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Feb 2018 17:20:08 -0000 Hi Ahmed, Shally, > -----Original Message----- > From: Verma, Shally [mailto:Shally.Verma@cavium.com] > Sent: Thursday, February 15, 2018 5:53 AM > To: Ahmed Mansour ; Trahe, Fiona ; > dev@dpdk.org > Cc: Athreya, Narayana Prasad ; Gupta, = Ashish > ; Sahu, Sunila ; De Lara= Guarch, Pablo > ; Challa, Mahipal ; Jain, Deepak K > ; Hemant Agrawal ; Roy P= ledge > ; Youri Querry > Subject: RE: [RFC v2] doc compression API for DPDK >=20 >=20 >=20 > >-----Original Message----- > >From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com] > >Sent: 14 February 2018 22:25 > >To: Verma, Shally ; Trahe, Fiona ; dev@dpdk.org > >Cc: Athreya, Narayana Prasad ; Gupta,= Ashish > ; Sahu, Sunila > >; De Lara Guarch, Pablo ; Challa, > Mahipal > >; Jain, Deepak K ; H= emant Agrawal > ; Roy > >Pledge ; Youri Querry > >Subject: Re: [RFC v2] doc compression API for DPDK > > > >On 2/14/2018 12:41 AM, Verma, Shally wrote: > >> Hi Ahmed > >> > >>> -----Original Message----- > >>> From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com] > >>> Sent: 02 February 2018 02:20 > >>> To: Trahe, Fiona ; Verma, Shally ; > dev@dpdk.org > >>> Cc: Athreya, Narayana Prasad ; Gup= ta, Ashish > ; Sahu, Sunila > >>> ; De Lara Guarch, Pablo ; Challa, > Mahipal > >>> ; Jain, Deepak K = ; Hemant Agrawal > ; Roy > >>> Pledge ; Youri Querry > >>> Subject: Re: [RFC v2] doc compression API for DPDK > >>> > >>>>>> [Fiona] I propose if BFINAL bit is detected before end of input > >>>>>> the decompression should stop. In this case consumed will be < src= .length. > >>>>>> produced will be < dst buffer size. Do we need an extra STATUS res= ponse? > >>>>>> STATUS_BFINAL_DETECTED ? > >>>>> [Shally] @fiona, I assume you mean here decompressor stop after pro= cessing Final block right? > >>>> [Fiona] Yes. > >>>> > >>>> And if yes, > >>>>> and if it can process that final block successfully/unsuccessfully,= then status could simply be > >>>>> SUCCESS/FAILED. > >>>>> I don't see need of specific return code for this use case. Just to= share, in past, we have practically > run into > >>>>> such cases with boost lib, and decompressor has simply worked this = way. > >>>> [Fiona] I'm ok with this. > >>>> > >>>>>> Only thing I don't like this is it can impact on performance, as n= ormally > >>>>>> we can just look for STATUS =3D=3D SUCCESS. Anything else should b= e an exception. > >>>>>> Now the application would have to check for SUCCESS || BFINAL_DETE= CTED every time. > >>>>>> Do you have a suggestion on how we should handle this? > >>>>>> > >>> [Ahmed] This makes sense. So in all cases the PMD should assume that = it > >>> should stop as soon as a BFINAL is observed. > >>> > >>> A question. What happens ins stateful vs stateless modes when > >>> decompressing an op that encompasses multiple BFINALs. I assume the > >>> caller in that case will use the consumed=3Dx bytes to find out how f= ar in > >>> to the input is the end of the first stream and start from the next > >>> byte. Is this correct? > >> [Shally] As per my understanding, each op can be tied up to only one = stream as we have only one > stream pointer per op and one > >stream can have only one BFINAL (as stream is one complete compressed da= ta) but looks like you're > suggesting a case where one op > >can carry multiple independent streams? and thus multiple BFINAL?! , suc= h as, below here is op > pointing to more than one streams > >> > >> -------------------------------------------- > >> op --> |stream1|stream2| |stream3| > >> -------------------------------------------- > >> > >> Could you confirm if I understand your question correct? > >[Ahmed] Correct. We found that in some storage applications the user > >does not know where exactly the BFINAL is. They rely on zlib software > >today. zlib.net software halts at the first BFINAL. Users put multiple > >streams in one op and rely on zlib to stop and inform them of the end > >location of the first stream. >=20 > [Shally] Then this is practically case possible on decompressor and decom= pressor doesn't regard flush > flag. So in that case, I expect PMD to internally reset themselves (say i= n case of zlib going through cycle > of deflateEnd and deflateInit or deflateReset) and return with status =3D= SUCCESS with updated produced > and consumed. Now in such case, if previous stream also has some footer f= ollowed by start of next > stream, then I am not sure how PMD / lib can support that case. Have you = had practically run of such > use-case on zlib? If yes, how then such application handle it in your exp= erience? > I can imagine for such input zlib would return with Z_FLUSH_END after 1st= BFINAL is processed to the > user. Then application doing deflateReset() or Init-End() cycle before st= arting with next. But if it starts > with input that doesn't have valid zlib header, then likely it will throw= an error. >=20 [Fiona] The consumed and produced tell the Application hw much data was pro= cessed up to=20 the end of the first deflate block encountered with a bfinal set. If there is data, e.g. footer after the block with bfinal, then I think it = must be the responsibility of the application to know this, the PMD can't have any responsibility for thi= s. The next op sent to the PMD must start with a valid deflate block. > >> > >> Thanks > >> Shally > >>