From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00041.outbound.protection.outlook.com [40.107.0.41]) by dpdk.org (Postfix) with ESMTP id ED24E2BC8 for ; Thu, 1 Feb 2018 21:23:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=QuOBhcNT/8ZSrnPat4W1P0CrvJ9twYMX1+HAVswMZME=; b=rqoL48yNXd6OytFllnTQ1bBlLii4sIOnEgikekYpxUDtmi+9edAM7Md5u1x0ZNWwbVF386/LoQw5uWthF4AcSRq5BbJ+dXS6ex/YA5A3kvvHOz9q5EhPcAuN1qEmjbT4UI7h5d94fS05jlFafae3HIBPoYPvurQNswEOY/K/Kvs= Received: from DB3PR0402MB3852.eurprd04.prod.outlook.com (52.134.71.143) by DB3PR0402MB3769.eurprd04.prod.outlook.com (52.134.71.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.444.14; Thu, 1 Feb 2018 20:23:26 +0000 Received: from DB3PR0402MB3852.eurprd04.prod.outlook.com ([fe80::bcb3:a74f:d325:7467]) by DB3PR0402MB3852.eurprd04.prod.outlook.com ([fe80::bcb3:a74f:d325:7467%13]) with mapi id 15.20.0444.016; Thu, 1 Feb 2018 20:23:25 +0000 From: Ahmed Mansour To: "Trahe, Fiona" , "Verma, Shally" , "dev@dpdk.org" CC: "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "De Lara Guarch, Pablo" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , Roy Pledge , Youri Querry Thread-Topic: [RFC v2] doc compression API for DPDK Thread-Index: AdOFUW8Wdt99b3u6RKydGSrxJwvtHg== Date: Thu, 1 Feb 2018 20:23:25 +0000 Message-ID: References: <348A99DA5F5B7549AA880327E580B435892F589D@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B43589315232@IRSMSX101.ger.corp.intel.com> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=ahmed.mansour@nxp.com; x-originating-ip: [192.88.168.1] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0402MB3769; 6:oiber6UKYytyS0BVmnKVpPIUWGoA35NkkZQbMD7F9RN0cxIBJz3lua8EiD3y7J/4wv+R/jyDHXsrO+3hxS2SFQ57XKofD0qg2+xIyw8DFZ82OC7nyDJyYNUazuyEqaFPccGR8IyJh3r1cL29bbFrVk8oZucm7U16zA9Zi6b76akDV0MtuHecU39cAyJ7llU6vq6ipuzB264KIBOtOJlcw7h5rFkYgSIsdi47MXlq0ol9uSBCS/iimajDr+4y2B65NiHDk3BD9y2vGYlAoYhzfjfOhaTIf3ycqa6oDZqk+3kg3nBH98GvkIboN/qpO3J3HkH6jFexdetBATDmc8TiACNh4tls3ahmw9kjDLHtkIAGmhnfCyd+y5oWIZj1Nzp6; 5:Lm5vQe3X6BDCnGGaLwxONDR9PYzIn7g++N+DJ1qYOz0Snkwbh9GQ81h6xfs+KOGsnr1xbK8cMILN0XiEsM5PjbuifsykNnDY8MpZzYE4XobmdX4cP5GHkCBHoZ2ppA20f0pofHzI1ooO0bQR4bcK8UACJRtru4IEA2Lbn8qJY1k=; 24:tnDfiJgG+zyLQuBA4GWtY/6VokkRoKT8WlgeTLWhv763IJD6wLyHERfCyuzSzfbW2hfEuBhzTwTfwTSTeT9FFoinb9Mw7JRfrCEVk5ODets=; 7:A8im4nh1cYSiNOll7VXHhK6vSmF5Gwmw2LxhadhFEa7rlOBwh6A6Zr4w+lAF+n882JEUoYDuGajmk71ipI2y/UfSwmJLx4MJOc/CGnGgeHNea3Mdq5gG7riBpNdwz5Sc+KrJRzlfwKYyESgslUV5UDZuOB1GXnfVN22duOXtKlwgsYNOLPYjAOgpY/c2gWr/S3Oaaef4TlOZwIoQHdSUoSNIYLr0cSQKBH16V3ylt7WTkIcwhe9LxiPhhc0dddxA x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(376002)(366004)(346002)(39860400002)(39380400002)(396003)(51444003)(189003)(199004)(8676002)(3660700001)(110136005)(54906003)(316002)(3280700002)(68736007)(8936002)(81156014)(81166006)(2906002)(66066001)(97736004)(74316002)(7736002)(305945005)(6116002)(3846002)(93886005)(5250100002)(2501003)(5890100001)(14454004)(105586002)(106356001)(2900100001)(6246003)(478600001)(25786009)(4326008)(55016002)(102836004)(53546011)(6506007)(5660300001)(99286004)(86362001)(9686003)(59450400001)(229853002)(7696005)(26005)(33656002)(53936002)(186003)(76176011)(561944003)(6436002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0402MB3769; H:DB3PR0402MB3852.eurprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 94a90736-55e1-43b3-e71d-08d569b1a5dd x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(5600026)(4604075)(3008032)(2017052603307)(7153060)(7193020); SRVR:DB3PR0402MB3769; x-ms-traffictypediagnostic: DB3PR0402MB3769: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(60795455431006)(20558992708506)(278428928389397)(211171220733660); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(3231101)(2400082)(944501161)(10201501046)(6055026)(6041288)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123562045)(6072148)(201708071742011); SRVR:DB3PR0402MB3769; BCL:0; PCL:0; RULEID:; SRVR:DB3PR0402MB3769; x-forefront-prvs: 0570F1F193 received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: hkHYSSOTixQWv/mllWORmvoRNg+S3iw9BG4fL+EPuiInz+tL24TJnxxjsDP49SpdFIdr3mpzt4rKR/IdVcpMKA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 94a90736-55e1-43b3-e71d-08d569b1a5dd X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Feb 2018 20:23:25.4766 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0402MB3769 Subject: Re: [dpdk-dev] [RFC v2] doc compression API for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Feb 2018 20:23:28 -0000 On 1/31/2018 2:03 PM, Trahe, Fiona wrote:=0A= > Hi Ahmed, Shally,=0A= >=0A= > ///snip///=0A= >>>>>>> D.1.1 Stateless and OUT_OF_SPACE=0A= >>>>>>> ------------------------------------------------=0A= >>>>>>> OUT_OF_SPACE is a condition when output buffer runs out of space=0A= >>>> and=0A= >>>>>> where PMD still has more data to produce. If PMD run into such=0A= >>>> condition,=0A= >>>>>> then it's an error condition in stateless processing.=0A= >>>>>>> In such case, PMD resets itself and return with status=0A= >>>>>> RTE_COMP_OP_STATUS_OUT_OF_SPACE with produced=3Dconsumed=3D0=0A= >>>> i.e.=0A= >>>>>> no input read, no output written.=0A= >>>>>>> Application can resubmit an full input with larger output buffer si= ze.=0A= >>>>>> [Ahmed] Can we add an option to allow the user to read the data that= =0A= >>>> was=0A= >>>>>> produced while still reporting OUT_OF_SPACE? this is mainly useful f= or=0A= >>>>>> decompression applications doing search.=0A= >>>>> [Shally] It is there but applicable for stateful operation type (plea= se refer to=0A= >>>> handling out_of_space under=0A= >>>>> "Stateful Section").=0A= >>>>> By definition, "stateless" here means that application (such as IPCOM= P)=0A= >>>> knows maximum output size=0A= >>>>> guaranteedly and ensure that uncompressed data size cannot grow more= =0A= >>>> than provided output buffer.=0A= >>>>> Such apps can submit an op with type =3D STATELESS and provide full i= nput,=0A= >>>> then PMD assume it has=0A= >>>>> sufficient input and output and thus doesn't need to maintain any con= texts=0A= >>>> after op is processed.=0A= >>>>> If application doesn't know about max output size, then it should pro= cess it=0A= >>>> as stateful op i.e. setup op=0A= >>>>> with type =3D STATEFUL and attach a stream so that PMD can maintain= =0A= >>>> relevant context to handle such=0A= >>>>> condition.=0A= >>>> [Fiona] There may be an alternative that's useful for Ahmed, while sti= ll=0A= >>>> respecting the stateless concept.=0A= >>>> In Stateless case where a PMD reports OUT_OF_SPACE in decompression=0A= >>>> case=0A= >>>> it could also return consumed=3D0, produced =3D x, where x>0. X indica= tes the=0A= >>>> amount of valid data which has=0A= >>>> been written to the output buffer. It is not complete, but if an appl= ication=0A= >>>> wants to search it it may be sufficient.=0A= >>>> If the application still wants the data it must resubmit the whole inp= ut with a=0A= >>>> bigger output buffer, and=0A= >>>> decompression will be repeated from the start, it=0A= >>>> cannot expect to continue on as the PMD has not maintained state, his= tory=0A= >>>> or data.=0A= >>>> I don't think there would be any need to indicate this in capabilities= , PMDs=0A= >>>> which cannot provide this=0A= >>>> functionality would always return produced=3Dconsumed=3D0, while PMDs = which=0A= >>>> can could set produced > 0.=0A= >>>> If this works for you both, we could consider a similar case for compr= ession.=0A= >>>>=0A= >>> [Shally] Sounds Fine to me. Though then in that case, consume should al= so be updated to actual=0A= >> consumed by PMD.=0A= >>> Setting consumed =3D 0 with produced > 0 doesn't correlate.=0A= >> [Ahmed]I like Fiona's suggestion, but I also do not like the implication= =0A= >> of returning consumed =3D 0. At the same time returning consumed =3D y= =0A= >> implies to the user that it can proceed from the middle. I prefer the=0A= >> consumed =3D 0 implementation, but I think a different return is needed = to=0A= >> distinguish it from OUT_OF_SPACE that the use can recover from. Perhaps= =0A= >> OUT_OF_SPACE_RECOVERABLE and OUT_OF_SPACE_TERMINATED. This also allows= =0A= >> future PMD implementations to provide recover-ability even in STATELESS= =0A= >> mode if they so wish. In this model STATELESS or STATEFUL would be a=0A= >> hint for the PMD implementation to make optimizations for each case, but= =0A= >> it does not force the PMD implementation to limit functionality if it=0A= >> can provide recover-ability.=0A= > [Fiona] So you're suggesting the following:=0A= > OUT_OF_SPACE - returned only on stateful operation. Not an error. Op.prod= uced=0A= > can be used and next op in stream should continue on from op.consumed= +1.=0A= > OUT_OF_SPACE_TERMINATED - returned only on stateless operation. =0A= > Error condition, no recovery possible.=0A= > consumed=3Dproduced=3D0. Application must resubmit all input data wit= h=0A= > a bigger output buffer.=0A= > OUT_OF_SPACE_RECOVERABLE - returned only on stateless operation, some rec= overy possible.=0A= > - consumed =3D 0, produced > 0. Application must resubmit all input = data with=0A= > a bigger output buffer. However in decompression case, data up to= produced =0A= > in dst buffer may be inspected/searched. Never happens in compres= sion =0A= > case as output data would be meaningless.=0A= > - consumed > 0, produced > 0. PMD has stored relevant state and hist= ory and so=0A= > can convert to stateful, using op.produced and continuing from co= nsumed+1. =0A= > I don't expect our PMDs to use this last case, but maybe this works for o= thers?=0A= > I'm not convinced it's not just adding complexity. It sounds like a versi= on of stateful =0A= > without a stream, and maybe less efficient?=0A= > If so should it respect the FLUSH flag? Which would have been FULL or FIN= AL in the op.=0A= > Or treat it as FLUSH_NONE or SYNC? I don't know why an application would = not=0A= > simply have submitted a STATEFUL request if this is the behaviour it want= s?=0A= [Ahmed] I was actually suggesting the removal of OUT_OF_SPACE entirely=0A= and replacing it with=0A= OUT_OF_SPACE_TERMINATED - returned only on stateless operation.=0A= Error condition, no recovery possible.=0A= - consumed=3D0 produced=3Damount of data produced. Application must= =0A= resubmit all input data with=0A= a bigger output buffer to process all of the op=0A= OUT_OF_SPACE_RECOVERABLE - Normally returned on stateful operation. Not=0A= an error. Op.produced=0A= can be used and next op in stream should continue on from op.consumed+1= .=0A= - consumed > 0, produced > 0. PMD has stored relevant state and=0A= history and so=0A= can continue using op.produced and continuing from consumed+1.= =0A= =0A= We would not return OUT_OF_SPACE_RECOVERABLE in stateless mode in our=0A= implementation either.=0A= =0A= Regardless of speculative future PMDs. The more important aspect of this=0A= for today is that the return status clearly determines=0A= the meaning of "consumed". If it is RECOVERABLE then consumed is=0A= meaningful. if it is TERMINATED then consumed in meaningless.=0A= This way we take away the ambiguity of having OUT_OF_SPACE mean two=0A= different user work flows.=0A= =0A= A speculative future PMD may be designed to return RECOVERABLE for=0A= stateless ops that are attached to streams.=0A= A future PMD may look to see if an op has a stream is attached and write=0A= out the state there and go into recoverable mode.=0A= in essence this leaves the choice up to the implementation and allows=0A= the PMD to take advantage of stateless optimizations=0A= so long as a "RECOVERABLE" scenario is rarely hit. The PMD will dump=0A= context as soon as it fully processes an op. It will only=0A= write context out in cases where the op chokes.=0A= This futuristic PMD should ignore the FLUSH since this STATELESS mode as=0A= indicated by the user and optimize=0A= >>>>>>> D.2 Compression API Stateful operation=0A= >>>>>>> ----------------------------------------------------------=0A= >>>>>>> A Stateful operation in DPDK compression means application invokes= =0A= >>>>>> enqueue burst() multiple times to process related chunk of data eith= er=0A= >>>>>> because=0A= >>>>>>> - Application broke data into several ops, and/or=0A= >>>>>>> - PMD ran into out_of_space situation during input processing=0A= >>>>>>>=0A= >>>>>>> In case of either one or all of the above conditions, PMD is requir= ed to=0A= >>>>>> maintain state of op across enque_burst() calls and=0A= >>>>>>> ops are setup with op_type RTE_COMP_OP_STATEFUL, and begin with=0A= >>>>>> flush value =3D RTE_COMP_NO/SYNC_FLUSH and end at flush value=0A= >>>>>> RTE_COMP_FULL/FINAL_FLUSH.=0A= >>>>>>> D.2.1 Stateful operation state maintenance=0A= >>>>>>> ---------------------------------------------------------------=0A= >>>>>>> It is always an ideal expectation from application that it should p= arse=0A= >>>>>> through all related chunk of source data making its mbuf-chain and= =0A= >>>> enqueue=0A= >>>>>> it for stateless processing.=0A= >>>>>>> However, if it need to break it into several enqueue_burst() calls,= then=0A= >>>> an=0A= >>>>>> expected call flow would be something like:=0A= >>>>>>> enqueue_burst( |op.no_flush |)=0A= >>>>>> [Ahmed] The work is now in flight to the PMD.The user will call dequ= eue=0A= >>>>>> burst in a loop until all ops are received. Is this correct?=0A= >>>>>>=0A= >>>>>>> deque_burst(op) // should dequeue before we enqueue next=0A= >>>>> [Shally] Yes. Ideally every submitted op need to be dequeued. However= =0A= >>>> this illustration is specifically in=0A= >>>>> context of stateful op processing to reflect if a stream is broken in= to=0A= >>>> chunks, then each chunk should be=0A= >>>>> submitted as one op at-a-time with type =3D STATEFUL and need to be= =0A= >>>> dequeued first before next chunk is=0A= >>>>> enqueued.=0A= >>>>>=0A= >>>>>>> enqueue_burst( |op.no_flush |)=0A= >>>>>>> deque_burst(op) // should dequeue before we enqueue next=0A= >>>>>>> enqueue_burst( |op.full_flush |)=0A= >>>>>> [Ahmed] Why now allow multiple work items in flight? I understand th= at=0A= >>>>>> occasionaly there will be OUT_OF_SPACE exception. Can we just=0A= >>>> distinguish=0A= >>>>>> the response in exception cases?=0A= >>>>> [Shally] Multiples ops are allowed in flight, however condition is ea= ch op in=0A= >>>> such case is independent of=0A= >>>>> each other i.e. belong to different streams altogether.=0A= >>>>> Earlier (as part of RFC v1 doc) we did consider the proposal to proce= ss all=0A= >>>> related chunks of data in single=0A= >>>>> burst by passing them as ops array but later found that as not-so-use= ful for=0A= >>>> PMD handling for various=0A= >>>>> reasons. You may please refer to RFC v1 doc review comments for same.= =0A= >>>> [Fiona] Agree with Shally. In summary, as only one op can be processed= at a=0A= >>>> time, since each needs the=0A= >>>> state of the previous, to allow more than 1 op to be in-flight at a ti= me would=0A= >>>> force PMDs to implement internal queueing and exception handling for= =0A= >>>> OUT_OF_SPACE conditions you mention.=0A= >> [Ahmed] But we are putting the ops on qps which would make them=0A= >> sequential. Handling OUT_OF_SPACE conditions would be a little bit more= =0A= >> complex but doable. =0A= > [Fiona] In my opinion this is not doable, could be very inefficient.=0A= > There may be many streams.=0A= > The PMD would have to have an internal queue per stream so=0A= > it could adjust the next src offset and length in the OUT_OF_SPACE case.= =0A= > And this may ripple back though all subsequent ops in the stream as each= =0A= > source len is increased and its dst buffer is not big enough.=0A= [Ahmed] Regarding multi op OUT_OF_SPACE handling.=0A= The caller would still need to adjust=0A= the src length/output buffer as you say. The PMD cannot handle=0A= OUT_OF_SPACE internally.=0A= After OUT_OF_SPACE occurs, the PMD should reject all ops in this stream=0A= until it gets explicit=0A= confirmation from the caller to continue working on this stream. Any ops=0A= received by=0A= the PMD should be returned to the caller with status STREAM_PAUSED since=0A= the caller did not=0A= explicitly acknowledge that it has solved the OUT_OF_SPACE issue.=0A= These semantics can be enabled by adding a new function to the API=0A= perhaps stream_resume().=0A= This allows the caller to indicate that it acknowledges that it has seen=0A= the issue and this op=0A= should be used to resolve the issue. Implementations that do not support=0A= this mode of use=0A= can push back immediately after one op is in flight. Implementations=0A= that support this use=0A= mode can allow many ops from the same session=0A= =0A= Regarding the ordering of ops=0A= We do force serialization of ops belonging to a stream in STATEFUL=0A= operation. Related ops do=0A= not go out of order and are given to available PMDs one at a time.=0A= =0A= >> The question is this mode of use useful for real=0A= >> life applications or would we be just adding complexity? The technical= =0A= >> advantage of this is that processing of Stateful ops is interdependent= =0A= >> and PMDs can take advantage of caching and other optimizations to make= =0A= >> processing related ops much faster than switching on every op. PMDs have= =0A= >> maintain state of more than 32 KB for DEFLATE for every stream.=0A= >>>> If the application has all the data, it can put it into chained mbufs = in a single=0A= >>>> op rather than=0A= >>>> multiple ops, which avoids pushing all that complexity down to the PMD= s.=0A= >> [Ahmed] I think that your suggested scheme of putting all related mbufs= =0A= >> into one op may be the best solution without the extra complexity of=0A= >> handling OUT_OF_SPACE cases, while still allowing the enqueuer extra=0A= >> time If we have a way of marking mbufs as ready for consumption. The=0A= >> enqueuer may not have all the data at hand but can enqueue the op with a= =0A= >> couple of empty mbus marked as not ready for consumption. The enqueuer= =0A= >> will then update the rest of the mbufs to ready for consumption once the= =0A= >> data is added. This introduces a race condition. A second flag for each= =0A= >> mbuf can be updated by the PMD to indicate that it processed it or not.= =0A= >> This way in cases where the PMD beat the application to the op, the=0A= >> application will just update the op to point to the first unprocessed=0A= >> mbuf and resend it to the PMD.=0A= > [Fiona] This doesn't sound safe. You want to add data to a stream after y= ou've=0A= > enqueued the op. You would have to write to op.src.length at a time when = the PMD=0A= > might be reading it. Sounds like a lock would be necessary.=0A= > Once the op has been enqueued, my understanding is its ownership is hande= d=0A= > over to the PMD and the application should not touch it until it has been= dequeued.=0A= > I don't think it's a good idea to change this model.=0A= > Can't the application just collect a stream of data in chained mbufs unti= l it has=0A= > enough to send an op, then construct the op and while waiting for that op= to=0A= > complete, accumulate the next batch of chained mbufs? Only construct the = next op=0A= > after the previous one is complete, based on the result of the previous o= ne. =0A= >=0A= [Ahmed] Fair enough. I agree with you. I imagined it in a different way=0A= in which each mbuf would have its own length.=0A= The advantage to gain is in applications where there is one PMD user,=0A= the down time between ops can be significant and setting up a single=0A= producer consumer pair significantly reduces the CPU cycles and PMD down=0A= time.=0A= =0A= ////snip////=0A=