From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id B9F68548B for ; Wed, 14 Mar 2018 19:39:16 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Mar 2018 11:39:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,306,1517904000"; d="scan'208";a="33867302" Received: from irsmsx151.ger.corp.intel.com ([163.33.192.59]) by FMSMGA003.fm.intel.com with ESMTP; 14 Mar 2018 11:39:12 -0700 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.5]) by IRSMSX151.ger.corp.intel.com ([169.254.4.16]) with mapi id 14.03.0319.002; Wed, 14 Mar 2018 18:39:12 +0000 From: "Trahe, Fiona" To: "Verma, Shally" , Ahmed Mansour , "dev@dpdk.org" CC: "De Lara Guarch, Pablo" , "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , "Roy Pledge" , Youri Querry , "Daly, Lee" , "Jozwiak, TomaszX" Thread-Topic: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative Thread-Index: AdO6FqZGVQESVGJPRLukdFdKpdlnigAioyWQABAgLEAALDY6IAAL6sJw Date: Wed, 14 Mar 2018 18:39:11 +0000 Message-ID: <348A99DA5F5B7549AA880327E580B4358934B32C@IRSMSX101.ger.corp.intel.com> References: <348A99DA5F5B7549AA880327E580B435893478BA@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B4358934A600@IRSMSX101.ger.corp.intel.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODI1N2JiMzQtNjg1Ny00YmEzLWIwNWItYWZkOTZhNzBkYTI2IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IlwvWEMxdVRKVVFGYU5iNHBPUGRcLzJuVUNPeUlhcThRdkV2c3FPK0tWVjk0Zz0ifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Mar 2018 18:39:17 -0000 Hi Shally, Ahmed, et al, Following internal and community feedback we've decided that there's still = too much churn in this.=20 We're proposing, in the interest of getting the API out in 18.05, to stick = with mbufs - acknowledging that they're not optimal for storage and we may propose changes in 18.08.=20 Compressdev will start as an experimental API in 18.05 - we'll POC and benc= hmark alternatives=20 or API extensions once we get time to do so. Regards, Fiona > -----Original Message----- > From: Verma, Shally [mailto:Shally.Verma@cavium.com] > Sent: Wednesday, March 14, 2018 12:51 PM > To: Trahe, Fiona ; Ahmed Mansour ; dev@dpdk.org > Cc: De Lara Guarch, Pablo ; Athreya, Nara= yana Prasad > ; Gupta, Ashish ; Sahu, Sunila > ; Challa, Mahipal ; Ja= in, Deepak K > ; Hemant Agrawal ; Roy P= ledge > ; Youri Querry ; Daly, Lee ; > Jozwiak, TomaszX > Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alterna= tive >=20 >=20 >=20 > >-----Original Message----- > >From: Trahe, Fiona [mailto:fiona.trahe@intel.com] > >Sent: 13 March 2018 21:22 > >To: Verma, Shally ; Ahmed Mansour ; > dev@dpdk.org > >Cc: De Lara Guarch, Pablo ; Athreya, Nar= ayana Prasad > ; > >Gupta, Ashish ; Sahu, Sunila ; Challa, Mahipal > >; Jain, Deepak K ; H= emant Agrawal > ; Roy > >Pledge ; Youri Querry ; Daly= , Lee > ; Jozwiak, TomaszX > >; Trahe, Fiona > >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf altern= ative > > > >Hi Shally, > > > >> -----Original Message----- > >> From: Verma, Shally [mailto:Shally.Verma@cavium.com] > >> Sent: Tuesday, March 13, 2018 8:15 AM > >> To: Trahe, Fiona ; Ahmed Mansour ; > dev@dpdk.org > >> Cc: De Lara Guarch, Pablo ; Athreya, N= arayana Prasad > >> ; Gupta, Ashish ; Sahu, Sunila > >> ; Challa, Mahipal ;= Jain, Deepak K > >> ; Hemant Agrawal ; Ro= y Pledge > >> ; Youri Querry ; fiona.tra= he@gmail.com; Daly, Lee > >> ; Jozwiak, TomaszX > >> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alte= rnative > >> > >> HI Fiona > >> > >> So I understand we're moving away from mbufs because of its size limit= ation (64k) and cacheline > overhead > >> and their more suitability to n/w applications. Given that, I understa= nd benefit of having another > structure > >> to input data but then what is proposal for ipcomp like application wh= ere mbuf usage may be a better > >> option? Should we keep support for both (mbuf and this structure) so t= hat apps can use appropriate > data > >> structure depending on their requirement. > >[Fiona] An application can use pass buffers from an mbuf or mbuf chain t= o compressdev by filling in the > >compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_dat= a_len(), > >rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc > >For simplicity I'd prefer to offer only 1 rather than 2 data formats on = the API. > >We see storage applications rather than IPComp as the main use-case for = compressdev, so would prefer > >to optimise for that. > >Do you think otherwise? >=20 > [Shally] Yea. We plan to use it for ipcomp and other such possible n/w ap= ps. So, we envision mbuf support > as necessary. So, I think we can add two APIs one which process on rte_co= mp_op and other on rte_mbufs > to make it simpler. >=20 > > > >> > >> Further comments, on github. > >> > >> Thanks > >> Shally > >> > >> >-----Original Message----- > >> >From: Trahe, Fiona [mailto:fiona.trahe@intel.com] > >> >Sent: 12 March 2018 21:31 > >> >To: Ahmed Mansour ; Verma, Shally ; > >> dev@dpdk.org > >> >Cc: De Lara Guarch, Pablo ; Athreya, = Narayana Prasad > >> ; > >> >Gupta, Ashish ; Sahu, Sunila ; Challa, > Mahipal > >> >; Jain, Deepak K = ; Hemant Agrawal > >> ; Roy > >> >Pledge ; Youri Querry ; f= iona.trahe@gmail.com; > Daly, > >> Lee ; > >> >Jozwiak, TomaszX > >> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alt= ernative > >> > > >> >Hi Shally, Ahmed, and anyone else interested in compressdev, > >> > > >> >I mentioned last week that we've been exploring using something other= than mbufs to pass src/dst > >> buffers to compressdev PMDs. > >> > > >> >Reasons: > >> > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. D= ata for compression > >> > can be greater and it would add cycles to have to break up into s= maller segments. > >> > - data may originate in mbufs, but is more likely, particularly for = storage use-cases, to > >> > originate in other data structures. > >> > - There's a 2 cache-line overhead for every segment in a chain, most= of this data > >> > is network-related, not needed by compressdev > >> >So moving to a custom structure would minimise memory overhead, remov= e restriction on 64k-1 size > and > >> give more flexibility if > >> >compressdev ever needs any comp-specific meta-data. > >> > > >> >We've come up with a compressdev-specific structure using the struct = iovec from sys/uio.h, which is > >> commonly used by storage > >> >applications. This would replace the src and dest mbufs in the op. > >> >I'll not include the code here - Pablo will push that to github short= ly and we'd appreciate review > >> comments there. > >> >https://github.com/pablodelara/dpdk-draft-compressdev > >> >Just posting on the mailing list to give a heads-up and ensure this r= eaches a wider audience than may > see > >> it on github. > >> > > >> >Note : We also considered having no data structures in the op, instea= d the application > >> >would supply a callback which the PMD would use to retrieve meta-data= (virt address, iova, length) > >> >for each next segment as needed. While this is quite flexible and all= ow the application > >> >to keep its data in its native structures, it's likely to cost more c= ycles. > >> >So we're not proposing this at the moment, but hope to benchmark it l= ater while the API is still > >> experimental. > >> > > >> >General feedback on direction is welcome here on the mailing list. > >> >For feedback on the details of implementation we would appreciate com= ments on github. > >> > > >> >Regards, > >> >Fiona.