From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0068.outbound.protection.outlook.com [104.47.40.68]) by dpdk.org (Postfix) with ESMTP id D3DF3AA99 for ; Wed, 14 Mar 2018 13:50:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=jIhw0RJdAco3iKDY4+18qbkvb47KXlWNSGTy4hTgdaw=; b=Gw4wK3jDW4hs2x7nwbPRcJTpiZQHhJkTjfl4OO8KL0JqEMxzCLf6bXdb40YSTTYHy5w1eUVFmXJKTy7ef0ZUsuZZLl0pYAgc42Z1Hp82Hah7jEgY9q+NqBHlQfGeCMb4wF/aBnP8Eh3Pim4D9SW2k/DmPXVElZ26cOj5TDSFn8U= Received: from MWHPR0701MB3641.namprd07.prod.outlook.com (10.167.236.162) by MWHPR0701MB3753.namprd07.prod.outlook.com (10.167.237.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.567.14; Wed, 14 Mar 2018 12:50:34 +0000 Received: from MWHPR0701MB3641.namprd07.prod.outlook.com ([fe80::2da5:ed9e:cf18:d138]) by MWHPR0701MB3641.namprd07.prod.outlook.com ([fe80::2da5:ed9e:cf18:d138%13]) with mapi id 15.20.0567.018; Wed, 14 Mar 2018 12:50:33 +0000 From: "Verma, Shally" To: "Trahe, Fiona" , Ahmed Mansour , "dev@dpdk.org" CC: "De Lara Guarch, Pablo" , "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , Roy Pledge , Youri Querry , "Daly, Lee" , "Jozwiak, TomaszX" Thread-Topic: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative Thread-Index: AdO6FqZGVQESVGJPRLukdFdKpdlnigAioyWQABAgLEAALDY6IA== Date: Wed, 14 Mar 2018 12:50:33 +0000 Message-ID: References: <348A99DA5F5B7549AA880327E580B435893478BA@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B4358934A600@IRSMSX101.ger.corp.intel.com> In-Reply-To: <348A99DA5F5B7549AA880327E580B4358934A600@IRSMSX101.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [115.113.156.2] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; MWHPR0701MB3753; 7:GU8CV5j7V+I478lI9BZNnnWN/qBs9/U/OJ9p1FdNiBZYAYcmq0xudlfpcjR9m5GnKN0uj54xjb5nQXqhVmhHNAFEk3xfWMDH+fXipGAwqJhwR8CELr8/aVt7C+6i4RacoIYfrijc4yG5Tc7+UXbgo9fpT5lPhW4cP9gD9z3dkSeVKlb8xZrxJL1zM3mUYBNGevMfbQV0W00Va+YQJeX8ufpucKjFszNXuGaYlK7IOSrZzEtBCKMgB+QC9UJj+j5m x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(39380400002)(39860400002)(346002)(396003)(366004)(376002)(189003)(199004)(13464003)(5250100002)(9686003)(8656006)(3660700001)(6116002)(5660300001)(6246003)(68736007)(7416002)(3846002)(86362001)(4326008)(97736004)(7736002)(105586002)(25786009)(55236004)(102836004)(305945005)(74316002)(14454004)(53546011)(2501003)(186003)(478600001)(316002)(561944003)(110136005)(59450400001)(6436002)(33656002)(26005)(6306002)(2950100002)(72206003)(3280700002)(6506007)(54906003)(8676002)(8936002)(106356001)(81156014)(55016002)(76176011)(53936002)(229853002)(7696005)(81166006)(2900100001)(99286004)(2906002)(66066001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR0701MB3753; H:MWHPR0701MB3641.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 4c76ee44-9491-4020-cbfe-08d589aa2cf8 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:MWHPR0701MB3753; x-ms-traffictypediagnostic: MWHPR0701MB3753: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(278428928389397)(166708455590820)(185117386973197)(85827821059158)(788757137089)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231221)(944501244)(52105095)(93006095)(93001095)(3002001)(10201501046)(6041310)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:MWHPR0701MB3753; BCL:0; PCL:0; RULEID:; SRVR:MWHPR0701MB3753; x-forefront-prvs: 0611A21987 received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 8KviqZNGkcmi7bJm3l7uTBLQIZQFKjd71mmEigSsUrA1wS/FKgQifgzmoYrz8TvSqJEbNkJKe9lMsUzFdHGyN2zPkxlHm9j1qXSM2o/B1+xWhnjIHLqLMeOcTw/wKVEvfNasj4wDDa08SO5wmCkOrDBR0ewYOeUwcYX6WD12JwEIKtbUxtKtC1oLdX63o8BUts0vtBp86jCbz9D38mjKFVLeZ5SkEXdbasiKZQS91HliL1i7u75gtneKrdDe2S/75WmGRr9tpy8L6CpLmc6Ixqv61ziASVgu4rhYJ/tFvvgzLpXsXIxuOv9Zomax32fT8LfhzVqmL/q5ASJrRG/XVA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4c76ee44-9491-4020-cbfe-08d589aa2cf8 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2018 12:50:33.3491 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR0701MB3753 Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Mar 2018 12:50:37 -0000 >-----Original Message----- >From: Trahe, Fiona [mailto:fiona.trahe@intel.com] >Sent: 13 March 2018 21:22 >To: Verma, Shally ; Ahmed Mansour ; dev@dpdk.org >Cc: De Lara Guarch, Pablo ; Athreya, Naray= ana Prasad ; >Gupta, Ashish ; Sahu, Sunila ; Challa, Mahipal >; Jain, Deepak K ; Hem= ant Agrawal ; Roy >Pledge ; Youri Querry ; Daly, = Lee ; Jozwiak, TomaszX >; Trahe, Fiona >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternat= ive > >Hi Shally, > >> -----Original Message----- >> From: Verma, Shally [mailto:Shally.Verma@cavium.com] >> Sent: Tuesday, March 13, 2018 8:15 AM >> To: Trahe, Fiona ; Ahmed Mansour ; dev@dpdk.org >> Cc: De Lara Guarch, Pablo ; Athreya, Nar= ayana Prasad >> ; Gupta, Ashish ; Sahu, Sunila >> ; Challa, Mahipal ; J= ain, Deepak K >> ; Hemant Agrawal ; Roy = Pledge >> ; Youri Querry ; fiona.trahe= @gmail.com; Daly, Lee >> ; Jozwiak, TomaszX >> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf altern= ative >> >> HI Fiona >> >> So I understand we're moving away from mbufs because of its size limitat= ion (64k) and cacheline overhead >> and their more suitability to n/w applications. Given that, I understand= benefit of having another structure >> to input data but then what is proposal for ipcomp like application wher= e mbuf usage may be a better >> option? Should we keep support for both (mbuf and this structure) so tha= t apps can use appropriate data >> structure depending on their requirement. >[Fiona] An application can use pass buffers from an mbuf or mbuf chain to = compressdev by filling in the >compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_= len(), >rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc >For simplicity I'd prefer to offer only 1 rather than 2 data formats on th= e API. >We see storage applications rather than IPComp as the main use-case for co= mpressdev, so would prefer >to optimise for that. >Do you think otherwise? [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps= . So, we envision mbuf support as necessary. So, I think we can add two API= s one which process on rte_comp_op and other on rte_mbufs to make it simple= r. > >> >> Further comments, on github. >> >> Thanks >> Shally >> >> >-----Original Message----- >> >From: Trahe, Fiona [mailto:fiona.trahe@intel.com] >> >Sent: 12 March 2018 21:31 >> >To: Ahmed Mansour ; Verma, Shally ; >> dev@dpdk.org >> >Cc: De Lara Guarch, Pablo ; Athreya, Na= rayana Prasad >> ; >> >Gupta, Ashish ; Sahu, Sunila ; Challa, Mahipal >> >; Jain, Deepak K ; = Hemant Agrawal >> ; Roy >> >Pledge ; Youri Querry ; fio= na.trahe@gmail.com; Daly, >> Lee ; >> >Jozwiak, TomaszX >> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alter= native >> > >> >Hi Shally, Ahmed, and anyone else interested in compressdev, >> > >> >I mentioned last week that we've been exploring using something other t= han mbufs to pass src/dst >> buffers to compressdev PMDs. >> > >> >Reasons: >> > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Dat= a for compression >> > can be greater and it would add cycles to have to break up into sma= ller segments. >> > - data may originate in mbufs, but is more likely, particularly for st= orage use-cases, to >> > originate in other data structures. >> > - There's a 2 cache-line overhead for every segment in a chain, most o= f this data >> > is network-related, not needed by compressdev >> >So moving to a custom structure would minimise memory overhead, remove = restriction on 64k-1 size and >> give more flexibility if >> >compressdev ever needs any comp-specific meta-data. >> > >> >We've come up with a compressdev-specific structure using the struct io= vec from sys/uio.h, which is >> commonly used by storage >> >applications. This would replace the src and dest mbufs in the op. >> >I'll not include the code here - Pablo will push that to github shortly= and we'd appreciate review >> comments there. >> >https://github.com/pablodelara/dpdk-draft-compressdev >> >Just posting on the mailing list to give a heads-up and ensure this rea= ches a wider audience than may see >> it on github. >> > >> >Note : We also considered having no data structures in the op, instead = the application >> >would supply a callback which the PMD would use to retrieve meta-data (= virt address, iova, length) >> >for each next segment as needed. While this is quite flexible and allow= the application >> >to keep its data in its native structures, it's likely to cost more cyc= les. >> >So we're not proposing this at the moment, but hope to benchmark it lat= er while the API is still >> experimental. >> > >> >General feedback on direction is welcome here on the mailing list. >> >For feedback on the details of implementation we would appreciate comme= nts on github. >> > >> >Regards, >> >Fiona.