From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0079.outbound.protection.outlook.com [104.47.38.79]) by dpdk.org (Postfix) with ESMTP id 7684F14E8 for ; Tue, 13 Mar 2018 09:14:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=2giZWBhSu2R1ZRBgYUL2e2e2Kv2gufHmd7sYobyxwRI=; b=ZHvZ4tmlZbXiGjVWO2v4W+g1Gn6PwPQiwG5mt76Vz5uwdJOsC/ilhBPC+p0IQtpP2NKgG/DoXbrLeU5IZ+3BJR25frexDBxBC7duAtYuYFiGOD1LohN9juBtcqlyc/pd21UnfdyjbCOfDPzVS9ZydAe5fiQ8PipmlN3CMdr/pzM= Received: from CY4PR0701MB3634.namprd07.prod.outlook.com (52.132.101.164) by CY4PR0701MB3811.namprd07.prod.outlook.com (52.132.102.157) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.548.13; Tue, 13 Mar 2018 08:14:50 +0000 Received: from CY4PR0701MB3634.namprd07.prod.outlook.com ([fe80::e842:2eb2:2ff5:5dd6]) by CY4PR0701MB3634.namprd07.prod.outlook.com ([fe80::e842:2eb2:2ff5:5dd6%13]) with mapi id 15.20.0548.021; Tue, 13 Mar 2018 08:14:49 +0000 From: "Verma, Shally" To: "Trahe, Fiona" , Ahmed Mansour , "dev@dpdk.org" CC: "De Lara Guarch, Pablo" , "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , Roy Pledge , Youri Querry , "fiona.trahe@gmail.com" , "Daly, Lee" , "Jozwiak, TomaszX" Thread-Topic: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative Thread-Index: AdO6FqZGVQESVGJPRLukdFdKpdlnigAioyWQ Date: Tue, 13 Mar 2018 08:14:49 +0000 Message-ID: References: <348A99DA5F5B7549AA880327E580B435893478BA@IRSMSX101.ger.corp.intel.com> In-Reply-To: <348A99DA5F5B7549AA880327E580B435893478BA@IRSMSX101.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [115.113.156.2] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY4PR0701MB3811; 7:E6oY9VAJBZAoD1snI1p8nLFW7qdqQ68gSjjaUy6yoMKh8IEbNdm4oUdN/MTKVQv8Q6wfHFi1vrkhqQQfoOzlXRY9tboFuwZo1YJ7/w1Y7KGoYCzXBBVkK2FKP2sp5QhUyIN12wXJ9IV34Fug0uMD6uRBkMMP1UYLGGBiNlRQYk7NhHnOGP8wmYTwBmlZ0Co8h51SBZVBvzHb43xHZP8eVnMLaiGq9xieUbSkkmB0OuLspMHjX5H1nEvr/erDePVU x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(346002)(366004)(376002)(39380400002)(396003)(39860400002)(189003)(199004)(13464003)(229853002)(6436002)(478600001)(72206003)(8676002)(2900100001)(3280700002)(14454004)(33656002)(5250100002)(2501003)(561944003)(66066001)(53936002)(55016002)(97736004)(6306002)(5660300001)(4326008)(6246003)(9686003)(59450400001)(25786009)(39060400002)(186003)(26005)(102836004)(55236004)(6506007)(105586002)(8936002)(81156014)(81166006)(110136005)(74316002)(99286004)(76176011)(86362001)(68736007)(106356001)(7416002)(8656006)(316002)(54906003)(7696005)(305945005)(2906002)(2950100002)(3660700001)(3846002)(6116002)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR0701MB3811; H:CY4PR0701MB3634.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 94f1d51f-03cf-4553-8751-08d588ba7d6f x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:CY4PR0701MB3811; x-ms-traffictypediagnostic: CY4PR0701MB3811: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(166708455590820)(185117386973197)(85827821059158)(788757137089)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(10201501046)(3231221)(944501244)(52105095)(6041310)(20161123564045)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(6072148)(201708071742011); SRVR:CY4PR0701MB3811; BCL:0; PCL:0; RULEID:; SRVR:CY4PR0701MB3811; x-forefront-prvs: 0610D16BBE received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: bJyoj+m/1f+cz93VoqI826LlzRMkZATSuk3sjFjGTnJsTGT2b2ACOkjHhBAvKRw2kGy1OxsP0VrjDWRp+wxRJB/qxBBAHjkl6TOJIKM8pqvEaWDn2GtSd2kHZ6s2G8I63guGZm2pG0/PAsf+fUD8KwFXfG2w1nj3r+OHggQ/YCbUrGkyZh8KbOcv7wIYw/1toxn30q97yhuFwtxZlDP708fBsxJMRMYDWx9Of6WaysL34bhWEdiBxMLzBWiS5gsCQ2nsedQcmouDerKQYWvqjVV54bVtW2g/Iolnb6ZzhLSLxGHPtOEG2HPAruBEKtaut0P+D+4sBxYenrMd2fl5xw== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 94f1d51f-03cf-4553-8751-08d588ba7d6f X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Mar 2018 08:14:49.1767 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR0701MB3811 Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Mar 2018 08:14:53 -0000 HI Fiona So I understand we're moving away from mbufs because of its size limitation= (64k) and cacheline overhead and their more suitability to n/w application= s. Given that, I understand benefit of having another structure to input da= ta but then what is proposal for ipcomp like application where mbuf usage m= ay be a better option? Should we keep support for both (mbuf and this struc= ture) so that apps can use appropriate data structure depending on their re= quirement. Further comments, on github. Thanks Shally >-----Original Message----- >From: Trahe, Fiona [mailto:fiona.trahe@intel.com] >Sent: 12 March 2018 21:31 >To: Ahmed Mansour ; Verma, Shally ; dev@dpdk.org >Cc: De Lara Guarch, Pablo ; Athreya, Naray= ana Prasad ; >Gupta, Ashish ; Sahu, Sunila ; Challa, Mahipal >; Jain, Deepak K ; Hem= ant Agrawal ; Roy >Pledge ; Youri Querry ; fiona.= trahe@gmail.com; Daly, Lee ; >Jozwiak, TomaszX >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternat= ive > >Hi Shally, Ahmed, and anyone else interested in compressdev, > >I mentioned last week that we've been exploring using something other than= mbufs to pass src/dst buffers to compressdev PMDs. > >Reasons: > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data f= or compression > can be greater and it would add cycles to have to break up into smalle= r segments. > - data may originate in mbufs, but is more likely, particularly for stora= ge use-cases, to > originate in other data structures. > - There's a 2 cache-line overhead for every segment in a chain, most of t= his data > is network-related, not needed by compressdev >So moving to a custom structure would minimise memory overhead, remove res= triction on 64k-1 size and give more flexibility if >compressdev ever needs any comp-specific meta-data. > >We've come up with a compressdev-specific structure using the struct iovec= from sys/uio.h, which is commonly used by storage >applications. This would replace the src and dest mbufs in the op. >I'll not include the code here - Pablo will push that to github shortly an= d we'd appreciate review comments there. >https://github.com/pablodelara/dpdk-draft-compressdev >Just posting on the mailing list to give a heads-up and ensure this reache= s a wider audience than may see it on github. > >Note : We also considered having no data structures in the op, instead the= application >would supply a callback which the PMD would use to retrieve meta-data (vir= t address, iova, length) >for each next segment as needed. While this is quite flexible and allow th= e application >to keep its data in its native structures, it's likely to cost more cycles= . >So we're not proposing this at the moment, but hope to benchmark it later = while the API is still experimental. > >General feedback on direction is welcome here on the mailing list. >For feedback on the details of implementation we would appreciate comments= on github. > >Regards, >Fiona.