From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 42F904C74 for ; Tue, 13 Mar 2018 12:16:07 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Mar 2018 04:16:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,464,1515484800"; d="scan'208";a="36900987" Received: from irsmsx153.ger.corp.intel.com ([163.33.192.75]) by fmsmga004.fm.intel.com with ESMTP; 13 Mar 2018 04:16:03 -0700 Received: from irsmsx111.ger.corp.intel.com (10.108.20.4) by IRSMSX153.ger.corp.intel.com (163.33.192.75) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 13 Mar 2018 11:16:03 +0000 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.221]) by irsmsx111.ger.corp.intel.com ([169.254.2.246]) with mapi id 14.03.0319.002; Tue, 13 Mar 2018 11:16:03 +0000 From: "Ananyev, Konstantin" To: "Trahe, Fiona" , Ahmed Mansour , "Verma, Shally" , "dev@dpdk.org" CC: "De Lara Guarch, Pablo" , "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , "Roy Pledge" , Youri Querry , "fiona.trahe@gmail.com" , "Daly, Lee" , "Jozwiak, TomaszX" Thread-Topic: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative Thread-Index: AdO6FqZGVQESVGJPRLukdFdKpdlnigApZaaQ Date: Tue, 13 Mar 2018 11:16:02 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772589E28F2B6@irsmsx105.ger.corp.intel.com> References: <348A99DA5F5B7549AA880327E580B435893478BA@IRSMSX101.ger.corp.intel.com> In-Reply-To: <348A99DA5F5B7549AA880327E580B435893478BA@IRSMSX101.ger.corp.intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNWEwYzRkZmUtYWIyZi00MTkzLWE0MGYtMTJiMGY0MTA4ZGQ5IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6Imoyb0h3ejVON2w3M2tkdEZHNkVcL0tDa1duV2lLZEJIUWpoeXRMQU9OUjI0PSJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Mar 2018 11:16:08 -0000 Hi Fiona, > Hi Shally, Ahmed, and anyone else interested in compressdev, >=20 > I mentioned last week that we've been exploring using something other tha= n mbufs to pass src/dst buffers to compressdev PMDs. >=20 > Reasons: > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data = for compression > can be greater and it would add cycles to have to break up into small= er segments. > - data may originate in mbufs, but is more likely, particularly for stor= age use-cases, to > originate in other data structures. > - There's a 2 cache-line overhead for every segment in a chain, most of = this data > is network-related, not needed by compressdev > So moving to a custom structure would minimise memory overhead, remove re= striction on 64k-1 size and give more flexibility if > compressdev ever needs any comp-specific meta-data. >=20 > We've come up with a compressdev-specific structure using the struct iove= c from sys/uio.h, which is commonly used by storage > applications. This would replace the src and dest mbufs in the op. > I'll not include the code here - Pablo will push that to github shortly a= nd we'd appreciate review comments there. > https://github.com/pablodelara/dpdk-draft-compressdev > Just posting on the mailing list to give a heads-up and ensure this reach= es a wider audience than may see it on github. >=20 > Note : We also considered having no data structures in the op, instead th= e application > would supply a callback which the PMD would use to retrieve meta-data (vi= rt address, iova, length) > for each next segment as needed. While this is quite flexible and allow t= he application > to keep its data in its native structures, it's likely to cost more cycle= s. As I said in different thread it will not only slowdown things, but will ma= ke it difficult (if possible at all) for compressdev PMDs to support DPDK MP model. Konstantin > So we're not proposing this at the moment, but hope to benchmark it later= while the API is still experimental. >=20 > General feedback on direction is welcome here on the mailing list. > For feedback on the details of implementation we would appreciate comment= s on github. >=20 > Regards, > Fiona.