From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0062.outbound.protection.outlook.com [104.47.0.62]) by dpdk.org (Postfix) with ESMTP id 5546D7CF1 for ; Mon, 18 Dec 2017 22:07:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=lpFxZefSgyl530BxaUjFKyJSU50UrW+buTxjTuifdPg=; b=KY19WXinYf6Km7XQgy6FfnQN1U88gvimvHA3veO7UmZfa88/ONY7U1Fp1AWeJ2HnlnABDTYc8JasX3Hs9/a2ZAyXY/p/JLvuXJa7+2dXuL+QUNuRg6l+p55pTqEbWz1iPTirgeym2oVjB1QwKnC63wtQ1NI/TeNZBsOilhtyAUo= Received: from DB3PR0402MB3852.eurprd04.prod.outlook.com (52.134.71.143) by AM5PR04MB2996.eurprd04.prod.outlook.com (10.173.254.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.323.15; Mon, 18 Dec 2017 21:07:05 +0000 Received: from DB3PR0402MB3852.eurprd04.prod.outlook.com ([fe80::2cde:683a:8f96:1a19]) by DB3PR0402MB3852.eurprd04.prod.outlook.com ([fe80::2cde:683a:8f96:1a19%13]) with mapi id 15.20.0282.012; Mon, 18 Dec 2017 21:07:03 +0000 From: Ahmed Mansour To: "dev@dpdk.org" , "Shally.Verma@cavium.com" , Hemant Agrawal CC: Hemant Agrawal , "Mahipal.Challa@cavium.com" , "fiona.trahe@intel.com" , "NarayanaPrasad.Athreya@cavium.com" , "pablo.de.lara.guarch@intel.com" , Roy Pledge , Youri Querry Thread-Topic: [RFC v3 0/1] Compression API in DPDK Thread-Index: AQHTeEQmcjdXftTxmkKPJr/pHav7/g== Date: Mon, 18 Dec 2017 21:07:03 +0000 Message-ID: References: <1511542480-389-1-git-send-email-fiona.trahe@intel.com> <1513359963-13817-1-git-send-email-fiona.trahe@intel.com> <1513359963-13817-1-git-send-email-fiona.trahe@intel.com> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=ahmed.mansour@nxp.com; x-originating-ip: [192.88.168.1] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM5PR04MB2996; 6:iIL1IX0u3IymrAmClYVHkGKY13MDkz+yEl+KyUByAP5obdoEmDD7i6+tm87Geo2md/3yZYDjhCSeZvTmcn98HF3pUCzb8/L3vPsyUrnVixSG2IleY6bBcDunpSTxbcUhNvxNmElEDRW9rrvnk6L+nQT2bKDzmgitsnMdNqSog68V7I2Kh12YJ0JLlMqySPBdSScPJ/10mp7oIdy4LiDoVohqENLmuYT63TkugR7BfagQpVZXt7s8X0uXPs7ULF3B6GB7cKIyqJGvF//Vh8pvcwNqm/K0PCIP4gwY+Q6C+ACgqKNdI2aKS5ilQb7lG9j/nSoFF9TWFAhDaFPIE9Oszum6HUjxqLM5dHqHYPlWpdM=; 5:gj5AuD8mr+NhmxVO1ll4o6TPxJPefS3pIY12i7pcP3Z8d6LJ867vTA6/ZXS8w03SMIfyq9jTqC1WJ0cUwKTy5029lnVbdqDcpehgqVJfyeDxJhL/4jpfiMYgA75Eizd7YcafqI0gmOZmX02iZwljrOTiaf1rQ84YZH7yajuAPXo=; 24:xrbTCCLdJAATpgvLLfG8oevuTUIgR1NJ1O02kOA3cu+y/piCYWk8ZkFwMG8YrC2yVjSvUIWnLLgOHDPxwUZQR5Oio8nc6rXob2eTd4D1ZMM=; 7:tnFtm7FQC8PnpLZAHk71WRITPZXK6WEP6tXo0jXXqejqCUUmF3H0U5/FhE4BM1YuW528AyEeZDdbJpYcgVrkRnmTFvzKJ7ctwZJAo60diHGbYDzZEHotSMxsS5B5e8Ibjg75qAtPBQAnz1F5Ud7eKTJlLESR6L2OritoXA5pt5n4qfcRd9B9nctJKRgsLKdRIAPBkjxPivJNDfFEsRHMG9n9BPZSnpIVMz+gfrNsdCeUp6OkI88fuc3YghJPDnPd x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(396003)(39860400002)(346002)(376002)(366004)(189003)(199004)(24454002)(97736004)(74316002)(229853002)(110136005)(106356001)(14454004)(105586002)(68736007)(305945005)(316002)(53936002)(5250100002)(8936002)(66066001)(8676002)(7736002)(478600001)(45080400002)(2900100001)(76176011)(81156014)(6436002)(81166006)(54906003)(6636002)(3846002)(6116002)(2501003)(2906002)(53546011)(33656002)(9686003)(25786009)(7696005)(59450400001)(3280700002)(86362001)(3660700001)(6506007)(6246003)(6306002)(102836003)(99286004)(55016002)(4326008)(5660300001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR04MB2996; H:DB3PR0402MB3852.eurprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 1c80adef-b578-4148-a2b6-08d5465b49d3 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603307); SRVR:AM5PR04MB2996; x-ms-traffictypediagnostic: AM5PR04MB2996: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(189930954265078)(45079756050767); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(3002001)(3231023)(10201501046)(93006095)(93001095)(6055026)(6041248)(20161123562025)(20161123558100)(20161123560025)(20161123555025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:AM5PR04MB2996; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:AM5PR04MB2996; x-forefront-prvs: 0525BB0ADF received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1c80adef-b578-4148-a2b6-08d5465b49d3 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2017 21:07:03.6078 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR04MB2996 Subject: Re: [dpdk-dev] [RFC v3 0/1] Compression API in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2017 21:07:07 -0000 Hi Fiona,=0A= =0A= On 12/15/2017 11:16 PM, Trahe, Fiona wrote:=0A= =0A= > With the vast amounts of data being transported around networks and store= d in=0A= > storage systems, reducing data size is becoming ever more important. Ther= e=0A= > are both software libraries and hardware devices available that provide= =0A= > compression, but no common API. This RFC proposes a compression API for= =0A= > DPDK to address this need.=0A= >=0A= > Features:=0A= > =E2=80=A2 Deflate Algorithm (https://emea01.safelinks.protection.outlook.= com/?url=3Dhttps%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc1951&data=3D02%7C01%7Ca= hmed.mansour%40nxp.com%7C76241a2796db4701ef0108d54634a70f%7C686ea1d3bc2b4c6= fa92cd99c5c301635%7C0%7C0%7C636492114308805852&sdata=3Dyglh48%2F8IuEn%2F7YC= L49FlyhGyCnNRX4g4xx2WJQesFs%3D&reserved=3D0)=0A= > =E2=80=A2 LZS algorithm (https://emea01.safelinks.protection.outlook.com/= ?url=3Dhttps%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc2395&data=3D02%7C01%7Cahmed= .mansour%40nxp.com%7C76241a2796db4701ef0108d54634a70f%7C686ea1d3bc2b4c6fa92= cd99c5c301635%7C0%7C0%7C636492114308805852&sdata=3DBlfNe3pRXpEFJE4KS8Spk8bJ= 5GWPDqADhZYoD7SKvjk%3D&reserved=3D0)=0A= > =E2=80=A2 Static and Dynamic Huffman encoding.=0A= > =E2=80=A2 Compression levels=0A= > =E2=80=A2 Checksum generation=0A= > =E2=80=A2 Asynchronous burst API=0A= > =E2=80=A2 Session-based (a session contains immutable data only and is us= eable across devices)=0A= > =E2=80=A2 stream-based to maintain state and history data for stateful fl= ows.=0A= >=0A= > Note 1: Split of functionality above/below API=0A= > When considering whether features should be supported on the API or not, = the=0A= > decision was based on the following:=0A= > The purpose of the API is to decouple the application from the compute-in= tensive=0A= > functions needed for compression by abstracting them under a common API. = These=0A= > can then be implemented by either hardware accelerators or optimised soft= ware=0A= > libraries. Where features are not compute-intensive and unlikely to be=0A= > offloaded or optimised, there=E2=80=99s nothing to be gained by each PMD = having=0A= > to separately implement them, and it makes more sense for them to be done= =0A= > above the API. So the following are not handled on the API and can be don= e above.=0A= > =E2=80=A2 Prepending/appending protocol headers (gzip, zlib)=0A= =0A= Agreed with the notion, however the header and footer handling can be =0A= added as an option. PMDs can support or not support each format. We =0A= (NXP) support auto padding of gzip and zlib headers as well as DEFLATE only= .=0A= =0A= > =E2=80=A2 File-handling, breaking files up into packets, reassembling.=0A= > =E2=80=A2 Synchronous API=0A= > =E2=80=A2 Serialisation of stateful requests=0A= >=0A= >=0A= Is stateful planned for next phase? During design discussions we uncovered = many API design=0A= considerations necessary for stateful use. Chained stateful support in the = future might not be=0A= possible without compatibility breaking =0A= =0A= > Note 2: The tricky question of where the API belongs=0A= > We considered=0A= > 1. Extending cryptodev=0A= > 2. New acceldev APIs for device handling + compressdev APIs for data path= =0A= > 3. New acceldev for all APIs=0A= > 4. New compressdev API=0A= > We've gone with option 4, a compressdev API. See original RFC [1] for re= asons.=0A= > We explored wrapping this around a generic acceldev that would be hidden = from the API=0A= > but could be common to cryptodev, compressdev and other accelerators on t= he PMD interface,=0A= > but this added complexity and indirection and didn't add enough value, so= we've abandoned it.=0A= >=0A= Makes sense. compression is common enough to be attempted to be a =0A= different device category.=0A= =0A= =0A= > Opens:=0A= > - Define structures and API for proposed hash functionality=0A= > - Agree on stateful behaviour=0A= =0A= What are the the current thoughts for stateful behavior?=0A= =0A= > - Complete capability APIs=0A= =0A= A capability API is very much required as different HW/SW can have =0A= different capabilities.=0A= =0A= =0A= regards,=0A= Ahmed=0A= =0A= =0A=