From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0040.outbound.protection.outlook.com [104.47.0.40]) by dpdk.org (Postfix) with ESMTP id 1EC8E1D7 for ; Mon, 18 Dec 2017 22:43:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=NEWudNb+oUlAcrUjJ/WlxEWOokTehDXqL2Nc/rM5tBE=; b=FiRFHSvGDdGNtTiftVAqy82At2UMk8sBNcaNXf2vhvqEpXtIZUklTaC4a/k79BwwJQHGIB1bIJ+7KjxUWcnijfdQ6X/HIjgouLZ9Tsm38Cmfej7p2r+5Q/SxVryzEsmq1V80S1Fkh5pY/Pj2/bvVgBy66StS2hmciLf5h8sN0IA= Received: from DB3PR0402MB3852.eurprd04.prod.outlook.com (52.134.71.143) by HE1PR04MB3001.eurprd04.prod.outlook.com (10.170.255.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.302.9; Mon, 18 Dec 2017 21:43:36 +0000 Received: from DB3PR0402MB3852.eurprd04.prod.outlook.com ([fe80::2cde:683a:8f96:1a19]) by DB3PR0402MB3852.eurprd04.prod.outlook.com ([fe80::2cde:683a:8f96:1a19%13]) with mapi id 15.20.0282.012; Mon, 18 Dec 2017 21:43:35 +0000 From: Ahmed Mansour To: "dev@dpdk.org" , "Shally.Verma@cavium.com" CC: "Mahipal.Challa@cavium.com" , "NarayanaPrasad.Athreya@cavium.com" , "pablo.de.lara.guarch@intel.com" , "fiona.trahe@intel.com" , Roy Pledge , Youri Querry , Hemant Agrawal Thread-Topic: [RFC v3 1/1] lib: add compressdev API Thread-Index: AQHTeElBzPqMCHpj+U2ot72hAgksuw== Date: Mon, 18 Dec 2017 21:43:35 +0000 Message-ID: References: <1511542566-10455-1-git-send-email-fiona.trahe@intel.com> <1513360153-15036-1-git-send-email-fiona.trahe@intel.com> Accept-Language: en-CA, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=ahmed.mansour@nxp.com; x-originating-ip: [192.88.168.1] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR04MB3001; 6:X03aAhwm3qOW5jwxUvsNjOg4omIqX5XR8Nw9VDjFotoAxtXgkTv1RrAtpPfe3bAiFUr5xkT2YZXTjWioUZQtM+BIOEKV+q+NXnlLjZckBvVG/5P6ix1osMl3hn3Hqee+qIYVIIUKy4joODAxlZElFQO0TZDYn7pcDKbTfqb7KLPMT1Yudlvgu8gFR0Z4KYJOrt0sbj0uR3iz0hHhPJL9Pd+3+xzIUlABCgRdFfshYwWFz2hmZvMr58MIED5XRtbmH7/zeMl4b4PcJI9O45wX8xHv5m3v8kSwkcFiVrsjwli+PCs5AbeBasRrFZg1YDxCgOvrTwkj2Qzkk5veO5JMi9vB10qduRb8d8BGxuoZGIQ=; 5:DfarQt+QgDjuNhsYRkl8GV2mlMEBR64Yj8bM4vU/Yre4TA3zbcdlYaTisul9orfyAa4vQMppcc4Po+NuQTYlDl+R0GbVsxcCsIsQEbUo49l5YuYVtMGn8GQUKInC5V0NWVG8EGJEVHoDNRwD9/tgsgRXXOqpcetn4Awq7AOvmA8=; 24:gnLwDi/Y5/bmqvL1a1h5Xxfjl7Rbgf5E450iWKklvNxuYpnp0XtNtDEhaoMvaeQ5Bf0fH4BlU405pg7d5Tau00JgjWHXcttDyP5gXPzK0tU=; 7:cFhAmy/+3oSLHFMvcKSjBgUYkEkwPDfNzFrVmvqR/M8SGiwlZbUy1COPTOGX377VatPieAEba1CIjE9scI/hL0oZzKG8jWwNZ0Dm2LJtDpqRNyyDofpxP0MxLz/nahifzjm39Q75GISQTg2dRNJL9Vww4k7BqTe+hFs5TdxanWBWNS+3zvyPKhnCnKUxlDXYySx3CptofPEBjKAGmw8BIGYWtZBjueMd+yRtMSNKTEa48VRdGeWgQKMqennyT0WK x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(396003)(376002)(39860400002)(366004)(346002)(189003)(199004)(24454002)(2906002)(45080400002)(53936002)(68736007)(3280700002)(33656002)(229853002)(966005)(106356001)(99286004)(9686003)(25786009)(6246003)(55016002)(478600001)(14454004)(105586002)(5250100002)(6306002)(53546011)(2501003)(575784001)(6116002)(4326008)(102836003)(305945005)(66066001)(97736004)(8676002)(5660300001)(6506007)(7696005)(86362001)(7736002)(3660700001)(6436002)(316002)(3846002)(74316002)(8936002)(110136005)(81166006)(2900100001)(81156014)(54906003)(76176011); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR04MB3001; H:DB3PR0402MB3852.eurprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 6401c1aa-6cb9-4c51-bbe5-08d54660645c x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603307); SRVR:HE1PR04MB3001; x-ms-traffictypediagnostic: HE1PR04MB3001: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(189930954265078)(45079756050767); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(3231023)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041248)(20161123560025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123558100)(20161123564025)(6072148)(201708071742011); SRVR:HE1PR04MB3001; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:HE1PR04MB3001; x-forefront-prvs: 0525BB0ADF received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6401c1aa-6cb9-4c51-bbe5-08d54660645c X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2017 21:43:35.6303 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR04MB3001 Subject: Re: [dpdk-dev] [RFC v3 1/1] lib: add compressdev API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Dec 2017 21:43:39 -0000 On 12/15/2017 11:19 PM, Trahe, Fiona wrote:=0A= .. =0A= =0A= > +=0A= > +/** Compression Algorithms */=0A= > +enum rte_comp_algorithm {=0A= > + RTE_COMP_NULL =3D 0,=0A= > + /**< No compression.=0A= > + * Pass-through, data is copied unchanged from source buffer to=0A= > + * destination buffer.=0A= > + */=0A= > + RTE_COMP_DEFLATE,=0A= > + /**< DEFLATE compression algorithm=0A= > + * https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2= Ftools.ietf.org%2Fhtml%2Frfc1951&data=3D02%7C01%7Cahmed.mansour%40nxp.com%7= Cf3edbd70b38b49eb1f0308d54634e444%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C= 0%7C636492115333128845&sdata=3DB3G0aIncVAK17dXlnSivXi0e56h2D7pEQZ9gK%2Fh3qZ= Q%3D&reserved=3D0=0A= > + */=0A= > + RTE_COMP_LZS,=0A= > + /**< LZS compression algorithm=0A= > + * https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2= Ftools.ietf.org%2Fhtml%2Frfc2395&data=3D02%7C01%7Cahmed.mansour%40nxp.com%7= Cf3edbd70b38b49eb1f0308d54634e444%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C= 0%7C636492115333128845&sdata=3DaNRFIfkelXlCUUgpp%2BzCYaTu28tp6fF0m6k7F13w1P= s%3D&reserved=3D0=0A= > + */=0A= > + RTE_COMP_ALGO_LIST_END=0A= > +};=0A= > +=0A= > +/**< Compression Level.=0A= > + * The number is interpreted by each PMD differently. However, lower num= bers=0A= > + * give fastest compression, at the expense of compression ratio while= =0A= > + * higher numbers may give better compression ratios but are likely slow= er.=0A= > + */=0A= > +#define RTE_COMP_LEVEL_PMD_DEFAULT (-1)=0A= > +/** Use PMD Default */=0A= > +#define RTE_COMP_LEVEL_NONE (0)=0A= > +/** Output uncompressed blocks if supported by the specified algorithm *= /=0A= > +#define RTE_COMP_LEVEL_MIN (1)=0A= > +/** Use minimum compression level supported by the PMD */=0A= > +#define RTE_COMP_LEVEL_MAX (9)=0A= > +/** Use maximum compression level supported by the PMD */=0A= > +=0A= > +/** Compression checksum types */=0A= > +enum rte_comp_checksum_type {=0A= > + RTE_COMP_NONE,=0A= > + /**< No checksum generated */=0A= > + RTE_COMP_CRC32,=0A= > + /**< Generates a CRC32 checksum, as used by gzip */=0A= > + RTE_COMP_ADLER32,=0A= > + /**< Generates an Adler-32 checksum, as used by zlib */=0A= > + RTE_COMP_CRC32_ADLER32,=0A= > + /**< Generates both Adler-32 and CRC32 checksums, concatenated.=0A= > + * CRC32 is in the lower 32bits, Adler-32 in the upper 32 bits.=0A= > + */=0A= =0A= What would be a real life use case for returning both CRC32 and ADLER32?=0A= Packaging the data once as Gzip and once as zlib?=0A= =0A= > +};=0A= > +=0A= > +/*=0A= > + * enum rte_comp_hash_algo {=0A= > + * RTE_COMP_HASH_NONE,=0A= > + * RTE_COMP_HASH_SHA1,=0A= > + * RTE_COMP_HASH_SHA256,=0A= > + * };=0A= > + * Need further input from cavium on this=0A= > + * xform will need a flag with above enum value=0A= > + * op will need to provide a virt/phys ptr to a data buffer of appropria= te size.=0A= > + * And via capability PMD can say whether supported or not.=0A= > + */=0A= > +=0A= > +/** Compression Huffman Type - used by DEFLATE algorithm */=0A= > +enum rte_comp_huffman {=0A= > + RTE_COMP_DEFAULT,=0A= > + /**< PMD may choose which Huffman codes to use */=0A= > + RTE_COMP_FIXED,=0A= > + /**< Use Fixed Huffman codes */=0A= > + RTE_COMP_DYNAMIC,=0A= > + /**< Use Dynamic Huffman codes */=0A= > +};=0A= > +=0A= > +=0A= > +enum rte_comp_flush_flag {=0A= > + RTE_COMP_FLUSH_NONE,=0A= > + /**< Data is not flushed. Output may remain in the compressor and be=0A= > + * processed during a following op. It may not be possible to decompres= s=0A= > + * output until a later op with some other flush flag has been sent.=0A= > + */=0A= > + RTE_COMP_FLUSH_SYNC,=0A= > + /**< All data should be flushed to output buffer. Output data can be=0A= > + * decompressed. However state and history is not cleared, so future=0A= > + * ops may use history from this op */=0A= > + RTE_COMP_FLUSH_FULL,=0A= > + /**< All data should be flushed to output buffer. Output data can be=0A= > + * decompressed. State and history data is cleared, so future=0A= > + * ops will be independent of ops processed before this.=0A= > + */=0A= > + RTE_COMP_FLUSH_FINAL=0A= > + /**< Same as RTE_COMP_FLUSH_FULL but also bfinal bit is set in last blo= ck=0A= > + */=0A= > + /* TODO:=0A= > + * describe flag meanings for decompression.=0A= > + * describe behavous in OUT_OF_SPACE case.=0A= > + * At least the last flag is specific to deflate algo. Should this be= =0A= > + * called rte_comp_deflate_flush_flag? And should there be=0A= > + * comp_op_deflate_params in the op? */=0A= =0A= What about Z_BLOCK and Z_TREES? Those are needed for sfwr zlib.net =0A= replacements.=0A= =0A= > +};=0A= > +=0A= > +/** Compression transform types */=0A= > +enum rte_comp_xform_type {=0A= > + RTE_COMP_COMPRESS,=0A= > + /**< Compression service - compress */=0A= > + RTE_COMP_DECOMPRESS,=0A= > + /**< Compression service - decompress */=0A= > +};=0A= > +=0A= =0A= ... =0A= =0A= > +struct rte_comp_session;=0A= > +/**=0A= > + * Compression Operation.=0A= > + *=0A= > + * This structure contains data relating to performing a compression=0A= > + * operation on the referenced mbuf data buffers.=0A= > + *=0A= > + * All compression operations are Out-of-place (OOP) operations,=0A= > + * as the size of the output data is different to the size of the input = data.=0A= > + *=0A= > + * Comp operations are enqueued and dequeued in comp PMDs using the=0A= > + * rte_compressdev_enqueue_burst() / rte_compressdev_dequeue_burst() API= s=0A= > + */=0A= > +struct rte_comp_op {=0A= > +=0A= > + enum rte_comp_op_type op_type;=0A= > + void * stream_private;=0A= > + /* location where PMD maintains stream state=0A= > + * only required if op_type is STATEFUL, else should be NULL=0A= > + */=0A= > + struct rte_comp_session *session;=0A= > + /**< Handle for the initialised session context */=0A= > + struct rte_mempool *mempool;=0A= > + /**< mempool from which operation is allocated */=0A= > + phys_addr_t phys_addr;=0A= > + /**< physical address of this operation */=0A= =0A= iova_addr?=0A= =0A= > + struct rte_mbuf *m_src;=0A= > + /**< source mbuf=0A= > + * The total size of the input buffer(s) can be retrieved using=0A= > + * rte_pktmbuf_data_len(m_src)=0A= > + */=0A= > + struct rte_mbuf *m_dst;=0A= > + /**< destination mbuf=0A= > + * The total size of the output buffer(s) can be retrieved using=0A= > + * rte_pktmbuf_data_len(m_dst)=0A= > + */=0A= > +=0A= > + struct {=0A= > + uint32_t offset;=0A= > + /**< Starting point for compression or decompression,=0A= > + * specified as number of bytes from start of packet in=0A= > + * source buffer.=0A= > + * Starting point for checksum generation in compress direction.=0A= > + */=0A= =0A= Why should offset have two different meanings for compression and=0A= decompression. It seems that the use case of offset on input is=0A= applicable to both use modes=0A= =0A= =0A= ...=0A= =0A= =0A= =0A=