From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 14C4C1B375 for ; Fri, 22 Dec 2017 15:16:00 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Dec 2017 06:15:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,441,1508828400"; d="scan'208";a="4045660" Received: from irsmsx102.ger.corp.intel.com ([163.33.3.155]) by fmsmga002.fm.intel.com with ESMTP; 22 Dec 2017 06:15:57 -0800 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.46]) by IRSMSX102.ger.corp.intel.com ([169.254.2.180]) with mapi id 14.03.0319.002; Fri, 22 Dec 2017 14:15:55 +0000 From: "Trahe, Fiona" To: Ahmed Mansour , "dev@dpdk.org" , "Shally.Verma@cavium.com" CC: "Mahipal.Challa@cavium.com" , "NarayanaPrasad.Athreya@cavium.com" , "De Lara Guarch, Pablo" , Roy Pledge , Youri Querry , Hemant Agrawal , "Trahe, Fiona" Thread-Topic: [RFC v3 1/1] lib: add compressdev API Thread-Index: AQHTdc0UinnGLuzhCUCQkELi5wsFSqNPawxQ Date: Fri, 22 Dec 2017 14:15:54 +0000 Message-ID: <348A99DA5F5B7549AA880327E580B435892E9968@IRSMSX101.ger.corp.intel.com> References: <1511542566-10455-1-git-send-email-fiona.trahe@intel.com> <1513360153-15036-1-git-send-email-fiona.trahe@intel.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYjI0YmRjMzctNDIzZi00NjBmLWIyYjctZWYxMzVhMjE1NzJmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IlkzUzRaajlRd1FuNGUxSmU4VDE5c3N6eGVzeDlJTXk4RFh3MWFJYk9FRVU9In0= x-ctpclassification: CTP_IC dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC v3 1/1] lib: add compressdev API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Dec 2017 14:16:01 -0000 Hi Ahmed, > -----Original Message----- > From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com] > Sent: Monday, December 18, 2017 9:44 PM > To: dev@dpdk.org; Shally.Verma@cavium.com > Cc: Mahipal.Challa@cavium.com; NarayanaPrasad.Athreya@cavium.com; De Lara= Guarch, Pablo > ; Trahe, Fiona ; R= oy Pledge > ; Youri Querry ; Hemant Agraw= al > > Subject: Re: [RFC v3 1/1] lib: add compressdev API >=20 > On 12/15/2017 11:19 PM, Trahe, Fiona wrote: > .. >=20 > > + > > +/** Compression Algorithms */ > > +enum rte_comp_algorithm { > > + RTE_COMP_NULL =3D 0, > > + /**< No compression. > > + * Pass-through, data is copied unchanged from source buffer to > > + * destination buffer. > > + */ > > + RTE_COMP_DEFLATE, > > + /**< DEFLATE compression algorithm > > + * > https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Ftool= s.ietf.org%2Fhtml%2Frfc195 > 1&data=3D02%7C01%7Cahmed.mansour%40nxp.com%7Cf3edbd70b38b49eb1f0308d54634= e444%7C686ea > 1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636492115333128845&sdata=3DB3G0aInc= VAK17dXlnSivXi0 > e56h2D7pEQZ9gK%2Fh3qZQ%3D&reserved=3D0 > > + */ > > + RTE_COMP_LZS, > > + /**< LZS compression algorithm > > + * > https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Ftool= s.ietf.org%2Fhtml%2Frfc239 > 5&data=3D02%7C01%7Cahmed.mansour%40nxp.com%7Cf3edbd70b38b49eb1f0308d54634= e444%7C686ea > 1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636492115333128845&sdata=3DaNRFIfke= lXlCUUgpp%2BzC > YaTu28tp6fF0m6k7F13w1Ps%3D&reserved=3D0 > > + */ > > + RTE_COMP_ALGO_LIST_END > > +}; > > + > > +/**< Compression Level. > > + * The number is interpreted by each PMD differently. However, lower n= umbers > > + * give fastest compression, at the expense of compression ratio while > > + * higher numbers may give better compression ratios but are likely sl= ower. > > + */ > > +#define RTE_COMP_LEVEL_PMD_DEFAULT (-1) > > +/** Use PMD Default */ > > +#define RTE_COMP_LEVEL_NONE (0) > > +/** Output uncompressed blocks if supported by the specified algorithm= */ > > +#define RTE_COMP_LEVEL_MIN (1) > > +/** Use minimum compression level supported by the PMD */ > > +#define RTE_COMP_LEVEL_MAX (9) > > +/** Use maximum compression level supported by the PMD */ > > + > > +/** Compression checksum types */ > > +enum rte_comp_checksum_type { > > + RTE_COMP_NONE, > > + /**< No checksum generated */ > > + RTE_COMP_CRC32, > > + /**< Generates a CRC32 checksum, as used by gzip */ > > + RTE_COMP_ADLER32, > > + /**< Generates an Adler-32 checksum, as used by zlib */ > > + RTE_COMP_CRC32_ADLER32, > > + /**< Generates both Adler-32 and CRC32 checksums, concatenated. > > + * CRC32 is in the lower 32bits, Adler-32 in the upper 32 bits. > > + */ >=20 > What would be a real life use case for returning both CRC32 and ADLER32? > Packaging the data once as Gzip and once as zlib? [Fiona] We've had requests for this from customers. >=20 > > +}; > > + > > +/* > > + * enum rte_comp_hash_algo { > > + * RTE_COMP_HASH_NONE, > > + * RTE_COMP_HASH_SHA1, > > + * RTE_COMP_HASH_SHA256, > > + * }; > > + * Need further input from cavium on this > > + * xform will need a flag with above enum value > > + * op will need to provide a virt/phys ptr to a data buffer of appropr= iate size. > > + * And via capability PMD can say whether supported or not. > > + */ > > + > > +/** Compression Huffman Type - used by DEFLATE algorithm */ > > +enum rte_comp_huffman { > > + RTE_COMP_DEFAULT, > > + /**< PMD may choose which Huffman codes to use */ > > + RTE_COMP_FIXED, > > + /**< Use Fixed Huffman codes */ > > + RTE_COMP_DYNAMIC, > > + /**< Use Dynamic Huffman codes */ > > +}; > > + > > + > > +enum rte_comp_flush_flag { > > + RTE_COMP_FLUSH_NONE, > > + /**< Data is not flushed. Output may remain in the compressor and be > > + * processed during a following op. It may not be possible to decompr= ess > > + * output until a later op with some other flush flag has been sent. > > + */ > > + RTE_COMP_FLUSH_SYNC, > > + /**< All data should be flushed to output buffer. Output data can be > > + * decompressed. However state and history is not cleared, so future > > + * ops may use history from this op */ > > + RTE_COMP_FLUSH_FULL, > > + /**< All data should be flushed to output buffer. Output data can be > > + * decompressed. State and history data is cleared, so future > > + * ops will be independent of ops processed before this. > > + */ > > + RTE_COMP_FLUSH_FINAL > > + /**< Same as RTE_COMP_FLUSH_FULL but also bfinal bit is set in last b= lock > > + */ > > + /* TODO: > > + * describe flag meanings for decompression. > > + * describe behavous in OUT_OF_SPACE case. > > + * At least the last flag is specific to deflate algo. Should this b= e > > + * called rte_comp_deflate_flush_flag? And should there be > > + * comp_op_deflate_params in the op? */ >=20 > What about Z_BLOCK and Z_TREES? Those are needed for sfwr zlib.net > replacements. [Fiona] We haven't seen a need for those. I would suggest proposing a patch= later once the initial API is stabilised.=20 > > +}; > > + > > +/** Compression transform types */ > > +enum rte_comp_xform_type { > > + RTE_COMP_COMPRESS, > > + /**< Compression service - compress */ > > + RTE_COMP_DECOMPRESS, > > + /**< Compression service - decompress */ > > +}; > > + >=20 > ... >=20 > > +struct rte_comp_session; > > +/** > > + * Compression Operation. > > + * > > + * This structure contains data relating to performing a compression > > + * operation on the referenced mbuf data buffers. > > + * > > + * All compression operations are Out-of-place (OOP) operations, > > + * as the size of the output data is different to the size of the inpu= t data. > > + * > > + * Comp operations are enqueued and dequeued in comp PMDs using the > > + * rte_compressdev_enqueue_burst() / rte_compressdev_dequeue_burst() A= PIs > > + */ > > +struct rte_comp_op { > > + > > + enum rte_comp_op_type op_type; > > + void * stream_private; > > + /* location where PMD maintains stream state > > + * only required if op_type is STATEFUL, else should be NULL > > + */ > > + struct rte_comp_session *session; > > + /**< Handle for the initialised session context */ > > + struct rte_mempool *mempool; > > + /**< mempool from which operation is allocated */ > > + phys_addr_t phys_addr; > > + /**< physical address of this operation */ >=20 > iova_addr? [Fiona] Yes, this should be rte_iova_t. will fix this in the v1. >=20 > > + struct rte_mbuf *m_src; > > + /**< source mbuf > > + * The total size of the input buffer(s) can be retrieved using > > + * rte_pktmbuf_data_len(m_src) > > + */ > > + struct rte_mbuf *m_dst; > > + /**< destination mbuf > > + * The total size of the output buffer(s) can be retrieved using > > + * rte_pktmbuf_data_len(m_dst) > > + */ > > + > > + struct { > > + uint32_t offset; > > + /**< Starting point for compression or decompression, > > + * specified as number of bytes from start of packet in > > + * source buffer. > > + * Starting point for checksum generation in compress direction. > > + */ >=20 > Why should offset have two different meanings for compression and > decompression. It seems that the use case of offset on input is > applicable to both use modes=20 [Fiona] It does have the same meaning for compression/decompression? Just for checksum it's different. Checksum is usually required on the uncom= pressed data. In decompression the input buffer contains compressed data. =20 > > ...