From: "Trahe, Fiona" <fiona.trahe@intel.com>
To: Ahmed Mansour <ahmed.mansour@nxp.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Shally.Verma@cavium.com" <Shally.Verma@cavium.com>
Cc: "Mahipal.Challa@cavium.com" <Mahipal.Challa@cavium.com>,
"NarayanaPrasad.Athreya@cavium.com"
<NarayanaPrasad.Athreya@cavium.com>,
"De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>,
Roy Pledge <roy.pledge@nxp.com>,
Youri Querry <youri.querry_1@nxp.com>,
Hemant Agrawal <hemant.agrawal@nxp.com>,
"Trahe, Fiona" <fiona.trahe@intel.com>
Subject: Re: [dpdk-dev] [RFC v3 1/1] lib: add compressdev API
Date: Fri, 22 Dec 2017 14:15:54 +0000 [thread overview]
Message-ID: <348A99DA5F5B7549AA880327E580B435892E9968@IRSMSX101.ger.corp.intel.com> (raw)
In-Reply-To: <DB3PR0402MB3852E8D80A5A5EDAB9121577E10E0@DB3PR0402MB3852.eurprd04.prod.outlook.com>
Hi Ahmed,
> -----Original Message-----
> From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com]
> Sent: Monday, December 18, 2017 9:44 PM
> To: dev@dpdk.org; Shally.Verma@cavium.com
> Cc: Mahipal.Challa@cavium.com; NarayanaPrasad.Athreya@cavium.com; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>; Roy Pledge
> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>
> Subject: Re: [RFC v3 1/1] lib: add compressdev API
>
> On 12/15/2017 11:19 PM, Trahe, Fiona wrote:
> .. <snip>
>
> > +
> > +/** Compression Algorithms */
> > +enum rte_comp_algorithm {
> > + RTE_COMP_NULL = 0,
> > + /**< No compression.
> > + * Pass-through, data is copied unchanged from source buffer to
> > + * destination buffer.
> > + */
> > + RTE_COMP_DEFLATE,
> > + /**< DEFLATE compression algorithm
> > + *
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc195
> 1&data=02%7C01%7Cahmed.mansour%40nxp.com%7Cf3edbd70b38b49eb1f0308d54634e444%7C686ea
> 1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636492115333128845&sdata=B3G0aIncVAK17dXlnSivXi0
> e56h2D7pEQZ9gK%2Fh3qZQ%3D&reserved=0
> > + */
> > + RTE_COMP_LZS,
> > + /**< LZS compression algorithm
> > + *
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc239
> 5&data=02%7C01%7Cahmed.mansour%40nxp.com%7Cf3edbd70b38b49eb1f0308d54634e444%7C686ea
> 1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636492115333128845&sdata=aNRFIfkelXlCUUgpp%2BzC
> YaTu28tp6fF0m6k7F13w1Ps%3D&reserved=0
> > + */
> > + RTE_COMP_ALGO_LIST_END
> > +};
> > +
> > +/**< Compression Level.
> > + * The number is interpreted by each PMD differently. However, lower numbers
> > + * give fastest compression, at the expense of compression ratio while
> > + * higher numbers may give better compression ratios but are likely slower.
> > + */
> > +#define RTE_COMP_LEVEL_PMD_DEFAULT (-1)
> > +/** Use PMD Default */
> > +#define RTE_COMP_LEVEL_NONE (0)
> > +/** Output uncompressed blocks if supported by the specified algorithm */
> > +#define RTE_COMP_LEVEL_MIN (1)
> > +/** Use minimum compression level supported by the PMD */
> > +#define RTE_COMP_LEVEL_MAX (9)
> > +/** Use maximum compression level supported by the PMD */
> > +
> > +/** Compression checksum types */
> > +enum rte_comp_checksum_type {
> > + RTE_COMP_NONE,
> > + /**< No checksum generated */
> > + RTE_COMP_CRC32,
> > + /**< Generates a CRC32 checksum, as used by gzip */
> > + RTE_COMP_ADLER32,
> > + /**< Generates an Adler-32 checksum, as used by zlib */
> > + RTE_COMP_CRC32_ADLER32,
> > + /**< Generates both Adler-32 and CRC32 checksums, concatenated.
> > + * CRC32 is in the lower 32bits, Adler-32 in the upper 32 bits.
> > + */
>
> What would be a real life use case for returning both CRC32 and ADLER32?
> Packaging the data once as Gzip and once as zlib?
[Fiona] We've had requests for this from customers.
>
> > +};
> > +
> > +/*
> > + * enum rte_comp_hash_algo {
> > + * RTE_COMP_HASH_NONE,
> > + * RTE_COMP_HASH_SHA1,
> > + * RTE_COMP_HASH_SHA256,
> > + * };
> > + * Need further input from cavium on this
> > + * xform will need a flag with above enum value
> > + * op will need to provide a virt/phys ptr to a data buffer of appropriate size.
> > + * And via capability PMD can say whether supported or not.
> > + */
> > +
> > +/** Compression Huffman Type - used by DEFLATE algorithm */
> > +enum rte_comp_huffman {
> > + RTE_COMP_DEFAULT,
> > + /**< PMD may choose which Huffman codes to use */
> > + RTE_COMP_FIXED,
> > + /**< Use Fixed Huffman codes */
> > + RTE_COMP_DYNAMIC,
> > + /**< Use Dynamic Huffman codes */
> > +};
> > +
> > +
> > +enum rte_comp_flush_flag {
> > + RTE_COMP_FLUSH_NONE,
> > + /**< Data is not flushed. Output may remain in the compressor and be
> > + * processed during a following op. It may not be possible to decompress
> > + * output until a later op with some other flush flag has been sent.
> > + */
> > + RTE_COMP_FLUSH_SYNC,
> > + /**< All data should be flushed to output buffer. Output data can be
> > + * decompressed. However state and history is not cleared, so future
> > + * ops may use history from this op */
> > + RTE_COMP_FLUSH_FULL,
> > + /**< All data should be flushed to output buffer. Output data can be
> > + * decompressed. State and history data is cleared, so future
> > + * ops will be independent of ops processed before this.
> > + */
> > + RTE_COMP_FLUSH_FINAL
> > + /**< Same as RTE_COMP_FLUSH_FULL but also bfinal bit is set in last block
> > + */
> > + /* TODO:
> > + * describe flag meanings for decompression.
> > + * describe behavous in OUT_OF_SPACE case.
> > + * At least the last flag is specific to deflate algo. Should this be
> > + * called rte_comp_deflate_flush_flag? And should there be
> > + * comp_op_deflate_params in the op? */
>
> What about Z_BLOCK and Z_TREES? Those are needed for sfwr zlib.net
> replacements.
[Fiona] We haven't seen a need for those. I would suggest proposing a patch later once the initial API is stabilised.
> > +};
> > +
> > +/** Compression transform types */
> > +enum rte_comp_xform_type {
> > + RTE_COMP_COMPRESS,
> > + /**< Compression service - compress */
> > + RTE_COMP_DECOMPRESS,
> > + /**< Compression service - decompress */
> > +};
> > +
>
> ... <snip>
>
> > +struct rte_comp_session;
> > +/**
> > + * Compression Operation.
> > + *
> > + * This structure contains data relating to performing a compression
> > + * operation on the referenced mbuf data buffers.
> > + *
> > + * All compression operations are Out-of-place (OOP) operations,
> > + * as the size of the output data is different to the size of the input data.
> > + *
> > + * Comp operations are enqueued and dequeued in comp PMDs using the
> > + * rte_compressdev_enqueue_burst() / rte_compressdev_dequeue_burst() APIs
> > + */
> > +struct rte_comp_op {
> > +
> > + enum rte_comp_op_type op_type;
> > + void * stream_private;
> > + /* location where PMD maintains stream state
> > + * only required if op_type is STATEFUL, else should be NULL
> > + */
> > + struct rte_comp_session *session;
> > + /**< Handle for the initialised session context */
> > + struct rte_mempool *mempool;
> > + /**< mempool from which operation is allocated */
> > + phys_addr_t phys_addr;
> > + /**< physical address of this operation */
>
> iova_addr?
[Fiona] Yes, this should be rte_iova_t. will fix this in the v1.
>
> > + struct rte_mbuf *m_src;
> > + /**< source mbuf
> > + * The total size of the input buffer(s) can be retrieved using
> > + * rte_pktmbuf_data_len(m_src)
> > + */
> > + struct rte_mbuf *m_dst;
> > + /**< destination mbuf
> > + * The total size of the output buffer(s) can be retrieved using
> > + * rte_pktmbuf_data_len(m_dst)
> > + */
> > +
> > + struct {
> > + uint32_t offset;
> > + /**< Starting point for compression or decompression,
> > + * specified as number of bytes from start of packet in
> > + * source buffer.
> > + * Starting point for checksum generation in compress direction.
> > + */
>
> Why should offset have two different meanings for compression and
> decompression. It seems that the use case of offset on input is
> applicable to both use modes
[Fiona] It does have the same meaning for compression/decompression?
Just for checksum it's different. Checksum is usually required on the uncompressed data.
In decompression the input buffer contains compressed data.
> <snip>
> ...
next prev parent reply other threads:[~2017-12-22 14:16 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-24 16:56 [dpdk-dev] [RFC v2] " Trahe, Fiona
2017-12-07 9:58 ` Verma, Shally
2017-12-11 18:22 ` Trahe, Fiona
2017-12-12 4:43 ` Verma, Shally
2017-12-15 17:49 ` [dpdk-dev] [RFC v3 1/1] " Trahe, Fiona
2017-12-18 21:43 ` Ahmed Mansour
2017-12-22 14:15 ` Trahe, Fiona [this message]
2018-01-18 12:53 ` Verma, Shally
2018-01-19 12:00 ` Trahe, Fiona
2018-01-23 11:58 ` Verma, Shally
2018-01-24 19:36 ` Ahmed Mansour
2018-01-25 10:24 ` Verma, Shally
2018-01-25 18:43 ` Trahe, Fiona
2018-01-29 12:26 ` Verma, Shally
2018-01-29 17:16 ` Ahmed Mansour
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=348A99DA5F5B7549AA880327E580B435892E9968@IRSMSX101.ger.corp.intel.com \
--to=fiona.trahe@intel.com \
--cc=Mahipal.Challa@cavium.com \
--cc=NarayanaPrasad.Athreya@cavium.com \
--cc=Shally.Verma@cavium.com \
--cc=ahmed.mansour@nxp.com \
--cc=dev@dpdk.org \
--cc=hemant.agrawal@nxp.com \
--cc=pablo.de.lara.guarch@intel.com \
--cc=roy.pledge@nxp.com \
--cc=youri.querry_1@nxp.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).