From: Pablo de Lara <pablo.de.lara.guarch@intel.com>
To: dev@dpdk.org
Cc: fiona.trahe@intel.com, shally.verma@cavium.com,
ahmed.mansour@nxp.com, Ashish.Gupta@cavium.com,
Pablo de Lara <pablo.de.lara.guarch@intel.com>,
Pablo, Shally Verma <shally.verma@caviumnetworks.com>,
Ashish Gupta <ashish.gupta@caviumnetworks.com>
Subject: [dpdk-dev] [PATCH v3 00/13] Implement compression API
Date: Fri, 6 Apr 2018 19:04:59 +0100 [thread overview]
Message-ID: <20180406180512.40154-1-pablo.de.lara.guarch@intel.com> (raw)
In-Reply-To: <1522166672-19415-1-git-send-email-fiona.trahe@intel.com>
With the vast amounts of data being transported around networks
and stored in storage systems, reducing data size is becoming ever more important.
There are both software libraries and hardware devices available that
provide compression, but no common API.
Such an API is proposed in this patchset, which supports the following features:
- Deflate Algorithm (https://tools.ietf.org/html/rfc1951)
- LZS algorithm (https://tools.ietf.org/html/rfc2395)
- Static and Dynamic Huffman encoding.
- Compression levels
- Checksum generation
- Asynchronous burst API
- private_xform - a place for PMDs to hold private data derived from
a xform and used by stateless operations.
- stream - a place for PMDs to hold private data derived from
a xform and also maintain state and history data. For
stateful flows.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Pablo de Lara pablo.de.lara.guarch@intel.com
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
Opens:
- creation of private_xform and stream mempools. In v3, config
added to give PMD the size params needed to create mempools.
Still open as to whether the pools should be created by
application, in API layer or by PMDs. Expect to resolve in a v4.
- alternative to mbufs for passing data between application
and accelerator. A lot of varied opinions on this, expect to
address in a future release.
- addition of feature to add capability to PMD to allow more than one
inflight operation from a stateful stream - to be added by NXP
Changes in v3:
- Remove rte_comp_op_ctod helper functions
- Remove param_range_check macro
- Rename from phys_addr to iova_addr
- Remove rte_comp_algo_strings
- Rename rte_compressdev_pmd_is_valid_dev to rte_compressdev_is_valid_dev
- Remove feature flags from compressdev
- Support hash operations
- Add shareable priv xform in feature flags, instead of returnin it
on xform creation
- Allow max number of queue pairs to be 0, meaning that there is no
limit.
- Add invalid configuration checks
- Add capability helper functions
Changes in v2:
- Add stream APIs
- Remove session
- Add SHAREABLE / NON_SHAREABLE private_xform types
- Add algo enum 'UNSPECIFIED' to fix warning in capabilities
- Change one remaining log to use dynamic logging.
- Add rte_cache_aligned keyword to op
- Rename enums with better names _ALGO, __CHECKSUM, _HUFFMAN_
- Use const keyword when passing xform
- Remove qp_count fn from dev_ops as never used
- Remove max_nb_queue-pairs from compressdev_init_param as never used
- Clarify device configure and start sequence
- Replace OUT_OF_SPACE with OUT_OF_SPACE_RECOVERABLE and TERMINATED
and clarified usage.
- Add stream and private_xform sizes to device config for use in
mempool creation
- Add capability helper fn
- Use Base2 log value for window size on xforms
- Add Meson build
- Update MAINTAINERS
- Update Map file
- Change order in doxy file
- Update Release note
Fiona Trahe (12):
compressdev: add basic device management
compressdev: add queue pair management
compressdev: add compression specific data
compressdev: add enqueue/dequeue functions
compressdev: add operation management
compressdev: support stateless operations
compressdev: support stateful operations
compressdev: add device feature flags
compressdev: add compression service feature flags
compressdev: add device stats
compressdev: add device capabilities
compressdev: get device id from name
Shally Verma (1):
compressdev: support hash operations
MAINTAINERS | 7 +
config/common_base | 6 +
config/rte_config.h | 3 +
doc/api/doxy-api-index.md | 1 +
doc/api/doxy-api.conf | 1 +
doc/guides/rel_notes/release_18_05.rst | 6 +
lib/Makefile | 3 +
lib/librte_compressdev/Makefile | 29 +
lib/librte_compressdev/meson.build | 9 +
lib/librte_compressdev/rte_comp.h | 539 ++++++++++++
lib/librte_compressdev/rte_compressdev.c | 902 +++++++++++++++++++++
lib/librte_compressdev/rte_compressdev.h | 701 ++++++++++++++++
lib/librte_compressdev/rte_compressdev_pmd.c | 157 ++++
lib/librte_compressdev/rte_compressdev_pmd.h | 437 ++++++++++
lib/librte_compressdev/rte_compressdev_version.map | 44 +
lib/meson.build | 2 +-
mk/rte.app.mk | 1 +
17 files changed, 2847 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_compressdev/Makefile
create mode 100644 lib/librte_compressdev/meson.build
create mode 100644 lib/librte_compressdev/rte_comp.h
create mode 100644 lib/librte_compressdev/rte_compressdev.c
create mode 100644 lib/librte_compressdev/rte_compressdev.h
create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.c
create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.h
create mode 100644 lib/librte_compressdev/rte_compressdev_version.map
--
2.14.3
next prev parent reply other threads:[~2018-04-06 18:05 UTC|newest]
Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-02 18:25 [dpdk-dev] [PATCH] compressdev: implement API Fiona Trahe
2018-02-04 14:18 ` Thomas Monjalon
2018-02-21 19:11 ` Trahe, Fiona
2018-02-24 1:17 ` Ahmed Mansour
2018-02-26 11:24 ` Trahe, Fiona
2018-02-26 19:15 ` Ahmed Mansour
2018-02-27 5:48 ` Verma, Shally
2018-02-26 11:25 ` Verma, Shally
2018-02-26 21:35 ` Ahmed Mansour
2018-02-27 5:53 ` Verma, Shally
2018-02-28 18:39 ` Trahe, Fiona
2018-03-01 6:58 ` Verma, Shally
2018-03-01 14:41 ` Trahe, Fiona
2018-03-02 0:55 ` Ahmed Mansour
2018-03-02 9:53 ` Trahe, Fiona
2018-03-02 19:48 ` Ahmed Mansour
2018-03-05 14:32 ` Verma, Shally
2018-03-06 23:33 ` Ahmed Mansour
2018-03-03 0:52 ` Ahmed Mansour
2018-02-04 14:24 ` Thomas Monjalon
2018-03-23 18:08 ` Trahe, Fiona
2018-03-24 1:02 ` Thomas Monjalon
2018-03-26 11:44 ` Trahe, Fiona
2018-03-27 16:04 ` [dpdk-dev] [PATCH v2 0/3] implement compression API Fiona Trahe
2018-04-06 18:04 ` Pablo de Lara [this message]
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 01/13] compressdev: add basic device management Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 02/13] compressdev: add queue pair management Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 03/13] compressdev: add compression specific data Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 04/13] compressdev: add enqueue/dequeue functions Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 05/13] compressdev: add operation management Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 06/13] compressdev: support stateless operations Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 07/13] compressdev: support stateful operations Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 08/13] compressdev: support hash operations Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 09/13] compressdev: add device feature flags Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 10/13] compressdev: add compression service " Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 11/13] compressdev: add device stats Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 12/13] compressdev: add device capabilities Pablo de Lara
2018-04-06 18:05 ` [dpdk-dev] [PATCH v3 13/13] compressdev: get device id from name Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 00/13] Implement compression API Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 01/13] compressdev: add basic device management Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 02/13] compressdev: add queue pair management Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 03/13] compressdev: add compression specific data Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 04/13] compressdev: add enqueue/dequeue functions Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 05/13] compressdev: add operation management Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 06/13] compressdev: support stateless operations Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 07/13] compressdev: support stateful operations Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 08/13] compressdev: support hash operations Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 09/13] compressdev: add device feature flags Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 10/13] compressdev: add compression service " Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 11/13] compressdev: add device stats Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 12/13] compressdev: add device capabilities Pablo de Lara
2018-04-08 12:58 ` [dpdk-dev] [PATCH v4 13/13] compressdev: get device id from name Pablo de Lara
2018-03-27 16:04 ` [dpdk-dev] [PATCH v2 1/3] compressdev: add structs and enum for compression service Fiona Trahe
2018-03-27 16:04 ` [dpdk-dev] [PATCH v2 2/3] compressdev: implement API Fiona Trahe
2018-03-27 16:04 ` [dpdk-dev] [PATCH v2 3/3] doc: update doxy and release note for compressdev Fiona Trahe
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 00/13] Implement compression API Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 01/13] compressdev: add basic device management Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 02/13] compressdev: add queue pair management Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 03/13] compressdev: add compression specific data Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 04/13] compressdev: add enqueue/dequeue functions Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 05/13] compressdev: add operation management Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 06/13] compressdev: support stateless operations Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 07/13] compressdev: support stateful operations Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 08/13] compressdev: support hash operations Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 09/13] compressdev: add device feature flags Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 10/13] compressdev: add compression service " Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 11/13] compressdev: add device stats Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 12/13] compressdev: add device capabilities Pablo de Lara
2018-04-13 18:18 ` [dpdk-dev] [PATCH v5 13/13] compressdev: get device id from name Pablo de Lara
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 00/14] Implement compression API Pablo de Lara
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 01/14] compressdev: add basic device management Pablo de Lara
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 02/14] compressdev: add queue pair management Pablo de Lara
2018-05-04 2:48 ` Verma, Shally
2018-05-04 8:43 ` Trahe, Fiona
2018-05-14 7:53 ` Verma, Shally
2018-05-14 8:04 ` De Lara Guarch, Pablo
2018-05-14 8:16 ` Verma, Shally
2018-05-14 8:30 ` De Lara Guarch, Pablo
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 03/14] compressdev: add compression specific data Pablo de Lara
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 04/14] compressdev: add enqueue/dequeue functions Pablo de Lara
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 05/14] compressdev: add operation management Pablo de Lara
2018-04-27 13:23 ` [dpdk-dev] [PATCH v6 06/14] compressdev: support stateless operations Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 07/14] compressdev: support stateful operations Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 08/14] compressdev: support hash operations Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 09/14] compressdev: add device feature flags Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 10/14] compressdev: add compression service " Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 11/14] compressdev: add device stats Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 12/14] compressdev: add device capabilities Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 13/14] compressdev: get device id from name Pablo de Lara
2018-04-27 13:24 ` [dpdk-dev] [PATCH v6 14/14] doc: add compressdev library guide Pablo de Lara
2018-05-08 21:25 ` [dpdk-dev] [PATCH v6 00/14] Implement compression API De Lara Guarch, Pablo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180406180512.40154-1-pablo.de.lara.guarch@intel.com \
--to=pablo.de.lara.guarch@intel.com \
--cc=Ashish.Gupta@cavium.com \
--cc=ahmed.mansour@nxp.com \
--cc=ashish.gupta@caviumnetworks.com \
--cc=dev@dpdk.org \
--cc=fiona.trahe@intel.com \
--cc=shally.verma@cavium.com \
--cc=shally.verma@caviumnetworks.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).