From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id DEBB27CEB for ; Fri, 27 Apr 2018 15:23:55 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Apr 2018 06:23:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,335,1520924400"; d="scan'208";a="223862486" Received: from silpixa00399464.ir.intel.com (HELO silpixa00399464.ger.corp.intel.com) ([10.237.222.157]) by fmsmga005.fm.intel.com with ESMTP; 27 Apr 2018 06:23:52 -0700 From: Pablo de Lara To: dev@dpdk.org Cc: fiona.trahe@intel.com, shally.verma@cavium.com, ahmed.mansour@nxp.com, Ashish.Gupta@cavium.com, Pablo de Lara , Shally Verma , Ashish Gupta Date: Fri, 27 Apr 2018 14:23:53 +0100 Message-Id: <20180427132407.13385-1-pablo.de.lara.guarch@intel.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <1517595924-25963-1-git-send-email-fiona.trahe@intel.com> References: <1517595924-25963-1-git-send-email-fiona.trahe@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v6 00/14] Implement compression API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 13:23:56 -0000 With the vast amounts of data being transported around networks and stored in storage systems, reducing data size is becoming ever more important. There are both software libraries and hardware devices available that provide compression, but no common API. Such an API is proposed in this patchset, which supports the following features: - Deflate Algorithm (https://tools.ietf.org/html/rfc1951) - LZS algorithm (https://tools.ietf.org/html/rfc2395) - Static and Dynamic Huffman encoding. - Compression levels - Checksum generation - Asynchronous burst API - private_xform - a place for PMDs to hold private data derived from a xform and used by stateless operations. - stream - a place for PMDs to hold private data derived from a xform and also maintain state and history data. For stateful flows. Signed-off-by: Fiona Trahe Signed-off-by: Pablo de Lara Signed-off-by: Shally Verma Signed-off-by: Ashish Gupta Changes in v6: - Add programmer's guide section for library - Change macro for no hash - Fix compilation in patch 4 - Remove next pointer in xform - Add queue pair release in device closure Changes in v5: - Make rte_compressdev_is_valid_dev, rte_compressdev_pmd_get_dev functions internal - Add rte_comp.h to API doc - Remove unused rte_comp_op_get_user_data, rte_comp_op_get_user_data_size functions - Fix comment - Move private data size arg out of PMD init params structure - Split out of space error status into two - Make rte_comp_op_alloc, rte_comp_op_bulk_alloc, rte_comp_free, rte_compressdev_enqueue_burst and rte_compressdev_dequeue_burst functions not inline. - Move rte_compressdev and rte_compressdev_data structures out of rte_compressdev.h, making them internal to the API - Remove unneeded driver registration (used in cryptodev, for multi PMD sessions) - Remove unneeded includes - rte_comp_op_bulk_alloc returns now 0 on success, instead of the number of allocated operations - Move internals macros to rte_compressdev_internals.h, to avoid exposing them unnecessarily Changes in v4: - Fix build (missing ";") - Change order in common_base, so compression is placed after security Changes in v3: - Remove rte_comp_op_ctod helper functions - Remove param_range_check macro - Rename from phys_addr to iova_addr - Remove rte_comp_algo_strings - Rename rte_compressdev_pmd_is_valid_dev to rte_compressdev_is_valid_dev - Remove feature flags from compressdev - Support hash operations - Add shareable priv xform in feature flags, instead of returnin it on xform creation - Allow max number of queue pairs to be 0, meaning that there is no limit. - Add invalid configuration checks - Add capability helper functions Changes in v2: - Add stream APIs - Remove session - Add SHAREABLE / NON_SHAREABLE private_xform types - Add algo enum 'UNSPECIFIED' to fix warning in capabilities - Change one remaining log to use dynamic logging. - Add rte_cache_aligned keyword to op - Rename enums with better names _ALGO, __CHECKSUM, _HUFFMAN_ - Use const keyword when passing xform - Remove qp_count fn from dev_ops as never used - Remove max_nb_queue-pairs from compressdev_init_param as never used - Clarify device configure and start sequence - Replace OUT_OF_SPACE with OUT_OF_SPACE_RECOVERABLE and TERMINATED and clarified usage. - Add stream and private_xform sizes to device config for use in mempool creation - Add capability helper fn - Use Base2 log value for window size on xforms - Add Meson build - Update MAINTAINERS - Update Map file - Change order in doxy file - Update Release note Ashish Gupta (1): doc: add compressdev library guide Fiona Trahe (12): compressdev: add basic device management compressdev: add queue pair management compressdev: add compression specific data compressdev: add enqueue/dequeue functions compressdev: add operation management compressdev: support stateless operations compressdev: support stateful operations compressdev: add device feature flags compressdev: add compression service feature flags compressdev: add device stats compressdev: add device capabilities compressdev: get device id from name Shally Verma (1): compressdev: support hash operations MAINTAINERS | 8 + config/common_base | 6 + config/rte_config.h | 3 + doc/api/doxy-api-index.md | 2 + doc/api/doxy-api.conf | 1 + doc/guides/prog_guide/compressdev_lib.rst | 623 +++++++++++++++++ doc/guides/prog_guide/img/stateful-op.svg | 116 +++ doc/guides/prog_guide/img/stateless-op-shared.svg | 124 ++++ doc/guides/prog_guide/img/stateless-op.svg | 140 ++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/rel_notes/release_18_05.rst | 6 + lib/Makefile | 3 + lib/librte_compressdev/Makefile | 31 + lib/librte_compressdev/meson.build | 12 + lib/librte_compressdev/rte_comp.c | 207 ++++++ lib/librte_compressdev/rte_comp.h | 458 ++++++++++++ lib/librte_compressdev/rte_compressdev.c | 775 +++++++++++++++++++++ lib/librte_compressdev/rte_compressdev.h | 539 ++++++++++++++ lib/librte_compressdev/rte_compressdev_internal.h | 117 ++++ lib/librte_compressdev/rte_compressdev_pmd.c | 160 +++++ lib/librte_compressdev/rte_compressdev_pmd.h | 390 +++++++++++ lib/librte_compressdev/rte_compressdev_version.map | 42 ++ lib/meson.build | 2 +- mk/rte.app.mk | 1 + 24 files changed, 3766 insertions(+), 1 deletion(-) create mode 100644 doc/guides/prog_guide/compressdev_lib.rst create mode 100644 doc/guides/prog_guide/img/stateful-op.svg create mode 100644 doc/guides/prog_guide/img/stateless-op-shared.svg create mode 100644 doc/guides/prog_guide/img/stateless-op.svg create mode 100644 lib/librte_compressdev/Makefile create mode 100644 lib/librte_compressdev/meson.build create mode 100644 lib/librte_compressdev/rte_comp.c create mode 100644 lib/librte_compressdev/rte_comp.h create mode 100644 lib/librte_compressdev/rte_compressdev.c create mode 100644 lib/librte_compressdev/rte_compressdev.h create mode 100644 lib/librte_compressdev/rte_compressdev_internal.h create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.c create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.h create mode 100644 lib/librte_compressdev/rte_compressdev_version.map -- 2.14.3