From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 8652C2BD3 for ; Tue, 27 Mar 2018 18:13:33 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Mar 2018 09:04:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,367,1517904000"; d="scan'208";a="45728922" Received: from sivswdev01.ir.intel.com (HELO localhost.localdomain) ([10.237.217.45]) by orsmga002.jf.intel.com with ESMTP; 27 Mar 2018 09:04:47 -0700 From: Fiona Trahe To: dev@dpdk.org Cc: pablo.de.lara.guarch@intel.com, fiona.trahe@intel.com, Shally.Verma@cavium.com, ahmed.mansour@nxp.com, Ashish.Gupta@cavium.com, Shally Verma , Ashish Gupta Date: Tue, 27 Mar 2018 17:04:29 +0100 Message-Id: <1522166672-19415-1-git-send-email-fiona.trahe@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1517595924-25963-1-git-send-email-fiona.trahe@intel.com> References: <1517595924-25963-1-git-send-email-fiona.trahe@intel.com> Subject: [dpdk-dev] [PATCH v2 0/3] implement compression API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Mar 2018 16:13:34 -0000 With the vast amounts of data being transported around networks and stored in storage systems, reducing data size is becoming ever more important. There are both software libraries and hardware devices available that provide compression, but no common API. Such an API is proposed in this commit, which supports the following features: - Deflate Algorithm (https://tools.ietf.org/html/rfc1951) - LZS algorithm (https://tools.ietf.org/html/rfc2395) - Static and Dynamic Huffman encoding. - Compression levels - Checksum generation - Asynchronous burst API - private_xform - a place for PMDs to hold private data derived from a xform and used by stateless operations. - stream - a place for PMDs to hold private data derived from a xform and also maintain state and history data. For stateful flows. Signed-off-by: Fiona Trahe Signed-off-by: Pablo de Lara pablo.de.lara.guarch@intel.com Signed-off-by: Shally Verma Signed-off-by: Ashish Gupta Opens: - creation of private_xform and stream mempools. In v2, config added to give PMD the size params needed to create mempools. Still open as to whether the pools should be created by application, in API layer or by PMDs. Expect to resolve in a v3. - alternative to mbufs for passing data between application and accelerator. A lot of varied opinions on this, expect to address in a future release. - addition of hash feature - to be added by Cavium - addition of feature to add capability to PMD to allow more than one inflight operation from a stateful stream - to be added by NXP Changes in v2: - Add stream APIs - Remove session - Add SHAREABLE / NON_SHAREABLE private_xform types - Add algo enum 'UNSPECIFIED' to fix warning in capabilities - Change one remaining log to use dynamic logging. - Add rte_cache_aligned keyword to op - Rename enums with better names _ALGO, __CHECKSUM, _HUFFMAN_ - Use const keyword when passing xform - Remove qp_count fn from dev_ops as never used - Remove max_nb_queue-pairs from compressdev_init_param as never used - Clarify device configure and start sequence - Replace OUT_OF_SPACE with OUT_OF_SPACE_RECOVERABLE and TERMINATED and clarified usage. - Add stream and private_xform sizes to device config for use in mempool creation - Add capability helper fn - Use Base2 log value for window size on xforms - Add Meson build - Update MAINTAINERS - Update Map file - Change order in doxy file - Update Release note Fiona Trahe (3): compressdev: add structs and enum for compression service compressdev: implement API doc: update doxy and release note for compressdev MAINTAINERS | 7 + config/common_base | 6 + config/rte_config.h | 3 + doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf | 1 + doc/guides/rel_notes/release_18_05.rst | 6 + lib/Makefile | 3 + lib/librte_compressdev/Makefile | 29 + lib/librte_compressdev/meson.build | 9 + lib/librte_compressdev/rte_comp.h | 528 ++++++++++++ lib/librte_compressdev/rte_compressdev.c | 904 +++++++++++++++++++++ lib/librte_compressdev/rte_compressdev.h | 727 +++++++++++++++++ lib/librte_compressdev/rte_compressdev_pmd.c | 156 ++++ lib/librte_compressdev/rte_compressdev_pmd.h | 450 ++++++++++ lib/librte_compressdev/rte_compressdev_version.map | 43 + lib/meson.build | 2 +- mk/rte.app.mk | 1 + 17 files changed, 2875 insertions(+), 1 deletion(-) create mode 100644 lib/librte_compressdev/Makefile create mode 100644 lib/librte_compressdev/meson.build create mode 100644 lib/librte_compressdev/rte_comp.h create mode 100644 lib/librte_compressdev/rte_compressdev.c create mode 100644 lib/librte_compressdev/rte_compressdev.h create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.c create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.h create mode 100644 lib/librte_compressdev/rte_compressdev_version.map -- 2.7.4