[dpdk-dev] [PATCH v3 0/5] compress: add ZLIB compression PMD

[dpdk-dev] [PATCH v3 0/5] compress: add ZLIB compression PMD

[Shally Verma]

This patch series add software zlib library (http://zlib.net/)
based compression PMD in DPDK compress drivers.

Application must need to install zlib prior to enabling
this PMD to avail compression/decompression services.
Currently driver only tested for deflate, stateless
compression and decompression.

Changes in v3:
- added comp_feature_flags to compressdev_capabilities
- removed stream_create as stateful not supported
- changed compression strategy to DEFAULT_STRATEGY
- Simplified core processing logic
- fix doc build error
- other v2 feedbacks

TBDs
- No change to current qp enqueue_err stats update.
- PMD increment enqd_err stats, if it fails to push op
  into completion queue. This logic to be revisited based on
  further discussion

Changes in v2:
- removed unused variables
- corrected capability to reflect current support
- add lookup for internally maintained mempool during device_configure
- optimized core compression/decompression logic in enq/deq APIs
- updated documentation with correct feature support

v1 includes:
- build changes to build zlib PMD
- zlib PMD implementation
- zlib PMD documentation
- meson build support

This patchset is dependent upon compressdev API.


Ashish Gupta (2):
  compress/zlib: add ZLIB PMD
  compress/zlib: add device PMD ops

Shally Verma (1):
  doc: add ZLIB PMD guide

Sunila Sahu (2):
  compress/zlib: create private xform
  compress/zlib: support burst enqueue/dequeue

 MAINTAINERS                                    |   5 +
 config/common_base                             |   5 +
 doc/guides/compressdevs/features/zlib.ini      |  11 +
 doc/guides/compressdevs/index.rst              |   1 +
 doc/guides/compressdevs/zlib.rst               |  69 ++++
 drivers/compress/Makefile                      |   1 +
 drivers/compress/meson.build                   |   2 +-
 drivers/compress/zlib/Makefile                 |  29 ++
 drivers/compress/zlib/meson.build              |  14 +
 drivers/compress/zlib/rte_pmd_zlib_version.map |   3 +
 drivers/compress/zlib/zlib_pmd.c               | 425 +++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_ops.c           | 308 ++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_private.h       |  71 +++++
 mk/rte.app.mk                                  |   2 +
 14 files changed, 945 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/compressdevs/features/zlib.ini
 create mode 100644 doc/guides/compressdevs/zlib.rst
 create mode 100644 drivers/compress/zlib/Makefile
 create mode 100644 drivers/compress/zlib/meson.build
 create mode 100644 drivers/compress/zlib/rte_pmd_zlib_version.map
 create mode 100644 drivers/compress/zlib/zlib_pmd.c
 create mode 100644 drivers/compress/zlib/zlib_pmd_ops.c
 create mode 100644 drivers/compress/zlib/zlib_pmd_private.h

-- 
2.9.5

[dpdk-dev] [PATCH v3 1/5] compress/zlib: add ZLIB PMD

[Shally Verma]

From: Ashish Gupta <ashish.gupta@caviumnetworks.com>

Add initial PMD setup routines in compressdev
framework. ZLIB PMD appears as virtual compression
device. User would need to install zlib prior to
enabling this PMD.

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 MAINTAINERS                                    |  3 +
 config/common_base                             |  5 ++
 drivers/compress/Makefile                      |  1 +
 drivers/compress/meson.build                   |  2 +-
 drivers/compress/zlib/Makefile                 | 28 +++++++++
 drivers/compress/zlib/meson.build              | 14 +++++
 drivers/compress/zlib/rte_pmd_zlib_version.map |  3 +
 drivers/compress/zlib/zlib_pmd.c               | 78 ++++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_private.h       | 32 +++++++++++
 mk/rte.app.mk                                  |  2 +
 10 files changed, 167 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7b2414d..ca27c6f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -872,6 +872,9 @@ M: Fiona Trahe <fiona.trahe@intel.com>
 F: drivers/compress/qat/
 F: drivers/common/qat/
 
+ZLIB
+M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
+F: drivers/compress/zlib/
 
 Eventdev Drivers
 ----------------
diff --git a/config/common_base b/config/common_base
index a061c21..d9b238c 100644
--- a/config/common_base
+++ b/config/common_base
@@ -582,6 +582,11 @@ CONFIG_RTE_COMPRESSDEV_TEST=n
 CONFIG_RTE_LIBRTE_PMD_ISAL=n
 
 #
+# Compile PMD for ZLIB compression device
+#
+CONFIG_RTE_LIBRTE_PMD_ZLIB=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
diff --git a/drivers/compress/Makefile b/drivers/compress/Makefile
index 592497f..1f159a5 100644
--- a/drivers/compress/Makefile
+++ b/drivers/compress/Makefile
@@ -4,5 +4,6 @@
 include $(RTE_SDK)/mk/rte.vars.mk
 
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib
 
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/compress/meson.build b/drivers/compress/meson.build
index 2352ad5..d2ca8fc 100644
--- a/drivers/compress/meson.build
+++ b/drivers/compress/meson.build
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-drivers = ['isal', 'qat']
+drivers = ['isal', 'qat', 'zlib']
 
 std_deps = ['compressdev'] # compressdev pulls in all other needed deps
 config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
diff --git a/drivers/compress/zlib/Makefile b/drivers/compress/zlib/Makefile
new file mode 100644
index 0000000..bd322c9
--- /dev/null
+++ b/drivers/compress/zlib/Makefile
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Cavium Networks
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_zlib.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_zlib_version.map
+
+# external library dependencies
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring -lz
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib_pmd.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/compress/zlib/meson.build b/drivers/compress/zlib/meson.build
new file mode 100644
index 0000000..3f0a77b
--- /dev/null
+++ b/drivers/compress/zlib/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Cavium Networks
+
+dep = dependency('zlib', required: false)
+if not dep.found()
+	build = false
+endif
+
+deps += 'bus_vdev'
+sources = files('zlib_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lz'
+
+allow_experimental_apis = true
diff --git a/drivers/compress/zlib/rte_pmd_zlib_version.map b/drivers/compress/zlib/rte_pmd_zlib_version.map
new file mode 100644
index 0000000..ad6e191
--- /dev/null
+++ b/drivers/compress/zlib/rte_pmd_zlib_version.map
@@ -0,0 +1,3 @@
+DPDK_18.08 {
+	local: *;
+};
diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
new file mode 100644
index 0000000..39d3628
--- /dev/null
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Cavium Networks
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_common.h>
+
+#include "zlib_pmd_private.h"
+
+static int
+zlib_create(const char *name,
+		struct rte_vdev_device *vdev,
+		struct rte_compressdev_pmd_init_params *init_params)
+{
+	struct rte_compressdev *dev;
+
+	dev = rte_compressdev_pmd_create(name, &vdev->device,
+			sizeof(struct zlib_private), init_params);
+	if (dev == NULL) {
+		ZLIB_PMD_ERR("driver %s: create failed", init_params->name);
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int
+zlib_probe(struct rte_vdev_device *vdev)
+{
+	struct rte_compressdev_pmd_init_params init_params = {
+		"",
+		rte_socket_id()
+	};
+	const char *name;
+	const char *input_args;
+
+	name = rte_vdev_device_name(vdev);
+
+	if (name == NULL)
+		return -EINVAL;
+	input_args = rte_vdev_device_args(vdev);
+	rte_compressdev_pmd_parse_input_args(&init_params, input_args);
+
+	return zlib_create(name, vdev, &init_params);
+}
+
+static int
+zlib_remove(struct rte_vdev_device *vdev)
+{
+	struct rte_compressdev *compressdev;
+	const char *name;
+
+	name = rte_vdev_device_name(vdev);
+	if (name == NULL)
+		return -EINVAL;
+
+	compressdev = rte_compressdev_pmd_get_named_dev(name);
+	if (compressdev == NULL)
+		return -ENODEV;
+
+	return rte_compressdev_pmd_destroy(compressdev);
+}
+
+static struct rte_vdev_driver zlib_pmd_drv = {
+	.probe = zlib_probe,
+	.remove = zlib_remove
+};
+
+RTE_PMD_REGISTER_VDEV(COMPRESSDEV_NAME_ZLIB_PMD, zlib_pmd_drv);
+RTE_INIT(zlib_init_log);
+
+static void
+zlib_init_log(void)
+{
+	zlib_logtype_driver = rte_log_register("pmd.compress.zlib");
+	if (zlib_logtype_driver >= 0)
+		rte_log_set_level(zlib_logtype_driver, RTE_LOG_INFO);
+}
diff --git a/drivers/compress/zlib/zlib_pmd_private.h b/drivers/compress/zlib/zlib_pmd_private.h
new file mode 100644
index 0000000..d26a740
--- /dev/null
+++ b/drivers/compress/zlib/zlib_pmd_private.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Cavium Networks
+ */
+
+#ifndef _RTE_ZLIB_PMD_PRIVATE_H_
+#define _RTE_ZLIB_PMD_PRIVATE_H_
+
+#include <zlib.h>
+#include <rte_compressdev.h>
+#include <rte_compressdev_pmd.h>
+
+#define COMPRESSDEV_NAME_ZLIB_PMD	compress_zlib
+/**< ZLIB PMD device name */
+
+#define DEF_MEM_LEVEL			8
+
+int zlib_logtype_driver;
+#define ZLIB_PMD_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, zlib_logtype_driver, "%s(): "fmt "\n", \
+			__func__, ##args)
+
+#define ZLIB_PMD_INFO(fmt, args...) \
+	ZLIB_PMD_LOG(INFO, fmt, ## args)
+#define ZLIB_PMD_ERR(fmt, args...) \
+	ZLIB_PMD_LOG(ERR, fmt, ## args)
+#define ZLIB_PMD_WARN(fmt, args...) \
+	ZLIB_PMD_LOG(WARNING, fmt, ## args)
+
+struct zlib_private {
+};
+
+#endif /* _RTE_ZLIB_PMD_PRIVATE_H_ */
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 1590838..5b7c684 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -228,6 +228,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal
 ifeq ($(CONFIG_RTE_LIBRTE_PMD_QAT_SYM),n)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)  += -lrte_pmd_qat
 endif # CONFIG_RTE_LIBRTE_PMD_QAT_SYM
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += -lrte_pmd_zlib
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += -lz
 endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
 
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
-- 
2.9.5

[dpdk-dev] [PATCH v3 2/5] compress/zlib: add device PMD ops

[Shally Verma]

From: Ashish Gupta <ashish.gupta@caviumnetworks.com>

Implement device configure and queue pair
setup PMD ops

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 drivers/compress/zlib/Makefile           |   1 +
 drivers/compress/zlib/meson.build        |   2 +-
 drivers/compress/zlib/zlib_pmd.c         |   2 +
 drivers/compress/zlib/zlib_pmd_ops.c     | 238 +++++++++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_private.h |  35 +++++
 5 files changed, 277 insertions(+), 1 deletion(-)

diff --git a/drivers/compress/zlib/Makefile b/drivers/compress/zlib/Makefile
index bd322c9..5cf8de6 100644
--- a/drivers/compress/zlib/Makefile
+++ b/drivers/compress/zlib/Makefile
@@ -24,5 +24,6 @@ LDLIBS += -lrte_bus_vdev
 
 # library source files
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib_pmd_ops.c
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/compress/zlib/meson.build b/drivers/compress/zlib/meson.build
index 3f0a77b..7748de2 100644
--- a/drivers/compress/zlib/meson.build
+++ b/drivers/compress/zlib/meson.build
@@ -7,7 +7,7 @@ if not dep.found()
 endif
 
 deps += 'bus_vdev'
-sources = files('zlib_pmd.c')
+sources = files('zlib_pmd.c', 'zlib_pmd_ops.c')
 ext_deps += dep
 pkgconfig_extra_libs += '-lz'
 
diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
index 39d3628..9363808 100644
--- a/drivers/compress/zlib/zlib_pmd.c
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -21,6 +21,8 @@ zlib_create(const char *name,
 		return -ENODEV;
 	}
 
+	dev->dev_ops = rte_zlib_pmd_ops;
+
 	return 0;
 }
 
diff --git a/drivers/compress/zlib/zlib_pmd_ops.c b/drivers/compress/zlib/zlib_pmd_ops.c
new file mode 100644
index 0000000..645c5b1
--- /dev/null
+++ b/drivers/compress/zlib/zlib_pmd_ops.c
@@ -0,0 +1,238 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Cavium Networks
+ */
+
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+
+#include "zlib_pmd_private.h"
+
+static const struct rte_compressdev_capabilities zlib_pmd_capabilities[] = {
+	{   /* Deflate */
+		.algo = RTE_COMP_ALGO_DEFLATE,
+		.comp_feature_flags = (RTE_COMP_FF_NONCOMPRESSED_BLOCKS |
+					RTE_COMP_FF_HUFFMAN_FIXED |
+					RTE_COMP_FF_HUFFMAN_DYNAMIC |
+					RTE_COMP_FF_OOP_SGL_IN_SGL_OUT),
+		.window_size = {
+			.min = 8,
+			.max = 15,
+			.increment = 1
+		},
+	},
+
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+
+};
+
+/** Configure device */
+static int
+zlib_pmd_config(struct rte_compressdev *dev,
+		struct rte_compressdev_config *config)
+{
+	struct rte_mempool *mp;
+	char mp_name[RTE_MEMPOOL_NAMESIZE];
+	struct zlib_private *internals = dev->data->dev_private;
+
+	snprintf(mp_name, RTE_MEMPOOL_NAMESIZE,
+			"stream_mp_%u", dev->data->dev_id);
+	mp = internals->mp;
+	if (mp == NULL) {
+		mp = rte_mempool_create(mp_name,
+				config->max_nb_priv_xforms +
+				config->max_nb_streams,
+				sizeof(struct zlib_priv_xform),
+				0, 0, NULL, NULL, NULL,
+				NULL, config->socket_id,
+				0);
+		if (mp == NULL) {
+			ZLIB_PMD_ERR("Cannot create private xform pool on "
+			"socket %d\n", config->socket_id);
+			return -ENOMEM;
+		}
+		internals->mp = mp;
+	}
+	return 0;
+}
+
+/** Start device */
+static int
+zlib_pmd_start(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Stop device */
+static void
+zlib_pmd_stop(__rte_unused struct rte_compressdev *dev)
+{
+}
+
+/** Close device */
+static int
+zlib_pmd_close(struct rte_compressdev *dev)
+{
+	struct zlib_private *internals = dev->data->dev_private;
+	rte_mempool_free(internals->mp);
+	internals->mp = NULL;
+	return 0;
+}
+
+/** Get device statistics */
+static void
+zlib_pmd_stats_get(struct rte_compressdev *dev,
+		struct rte_compressdev_stats *stats)
+{
+	int qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct zlib_qp *qp = dev->data->queue_pairs[qp_id];
+
+		stats->enqueued_count += qp->qp_stats.enqueued_count;
+		stats->dequeued_count += qp->qp_stats.dequeued_count;
+
+		stats->enqueue_err_count += qp->qp_stats.enqueue_err_count;
+		stats->dequeue_err_count += qp->qp_stats.dequeue_err_count;
+	}
+}
+
+/** Reset device statistics */
+static void
+zlib_pmd_stats_reset(struct rte_compressdev *dev)
+{
+	int qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct zlib_qp *qp = dev->data->queue_pairs[qp_id];
+
+		memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	}
+}
+
+/** Get device info */
+static void
+zlib_pmd_info_get(struct rte_compressdev *dev,
+		struct rte_compressdev_info *dev_info)
+{
+	if (dev_info != NULL) {
+		dev_info->driver_name = dev->device->name;
+		dev_info->feature_flags = dev->feature_flags;
+		dev_info->capabilities = zlib_pmd_capabilities;
+	}
+}
+
+/** Release queue pair */
+static int
+zlib_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
+{
+	struct zlib_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp != NULL) {
+		rte_ring_free(qp->processed_pkts);
+		rte_free(qp);
+		dev->data->queue_pairs[qp_id] = NULL;
+	}
+	return 0;
+}
+
+/** set a unique name for the queue pair based on its name, dev_id and qp_id */
+static int
+zlib_pmd_qp_set_unique_name(struct rte_compressdev *dev,
+		struct zlib_qp *qp)
+{
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+				"zlib_pmd_%u_qp_%u",
+				dev->data->dev_id, qp->id);
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */
+static struct rte_ring *
+zlib_pmd_qp_create_processed_pkts_ring(struct zlib_qp *qp,
+		unsigned int ring_size, int socket_id)
+{
+	struct rte_ring *r = qp->processed_pkts;
+
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {
+			ZLIB_PMD_INFO("Reusing existing ring %s for processed"
+					" packets", qp->name);
+			return r;
+		}
+
+		ZLIB_PMD_ERR("Unable to reuse existing ring %s for processed"
+				" packets", qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+						RING_F_EXACT_SZ);
+}
+
+/** Setup a queue pair */
+static int
+zlib_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
+		uint32_t max_inflight_ops, int socket_id)
+{
+	struct zlib_qp *qp = NULL;
+
+	/* Free memory prior to re-allocation if needed. */
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		zlib_pmd_qp_release(dev, qp_id);
+
+	/* Allocate the queue pair data structure. */
+	qp = rte_zmalloc_socket("ZLIB PMD Queue Pair", sizeof(*qp),
+					RTE_CACHE_LINE_SIZE, socket_id);
+	if (qp == NULL)
+		return (-ENOMEM);
+
+	qp->id = qp_id;
+	dev->data->queue_pairs[qp_id] = qp;
+
+	if (zlib_pmd_qp_set_unique_name(dev, qp))
+		goto qp_setup_cleanup;
+
+	qp->processed_pkts = zlib_pmd_qp_create_processed_pkts_ring(qp,
+			max_inflight_ops, socket_id);
+	if (qp->processed_pkts == NULL)
+		goto qp_setup_cleanup;
+
+	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	return 0;
+
+qp_setup_cleanup:
+	if (qp) {
+		rte_free(qp);
+		qp = NULL;
+	}
+	return -1;
+}
+
+struct rte_compressdev_ops zlib_pmd_ops = {
+		.dev_configure		= zlib_pmd_config,
+		.dev_start		= zlib_pmd_start,
+		.dev_stop		= zlib_pmd_stop,
+		.dev_close		= zlib_pmd_close,
+
+		.stats_get		= zlib_pmd_stats_get,
+		.stats_reset		= zlib_pmd_stats_reset,
+
+		.dev_infos_get		= zlib_pmd_info_get,
+
+		.queue_pair_setup	= zlib_pmd_qp_setup,
+		.queue_pair_release	= zlib_pmd_qp_release,
+
+		.private_xform_create	= NULL,
+		.private_xform_free	= NULL,
+
+		.stream_create	= NULL,
+		.stream_free	= NULL
+};
+
+struct rte_compressdev_ops *rte_zlib_pmd_ops = &zlib_pmd_ops;
diff --git a/drivers/compress/zlib/zlib_pmd_private.h b/drivers/compress/zlib/zlib_pmd_private.h
index d26a740..0e391a4 100644
--- a/drivers/compress/zlib/zlib_pmd_private.h
+++ b/drivers/compress/zlib/zlib_pmd_private.h
@@ -27,6 +27,41 @@ int zlib_logtype_driver;
 	ZLIB_PMD_LOG(WARNING, fmt, ## args)
 
 struct zlib_private {
+	struct rte_mempool *mp;
 };
 
+struct zlib_qp {
+	struct rte_ring *processed_pkts;
+	/**< Ring for placing process packets */
+	struct rte_compressdev_stats qp_stats;
+	/**< Queue pair statistics */
+	uint16_t id;
+	/**< Queue Pair Identifier */
+	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/**< Unique Queue Pair Name */
+} __rte_cache_aligned;
+
+/* Algorithm handler function prototype */
+typedef void (*comp_func_t)(struct rte_comp_op *op, z_stream *strm);
+
+typedef int (*comp_free_t)(z_stream *strm);
+
+/** ZLIB Stream structure */
+struct zlib_stream {
+	z_stream strm;
+	/**< zlib stream structure */
+	comp_func_t comp;
+	/**< Operation (compression/decompression) */
+	comp_free_t free;
+	/**< Free Operation (compression/decompression) */
+} __rte_cache_aligned;
+
+/** ZLIB private xform structure */
+struct zlib_priv_xform {
+	struct zlib_stream stream;
+} __rte_cache_aligned;
+
+/** Device specific operations function pointer structure */
+extern struct rte_compressdev_ops *rte_zlib_pmd_ops;
+
 #endif /* _RTE_ZLIB_PMD_PRIVATE_H_ */
-- 
2.9.5

[dpdk-dev] [PATCH v3 3/5] compress/zlib: create private xform

[Shally Verma]

From: Sunila Sahu <ssahu@caviumnetworks.com>

Create non-shareable private xform for stateless
operation processing

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 drivers/compress/zlib/zlib_pmd.c         | 93 ++++++++++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_ops.c     | 74 ++++++++++++++++++++++++-
 drivers/compress/zlib/zlib_pmd_private.h |  4 ++
 3 files changed, 169 insertions(+), 2 deletions(-)

diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
index 9363808..9a464ee 100644
--- a/drivers/compress/zlib/zlib_pmd.c
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -7,6 +7,99 @@
 
 #include "zlib_pmd_private.h"
 
+/** Parse comp xform and set private xform/stream parameters */
+int
+zlib_set_stream_parameters(const struct rte_comp_xform *xform,
+		struct zlib_stream *stream)
+{
+	int strategy, level, wbits;
+	z_stream *strm = &stream->strm;
+
+	/* allocate deflate state */
+	strm->zalloc = Z_NULL;
+	strm->zfree = Z_NULL;
+	strm->opaque = Z_NULL;
+
+	switch (xform->type) {
+	case RTE_COMP_COMPRESS:
+		/** Compression window bits */
+		switch (xform->compress.algo) {
+		case RTE_COMP_ALGO_DEFLATE:
+			wbits = -(xform->compress.window_size);
+			break;
+		default:
+			ZLIB_PMD_ERR("Compression algorithm not supported\n");
+			return -1;
+		}
+		/** Compression Level */
+		switch (xform->compress.level) {
+		case RTE_COMP_LEVEL_PMD_DEFAULT:
+			level = Z_DEFAULT_COMPRESSION;
+			break;
+		case RTE_COMP_LEVEL_NONE:
+			level = Z_NO_COMPRESSION;
+			break;
+		case RTE_COMP_LEVEL_MIN:
+			level = Z_BEST_SPEED;
+			break;
+		case RTE_COMP_LEVEL_MAX:
+			level = Z_BEST_COMPRESSION;
+			break;
+		default:
+			level = xform->compress.level;
+			if (level < RTE_COMP_LEVEL_MIN ||
+					level > RTE_COMP_LEVEL_MAX) {
+				ZLIB_PMD_ERR("Compression level %d "
+						"not supported\n",
+						level);
+				return -1;
+			}
+		}
+		/** Compression strategy */
+		switch (xform->compress.deflate.huffman) {
+		case RTE_COMP_HUFFMAN_DEFAULT:
+			strategy = Z_DEFAULT_STRATEGY;
+			break;
+		case RTE_COMP_HUFFMAN_FIXED:
+			strategy = Z_FIXED;
+			break;
+		case RTE_COMP_HUFFMAN_DYNAMIC:
+			strategy = Z_DEFAULT_STRATEGY;
+			break;
+		default:
+			ZLIB_PMD_ERR("Compression strategy not supported\n");
+			return -1;
+		}
+		if (deflateInit2(strm, level,
+					Z_DEFLATED, wbits,
+					DEF_MEM_LEVEL, strategy) != Z_OK) {
+			ZLIB_PMD_ERR("Deflate init failed\n");
+			return -1;
+		}
+	break;
+
+	case RTE_COMP_DECOMPRESS:
+		/** window bits */
+		switch (xform->decompress.algo) {
+		case RTE_COMP_ALGO_DEFLATE:
+			wbits = -(xform->decompress.window_size);
+			break;
+		default:
+			ZLIB_PMD_ERR("Compression algorithm not supported\n");
+			return -1;
+		}
+
+		if (inflateInit2(strm, wbits) != Z_OK) {
+			ZLIB_PMD_ERR("Inflate init failed\n");
+			return -1;
+		}
+		break;
+	default:
+		return -1;
+	}
+	return 0;
+}
+
 static int
 zlib_create(const char *name,
 		struct rte_vdev_device *vdev,
diff --git a/drivers/compress/zlib/zlib_pmd_ops.c b/drivers/compress/zlib/zlib_pmd_ops.c
index 645c5b1..2fb6a85 100644
--- a/drivers/compress/zlib/zlib_pmd_ops.c
+++ b/drivers/compress/zlib/zlib_pmd_ops.c
@@ -214,6 +214,76 @@ zlib_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 	return -1;
 }
 
+/** Configure stream */
+static int
+zlib_pmd_stream_create(struct rte_compressdev *dev,
+		const struct rte_comp_xform *xform,
+		void **zstream)
+{
+	int ret = 0;
+	struct zlib_stream *stream;
+	struct zlib_private *internals = dev->data->dev_private;
+
+	if (xform == NULL) {
+		ZLIB_PMD_ERR("invalid xform struct");
+		return -EINVAL;
+	}
+
+	if (rte_mempool_get(internals->mp, zstream)) {
+		ZLIB_PMD_ERR("Couldn't get object from session mempool");
+		return -ENOMEM;
+	}
+	stream = *((struct zlib_stream **)zstream);
+
+	ret = zlib_set_stream_parameters(xform, stream);
+
+	if (ret < 0) {
+		ZLIB_PMD_ERR("failed configure session parameters");
+
+		memset(stream, 0, sizeof(struct zlib_stream));
+		/* Return session to mempool */
+		rte_mempool_put(internals->mp, stream);
+		return ret;
+	}
+
+	return 0;
+}
+
+/** Configure private xform */
+static int
+zlib_pmd_private_xform_create(struct rte_compressdev *dev,
+		const struct rte_comp_xform *xform,
+		void **private_xform)
+{
+	return zlib_pmd_stream_create(dev, xform, private_xform);
+}
+
+/** Clear the memory of stream so it doesn't leave key material behind */
+static int
+zlib_pmd_stream_free(__rte_unused struct rte_compressdev *dev,
+		void *zstream)
+{
+	struct zlib_stream *stream = (struct zlib_stream *)zstream;
+	if (!stream)
+		return -EINVAL;
+
+	stream->free(&stream->strm);
+	/* Zero out the whole structure */
+	memset(stream, 0, sizeof(struct zlib_stream));
+	struct rte_mempool *mp = rte_mempool_from_obj(stream);
+	rte_mempool_put(mp, stream);
+
+	return 0;
+}
+
+/** Clear the memory of stream so it doesn't leave key material behind */
+static int
+zlib_pmd_private_xform_free(struct rte_compressdev *dev,
+		void *private_xform)
+{
+	return zlib_pmd_stream_free(dev, private_xform);
+}
+
 struct rte_compressdev_ops zlib_pmd_ops = {
 		.dev_configure		= zlib_pmd_config,
 		.dev_start		= zlib_pmd_start,
@@ -228,8 +298,8 @@ struct rte_compressdev_ops zlib_pmd_ops = {
 		.queue_pair_setup	= zlib_pmd_qp_setup,
 		.queue_pair_release	= zlib_pmd_qp_release,
 
-		.private_xform_create	= NULL,
-		.private_xform_free	= NULL,
+		.private_xform_create	= zlib_pmd_private_xform_create,
+		.private_xform_free	= zlib_pmd_private_xform_free,
 
 		.stream_create	= NULL,
 		.stream_free	= NULL
diff --git a/drivers/compress/zlib/zlib_pmd_private.h b/drivers/compress/zlib/zlib_pmd_private.h
index 0e391a4..2c6e83d 100644
--- a/drivers/compress/zlib/zlib_pmd_private.h
+++ b/drivers/compress/zlib/zlib_pmd_private.h
@@ -61,6 +61,10 @@ struct zlib_priv_xform {
 	struct zlib_stream stream;
 } __rte_cache_aligned;
 
+int
+zlib_set_stream_parameters(const struct rte_comp_xform *xform,
+		struct zlib_stream *stream);
+
 /** Device specific operations function pointer structure */
 extern struct rte_compressdev_ops *rte_zlib_pmd_ops;
 
-- 
2.9.5

[dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[Shally Verma]

From: Sunila Sahu <ssahu@caviumnetworks.com>

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 drivers/compress/zlib/zlib_pmd.c | 254 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 253 insertions(+), 1 deletion(-)

diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
index 9a464ee..44baa67 100644
--- a/drivers/compress/zlib/zlib_pmd.c
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -7,7 +7,213 @@
 
 #include "zlib_pmd_private.h"
 
-/** Parse comp xform and set private xform/stream parameters */
+/** Compute next mbuf in the list, assign data buffer and length,
+ *  returns 0 if mbuf is NULL
+ */
+#define COMPUTE_BUF(mbuf, data, len)		\
+		((mbuf = mbuf->next) ?		\
+		(data = rte_pktmbuf_mtod(mbuf, uint8_t *)),	\
+		(len = rte_pktmbuf_data_len(mbuf)) : 0)
+
+static void
+process_zlib_deflate(struct rte_comp_op *op, z_stream *strm)
+{
+	int ret, flush, fin_flush;
+	struct rte_mbuf *mbuf_src = op->m_src;
+	struct rte_mbuf *mbuf_dst = op->m_dst;
+
+	switch (op->flush_flag) {
+	case RTE_COMP_FLUSH_FULL:
+	case RTE_COMP_FLUSH_FINAL:
+		fin_flush = Z_FINISH;
+		break;
+	default:
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid flush value\n");
+	}
+
+	if (unlikely(!strm)) {
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid z_stream\n");
+		return;
+	}
+	/* Update z_stream with the inputs provided by application */
+	strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
+			op->src.offset);
+
+	strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
+
+	strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
+			op->dst.offset);
+
+	strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
+
+	/* Set flush value to NO_FLUSH unless it is last mbuf */
+	flush = Z_NO_FLUSH;
+	/* Initialize status to SUCCESS */
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	do {
+		/* Set flush value to Z_FINISH for last block */
+		if ((op->src.length - strm->total_in) <= strm->avail_in) {
+			strm->avail_in = (op->src.length - strm->total_in);
+			flush = fin_flush;
+		}
+		do {
+			ret = deflate(strm, flush);
+			if (unlikely(ret == Z_STREAM_ERROR)) {
+				/* error return, do not process further */
+				op->status =  RTE_COMP_OP_STATUS_ERROR;
+				goto def_end;
+			}
+			/* Break if Z_STREAM_END is encountered */
+			if (ret == Z_STREAM_END)
+				goto def_end;
+
+		/* Keep looping until input mbuf is consumed.
+		 * Exit if destination mbuf gets exhausted.
+		 */
+		} while ((strm->avail_out == 0) &&
+			COMPUTE_BUF(mbuf_dst, strm->next_out, strm->avail_out));
+
+		if (!strm->avail_out) {
+			/* there is no space for compressed output */
+			op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+			break;
+		}
+
+	/* Update source buffer to next mbuf
+	 * Exit if input buffers are fully consumed
+	 */
+	} while (COMPUTE_BUF(mbuf_src, strm->next_in, strm->avail_in));
+
+def_end:
+	/* Update op stats */
+	switch (op->status) {
+	case RTE_COMP_OP_STATUS_SUCCESS:
+		op->consumed += strm->total_in;
+	case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
+		op->produced += strm->total_out;
+		break;
+	default:
+		ZLIB_PMD_ERR("stats not updated for status:%d\n",
+				op->status);
+	}
+
+	deflateReset(strm);
+}
+
+static void
+process_zlib_inflate(struct rte_comp_op *op, z_stream *strm)
+{
+	int ret, flush;
+	struct rte_mbuf *mbuf_src = op->m_src;
+	struct rte_mbuf *mbuf_dst = op->m_dst;
+
+	if (unlikely(!strm)) {
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid z_stream\n");
+		return;
+	}
+	strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
+			op->src.offset);
+
+	strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
+
+	strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
+			op->dst.offset);
+
+	strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
+
+	/** Ignoring flush value provided from application for decompression */
+	flush = Z_NO_FLUSH;
+	/* initialize status to SUCCESS */
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	do {
+		do {
+			ret = inflate(strm, flush);
+
+			switch (ret) {
+			/* Fall-through */
+			case Z_NEED_DICT:
+				ret = Z_DATA_ERROR;
+			/* Fall-through */
+			case Z_DATA_ERROR:
+			/* Fall-through */
+			case Z_MEM_ERROR:
+			/* Fall-through */
+			case Z_STREAM_ERROR:
+				op->status = RTE_COMP_OP_STATUS_ERROR;
+			/* Fall-through */
+			case Z_STREAM_END:
+				/* no further computation needed if
+				 * Z_STREAM_END is encountered
+				 */
+				goto inf_end;
+			default:
+				/* success */
+				break;
+
+			}
+		/* Keep looping until input mbuf is consumed.
+		 * Exit if destination mbuf gets exhausted.
+		 */
+		} while ((strm->avail_out == 0) &&
+			COMPUTE_BUF(mbuf_dst, strm->next_out, strm->avail_out));
+
+		if (!strm->avail_out) {
+			/* there is no more space for decompressed output */
+			op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+			break;
+		}
+	/* Read next input buffer to be processed, exit if compressed
+	 * blocks are fully read
+	 */
+	} while (COMPUTE_BUF(mbuf_src, strm->next_in, strm->avail_in));
+
+inf_end:
+	/* Update op stats */
+	switch (op->status) {
+	case RTE_COMP_OP_STATUS_SUCCESS:
+		op->consumed += strm->total_in;
+	/* Fall-through */
+	case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
+		op->produced += strm->total_out;
+		break;
+	default:
+		ZLIB_PMD_ERR("stats not produced for status:%d\n",
+				op->status);
+	}
+
+	inflateReset(strm);
+}
+
+/** Process comp operation for mbuf */
+static inline int
+process_zlib_op(struct zlib_qp *qp, struct rte_comp_op *op)
+{
+	struct zlib_stream *stream;
+	struct zlib_priv_xform *private_xform;
+
+	if ((op->op_type == RTE_COMP_OP_STATEFUL) ||
+	    (op->src.offset > rte_pktmbuf_data_len(op->m_src)) ||
+	    (op->dst.offset > rte_pktmbuf_data_len(op->m_dst))) {
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid source or destination buffers or "
+			     "invalid Operation requested\n");
+	} else {
+		private_xform = (struct zlib_priv_xform *)op->private_xform;
+		stream = &private_xform->stream;
+		stream->comp(op, &stream->strm);
+	}
+	/* whatever is out of op, put it into completion queue with
+	 * its status
+	 */
+	return rte_ring_enqueue(qp->processed_pkts, (void *)op);
+}
+
+/** Parse comp xform and set private xform/Stream parameters */
 int
 zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 		struct zlib_stream *stream)
@@ -22,6 +228,8 @@ zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 
 	switch (xform->type) {
 	case RTE_COMP_COMPRESS:
+		stream->comp = process_zlib_deflate;
+		stream->free = deflateEnd;
 		/** Compression window bits */
 		switch (xform->compress.algo) {
 		case RTE_COMP_ALGO_DEFLATE:
@@ -79,6 +287,8 @@ zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 	break;
 
 	case RTE_COMP_DECOMPRESS:
+		stream->comp = process_zlib_inflate;
+		stream->free = inflateEnd;
 		/** window bits */
 		switch (xform->decompress.algo) {
 		case RTE_COMP_ALGO_DEFLATE:
@@ -100,6 +310,44 @@ zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 	return 0;
 }
 
+static uint16_t
+zlib_pmd_enqueue_burst(void *queue_pair,
+			struct rte_comp_op **ops, uint16_t nb_ops)
+{
+	struct zlib_qp *qp = queue_pair;
+	int ret;
+	uint16_t i;
+	uint16_t enqd = 0;
+	for (i = 0; i < nb_ops; i++) {
+		ret = process_zlib_op(qp, ops[i]);
+		if (unlikely(ret < 0)) {
+			/* increment count if failed to push to completion
+			 * queue
+			 */
+			qp->qp_stats.enqueue_err_count++;
+		} else {
+			qp->qp_stats.enqueued_count++;
+			enqd++;
+		}
+	}
+	return enqd;
+}
+
+static uint16_t
+zlib_pmd_dequeue_burst(void *queue_pair,
+			struct rte_comp_op **ops, uint16_t nb_ops)
+{
+	struct zlib_qp *qp = queue_pair;
+
+	unsigned int nb_dequeued = 0;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts,
+			(void **)ops, nb_ops, NULL);
+	qp->qp_stats.dequeued_count += nb_dequeued;
+
+	return nb_dequeued;
+}
+
 static int
 zlib_create(const char *name,
 		struct rte_vdev_device *vdev,
@@ -116,6 +364,10 @@ zlib_create(const char *name,
 
 	dev->dev_ops = rte_zlib_pmd_ops;
 
+	/* register rx/tx burst functions for data path */
+	dev->dequeue_burst = zlib_pmd_dequeue_burst;
+	dev->enqueue_burst = zlib_pmd_enqueue_burst;
+
 	return 0;
 }
 
-- 
2.9.5

[dpdk-dev] [PATCH v3 5/5] doc: add ZLIB PMD guide

[Shally Verma]

Add zlib pmd feature support and user guide with
build and run instructions

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 MAINTAINERS                               |  2 +
 doc/guides/compressdevs/features/zlib.ini | 11 +++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/zlib.rst          | 69 +++++++++++++++++++++++++++++++
 4 files changed, 83 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index ca27c6f..7e3c450 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -875,6 +875,8 @@ F: drivers/common/qat/
 ZLIB
 M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
 F: drivers/compress/zlib/
+F: doc/guides/compressdevs/zlib.rst
+F: doc/guides/compressdevs/features/zlib.ini
 
 Eventdev Drivers
 ----------------
diff --git a/doc/guides/compressdevs/features/zlib.ini b/doc/guides/compressdevs/features/zlib.ini
new file mode 100644
index 0000000..c794643
--- /dev/null
+++ b/doc/guides/compressdevs/features/zlib.ini
@@ -0,0 +1,11 @@
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; Supported features of 'ZLIB' compression driver.
+;
+[Features]
+Pass-through   = Y
+Deflate        = Y
+Fixed          = Y
+Dynamic        = Y
+OOP SGL In SGL Out  = Y
diff --git a/doc/guides/compressdevs/index.rst b/doc/guides/compressdevs/index.rst
index 4228768..6ba6641 100644
--- a/doc/guides/compressdevs/index.rst
+++ b/doc/guides/compressdevs/index.rst
@@ -12,3 +12,4 @@ Compression Device Drivers
     overview
     isal
     qat_comp
+    zlib
diff --git a/doc/guides/compressdevs/zlib.rst b/doc/guides/compressdevs/zlib.rst
new file mode 100644
index 0000000..c020792
--- /dev/null
+++ b/doc/guides/compressdevs/zlib.rst
@@ -0,0 +1,69 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Cavium Networks.
+
+ZLIB Compression Poll Mode Driver
+==================================
+
+The ZLIB PMD (**librte_pmd_zlib**) provides poll mode compression &
+decompression driver based on SW zlib library,
+
+Features
+--------
+
+ZLIB PMD has support for:
+
+Compression/Decompression algorithm:
+
+* DEFLATE
+
+Huffman code type:
+
+* FIXED
+* DYNAMIC
+
+Window size support:
+
+* Min - 256 bytes
+* Max - 32K
+
+Limitations
+-----------
+
+* Stateful  not supported.
+
+Installation
+------------
+
+* To build DPDK with ZLIB library, the user is required to download the ``libz`` library.
+* Use following command for installation.
+
+* For Fedora users::
+     sudo yum install zlib-devel
+* For Ubuntu users::
+     sudo apt-get install zlib1g-dev
+
+* Once downloaded, the user needs to build the library.
+
+* To build from sources
+  download zlib sources from http://zlib.net/ and do following before building DPDK::
+
+    make
+    sudo make install
+
+Initialization
+--------------
+
+In order to enable this virtual compression PMD, user must:
+
+* Set ``CONFIG_RTE_LIBRTE_PMD_ZLIB=y`` in config/common_base.
+
+To use the PMD in an application, user must:
+
+* Call ``rte_vdev_init("compress_zlib")`` within the application.
+
+* Use ``--vdev="compress_zlib"`` in the EAL options, which will call ``rte_vdev_init()`` internally.
+
+The following parameter (optional) can be provided in the previous two calls:
+
+* ``socket_id:`` Specify the socket where the memory for the device is going to be allocated
+  (by default, socket_id will be the socket where the core that is creating the PMD is running on).
-- 
2.9.5

Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
> Sent: Saturday, July 21, 2018 7:18 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> mchalla@caviumnetworks.com; ashish.gupta@caviumnetworks.com;
> sunila.sahu@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>
> Subject: [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
> 
> From: Sunila Sahu <ssahu@caviumnetworks.com>
> 
> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> ---
>  drivers/compress/zlib/zlib_pmd.c | 254

...

> +def_end:
> +	/* Update op stats */
> +	switch (op->status) {
> +	case RTE_COMP_OP_STATUS_SUCCESS:
> +		op->consumed += strm->total_in;

Compilation issue here:

drivers/compress/zlib/zlib_pmd.c:94:16: error: this statement may fall through [-Werror=implicit-fallthrough=]
   op->consumed += strm->total_in;
   ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~

I think you are missing a "/* Fall-through */" comment here.

Thanks,
Pablo

> +	case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
> +		op->produced += strm->total_out;
> +		break;

Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[Verma, Shally]

Hi Pablo

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 23 July 2018 18:07
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu, Sunila
><Sunila.Sahu@cavium.com>
>Subject: RE: [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
>
>External Email
>
>> -----Original Message-----
>> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
>> Sent: Saturday, July 21, 2018 7:18 PM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> mchalla@caviumnetworks.com; ashish.gupta@caviumnetworks.com;
>> sunila.sahu@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>
>> Subject: [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
>>
>> From: Sunila Sahu <ssahu@caviumnetworks.com>
>>
>> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>> ---
>>  drivers/compress/zlib/zlib_pmd.c | 254
>
>...
>
>> +def_end:
>> +     /* Update op stats */
>> +     switch (op->status) {
>> +     case RTE_COMP_OP_STATUS_SUCCESS:
>> +             op->consumed += strm->total_in;
>
>Compilation issue here:
>
>drivers/compress/zlib/zlib_pmd.c:94:16: error: this statement may fall through [-Werror=implicit-fallthrough=]
>   op->consumed += strm->total_in;
>   ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
>
>I think you are missing a "/* Fall-through */" comment here.
OK. I will send next patch immediately with this fix. But which compiler version you following, since this isn't seen during our build (which we did on multiple platforms) so want to understand what is the environment we need to have to avoid such regressions.

Thanks
Shally

>
>Thanks,
>Pablo
>
>> +     case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
>> +             op->produced += strm->total_out;
>> +             break;

Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Monday, July 23, 2018 1:53 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> <Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>;
> Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu, Sunila
> <Sunila.Sahu@cavium.com>
> Subject: RE: [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
> 
> Hi Pablo
> 
> >-----Original Message-----
> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >Sent: 23 July 2018 18:07
> >To: Verma, Shally <Shally.Verma@cavium.com>
> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>;
> >Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu, Sunila
> ><Sunila.Sahu@cavium.com>
> >Subject: RE: [PATCH v3 4/5] compress/zlib: support burst
> >enqueue/dequeue
> >
> >External Email
> >
> >> -----Original Message-----
> >> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
> >> Sent: Saturday, July 21, 2018 7:18 PM
> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> >> mchalla@caviumnetworks.com; ashish.gupta@caviumnetworks.com;
> >> sunila.sahu@caviumnetworks.com; Sunila Sahu
> >> <ssahu@caviumnetworks.com>
> >> Subject: [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
> >>
> >> From: Sunila Sahu <ssahu@caviumnetworks.com>
> >>
> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> >> ---
> >>  drivers/compress/zlib/zlib_pmd.c | 254
> >
> >...
> >
> >> +def_end:
> >> +     /* Update op stats */
> >> +     switch (op->status) {
> >> +     case RTE_COMP_OP_STATUS_SUCCESS:
> >> +             op->consumed += strm->total_in;
> >
> >Compilation issue here:
> >
> >drivers/compress/zlib/zlib_pmd.c:94:16: error: this statement may fall through
> [-Werror=implicit-fallthrough=]
> >   op->consumed += strm->total_in;
> >   ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
> >
> >I think you are missing a "/* Fall-through */" comment here.
> OK. I will send next patch immediately with this fix. But which compiler version
> you following, since this isn't seen during our build (which we did on multiple
> platforms) so want to understand what is the environment we need to have to
> avoid such regressions.

I see this with gcc 7.2.1 (on Fedora 27).

Pablo

> 
> Thanks
> Shally
> 
> >
> >Thanks,
> >Pablo
> >
> >> +     case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
> >> +             op->produced += strm->total_out;
> >> +             break;

Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[Stephen Hemminger]

On Sat, 21 Jul 2018 23:47:48 +0530
Shally Verma <shally.verma@caviumnetworks.com> wrote:

> -/** Parse comp xform and set private xform/stream parameters */
> +/** Compute next mbuf in the list, assign data buffer and length,
> + *  returns 0 if mbuf is NULL
> + */
> +#define COMPUTE_BUF(mbuf, data, len)		\
> +		((mbuf = mbuf->next) ?		\
> +		(data = rte_pktmbuf_mtod(mbuf, uint8_t *)),	\
> +		(len = rte_pktmbuf_data_len(mbuf)) : 0)
> +

Could this be an inline not a macro?

Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[Verma, Shally]

>-----Original Message-----
>From: Stephen Hemminger <stephen@networkplumber.org>
>Sent: 23 July 2018 22:24
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: pablo.de.lara.guarch@intel.com; dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa,
>Mahipal <Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
>Sahu, Sunila <Sunila.Sahu@cavium.com>
>Subject: Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
>
>External Email
>
>On Sat, 21 Jul 2018 23:47:48 +0530
>Shally Verma <shally.verma@caviumnetworks.com> wrote:
>
>> -/** Parse comp xform and set private xform/stream parameters */
>> +/** Compute next mbuf in the list, assign data buffer and length,
>> + *  returns 0 if mbuf is NULL
>> + */
>> +#define COMPUTE_BUF(mbuf, data, len)         \
>> +             ((mbuf = mbuf->next) ?          \
>> +             (data = rte_pktmbuf_mtod(mbuf, uint8_t *)),     \
>> +             (len = rte_pktmbuf_data_len(mbuf)) : 0)
>> +
>
>Could this be an inline not a macro?
[Shally] Again what goes in favour of inline here? Just curious to know if DPDK has any preferred guidelines regarding this?

Thanks
Shally 

Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue

[Stephen Hemminger]

On Mon, 23 Jul 2018 17:14:26 +0000
"Verma, Shally" <Shally.Verma@cavium.com> wrote:

> >-----Original Message-----
> >From: Stephen Hemminger <stephen@networkplumber.org>
> >Sent: 23 July 2018 22:24
> >To: Verma, Shally <Shally.Verma@cavium.com>
> >Cc: pablo.de.lara.guarch@intel.com; dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa,
> >Mahipal <Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> >Sahu, Sunila <Sunila.Sahu@cavium.com>
> >Subject: Re: [dpdk-dev] [PATCH v3 4/5] compress/zlib: support burst enqueue/dequeue
> >
> >External Email
> >
> >On Sat, 21 Jul 2018 23:47:48 +0530
> >Shally Verma <shally.verma@caviumnetworks.com> wrote:
> >  
> >> -/** Parse comp xform and set private xform/stream parameters */
> >> +/** Compute next mbuf in the list, assign data buffer and length,
> >> + *  returns 0 if mbuf is NULL
> >> + */
> >> +#define COMPUTE_BUF(mbuf, data, len)         \
> >> +             ((mbuf = mbuf->next) ?          \
> >> +             (data = rte_pktmbuf_mtod(mbuf, uint8_t *)),     \
> >> +             (len = rte_pktmbuf_data_len(mbuf)) : 0)
> >> +  
> >
> >Could this be an inline not a macro?  
> [Shally] Again what goes in favour of inline here? Just curious to know if DPDK has any preferred guidelines regarding this?
> 
> Thanks
> Shally 


Macros have no type checking and are harder to debug.
They should only be used when doing generic code (ie template like).