[dpdk-dev] [PATCH v4 0/5] compress: add ZLIB compression PMD

[dpdk-dev] [PATCH v4 0/5] compress: add ZLIB compression PMD

[Shally Verma]

This patch series add software zlib library (http://zlib.net/)
based compression PMD in DPDK compress drivers.

Application must need to install zlib prior to enabling
this PMD to avail compression/decompression services.
Currently driver only tested for deflate, stateless
compression and decompression.

Changes in v4:
- fixed -Wimplcit-fallthrough compilation error
 with gcc-7+ toolchain

Changes in v3:
- added comp_feature_flags to compressdev_capabilities
- removed stream_create as stateful not supported
- changed compression strategy to DEFAULT_STRATEGY
- Simplified core processing logic
- fix doc build error
- other v2 feedbacks

TBDs
- No change to current qp enqueue_err stats update.
- PMD increment enqd_err stats, if it fails to push op
  into completion queue. This logic to be revisited based on
  further discussion

Changes in v2:
- removed unused variables
- corrected capability to reflect current support
- add lookup for internally maintained mempool during device_configure
- optimized core compression/decompression logic in enq/deq APIs
- updated documentation with correct feature support

v1 includes:
- build changes to build zlib PMD
- zlib PMD implementation
- zlib PMD documentation
- meson build support

This patchset is dependent upon compressdev API.

Ashish Gupta (2):
  compress/zlib: add ZLIB PMD
  compress/zlib: add device PMD ops

Shally Verma (1):
  doc: add ZLIB PMD guide

Sunila Sahu (2):
  compress/zlib: create private xform
  compress/zlib: support burst enqueue/dequeue

 MAINTAINERS                                    |   5 +
 config/common_base                             |   5 +
 doc/guides/compressdevs/features/zlib.ini      |  11 +
 doc/guides/compressdevs/index.rst              |   1 +
 doc/guides/compressdevs/zlib.rst               |  69 ++++
 drivers/compress/Makefile                      |   1 +
 drivers/compress/meson.build                   |   2 +-
 drivers/compress/zlib/Makefile                 |  29 ++
 drivers/compress/zlib/meson.build              |  14 +
 drivers/compress/zlib/rte_pmd_zlib_version.map |   3 +
 drivers/compress/zlib/zlib_pmd.c               | 427 +++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_ops.c           | 308 ++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_private.h       |  71 ++++
 mk/rte.app.mk                                  |   2 +
 14 files changed, 947 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/compressdevs/features/zlib.ini
 create mode 100644 doc/guides/compressdevs/zlib.rst
 create mode 100644 drivers/compress/zlib/Makefile
 create mode 100644 drivers/compress/zlib/meson.build
 create mode 100644 drivers/compress/zlib/rte_pmd_zlib_version.map
 create mode 100644 drivers/compress/zlib/zlib_pmd.c
 create mode 100644 drivers/compress/zlib/zlib_pmd_ops.c
 create mode 100644 drivers/compress/zlib/zlib_pmd_private.h

-- 
2.9.5

[dpdk-dev] [PATCH v4 1/5] compress/zlib: add ZLIB PMD

[Shally Verma]

From: Ashish Gupta <ashish.gupta@caviumnetworks.com>

Add initial PMD setup routines in compressdev
framework. ZLIB PMD appears as virtual compression
device. User would need to install zlib prior to
enabling this PMD.

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 MAINTAINERS                                    |  3 +
 config/common_base                             |  5 ++
 drivers/compress/Makefile                      |  1 +
 drivers/compress/meson.build                   |  2 +-
 drivers/compress/zlib/Makefile                 | 28 +++++++++
 drivers/compress/zlib/meson.build              | 14 +++++
 drivers/compress/zlib/rte_pmd_zlib_version.map |  3 +
 drivers/compress/zlib/zlib_pmd.c               | 78 ++++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_private.h       | 32 +++++++++++
 mk/rte.app.mk                                  |  2 +
 10 files changed, 167 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7b2414d..ca27c6f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -872,6 +872,9 @@ M: Fiona Trahe <fiona.trahe@intel.com>
 F: drivers/compress/qat/
 F: drivers/common/qat/
 
+ZLIB
+M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
+F: drivers/compress/zlib/
 
 Eventdev Drivers
 ----------------
diff --git a/config/common_base b/config/common_base
index a061c21..d9b238c 100644
--- a/config/common_base
+++ b/config/common_base
@@ -582,6 +582,11 @@ CONFIG_RTE_COMPRESSDEV_TEST=n
 CONFIG_RTE_LIBRTE_PMD_ISAL=n
 
 #
+# Compile PMD for ZLIB compression device
+#
+CONFIG_RTE_LIBRTE_PMD_ZLIB=n
+
+#
 # Compile generic event device library
 #
 CONFIG_RTE_LIBRTE_EVENTDEV=y
diff --git a/drivers/compress/Makefile b/drivers/compress/Makefile
index 592497f..1f159a5 100644
--- a/drivers/compress/Makefile
+++ b/drivers/compress/Makefile
@@ -4,5 +4,6 @@
 include $(RTE_SDK)/mk/rte.vars.mk
 
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += isal
+DIRS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib
 
 include $(RTE_SDK)/mk/rte.subdir.mk
diff --git a/drivers/compress/meson.build b/drivers/compress/meson.build
index 2352ad5..d2ca8fc 100644
--- a/drivers/compress/meson.build
+++ b/drivers/compress/meson.build
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-drivers = ['isal', 'qat']
+drivers = ['isal', 'qat', 'zlib']
 
 std_deps = ['compressdev'] # compressdev pulls in all other needed deps
 config_flag_fmt = 'RTE_LIBRTE_@0@_PMD'
diff --git a/drivers/compress/zlib/Makefile b/drivers/compress/zlib/Makefile
new file mode 100644
index 0000000..bd322c9
--- /dev/null
+++ b/drivers/compress/zlib/Makefile
@@ -0,0 +1,28 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Cavium Networks
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_pmd_zlib.a
+
+# build flags
+CFLAGS += -O3
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+# library version
+LIBABIVER := 1
+
+# versioning export map
+EXPORT_MAP := rte_pmd_zlib_version.map
+
+# external library dependencies
+LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring -lz
+LDLIBS += -lrte_compressdev
+LDLIBS += -lrte_bus_vdev
+
+# library source files
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib_pmd.c
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/compress/zlib/meson.build b/drivers/compress/zlib/meson.build
new file mode 100644
index 0000000..3f0a77b
--- /dev/null
+++ b/drivers/compress/zlib/meson.build
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Cavium Networks
+
+dep = dependency('zlib', required: false)
+if not dep.found()
+	build = false
+endif
+
+deps += 'bus_vdev'
+sources = files('zlib_pmd.c')
+ext_deps += dep
+pkgconfig_extra_libs += '-lz'
+
+allow_experimental_apis = true
diff --git a/drivers/compress/zlib/rte_pmd_zlib_version.map b/drivers/compress/zlib/rte_pmd_zlib_version.map
new file mode 100644
index 0000000..ad6e191
--- /dev/null
+++ b/drivers/compress/zlib/rte_pmd_zlib_version.map
@@ -0,0 +1,3 @@
+DPDK_18.08 {
+	local: *;
+};
diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
new file mode 100644
index 0000000..39d3628
--- /dev/null
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Cavium Networks
+ */
+
+#include <rte_bus_vdev.h>
+#include <rte_common.h>
+
+#include "zlib_pmd_private.h"
+
+static int
+zlib_create(const char *name,
+		struct rte_vdev_device *vdev,
+		struct rte_compressdev_pmd_init_params *init_params)
+{
+	struct rte_compressdev *dev;
+
+	dev = rte_compressdev_pmd_create(name, &vdev->device,
+			sizeof(struct zlib_private), init_params);
+	if (dev == NULL) {
+		ZLIB_PMD_ERR("driver %s: create failed", init_params->name);
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int
+zlib_probe(struct rte_vdev_device *vdev)
+{
+	struct rte_compressdev_pmd_init_params init_params = {
+		"",
+		rte_socket_id()
+	};
+	const char *name;
+	const char *input_args;
+
+	name = rte_vdev_device_name(vdev);
+
+	if (name == NULL)
+		return -EINVAL;
+	input_args = rte_vdev_device_args(vdev);
+	rte_compressdev_pmd_parse_input_args(&init_params, input_args);
+
+	return zlib_create(name, vdev, &init_params);
+}
+
+static int
+zlib_remove(struct rte_vdev_device *vdev)
+{
+	struct rte_compressdev *compressdev;
+	const char *name;
+
+	name = rte_vdev_device_name(vdev);
+	if (name == NULL)
+		return -EINVAL;
+
+	compressdev = rte_compressdev_pmd_get_named_dev(name);
+	if (compressdev == NULL)
+		return -ENODEV;
+
+	return rte_compressdev_pmd_destroy(compressdev);
+}
+
+static struct rte_vdev_driver zlib_pmd_drv = {
+	.probe = zlib_probe,
+	.remove = zlib_remove
+};
+
+RTE_PMD_REGISTER_VDEV(COMPRESSDEV_NAME_ZLIB_PMD, zlib_pmd_drv);
+RTE_INIT(zlib_init_log);
+
+static void
+zlib_init_log(void)
+{
+	zlib_logtype_driver = rte_log_register("pmd.compress.zlib");
+	if (zlib_logtype_driver >= 0)
+		rte_log_set_level(zlib_logtype_driver, RTE_LOG_INFO);
+}
diff --git a/drivers/compress/zlib/zlib_pmd_private.h b/drivers/compress/zlib/zlib_pmd_private.h
new file mode 100644
index 0000000..d26a740
--- /dev/null
+++ b/drivers/compress/zlib/zlib_pmd_private.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Cavium Networks
+ */
+
+#ifndef _RTE_ZLIB_PMD_PRIVATE_H_
+#define _RTE_ZLIB_PMD_PRIVATE_H_
+
+#include <zlib.h>
+#include <rte_compressdev.h>
+#include <rte_compressdev_pmd.h>
+
+#define COMPRESSDEV_NAME_ZLIB_PMD	compress_zlib
+/**< ZLIB PMD device name */
+
+#define DEF_MEM_LEVEL			8
+
+int zlib_logtype_driver;
+#define ZLIB_PMD_LOG(level, fmt, args...) \
+	rte_log(RTE_LOG_ ## level, zlib_logtype_driver, "%s(): "fmt "\n", \
+			__func__, ##args)
+
+#define ZLIB_PMD_INFO(fmt, args...) \
+	ZLIB_PMD_LOG(INFO, fmt, ## args)
+#define ZLIB_PMD_ERR(fmt, args...) \
+	ZLIB_PMD_LOG(ERR, fmt, ## args)
+#define ZLIB_PMD_WARN(fmt, args...) \
+	ZLIB_PMD_LOG(WARNING, fmt, ## args)
+
+struct zlib_private {
+};
+
+#endif /* _RTE_ZLIB_PMD_PRIVATE_H_ */
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 1590838..5b7c684 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -228,6 +228,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ISAL) += -lisal
 ifeq ($(CONFIG_RTE_LIBRTE_PMD_QAT_SYM),n)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT)  += -lrte_pmd_qat
 endif # CONFIG_RTE_LIBRTE_PMD_QAT_SYM
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += -lrte_pmd_zlib
+_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += -lz
 endif # CONFIG_RTE_LIBRTE_COMPRESSDEV
 
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
-- 
2.9.5

[dpdk-dev] [PATCH v4 2/5] compress/zlib: add device PMD ops

[Shally Verma]

From: Ashish Gupta <ashish.gupta@caviumnetworks.com>

Implement device configure and queue pair
setup PMD ops

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 drivers/compress/zlib/Makefile           |   1 +
 drivers/compress/zlib/meson.build        |   2 +-
 drivers/compress/zlib/zlib_pmd.c         |   2 +
 drivers/compress/zlib/zlib_pmd_ops.c     | 238 +++++++++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_private.h |  35 +++++
 5 files changed, 277 insertions(+), 1 deletion(-)

diff --git a/drivers/compress/zlib/Makefile b/drivers/compress/zlib/Makefile
index bd322c9..5cf8de6 100644
--- a/drivers/compress/zlib/Makefile
+++ b/drivers/compress/zlib/Makefile
@@ -24,5 +24,6 @@ LDLIBS += -lrte_bus_vdev
 
 # library source files
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib_pmd.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_ZLIB) += zlib_pmd_ops.c
 
 include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/compress/zlib/meson.build b/drivers/compress/zlib/meson.build
index 3f0a77b..7748de2 100644
--- a/drivers/compress/zlib/meson.build
+++ b/drivers/compress/zlib/meson.build
@@ -7,7 +7,7 @@ if not dep.found()
 endif
 
 deps += 'bus_vdev'
-sources = files('zlib_pmd.c')
+sources = files('zlib_pmd.c', 'zlib_pmd_ops.c')
 ext_deps += dep
 pkgconfig_extra_libs += '-lz'
 
diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
index 39d3628..9363808 100644
--- a/drivers/compress/zlib/zlib_pmd.c
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -21,6 +21,8 @@ zlib_create(const char *name,
 		return -ENODEV;
 	}
 
+	dev->dev_ops = rte_zlib_pmd_ops;
+
 	return 0;
 }
 
diff --git a/drivers/compress/zlib/zlib_pmd_ops.c b/drivers/compress/zlib/zlib_pmd_ops.c
new file mode 100644
index 0000000..645c5b1
--- /dev/null
+++ b/drivers/compress/zlib/zlib_pmd_ops.c
@@ -0,0 +1,238 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Cavium Networks
+ */
+
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_malloc.h>
+
+#include "zlib_pmd_private.h"
+
+static const struct rte_compressdev_capabilities zlib_pmd_capabilities[] = {
+	{   /* Deflate */
+		.algo = RTE_COMP_ALGO_DEFLATE,
+		.comp_feature_flags = (RTE_COMP_FF_NONCOMPRESSED_BLOCKS |
+					RTE_COMP_FF_HUFFMAN_FIXED |
+					RTE_COMP_FF_HUFFMAN_DYNAMIC |
+					RTE_COMP_FF_OOP_SGL_IN_SGL_OUT),
+		.window_size = {
+			.min = 8,
+			.max = 15,
+			.increment = 1
+		},
+	},
+
+	RTE_COMP_END_OF_CAPABILITIES_LIST()
+
+};
+
+/** Configure device */
+static int
+zlib_pmd_config(struct rte_compressdev *dev,
+		struct rte_compressdev_config *config)
+{
+	struct rte_mempool *mp;
+	char mp_name[RTE_MEMPOOL_NAMESIZE];
+	struct zlib_private *internals = dev->data->dev_private;
+
+	snprintf(mp_name, RTE_MEMPOOL_NAMESIZE,
+			"stream_mp_%u", dev->data->dev_id);
+	mp = internals->mp;
+	if (mp == NULL) {
+		mp = rte_mempool_create(mp_name,
+				config->max_nb_priv_xforms +
+				config->max_nb_streams,
+				sizeof(struct zlib_priv_xform),
+				0, 0, NULL, NULL, NULL,
+				NULL, config->socket_id,
+				0);
+		if (mp == NULL) {
+			ZLIB_PMD_ERR("Cannot create private xform pool on "
+			"socket %d\n", config->socket_id);
+			return -ENOMEM;
+		}
+		internals->mp = mp;
+	}
+	return 0;
+}
+
+/** Start device */
+static int
+zlib_pmd_start(__rte_unused struct rte_compressdev *dev)
+{
+	return 0;
+}
+
+/** Stop device */
+static void
+zlib_pmd_stop(__rte_unused struct rte_compressdev *dev)
+{
+}
+
+/** Close device */
+static int
+zlib_pmd_close(struct rte_compressdev *dev)
+{
+	struct zlib_private *internals = dev->data->dev_private;
+	rte_mempool_free(internals->mp);
+	internals->mp = NULL;
+	return 0;
+}
+
+/** Get device statistics */
+static void
+zlib_pmd_stats_get(struct rte_compressdev *dev,
+		struct rte_compressdev_stats *stats)
+{
+	int qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct zlib_qp *qp = dev->data->queue_pairs[qp_id];
+
+		stats->enqueued_count += qp->qp_stats.enqueued_count;
+		stats->dequeued_count += qp->qp_stats.dequeued_count;
+
+		stats->enqueue_err_count += qp->qp_stats.enqueue_err_count;
+		stats->dequeue_err_count += qp->qp_stats.dequeue_err_count;
+	}
+}
+
+/** Reset device statistics */
+static void
+zlib_pmd_stats_reset(struct rte_compressdev *dev)
+{
+	int qp_id;
+
+	for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+		struct zlib_qp *qp = dev->data->queue_pairs[qp_id];
+
+		memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	}
+}
+
+/** Get device info */
+static void
+zlib_pmd_info_get(struct rte_compressdev *dev,
+		struct rte_compressdev_info *dev_info)
+{
+	if (dev_info != NULL) {
+		dev_info->driver_name = dev->device->name;
+		dev_info->feature_flags = dev->feature_flags;
+		dev_info->capabilities = zlib_pmd_capabilities;
+	}
+}
+
+/** Release queue pair */
+static int
+zlib_pmd_qp_release(struct rte_compressdev *dev, uint16_t qp_id)
+{
+	struct zlib_qp *qp = dev->data->queue_pairs[qp_id];
+
+	if (qp != NULL) {
+		rte_ring_free(qp->processed_pkts);
+		rte_free(qp);
+		dev->data->queue_pairs[qp_id] = NULL;
+	}
+	return 0;
+}
+
+/** set a unique name for the queue pair based on its name, dev_id and qp_id */
+static int
+zlib_pmd_qp_set_unique_name(struct rte_compressdev *dev,
+		struct zlib_qp *qp)
+{
+	unsigned int n = snprintf(qp->name, sizeof(qp->name),
+				"zlib_pmd_%u_qp_%u",
+				dev->data->dev_id, qp->id);
+
+	if (n >= sizeof(qp->name))
+		return -1;
+
+	return 0;
+}
+
+/** Create a ring to place process packets on */
+static struct rte_ring *
+zlib_pmd_qp_create_processed_pkts_ring(struct zlib_qp *qp,
+		unsigned int ring_size, int socket_id)
+{
+	struct rte_ring *r = qp->processed_pkts;
+
+	if (r) {
+		if (rte_ring_get_size(r) >= ring_size) {
+			ZLIB_PMD_INFO("Reusing existing ring %s for processed"
+					" packets", qp->name);
+			return r;
+		}
+
+		ZLIB_PMD_ERR("Unable to reuse existing ring %s for processed"
+				" packets", qp->name);
+		return NULL;
+	}
+
+	return rte_ring_create(qp->name, ring_size, socket_id,
+						RING_F_EXACT_SZ);
+}
+
+/** Setup a queue pair */
+static int
+zlib_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
+		uint32_t max_inflight_ops, int socket_id)
+{
+	struct zlib_qp *qp = NULL;
+
+	/* Free memory prior to re-allocation if needed. */
+	if (dev->data->queue_pairs[qp_id] != NULL)
+		zlib_pmd_qp_release(dev, qp_id);
+
+	/* Allocate the queue pair data structure. */
+	qp = rte_zmalloc_socket("ZLIB PMD Queue Pair", sizeof(*qp),
+					RTE_CACHE_LINE_SIZE, socket_id);
+	if (qp == NULL)
+		return (-ENOMEM);
+
+	qp->id = qp_id;
+	dev->data->queue_pairs[qp_id] = qp;
+
+	if (zlib_pmd_qp_set_unique_name(dev, qp))
+		goto qp_setup_cleanup;
+
+	qp->processed_pkts = zlib_pmd_qp_create_processed_pkts_ring(qp,
+			max_inflight_ops, socket_id);
+	if (qp->processed_pkts == NULL)
+		goto qp_setup_cleanup;
+
+	memset(&qp->qp_stats, 0, sizeof(qp->qp_stats));
+	return 0;
+
+qp_setup_cleanup:
+	if (qp) {
+		rte_free(qp);
+		qp = NULL;
+	}
+	return -1;
+}
+
+struct rte_compressdev_ops zlib_pmd_ops = {
+		.dev_configure		= zlib_pmd_config,
+		.dev_start		= zlib_pmd_start,
+		.dev_stop		= zlib_pmd_stop,
+		.dev_close		= zlib_pmd_close,
+
+		.stats_get		= zlib_pmd_stats_get,
+		.stats_reset		= zlib_pmd_stats_reset,
+
+		.dev_infos_get		= zlib_pmd_info_get,
+
+		.queue_pair_setup	= zlib_pmd_qp_setup,
+		.queue_pair_release	= zlib_pmd_qp_release,
+
+		.private_xform_create	= NULL,
+		.private_xform_free	= NULL,
+
+		.stream_create	= NULL,
+		.stream_free	= NULL
+};
+
+struct rte_compressdev_ops *rte_zlib_pmd_ops = &zlib_pmd_ops;
diff --git a/drivers/compress/zlib/zlib_pmd_private.h b/drivers/compress/zlib/zlib_pmd_private.h
index d26a740..0e391a4 100644
--- a/drivers/compress/zlib/zlib_pmd_private.h
+++ b/drivers/compress/zlib/zlib_pmd_private.h
@@ -27,6 +27,41 @@ int zlib_logtype_driver;
 	ZLIB_PMD_LOG(WARNING, fmt, ## args)
 
 struct zlib_private {
+	struct rte_mempool *mp;
 };
 
+struct zlib_qp {
+	struct rte_ring *processed_pkts;
+	/**< Ring for placing process packets */
+	struct rte_compressdev_stats qp_stats;
+	/**< Queue pair statistics */
+	uint16_t id;
+	/**< Queue Pair Identifier */
+	char name[RTE_COMPRESSDEV_NAME_MAX_LEN];
+	/**< Unique Queue Pair Name */
+} __rte_cache_aligned;
+
+/* Algorithm handler function prototype */
+typedef void (*comp_func_t)(struct rte_comp_op *op, z_stream *strm);
+
+typedef int (*comp_free_t)(z_stream *strm);
+
+/** ZLIB Stream structure */
+struct zlib_stream {
+	z_stream strm;
+	/**< zlib stream structure */
+	comp_func_t comp;
+	/**< Operation (compression/decompression) */
+	comp_free_t free;
+	/**< Free Operation (compression/decompression) */
+} __rte_cache_aligned;
+
+/** ZLIB private xform structure */
+struct zlib_priv_xform {
+	struct zlib_stream stream;
+} __rte_cache_aligned;
+
+/** Device specific operations function pointer structure */
+extern struct rte_compressdev_ops *rte_zlib_pmd_ops;
+
 #endif /* _RTE_ZLIB_PMD_PRIVATE_H_ */
-- 
2.9.5

[dpdk-dev] [PATCH v4 3/5] compress/zlib: create private xform

[Shally Verma]

From: Sunila Sahu <ssahu@caviumnetworks.com>

Create non-shareable private xform for stateless
operation processing

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 drivers/compress/zlib/zlib_pmd.c         | 94 ++++++++++++++++++++++++++++++++
 drivers/compress/zlib/zlib_pmd_ops.c     | 74 ++++++++++++++++++++++++-
 drivers/compress/zlib/zlib_pmd_private.h |  4 ++
 3 files changed, 170 insertions(+), 2 deletions(-)

diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
index 9363808..47bc73d 100644
--- a/drivers/compress/zlib/zlib_pmd.c
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -7,6 +7,100 @@
 
 #include "zlib_pmd_private.h"
 
+/** Parse comp xform and set private xform/stream parameters */
+int
+zlib_set_stream_parameters(const struct rte_comp_xform *xform,
+		struct zlib_stream *stream)
+{
+	int strategy, level, wbits;
+	z_stream *strm = &stream->strm;
+
+	/* allocate deflate state */
+	strm->zalloc = Z_NULL;
+	strm->zfree = Z_NULL;
+	strm->opaque = Z_NULL;
+
+	switch (xform->type) {
+	case RTE_COMP_COMPRESS:
+		/** Compression window bits */
+		switch (xform->compress.algo) {
+		case RTE_COMP_ALGO_DEFLATE:
+			wbits = -(xform->compress.window_size);
+			break;
+		default:
+			ZLIB_PMD_ERR("Compression algorithm not supported\n");
+			return -1;
+		}
+		/** Compression Level */
+		switch (xform->compress.level) {
+		case RTE_COMP_LEVEL_PMD_DEFAULT:
+			level = Z_DEFAULT_COMPRESSION;
+			break;
+		case RTE_COMP_LEVEL_NONE:
+			level = Z_NO_COMPRESSION;
+			break;
+		case RTE_COMP_LEVEL_MIN:
+			level = Z_BEST_SPEED;
+			break;
+		case RTE_COMP_LEVEL_MAX:
+			level = Z_BEST_COMPRESSION;
+			break;
+		default:
+			level = xform->compress.level;
+			if (level < RTE_COMP_LEVEL_MIN ||
+					level > RTE_COMP_LEVEL_MAX) {
+				ZLIB_PMD_ERR("Compression level %d "
+						"not supported\n",
+						level);
+				return -1;
+			}
+			break;
+		}
+		/** Compression strategy */
+		switch (xform->compress.deflate.huffman) {
+		case RTE_COMP_HUFFMAN_DEFAULT:
+			strategy = Z_DEFAULT_STRATEGY;
+			break;
+		case RTE_COMP_HUFFMAN_FIXED:
+			strategy = Z_FIXED;
+			break;
+		case RTE_COMP_HUFFMAN_DYNAMIC:
+			strategy = Z_DEFAULT_STRATEGY;
+			break;
+		default:
+			ZLIB_PMD_ERR("Compression strategy not supported\n");
+			return -1;
+		}
+		if (deflateInit2(strm, level,
+					Z_DEFLATED, wbits,
+					DEF_MEM_LEVEL, strategy) != Z_OK) {
+			ZLIB_PMD_ERR("Deflate init failed\n");
+			return -1;
+		}
+		break;
+
+	case RTE_COMP_DECOMPRESS:
+		/** window bits */
+		switch (xform->decompress.algo) {
+		case RTE_COMP_ALGO_DEFLATE:
+			wbits = -(xform->decompress.window_size);
+			break;
+		default:
+			ZLIB_PMD_ERR("Compression algorithm not supported\n");
+			return -1;
+		}
+
+		if (inflateInit2(strm, wbits) != Z_OK) {
+			ZLIB_PMD_ERR("Inflate init failed\n");
+			return -1;
+		}
+		break;
+	default:
+		return -1;
+	}
+	return 0;
+}
+
 static int
 zlib_create(const char *name,
 		struct rte_vdev_device *vdev,
diff --git a/drivers/compress/zlib/zlib_pmd_ops.c b/drivers/compress/zlib/zlib_pmd_ops.c
index 645c5b1..2fb6a85 100644
--- a/drivers/compress/zlib/zlib_pmd_ops.c
+++ b/drivers/compress/zlib/zlib_pmd_ops.c
@@ -214,6 +214,76 @@ zlib_pmd_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
 	return -1;
 }
 
+/** Configure stream */
+static int
+zlib_pmd_stream_create(struct rte_compressdev *dev,
+		const struct rte_comp_xform *xform,
+		void **zstream)
+{
+	int ret = 0;
+	struct zlib_stream *stream;
+	struct zlib_private *internals = dev->data->dev_private;
+
+	if (xform == NULL) {
+		ZLIB_PMD_ERR("invalid xform struct");
+		return -EINVAL;
+	}
+
+	if (rte_mempool_get(internals->mp, zstream)) {
+		ZLIB_PMD_ERR("Couldn't get object from session mempool");
+		return -ENOMEM;
+	}
+	stream = *((struct zlib_stream **)zstream);
+
+	ret = zlib_set_stream_parameters(xform, stream);
+
+	if (ret < 0) {
+		ZLIB_PMD_ERR("failed configure session parameters");
+
+		memset(stream, 0, sizeof(struct zlib_stream));
+		/* Return session to mempool */
+		rte_mempool_put(internals->mp, stream);
+		return ret;
+	}
+
+	return 0;
+}
+
+/** Configure private xform */
+static int
+zlib_pmd_private_xform_create(struct rte_compressdev *dev,
+		const struct rte_comp_xform *xform,
+		void **private_xform)
+{
+	return zlib_pmd_stream_create(dev, xform, private_xform);
+}
+
+/** Clear the memory of stream so it doesn't leave key material behind */
+static int
+zlib_pmd_stream_free(__rte_unused struct rte_compressdev *dev,
+		void *zstream)
+{
+	struct zlib_stream *stream = (struct zlib_stream *)zstream;
+	if (!stream)
+		return -EINVAL;
+
+	stream->free(&stream->strm);
+	/* Zero out the whole structure */
+	memset(stream, 0, sizeof(struct zlib_stream));
+	struct rte_mempool *mp = rte_mempool_from_obj(stream);
+	rte_mempool_put(mp, stream);
+
+	return 0;
+}
+
+/** Clear the memory of stream so it doesn't leave key material behind */
+static int
+zlib_pmd_private_xform_free(struct rte_compressdev *dev,
+		void *private_xform)
+{
+	return zlib_pmd_stream_free(dev, private_xform);
+}
+
 struct rte_compressdev_ops zlib_pmd_ops = {
 		.dev_configure		= zlib_pmd_config,
 		.dev_start		= zlib_pmd_start,
@@ -228,8 +298,8 @@ struct rte_compressdev_ops zlib_pmd_ops = {
 		.queue_pair_setup	= zlib_pmd_qp_setup,
 		.queue_pair_release	= zlib_pmd_qp_release,
 
-		.private_xform_create	= NULL,
-		.private_xform_free	= NULL,
+		.private_xform_create	= zlib_pmd_private_xform_create,
+		.private_xform_free	= zlib_pmd_private_xform_free,
 
 		.stream_create	= NULL,
 		.stream_free	= NULL
diff --git a/drivers/compress/zlib/zlib_pmd_private.h b/drivers/compress/zlib/zlib_pmd_private.h
index 0e391a4..2c6e83d 100644
--- a/drivers/compress/zlib/zlib_pmd_private.h
+++ b/drivers/compress/zlib/zlib_pmd_private.h
@@ -61,6 +61,10 @@ struct zlib_priv_xform {
 	struct zlib_stream stream;
 } __rte_cache_aligned;
 
+int
+zlib_set_stream_parameters(const struct rte_comp_xform *xform,
+		struct zlib_stream *stream);
+
 /** Device specific operations function pointer structure */
 extern struct rte_compressdev_ops *rte_zlib_pmd_ops;
 
-- 
2.9.5

[dpdk-dev] [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue

[Shally Verma]

From: Sunila Sahu <ssahu@caviumnetworks.com>

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 drivers/compress/zlib/zlib_pmd.c | 255 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 254 insertions(+), 1 deletion(-)

diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
index 47bc73d..dc1e230 100644
--- a/drivers/compress/zlib/zlib_pmd.c
+++ b/drivers/compress/zlib/zlib_pmd.c
@@ -7,7 +7,214 @@
 
 #include "zlib_pmd_private.h"
 
-/** Parse comp xform and set private xform/stream parameters */
+/** Compute next mbuf in the list, assign data buffer and length,
+ *  returns 0 if mbuf is NULL
+ */
+#define COMPUTE_BUF(mbuf, data, len)		\
+		((mbuf = mbuf->next) ?		\
+		(data = rte_pktmbuf_mtod(mbuf, uint8_t *)),	\
+		(len = rte_pktmbuf_data_len(mbuf)) : 0)
+
+static void
+process_zlib_deflate(struct rte_comp_op *op, z_stream *strm)
+{
+	int ret, flush, fin_flush;
+	struct rte_mbuf *mbuf_src = op->m_src;
+	struct rte_mbuf *mbuf_dst = op->m_dst;
+
+	switch (op->flush_flag) {
+	case RTE_COMP_FLUSH_FULL:
+	case RTE_COMP_FLUSH_FINAL:
+		fin_flush = Z_FINISH;
+		break;
+	default:
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid flush value\n");
+	}
+
+	if (unlikely(!strm)) {
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid z_stream\n");
+		return;
+	}
+	/* Update z_stream with the inputs provided by application */
+	strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
+			op->src.offset);
+
+	strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
+
+	strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
+			op->dst.offset);
+
+	strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
+
+	/* Set flush value to NO_FLUSH unless it is last mbuf */
+	flush = Z_NO_FLUSH;
+	/* Initialize status to SUCCESS */
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	do {
+		/* Set flush value to Z_FINISH for last block */
+		if ((op->src.length - strm->total_in) <= strm->avail_in) {
+			strm->avail_in = (op->src.length - strm->total_in);
+			flush = fin_flush;
+		}
+		do {
+			ret = deflate(strm, flush);
+			if (unlikely(ret == Z_STREAM_ERROR)) {
+				/* error return, do not process further */
+				op->status =  RTE_COMP_OP_STATUS_ERROR;
+				goto def_end;
+			}
+			/* Break if Z_STREAM_END is encountered */
+			if (ret == Z_STREAM_END)
+				goto def_end;
+
+		/* Keep looping until input mbuf is consumed.
+		 * Exit if destination mbuf gets exhausted.
+		 */
+		} while ((strm->avail_out == 0) &&
+			COMPUTE_BUF(mbuf_dst, strm->next_out, strm->avail_out));
+
+		if (!strm->avail_out) {
+			/* there is no space for compressed output */
+			op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+			break;
+		}
+
+	/* Update source buffer to next mbuf
+	 * Exit if input buffers are fully consumed
+	 */
+	} while (COMPUTE_BUF(mbuf_src, strm->next_in, strm->avail_in));
+
+def_end:
+	/* Update op stats */
+	switch (op->status) {
+	case RTE_COMP_OP_STATUS_SUCCESS:
+		op->consumed += strm->total_in;
+	/* Fall-through */
+	case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
+		op->produced += strm->total_out;
+		break;
+	default:
+		ZLIB_PMD_ERR("stats not updated for status:%d\n",
+				op->status);
+	}
+
+	deflateReset(strm);
+}
+
+static void
+process_zlib_inflate(struct rte_comp_op *op, z_stream *strm)
+{
+	int ret, flush;
+	struct rte_mbuf *mbuf_src = op->m_src;
+	struct rte_mbuf *mbuf_dst = op->m_dst;
+
+	if (unlikely(!strm)) {
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid z_stream\n");
+		return;
+	}
+	strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
+			op->src.offset);
+
+	strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
+
+	strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
+			op->dst.offset);
+
+	strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
+
+	/** Ignoring flush value provided from application for decompression */
+	flush = Z_NO_FLUSH;
+	/* initialize status to SUCCESS */
+	op->status = RTE_COMP_OP_STATUS_SUCCESS;
+
+	do {
+		do {
+			ret = inflate(strm, flush);
+
+			switch (ret) {
+			/* Fall-through */
+			case Z_NEED_DICT:
+				ret = Z_DATA_ERROR;
+			/* Fall-through */
+			case Z_DATA_ERROR:
+			/* Fall-through */
+			case Z_MEM_ERROR:
+			/* Fall-through */
+			case Z_STREAM_ERROR:
+				op->status = RTE_COMP_OP_STATUS_ERROR;
+			/* Fall-through */
+			case Z_STREAM_END:
+				/* no further computation needed if
+				 * Z_STREAM_END is encountered
+				 */
+				goto inf_end;
+			default:
+				/* success */
+				break;
+
+			}
+		/* Keep looping until input mbuf is consumed.
+		 * Exit if destination mbuf gets exhausted.
+		 */
+		} while ((strm->avail_out == 0) &&
+			COMPUTE_BUF(mbuf_dst, strm->next_out, strm->avail_out));
+
+		if (!strm->avail_out) {
+			/* there is no more space for decompressed output */
+			op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+			break;
+		}
+	/* Read next input buffer to be processed, exit if compressed
+	 * blocks are fully read
+	 */
+	} while (COMPUTE_BUF(mbuf_src, strm->next_in, strm->avail_in));
+
+inf_end:
+	/* Update op stats */
+	switch (op->status) {
+	case RTE_COMP_OP_STATUS_SUCCESS:
+		op->consumed += strm->total_in;
+	/* Fall-through */
+	case RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED:
+		op->produced += strm->total_out;
+		break;
+	default:
+		ZLIB_PMD_ERR("stats not produced for status:%d\n",
+				op->status);
+	}
+
+	inflateReset(strm);
+}
+
+/** Process comp operation for mbuf */
+static inline int
+process_zlib_op(struct zlib_qp *qp, struct rte_comp_op *op)
+{
+	struct zlib_stream *stream;
+	struct zlib_priv_xform *private_xform;
+
+	if ((op->op_type == RTE_COMP_OP_STATEFUL) ||
+	    (op->src.offset > rte_pktmbuf_data_len(op->m_src)) ||
+	    (op->dst.offset > rte_pktmbuf_data_len(op->m_dst))) {
+		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		ZLIB_PMD_ERR("Invalid source or destination buffers or "
+			     "invalid Operation requested\n");
+	} else {
+		private_xform = (struct zlib_priv_xform *)op->private_xform;
+		stream = &private_xform->stream;
+		stream->comp(op, &stream->strm);
+	}
+	/* whatever is out of op, put it into completion queue with
+	 * its status
+	 */
+	return rte_ring_enqueue(qp->processed_pkts, (void *)op);
+}
+
+/** Parse comp xform and set private xform/Stream parameters */
 int
 zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 		struct zlib_stream *stream)
@@ -22,6 +229,8 @@ zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 
 	switch (xform->type) {
 	case RTE_COMP_COMPRESS:
+		stream->comp = process_zlib_deflate;
+		stream->free = deflateEnd;
 		/** Compression window bits */
 		switch (xform->compress.algo) {
 		case RTE_COMP_ALGO_DEFLATE:
@@ -80,6 +289,8 @@ zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 		break;
 
 	case RTE_COMP_DECOMPRESS:
+		stream->comp = process_zlib_inflate;
+		stream->free = inflateEnd;
 		/** window bits */
 		switch (xform->decompress.algo) {
 		case RTE_COMP_ALGO_DEFLATE:
@@ -101,6 +312,44 @@ zlib_set_stream_parameters(const struct rte_comp_xform *xform,
 	return 0;
 }
 
+static uint16_t
+zlib_pmd_enqueue_burst(void *queue_pair,
+			struct rte_comp_op **ops, uint16_t nb_ops)
+{
+	struct zlib_qp *qp = queue_pair;
+	int ret;
+	uint16_t i;
+	uint16_t enqd = 0;
+	for (i = 0; i < nb_ops; i++) {
+		ret = process_zlib_op(qp, ops[i]);
+		if (unlikely(ret < 0)) {
+			/* increment count if failed to push to completion
+			 * queue
+			 */
+			qp->qp_stats.enqueue_err_count++;
+		} else {
+			qp->qp_stats.enqueued_count++;
+			enqd++;
+		}
+	}
+	return enqd;
+}
+
+static uint16_t
+zlib_pmd_dequeue_burst(void *queue_pair,
+			struct rte_comp_op **ops, uint16_t nb_ops)
+{
+	struct zlib_qp *qp = queue_pair;
+
+	unsigned int nb_dequeued = 0;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts,
+			(void **)ops, nb_ops, NULL);
+	qp->qp_stats.dequeued_count += nb_dequeued;
+
+	return nb_dequeued;
+}
+
 static int
 zlib_create(const char *name,
 		struct rte_vdev_device *vdev,
@@ -117,6 +366,10 @@ zlib_create(const char *name,
 
 	dev->dev_ops = rte_zlib_pmd_ops;
 
+	/* register rx/tx burst functions for data path */
+	dev->dequeue_burst = zlib_pmd_dequeue_burst;
+	dev->enqueue_burst = zlib_pmd_enqueue_burst;
+
 	return 0;
 }
 
-- 
2.9.5

[dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide

[Shally Verma]

Add zlib pmd feature support and user guide with
build and run instructions

Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
---
 MAINTAINERS                               |  2 +
 doc/guides/compressdevs/features/zlib.ini | 11 +++++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/zlib.rst          | 69 +++++++++++++++++++++++++++++++
 4 files changed, 83 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index ca27c6f..7e3c450 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -875,6 +875,8 @@ F: drivers/common/qat/
 ZLIB
 M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
 F: drivers/compress/zlib/
+F: doc/guides/compressdevs/zlib.rst
+F: doc/guides/compressdevs/features/zlib.ini
 
 Eventdev Drivers
 ----------------
diff --git a/doc/guides/compressdevs/features/zlib.ini b/doc/guides/compressdevs/features/zlib.ini
new file mode 100644
index 0000000..c794643
--- /dev/null
+++ b/doc/guides/compressdevs/features/zlib.ini
@@ -0,0 +1,11 @@
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+; Supported features of 'ZLIB' compression driver.
+;
+[Features]
+Pass-through   = Y
+Deflate        = Y
+Fixed          = Y
+Dynamic        = Y
+OOP SGL In SGL Out  = Y
diff --git a/doc/guides/compressdevs/index.rst b/doc/guides/compressdevs/index.rst
index 4228768..6ba6641 100644
--- a/doc/guides/compressdevs/index.rst
+++ b/doc/guides/compressdevs/index.rst
@@ -12,3 +12,4 @@ Compression Device Drivers
     overview
     isal
     qat_comp
+    zlib
diff --git a/doc/guides/compressdevs/zlib.rst b/doc/guides/compressdevs/zlib.rst
new file mode 100644
index 0000000..c020792
--- /dev/null
+++ b/doc/guides/compressdevs/zlib.rst
@@ -0,0 +1,69 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Cavium Networks.
+
+ZLIB Compression Poll Mode Driver
+==================================
+
+The ZLIB PMD (**librte_pmd_zlib**) provides poll mode compression &
+decompression driver based on SW zlib library,
+
+Features
+--------
+
+ZLIB PMD has support for:
+
+Compression/Decompression algorithm:
+
+* DEFLATE
+
+Huffman code type:
+
+* FIXED
+* DYNAMIC
+
+Window size support:
+
+* Min - 256 bytes
+* Max - 32K
+
+Limitations
+-----------
+
+* Stateful  not supported.
+
+Installation
+------------
+
+* To build DPDK with ZLIB library, the user is required to download the ``libz`` library.
+* Use following command for installation.
+
+* For Fedora users::
+     sudo yum install zlib-devel
+* For Ubuntu users::
+     sudo apt-get install zlib1g-dev
+
+* Once downloaded, the user needs to build the library.
+
+* To build from sources
+  download zlib sources from http://zlib.net/ and do following before building DPDK::
+
+    make
+    sudo make install
+
+Initialization
+--------------
+
+In order to enable this virtual compression PMD, user must:
+
+* Set ``CONFIG_RTE_LIBRTE_PMD_ZLIB=y`` in config/common_base.
+
+To use the PMD in an application, user must:
+
+* Call ``rte_vdev_init("compress_zlib")`` within the application.
+
+* Use ``--vdev="compress_zlib"`` in the EAL options, which will call ``rte_vdev_init()`` internally.
+
+The following parameter (optional) can be provided in the previous two calls:
+
+* ``socket_id:`` Specify the socket where the memory for the device is going to be allocated
+  (by default, socket_id will be the socket where the core that is creating the PMD is running on).
-- 
2.9.5

Re: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shally Verma
> Sent: Monday, July 23, 2018 3:51 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> mchalla@caviumnetworks.com; Sunila Sahu
> <sunila.sahu@caviumnetworks.com>; Ashish Gupta
> <ashish.gupta@caviumnetworks.com>
> Subject: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> 
> Add zlib pmd feature support and user guide with build and run instructions
> 
> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> ---
>  MAINTAINERS                               |  2 +
>  doc/guides/compressdevs/features/zlib.ini | 11 +++++
>  doc/guides/compressdevs/index.rst         |  1 +
>  doc/guides/compressdevs/zlib.rst          | 69
> +++++++++++++++++++++++++++++++
>  4 files changed, 83 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index ca27c6f..7e3c450 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -875,6 +875,8 @@ F: drivers/common/qat/  ZLIB
>  M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>  F: drivers/compress/zlib/
> +F: doc/guides/compressdevs/zlib.rst
> +F: doc/guides/compressdevs/features/zlib.ini
> 
>  Eventdev Drivers
>  ----------------
> diff --git a/doc/guides/compressdevs/features/zlib.ini
> b/doc/guides/compressdevs/features/zlib.ini
> new file mode 100644
> index 0000000..c794643
> --- /dev/null
> +++ b/doc/guides/compressdevs/features/zlib.ini
> @@ -0,0 +1,11 @@
> +;
> +; Refer to default.ini for the full list of available PMD features.
> +;
> +; Supported features of 'ZLIB' compression driver.
> +;
> +[Features]
> +Pass-through   = Y
> +Deflate        = Y
> +Fixed          = Y
> +Dynamic        = Y
> +OOP SGL In SGL Out  = Y

I assume that you support also "OOP SGL In LB Out" and "OOP LB In SGL Out", right?

Re: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 23 July 2018 23:28
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>
>Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
>
>External Email
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shally Verma
>> Sent: Monday, July 23, 2018 3:51 PM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> mchalla@caviumnetworks.com; Sunila Sahu
>> <sunila.sahu@caviumnetworks.com>; Ashish Gupta
>> <ashish.gupta@caviumnetworks.com>
>> Subject: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
>>
>> Add zlib pmd feature support and user guide with build and run instructions
>>
>> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>> ---
>>  MAINTAINERS                               |  2 +
>>  doc/guides/compressdevs/features/zlib.ini | 11 +++++
>>  doc/guides/compressdevs/index.rst         |  1 +
>>  doc/guides/compressdevs/zlib.rst          | 69
>> +++++++++++++++++++++++++++++++
>>  4 files changed, 83 insertions(+)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index ca27c6f..7e3c450 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -875,6 +875,8 @@ F: drivers/common/qat/  ZLIB
>>  M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>>  F: drivers/compress/zlib/
>> +F: doc/guides/compressdevs/zlib.rst
>> +F: doc/guides/compressdevs/features/zlib.ini
>>
>>  Eventdev Drivers
>>  ----------------
>> diff --git a/doc/guides/compressdevs/features/zlib.ini
>> b/doc/guides/compressdevs/features/zlib.ini
>> new file mode 100644
>> index 0000000..c794643
>> --- /dev/null
>> +++ b/doc/guides/compressdevs/features/zlib.ini
>> @@ -0,0 +1,11 @@
>> +;
>> +; Refer to default.ini for the full list of available PMD features.
>> +;
>> +; Supported features of 'ZLIB' compression driver.
>> +;
>> +[Features]
>> +Pass-through   = Y
>> +Deflate        = Y
>> +Fixed          = Y
>> +Dynamic        = Y
>> +OOP SGL In SGL Out  = Y
>
>I assume that you support also "OOP SGL In LB Out" and "OOP LB In SGL Out", right?
yes, but untested thus not claiming.

Thanks
Shally

Re: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide

[De Lara Guarch, Pablo]

Hi Shally,

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Monday, July 23, 2018 7:00 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> Gupta, Ashish <Ashish.Gupta@cavium.com>
> Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> 
> 
> 
> >-----Original Message-----
> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >Sent: 23 July 2018 23:28
> >To: Verma, Shally <Shally.Verma@cavium.com>
> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> >Gupta, Ashish <Ashish.Gupta@cavium.com>
> >Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> >
> >External Email
> >
> >> -----Original Message-----
> >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shally Verma
> >> Sent: Monday, July 23, 2018 3:51 PM
> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> >> mchalla@caviumnetworks.com; Sunila Sahu
> >> <sunila.sahu@caviumnetworks.com>; Ashish Gupta
> >> <ashish.gupta@caviumnetworks.com>
> >> Subject: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> >>
> >> Add zlib pmd feature support and user guide with build and run
> >> instructions
> >>
> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> >> ---
> >>  MAINTAINERS                               |  2 +
> >>  doc/guides/compressdevs/features/zlib.ini | 11 +++++
> >>  doc/guides/compressdevs/index.rst         |  1 +
> >>  doc/guides/compressdevs/zlib.rst          | 69
> >> +++++++++++++++++++++++++++++++
> >>  4 files changed, 83 insertions(+)
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS index ca27c6f..7e3c450 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -875,6 +875,8 @@ F: drivers/common/qat/  ZLIB
> >>  M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> >>  F: drivers/compress/zlib/
> >> +F: doc/guides/compressdevs/zlib.rst
> >> +F: doc/guides/compressdevs/features/zlib.ini
> >>
> >>  Eventdev Drivers
> >>  ----------------
> >> diff --git a/doc/guides/compressdevs/features/zlib.ini
> >> b/doc/guides/compressdevs/features/zlib.ini
> >> new file mode 100644
> >> index 0000000..c794643
> >> --- /dev/null
> >> +++ b/doc/guides/compressdevs/features/zlib.ini
> >> @@ -0,0 +1,11 @@
> >> +;
> >> +; Refer to default.ini for the full list of available PMD features.
> >> +;
> >> +; Supported features of 'ZLIB' compression driver.
> >> +;
> >> +[Features]
> >> +Pass-through   = Y
> >> +Deflate        = Y
> >> +Fixed          = Y
> >> +Dynamic        = Y
> >> +OOP SGL In SGL Out  = Y
> >
> >I assume that you support also "OOP SGL In LB Out" and "OOP LB In SGL Out",
> right?
> yes, but untested thus not claiming.

Right, but knowing that a Linear buffer is basically an SGL with just one segment,
I think it is safe to say that you support these two other cases.
The only reason why we have them is in case, you only support one of them, but you don't support SGL in SGL Out).

I won't have time to extend the test for those combinations in this release, I hope I can do that in the next one,
but as said, I think it is safe to claim that this PMD supports all the cases.

> 
> Thanks
> Shally

Re: [dpdk-dev] [PATCH v4 1/5] compress/zlib: add ZLIB PMD

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
> Sent: Monday, July 23, 2018 3:51 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> mchalla@caviumnetworks.com; Ashish Gupta
> <ashish.gupta@caviumnetworks.com>; Sunila Sahu
> <sunila.sahu@caviumnetworks.com>
> Subject: [PATCH v4 1/5] compress/zlib: add ZLIB PMD
> 
> From: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> 
> Add initial PMD setup routines in compressdev framework. ZLIB PMD appears as
> virtual compression device. User would need to install zlib prior to enabling this
> PMD.
> 
> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>

...

> +++ b/drivers/compress/zlib/zlib_pmd.c

...

> +static int
> +zlib_probe(struct rte_vdev_device *vdev) {
> +	struct rte_compressdev_pmd_init_params init_params = {
> +		"",
> +		rte_socket_id()
> +	};
> +	const char *name;
> +	const char *input_args;
> +
> +	name = rte_vdev_device_name(vdev);
> +
> +	if (name == NULL)
> +		return -EINVAL;
> +	input_args = rte_vdev_device_args(vdev);
> +	rte_compressdev_pmd_parse_input_args(&init_params, input_args);

Need to check the return value of this function.

Re: [dpdk-dev] [PATCH v4 2/5] compress/zlib: add device PMD ops

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
> Sent: Monday, July 23, 2018 3:51 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> mchalla@caviumnetworks.com; Ashish Gupta
> <ashish.gupta@caviumnetworks.com>; Sunila Sahu
> <sunila.sahu@caviumnetworks.com>
> Subject: [PATCH v4 2/5] compress/zlib: add device PMD ops
> 
> From: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> 
> Implement device configure and queue pair setup PMD ops
> 
> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>

...

> --- /dev/null
> +++ b/drivers/compress/zlib/zlib_pmd_ops.c
> @@ -0,0 +1,238 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2018 Cavium Networks
> + */
> +
> +#include <string.h>
> +
> +#include <rte_common.h>
> +#include <rte_malloc.h>
> +
> +#include "zlib_pmd_private.h"
> +
> +static const struct rte_compressdev_capabilities zlib_pmd_capabilities[] = {
> +	{   /* Deflate */
> +		.algo = RTE_COMP_ALGO_DEFLATE,
> +		.comp_feature_flags =
> (RTE_COMP_FF_NONCOMPRESSED_BLOCKS |
> +					RTE_COMP_FF_HUFFMAN_FIXED |
> +					RTE_COMP_FF_HUFFMAN_DYNAMIC |
> +
> 	RTE_COMP_FF_OOP_SGL_IN_SGL_OUT),

As said in the document patch, I think you should add the other two SGL cases here.

Re: [dpdk-dev] [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
> Sent: Monday, July 23, 2018 3:51 PM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> mchalla@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>;
> Sunila Sahu <sunila.sahu@caviumnetworks.com>; Ashish Gupta
> <ashish.gupta@caviumnetworks.com>
> Subject: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
> 
> From: Sunila Sahu <ssahu@caviumnetworks.com>
> 
> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> ---
>  drivers/compress/zlib/zlib_pmd.c | 255
> ++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 254 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
> index 47bc73d..dc1e230 100644
> --- a/drivers/compress/zlib/zlib_pmd.c
> +++ b/drivers/compress/zlib/zlib_pmd.c
> @@ -7,7 +7,214 @@
> 
>  #include "zlib_pmd_private.h"
> 
> -/** Parse comp xform and set private xform/stream parameters */
> +/** Compute next mbuf in the list, assign data buffer and length,
> + *  returns 0 if mbuf is NULL
> + */
> +#define COMPUTE_BUF(mbuf, data, len)		\
> +		((mbuf = mbuf->next) ?		\
> +		(data = rte_pktmbuf_mtod(mbuf, uint8_t *)),	\
> +		(len = rte_pktmbuf_data_len(mbuf)) : 0)
> +
> +static void
> +process_zlib_deflate(struct rte_comp_op *op, z_stream *strm) {

...

> +	/* Update z_stream with the inputs provided by application */
> +	strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
> +			op->src.offset);

This is assuming that src buffer is a linear buffer or that offset won't be large enough to cross boundaries between segments.
If an SGL is passed and offset is bigger than the first segment, next_in should point at a different segment, with the remaining part of the offset in that segment
(look at ISA-L SGL patch: http://patches.dpdk.org/patch/43283/). Same applies to avail_in, next_out and avail_out.

> +
> +	strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
> +
> +	strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
> +			op->dst.offset);
> +
> +	strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
> +

...

> +	strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
> +			op->src.offset);
> +
> +	strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
> +
> +	strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
> +			op->dst.offset);
> +
> +	strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;

Same comments as above (compression).

...

> +static inline int
> +process_zlib_op(struct zlib_qp *qp, struct rte_comp_op *op) {
> +	struct zlib_stream *stream;
> +	struct zlib_priv_xform *private_xform;
> +
> +	if ((op->op_type == RTE_COMP_OP_STATEFUL) ||
> +	    (op->src.offset > rte_pktmbuf_data_len(op->m_src)) ||
> +	    (op->dst.offset > rte_pktmbuf_data_len(op->m_dst))) {

Since m_src and m_dst could be SGLs, pkt_len should be checked, instead of data_len (which would be only for single segment).
Also, you should check the length too, in case of source buffers (src.offset + src.length > m_src->pkt_len).

Lastly, the two lines after the first if line should have double indentation to distinguish clearly against the body of the if.
If line is too long, consider storing the length of the buffers in variables.


> +		op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
> +		ZLIB_PMD_ERR("Invalid source or destination buffers or "
> +			     "invalid Operation requested\n");

Re: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 24 July 2018 02:49
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>
>Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
>
>External Email
>
>Hi Shally,
>
>> -----Original Message-----
>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>> Sent: Monday, July 23, 2018 7:00 PM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
>> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
>> Gupta, Ashish <Ashish.Gupta@cavium.com>
>> Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
>>
>>
>>
>> >-----Original Message-----
>> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> >Sent: 23 July 2018 23:28
>> >To: Verma, Shally <Shally.Verma@cavium.com>
>> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
>> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
>> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
>> >Gupta, Ashish <Ashish.Gupta@cavium.com>
>> >Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
>> >
>> >External Email
>> >
>> >> -----Original Message-----
>> >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shally Verma
>> >> Sent: Monday, July 23, 2018 3:51 PM
>> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> >> mchalla@caviumnetworks.com; Sunila Sahu
>> >> <sunila.sahu@caviumnetworks.com>; Ashish Gupta
>> >> <ashish.gupta@caviumnetworks.com>
>> >> Subject: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
>> >>
>> >> Add zlib pmd feature support and user guide with build and run
>> >> instructions
>> >>
>> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>> >> ---
>> >>  MAINTAINERS                               |  2 +
>> >>  doc/guides/compressdevs/features/zlib.ini | 11 +++++
>> >>  doc/guides/compressdevs/index.rst         |  1 +
>> >>  doc/guides/compressdevs/zlib.rst          | 69
>> >> +++++++++++++++++++++++++++++++
>> >>  4 files changed, 83 insertions(+)
>> >>
>> >> diff --git a/MAINTAINERS b/MAINTAINERS index ca27c6f..7e3c450 100644
>> >> --- a/MAINTAINERS
>> >> +++ b/MAINTAINERS
>> >> @@ -875,6 +875,8 @@ F: drivers/common/qat/  ZLIB
>> >>  M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> >>  F: drivers/compress/zlib/
>> >> +F: doc/guides/compressdevs/zlib.rst
>> >> +F: doc/guides/compressdevs/features/zlib.ini
>> >>
>> >>  Eventdev Drivers
>> >>  ----------------
>> >> diff --git a/doc/guides/compressdevs/features/zlib.ini
>> >> b/doc/guides/compressdevs/features/zlib.ini
>> >> new file mode 100644
>> >> index 0000000..c794643
>> >> --- /dev/null
>> >> +++ b/doc/guides/compressdevs/features/zlib.ini
>> >> @@ -0,0 +1,11 @@
>> >> +;
>> >> +; Refer to default.ini for the full list of available PMD features.
>> >> +;
>> >> +; Supported features of 'ZLIB' compression driver.
>> >> +;
>> >> +[Features]
>> >> +Pass-through   = Y
>> >> +Deflate        = Y
>> >> +Fixed          = Y
>> >> +Dynamic        = Y
>> >> +OOP SGL In SGL Out  = Y
>> >
>> >I assume that you support also "OOP SGL In LB Out" and "OOP LB In SGL Out",
>> right?
>> yes, but untested thus not claiming.
>
>Right, but knowing that a Linear buffer is basically an SGL with just one segment,
>I think it is safe to say that you support these two other cases.
>The only reason why we have them is in case, you only support one of them, but you don't support SGL in SGL Out).
>
>I won't have time to extend the test for those combinations in this release, I hope I can do that in the next one,
>but as said, I think it is safe to claim that this PMD supports all the cases.
Agree. However I would still wait to test them before claiming support.

Thanks
Shally
>
>>
>> Thanks
>> Shally

Re: [dpdk-dev] [PATCH v4 1/5] compress/zlib: add ZLIB PMD

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 24 July 2018 03:29
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>
>Subject: RE: [PATCH v4 1/5] compress/zlib: add ZLIB PMD
>
...
>
>> +++ b/drivers/compress/zlib/zlib_pmd.c
>
>...
>
>> +static int
>> +zlib_probe(struct rte_vdev_device *vdev) {
>> +     struct rte_compressdev_pmd_init_params init_params = {
>> +             "",
>> +             rte_socket_id()
>> +     };
>> +     const char *name;
>> +     const char *input_args;
>> +
>> +     name = rte_vdev_device_name(vdev);
>> +
>> +     if (name == NULL)
>> +             return -EINVAL;
>> +     input_args = rte_vdev_device_args(vdev);
>> +     rte_compressdev_pmd_parse_input_args(&init_params, input_args);
>
>Need to check the return value of this function.
Ack.

Thanks
Shally

Re: [dpdk-dev] [PATCH v4 2/5] compress/zlib: add device PMD ops

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 24 July 2018 03:33
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>
>Subject: RE: [PATCH v4 2/5] compress/zlib: add device PMD ops
>
>External Email
>
>> -----Original Message-----
>> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
>> Sent: Monday, July 23, 2018 3:51 PM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> mchalla@caviumnetworks.com; Ashish Gupta
>> <ashish.gupta@caviumnetworks.com>; Sunila Sahu
>> <sunila.sahu@caviumnetworks.com>
>> Subject: [PATCH v4 2/5] compress/zlib: add device PMD ops
>>
>> From: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>>
>> Implement device configure and queue pair setup PMD ops
>>
>> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>
>...
>
>> --- /dev/null
>> +++ b/drivers/compress/zlib/zlib_pmd_ops.c
>> @@ -0,0 +1,238 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2018 Cavium Networks
>> + */
>> +
>> +#include <string.h>
>> +
>> +#include <rte_common.h>
>> +#include <rte_malloc.h>
>> +
>> +#include "zlib_pmd_private.h"
>> +
>> +static const struct rte_compressdev_capabilities zlib_pmd_capabilities[] = {
>> +     {   /* Deflate */
>> +             .algo = RTE_COMP_ALGO_DEFLATE,
>> +             .comp_feature_flags =
>> (RTE_COMP_FF_NONCOMPRESSED_BLOCKS |
>> +                                     RTE_COMP_FF_HUFFMAN_FIXED |
>> +                                     RTE_COMP_FF_HUFFMAN_DYNAMIC |
>> +
>>       RTE_COMP_FF_OOP_SGL_IN_SGL_OUT),
>
>As said in the document patch, I think you should add the other two SGL cases here.
Would give thought to it however, prefer to hold back on their support until we evaluate them with proper testcases.

Thanks
Shally

Re: [dpdk-dev] [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 24 July 2018 03:55
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish
><Ashish.Gupta@cavium.com>
>Subject: RE: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>
>External Email
>
>> -----Original Message-----
>> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
>> Sent: Monday, July 23, 2018 3:51 PM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> mchalla@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>;
>> Sunila Sahu <sunila.sahu@caviumnetworks.com>; Ashish Gupta
>> <ashish.gupta@caviumnetworks.com>
>> Subject: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>>
>> From: Sunila Sahu <ssahu@caviumnetworks.com>
>>
>> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>> ---
>>  drivers/compress/zlib/zlib_pmd.c | 255
>> ++++++++++++++++++++++++++++++++++++++-
>>  1 file changed, 254 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/compress/zlib/zlib_pmd.c b/drivers/compress/zlib/zlib_pmd.c
>> index 47bc73d..dc1e230 100644
>> --- a/drivers/compress/zlib/zlib_pmd.c
>> +++ b/drivers/compress/zlib/zlib_pmd.c
>> @@ -7,7 +7,214 @@
>>
>>  #include "zlib_pmd_private.h"
>>
>> -/** Parse comp xform and set private xform/stream parameters */
>> +/** Compute next mbuf in the list, assign data buffer and length,
>> + *  returns 0 if mbuf is NULL
>> + */
>> +#define COMPUTE_BUF(mbuf, data, len)         \
>> +             ((mbuf = mbuf->next) ?          \
>> +             (data = rte_pktmbuf_mtod(mbuf, uint8_t *)),     \
>> +             (len = rte_pktmbuf_data_len(mbuf)) : 0)
>> +
>> +static void
>> +process_zlib_deflate(struct rte_comp_op *op, z_stream *strm) {
>
>...
>
>> +     /* Update z_stream with the inputs provided by application */
>> +     strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
>> +                     op->src.offset);
>
>This is assuming that src buffer is a linear buffer or that offset won't be large enough to cross boundaries between segments.
>If an SGL is passed and offset is bigger than the first segment, next_in should point at a different segment, with the remaining part of
>the offset in that segment
>(look at ISA-L SGL patch: http://patches.dpdk.org/patch/43283/). Same applies to avail_in, next_out and avail_out.

[Shally] as per my last knowledge, offset was expected to be belonging only to the first segment in chained mbuf. Isn't that the case anymore? Did I miss any update on its definition?
We had added the logic earlier that you're suggesting but removed that later, as I understood clarification about offset falling into any segment is still pending.

Thanks
Shally

>
>> +
>> +     strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
>> +
>> +     strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
>> +                     op->dst.offset);
>> +
>> +     strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
>> +
>
>...
>
>> +     strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
>> +                     op->src.offset);
>> +
>> +     strm->avail_in = rte_pktmbuf_data_len(mbuf_src) - op->src.offset;
>> +
>> +     strm->next_out = rte_pktmbuf_mtod_offset(mbuf_dst, uint8_t *,
>> +                     op->dst.offset);
>> +
>> +     strm->avail_out = rte_pktmbuf_data_len(mbuf_dst) - op->dst.offset;
>
>Same comments as above (compression).
>
>...
>
>> +static inline int
>> +process_zlib_op(struct zlib_qp *qp, struct rte_comp_op *op) {
>> +     struct zlib_stream *stream;
>> +     struct zlib_priv_xform *private_xform;
>> +
>> +     if ((op->op_type == RTE_COMP_OP_STATEFUL) ||
>> +         (op->src.offset > rte_pktmbuf_data_len(op->m_src)) ||
>> +         (op->dst.offset > rte_pktmbuf_data_len(op->m_dst))) {
>
>Since m_src and m_dst could be SGLs, pkt_len should be checked, instead of data_len (which would be only for single segment).
>Also, you should check the length too, in case of source buffers (src.offset + src.length > m_src->pkt_len).
>
>Lastly, the two lines after the first if line should have double indentation to distinguish clearly against the body of the if.
>If line is too long, consider storing the length of the buffers in variables.
>
>
>> +             op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
>> +             ZLIB_PMD_ERR("Invalid source or destination buffers or "
>> +                          "invalid Operation requested\n");

Re: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Tuesday, July 24, 2018 6:32 AM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> Gupta, Ashish <Ashish.Gupta@cavium.com>
> Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> 
> 
> 
> >-----Original Message-----
> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >Sent: 24 July 2018 02:49
> >To: Verma, Shally <Shally.Verma@cavium.com>
> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> >Gupta, Ashish <Ashish.Gupta@cavium.com>
> >Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> >
> >External Email
> >
> >Hi Shally,
> >
> >> -----Original Message-----
> >> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> >> Sent: Monday, July 23, 2018 7:00 PM
> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> Cc: dev@dpdk.org; Athreya, Narayana Prasad
> >> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> >> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> >> Gupta, Ashish <Ashish.Gupta@cavium.com>
> >> Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> >>
> >>
> >>
> >> >-----Original Message-----
> >> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> >Sent: 23 July 2018 23:28
> >> >To: Verma, Shally <Shally.Verma@cavium.com>
> >> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
> >> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> >> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> >> >Gupta, Ashish <Ashish.Gupta@cavium.com>
> >> >Subject: RE: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> >> >
> >> >External Email
> >> >
> >> >> -----Original Message-----
> >> >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shally Verma
> >> >> Sent: Monday, July 23, 2018 3:51 PM
> >> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> >> >> mchalla@caviumnetworks.com; Sunila Sahu
> >> >> <sunila.sahu@caviumnetworks.com>; Ashish Gupta
> >> >> <ashish.gupta@caviumnetworks.com>
> >> >> Subject: [dpdk-dev] [PATCH v4 5/5] doc: add ZLIB PMD guide
> >> >>
> >> >> Add zlib pmd feature support and user guide with build and run
> >> >> instructions
> >> >>
> >> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> >> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> >> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> >> >> ---
> >> >>  MAINTAINERS                               |  2 +
> >> >>  doc/guides/compressdevs/features/zlib.ini | 11 +++++
> >> >>  doc/guides/compressdevs/index.rst         |  1 +
> >> >>  doc/guides/compressdevs/zlib.rst          | 69
> >> >> +++++++++++++++++++++++++++++++
> >> >>  4 files changed, 83 insertions(+)
> >> >>
> >> >> diff --git a/MAINTAINERS b/MAINTAINERS index ca27c6f..7e3c450
> >> >> 100644
> >> >> --- a/MAINTAINERS
> >> >> +++ b/MAINTAINERS
> >> >> @@ -875,6 +875,8 @@ F: drivers/common/qat/  ZLIB
> >> >>  M: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> >> >>  F: drivers/compress/zlib/
> >> >> +F: doc/guides/compressdevs/zlib.rst
> >> >> +F: doc/guides/compressdevs/features/zlib.ini
> >> >>
> >> >>  Eventdev Drivers
> >> >>  ----------------
> >> >> diff --git a/doc/guides/compressdevs/features/zlib.ini
> >> >> b/doc/guides/compressdevs/features/zlib.ini
> >> >> new file mode 100644
> >> >> index 0000000..c794643
> >> >> --- /dev/null
> >> >> +++ b/doc/guides/compressdevs/features/zlib.ini
> >> >> @@ -0,0 +1,11 @@
> >> >> +;
> >> >> +; Refer to default.ini for the full list of available PMD features.
> >> >> +;
> >> >> +; Supported features of 'ZLIB' compression driver.
> >> >> +;
> >> >> +[Features]
> >> >> +Pass-through   = Y
> >> >> +Deflate        = Y
> >> >> +Fixed          = Y
> >> >> +Dynamic        = Y
> >> >> +OOP SGL In SGL Out  = Y
> >> >
> >> >I assume that you support also "OOP SGL In LB Out" and "OOP LB In
> >> >SGL Out",
> >> right?
> >> yes, but untested thus not claiming.
> >
> >Right, but knowing that a Linear buffer is basically an SGL with just
> >one segment, I think it is safe to say that you support these two other cases.
> >The only reason why we have them is in case, you only support one of them,
> but you don't support SGL in SGL Out).
> >
> >I won't have time to extend the test for those combinations in this
> >release, I hope I can do that in the next one, but as said, I think it is safe to
> claim that this PMD supports all the cases.
> Agree. However I would still wait to test them before claiming support.

Sure, no problem.

Thanks,
Pablo

> 
> Thanks
> Shally
> >
> >>
> >> Thanks
> >> Shally

Re: [dpdk-dev] [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Tuesday, July 24, 2018 8:45 AM
> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Cc: dev@dpdk.org; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu,
> Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>
> Subject: RE: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
> 
> 
> 
> >-----Original Message-----
> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >Sent: 24 July 2018 03:55
> >To: Verma, Shally <Shally.Verma@cavium.com>
> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
> >Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish
> ><Ashish.Gupta@cavium.com>
> >Subject: RE: [PATCH v4 4/5] compress/zlib: support burst
> >enqueue/dequeue
> >
> >External Email
> >
> >> -----Original Message-----
> >> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
> >> Sent: Monday, July 23, 2018 3:51 PM
> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
> >> mchalla@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>;
> >> Sunila Sahu <sunila.sahu@caviumnetworks.com>; Ashish Gupta
> >> <ashish.gupta@caviumnetworks.com>
> >> Subject: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
> >>
> >> From: Sunila Sahu <ssahu@caviumnetworks.com>
> >>
> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
> >> ---
> >>  drivers/compress/zlib/zlib_pmd.c | 255
> >> ++++++++++++++++++++++++++++++++++++++-
> >>  1 file changed, 254 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/compress/zlib/zlib_pmd.c
> >> b/drivers/compress/zlib/zlib_pmd.c
> >> index 47bc73d..dc1e230 100644
> >> --- a/drivers/compress/zlib/zlib_pmd.c
> >> +++ b/drivers/compress/zlib/zlib_pmd.c
> >> @@ -7,7 +7,214 @@
> >>
> >>  #include "zlib_pmd_private.h"
> >>
> >> -/** Parse comp xform and set private xform/stream parameters */
> >> +/** Compute next mbuf in the list, assign data buffer and length,
> >> + *  returns 0 if mbuf is NULL
> >> + */
> >> +#define COMPUTE_BUF(mbuf, data, len)         \
> >> +             ((mbuf = mbuf->next) ?          \
> >> +             (data = rte_pktmbuf_mtod(mbuf, uint8_t *)),     \
> >> +             (len = rte_pktmbuf_data_len(mbuf)) : 0)
> >> +
> >> +static void
> >> +process_zlib_deflate(struct rte_comp_op *op, z_stream *strm) {
> >
> >...
> >
> >> +     /* Update z_stream with the inputs provided by application */
> >> +     strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
> >> +                     op->src.offset);
> >
> >This is assuming that src buffer is a linear buffer or that offset won't be large
> enough to cross boundaries between segments.
> >If an SGL is passed and offset is bigger than the first segment,
> >next_in should point at a different segment, with the remaining part of
> >the offset in that segment (look at ISA-L SGL patch:
> http://patches.dpdk.org/patch/43283/). Same applies to avail_in, next_out and
> avail_out.
> 
> [Shally] as per my last knowledge, offset was expected to be belonging only to
> the first segment in chained mbuf. Isn't that the case anymore? Did I miss any
> update on its definition?
> We had added the logic earlier that you're suggesting but removed that later, as
> I understood clarification about offset falling into any segment is still pending.
> 

According to the comments:

                uint32_t offset;
                /**< Starting point for compression or decompression,
                 * specified as number of bytes from start of packet in
                 * source buffer.
                 * This offset starts from the first segment
                 * of the buffer, in case the m_src is a chain of mbufs.

It says that the offset starts from the first segment, but not that
it is only applicable for the first segment.
>From my point of view, an SGL should be seen like a contiguous (linear) buffer,
so if the offset crosses multiple segments, it is still valid, as it is still part of the buffer.

Thanks,
Pablo

> Thanks
> Shally

Re: [dpdk-dev] [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 24 July 2018 13:24
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish
><Ashish.Gupta@cavium.com>
>Subject: RE: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>
>External Email
>
>> -----Original Message-----
>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>> Sent: Tuesday, July 24, 2018 8:45 AM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
>> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu,
>> Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>
>> Subject: RE: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>>
>>
>>
>> >-----Original Message-----
>> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> >Sent: 24 July 2018 03:55
>> >To: Verma, Shally <Shally.Verma@cavium.com>
>> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
>> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
>> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
>> >Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish
>> ><Ashish.Gupta@cavium.com>
>> >Subject: RE: [PATCH v4 4/5] compress/zlib: support burst
>> >enqueue/dequeue
>> >
>> >External Email
>> >
>> >> -----Original Message-----
>> >> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
>> >> Sent: Monday, July 23, 2018 3:51 PM
>> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> >> mchalla@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>;
>> >> Sunila Sahu <sunila.sahu@caviumnetworks.com>; Ashish Gupta
>> >> <ashish.gupta@caviumnetworks.com>
>> >> Subject: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>> >>
>> >> From: Sunila Sahu <ssahu@caviumnetworks.com>
>> >>
>> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>> >> ---
>> >>  drivers/compress/zlib/zlib_pmd.c | 255
>> >> ++++++++++++++++++++++++++++++++++++++-
>> >>  1 file changed, 254 insertions(+), 1 deletion(-)
>> >>
>> >> diff --git a/drivers/compress/zlib/zlib_pmd.c
>> >> b/drivers/compress/zlib/zlib_pmd.c
>> >> index 47bc73d..dc1e230 100644
>> >> --- a/drivers/compress/zlib/zlib_pmd.c
>> >> +++ b/drivers/compress/zlib/zlib_pmd.c
>> >> @@ -7,7 +7,214 @@
>> >>
>> >>  #include "zlib_pmd_private.h"
>> >>
>> >> -/** Parse comp xform and set private xform/stream parameters */
>> >> +/** Compute next mbuf in the list, assign data buffer and length,
>> >> + *  returns 0 if mbuf is NULL
>> >> + */
>> >> +#define COMPUTE_BUF(mbuf, data, len)         \
>> >> +             ((mbuf = mbuf->next) ?          \
>> >> +             (data = rte_pktmbuf_mtod(mbuf, uint8_t *)),     \
>> >> +             (len = rte_pktmbuf_data_len(mbuf)) : 0)
>> >> +
>> >> +static void
>> >> +process_zlib_deflate(struct rte_comp_op *op, z_stream *strm) {
>> >
>> >...
>> >
>> >> +     /* Update z_stream with the inputs provided by application */
>> >> +     strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
>> >> +                     op->src.offset);
>> >
>> >This is assuming that src buffer is a linear buffer or that offset won't be large
>> enough to cross boundaries between segments.
>> >If an SGL is passed and offset is bigger than the first segment,
>> >next_in should point at a different segment, with the remaining part of
>> >the offset in that segment (look at ISA-L SGL patch:
>> http://patches.dpdk.org/patch/43283/). Same applies to avail_in, next_out and
>> avail_out.
>>
>> [Shally] as per my last knowledge, offset was expected to be belonging only to
>> the first segment in chained mbuf. Isn't that the case anymore? Did I miss any
>> update on its definition?
>> We had added the logic earlier that you're suggesting but removed that later, as
>> I understood clarification about offset falling into any segment is still pending.
>>
>
>According to the comments:
>
>                uint32_t offset;
>                /**< Starting point for compression or decompression,
>                 * specified as number of bytes from start of packet in
>                 * source buffer.
>                 * This offset starts from the first segment
>                 * of the buffer, in case the m_src is a chain of mbufs.
>
>It says that the offset starts from the first segment, but not that
>it is only applicable for the first segment.
>From my point of view, an SGL should be seen like a contiguous (linear) buffer,
>so if the offset crosses multiple segments, it is still valid, as it is still part of the buffer.
Ya. not saying that offset cannot belong to different segments. But I was misinformed here and assumed we're limiting to first one.
Will take care of it now.

Thanks
Shally
>
>Thanks,
>Pablo
>
>> Thanks
>> Shally

Re: [dpdk-dev] [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue

[Verma, Shally]

>-----Original Message-----
>From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Sent: 24 July 2018 13:24
>To: Verma, Shally <Shally.Verma@cavium.com>
>Cc: dev@dpdk.org; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish
><Ashish.Gupta@cavium.com>
>Subject: RE: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>
>External Email
>
>> -----Original Message-----
>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>> Sent: Tuesday, July 24, 2018 8:45 AM
>> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> Cc: dev@dpdk.org; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
>> <Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Sahu,
>> Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>
>> Subject: RE: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>>
>>
>>
>> >-----Original Message-----
>> >From: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> >Sent: 24 July 2018 03:55
>> >To: Verma, Shally <Shally.Verma@cavium.com>
>> >Cc: dev@dpdk.org; Athreya, Narayana Prasad
>> ><NarayanaPrasad.Athreya@cavium.com>; Challa, Mahipal
>> ><Mahipal.Challa@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>;
>> >Sahu, Sunila <Sunila.Sahu@cavium.com>; Gupta, Ashish
>> ><Ashish.Gupta@cavium.com>
>> >Subject: RE: [PATCH v4 4/5] compress/zlib: support burst
>> >enqueue/dequeue
>> >
>> >External Email
>> >
>> >> -----Original Message-----
>> >> From: Shally Verma [mailto:shally.verma@caviumnetworks.com]
>> >> Sent: Monday, July 23, 2018 3:51 PM
>> >> To: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>> >> Cc: dev@dpdk.org; pathreya@caviumnetworks.com;
>> >> mchalla@caviumnetworks.com; Sunila Sahu <ssahu@caviumnetworks.com>;
>> >> Sunila Sahu <sunila.sahu@caviumnetworks.com>; Ashish Gupta
>> >> <ashish.gupta@caviumnetworks.com>
>> >> Subject: [PATCH v4 4/5] compress/zlib: support burst enqueue/dequeue
>> >>
>> >> From: Sunila Sahu <ssahu@caviumnetworks.com>
>> >>
>> >> Signed-off-by: Sunila Sahu <sunila.sahu@caviumnetworks.com>
>> >> Signed-off-by: Shally Verma <shally.verma@caviumnetworks.com>
>> >> Signed-off-by: Ashish Gupta <ashish.gupta@caviumnetworks.com>
>> >> ---
>> >>  drivers/compress/zlib/zlib_pmd.c | 255
>> >> ++++++++++++++++++++++++++++++++++++++-
>> >>  1 file changed, 254 insertions(+), 1 deletion(-)
>> >>
>> >> diff --git a/drivers/compress/zlib/zlib_pmd.c
>> >> b/drivers/compress/zlib/zlib_pmd.c
>> >> index 47bc73d..dc1e230 100644
>> >> --- a/drivers/compress/zlib/zlib_pmd.c
>> >> +++ b/drivers/compress/zlib/zlib_pmd.c
>> >> @@ -7,7 +7,214 @@
>> >>
>> >>  #include "zlib_pmd_private.h"
>> >>
>> >> -/** Parse comp xform and set private xform/stream parameters */
>> >> +/** Compute next mbuf in the list, assign data buffer and length,
>> >> + *  returns 0 if mbuf is NULL
>> >> + */
>> >> +#define COMPUTE_BUF(mbuf, data, len)         \
>> >> +             ((mbuf = mbuf->next) ?          \
>> >> +             (data = rte_pktmbuf_mtod(mbuf, uint8_t *)),     \
>> >> +             (len = rte_pktmbuf_data_len(mbuf)) : 0)
>> >> +
>> >> +static void
>> >> +process_zlib_deflate(struct rte_comp_op *op, z_stream *strm) {
>> >
>> >...
>> >
>> >> +     /* Update z_stream with the inputs provided by application */
>> >> +     strm->next_in = rte_pktmbuf_mtod_offset(mbuf_src, uint8_t *,
>> >> +                     op->src.offset);
>> >
>> >This is assuming that src buffer is a linear buffer or that offset won't be large
>> enough to cross boundaries between segments.
>> >If an SGL is passed and offset is bigger than the first segment,
>> >next_in should point at a different segment, with the remaining part of
>> >the offset in that segment (look at ISA-L SGL patch:
>> http://patches.dpdk.org/patch/43283/). Same applies to avail_in, next_out and
>> avail_out.
>>
>> [Shally] as per my last knowledge, offset was expected to be belonging only to
>> the first segment in chained mbuf. Isn't that the case anymore? Did I miss any
>> update on its definition?
>> We had added the logic earlier that you're suggesting but removed that later, as
>> I understood clarification about offset falling into any segment is still pending.
>>
>
>According to the comments:
>
>                uint32_t offset;
>                /**< Starting point for compression or decompression,
>                 * specified as number of bytes from start of packet in
>                 * source buffer.
>                 * This offset starts from the first segment
>                 * of the buffer, in case the m_src is a chain of mbufs.
>
>It says that the offset starts from the first segment, but not that
>it is only applicable for the first segment.
>From my point of view, an SGL should be seen like a contiguous (linear) buffer,
>so if the offset crosses multiple segments, it is still valid, as it is still part of the buffer.
>
Also believe doc need an update, it's kind of misleading "start from first segment", should replace by "in case of chained mbuf, it can fall into any segments".

Thanks
Shally
>Thanks,
>Pablo
>
>> Thanks
>> Shally