[PATCH v1] event/dlb2: revise QE Weight Feature

[Abdullah Sevincer <abdullah.sevincer@intel.com>]

Revise QE weight feature to enable from command line
just passing  a flag. If QE weight feature is enabled
simply port cq weight will be same as dequeue depth.

Also, update DLB documentation for revised QE weight feature and
usage of eventdev application with DLB hardware.

Signed-off-by: Abdullah Sevincer <abdullah.sevincer@intel.com>
---
 doc/guides/eventdevs/dlb2.rst  | 107 ++++++++++++++++++++------
 drivers/event/dlb2/dlb2.c      | 132 +++++++++++----------------------
 drivers/event/dlb2/dlb2_priv.h |   4 +-
 3 files changed, 131 insertions(+), 112 deletions(-)

diff --git a/doc/guides/eventdevs/dlb2.rst b/doc/guides/eventdevs/dlb2.rst
index f5bf5757c6..c84d153a56 100644
--- a/doc/guides/eventdevs/dlb2.rst
+++ b/doc/guides/eventdevs/dlb2.rst
@@ -17,8 +17,8 @@ Configuration
 -------------
 
 The DLB PF PMD is a user-space PMD that uses VFIO to gain direct
-device access. To use this operation mode, the PCIe PF device must be bound
-to a DPDK-compatible VFIO driver, such as vfio-pci.
+device access. To use this operation mode, the PCIe PF device must
+be bound to a DPDK-compatible VFIO driver, such as vfio-pci.
 
 Eventdev API Notes
 ------------------
@@ -395,26 +395,6 @@ The depth must be between 32 and 1024, and must be a power of 2.
 
        --allow ea:00.0,max_enqueue_depth=<depth>
 
-QE Weight
-~~~~~~~~~
-
-DLB supports advanced scheduling mechanisms, such as CQ weight.
-Each load balanced CQ has a configurable work capacity (max 256)
-which corresponds to the total QE weight DLB will allow to be enqueued
-to that consumer. Every load balanced event/QE carries a weight of 0, 2, 4,
-or 8 and DLB will increment a (per CQ) load indicator when it schedules a
-QE to that CQ. The weight is also stored in the history list. When a
-completion arrives, the weight is popped from the history list and used to
-decrement the load indicator. This creates a new scheduling condition - a CQ
-whose load is equal to or in excess of capacity is not available for traffic.
-Note that the weight may not exceed the maximum CQ depth.
-
-    .. code-block:: console
-
-       --allow ea:00.0,cq_weight=all:<weight>
-       --allow ea:00.0,cq_weight=qidA-qidB:<weight>
-       --allow ea:00.0,cq_weight=qid:<weight>
-
 Producer Coremask
 ~~~~~~~~~~~~~~~~~
 
@@ -450,3 +430,86 @@ won't be used.
     .. code-block:: console
 
        --allow ea:00.0,default_port_allocation=<y/Y>
+
+QE Weight
+~~~~~~~~~
+
+DLB supports advanced scheduling mechanisms, such as CQ weight.
+Each load balanced CQ has a configurable work capacity (max 256)
+which corresponds to the total QE weight DLB will allow to be enqueued
+to that consumer. Every load balanced event/QE carries a weight of 0, 2, 4,
+or 8 and DLB will increment a (per CQ) load indicator when it schedules a
+QE to that CQ. The weight is also stored in the history list. When a
+completion arrives, the weight is popped from the history list and used to
+decrement the load indicator. This creates a new scheduling condition - a CQ
+whose load is equal to or in excess of capacity is not available for traffic.
+Note that the weight may not exceed the maximum CQ depth.
+
+Example command to enable QE Weight feature:
+
+    .. code-block:: console
+
+       --allow ea:00.0,enable_cq_weight=<y/Y>
+
+Running Eventdev Applications with DLB Device
+---------------------------------------------
+This section explains how to run eventdev applications
+with DLB hardware as well as difference in command line parameter
+to switch between a DLB hardware and a virtual eventdev device such as SW0, hence
+users can run applications with or without DLB device to compare performance of
+a DLB device.
+
+In order to run eventdev applications, DLB device must be bound
+to a DPDK-compatible VFIO driver, such as vfio-pci.
+
+Example command to bind DLB device to vfio-pci driver:
+
+    .. code-block:: console
+
+       ../usertools/dpdk-devbind.py -b vfio-pci ea:00.0
+
+Eventdev applications can be run with or without a DLB device.
+Below examples give details of running eventdev application without DLB device
+and with DLB device. Notice that the primary difference between two examples are
+passing the parameter ``--vdev <driver><id>``. The first example run uses a virtual
+eventdev device SW0 while second example run directly and picks DLB device from
+VFIO driver.
+
+Example command to run eventdev application without a DLB device:
+
+	.. code-block:: console
+
+	   sudo <build_dir>/app/dpdk-test-eventdev --vdev=event_sw0 -- \
+					--test=order_queue --plcores 1 --wlcores 2,3
+
+After binding DLB device to a supported pci driver such as vfio-pci,
+eventdev applications can be run on the DLB device.
+
+Example command to run eventdev application with a DLB device:
+
+	.. code-block:: console
+
+		sudo build/app/dpdk-test-eventdev -- --test=order_queue\
+			--plcores=1 --wlcores=2-7 --stlist=o --worker_deq_depth=128\
+			--prod_enq_burst_sz=64 --nb_flows=64 --nb_pkts=1000000
+
+A particular DLB device can also be picked from command line by passing
+	``--a`` or  ``--allow`` option:
+
+	.. code-block:: console
+
+		sudo build/app/dpdk-test-eventdev --allow ea:00.0 -- --test=order_queue\
+			--plcores=1 --wlcores=2-7 --stlist=o --worker_deq_depth=128\
+			--prod_enq_burst_sz=64 --nb_flows=64 --nb_pkts=1000000
+
+Debugging options
+~~~~~~~~~~~~~~~~~
+
+To specify log level for a DLB device use ``--log-level=dlb,8``.
+Example command to run eventdev application with a DLB device log level enabled:
+
+	.. code-block:: console
+
+		sudo build/app/dpdk-test-eventdev --allow ea:00.0 --log-level=dlb,8 -- --test=order_queue\
+			--plcores=1 --wlcores=2-7 --stlist=o --worker_deq_depth=128\
+			--prod_enq_burst_sz=64 --nb_flows=64 --nb_pkts=1000000
\ No newline at end of file
diff --git a/drivers/event/dlb2/dlb2.c b/drivers/event/dlb2/dlb2.c
index 60c5cd4804..07c9384950 100644
--- a/drivers/event/dlb2/dlb2.c
+++ b/drivers/event/dlb2/dlb2.c
@@ -117,63 +117,6 @@ dlb2_init_queue_depth_thresholds(struct dlb2_eventdev *dlb2,
 	}
 }
 
-/* override defaults with value(s) provided on command line */
-static void
-dlb2_init_cq_weight(struct dlb2_eventdev *dlb2, int *cq_weight)
-{
-	int q;
-
-	for (q = 0; q < DLB2_MAX_NUM_PORTS_ALL; q++)
-		dlb2->ev_ports[q].cq_weight = cq_weight[q];
-}
-
-static int
-set_cq_weight(const char *key __rte_unused,
-	      const char *value,
-	      void *opaque)
-{
-	struct dlb2_cq_weight *cq_weight = opaque;
-	int first, last, weight, i;
-
-	if (value == NULL || opaque == NULL) {
-		DLB2_LOG_ERR("NULL pointer\n");
-		return -EINVAL;
-	}
-
-	/* command line override may take one of the following 3 forms:
-	 * qid_depth_thresh=all:<threshold_value> ... all queues
-	 * qid_depth_thresh=qidA-qidB:<threshold_value> ... a range of queues
-	 * qid_depth_thresh=qid:<threshold_value> ... just one queue
-	 */
-	if (sscanf(value, "all:%d", &weight) == 1) {
-		first = 0;
-		last = DLB2_MAX_NUM_PORTS_ALL - 1;
-	} else if (sscanf(value, "%d-%d:%d", &first, &last, &weight) == 3) {
-		/* we have everything we need */
-	} else if (sscanf(value, "%d:%d", &first, &weight) == 2) {
-		last = first;
-	} else {
-		DLB2_LOG_ERR("Error parsing ldb port qe weight devarg. Should be all:val, qid-qid:val, or qid:val\n");
-		return -EINVAL;
-	}
-
-	if (first > last || first < 0 ||
-		last >= DLB2_MAX_NUM_PORTS_ALL) {
-		DLB2_LOG_ERR("Error parsing ldb port qe weight arg, invalid port value\n");
-		return -EINVAL;
-	}
-
-	if (weight < 0 || weight > DLB2_MAX_CQ_DEPTH_OVERRIDE) {
-		DLB2_LOG_ERR("Error parsing ldb port qe weight devarg, must be < cq depth\n");
-		return -EINVAL;
-	}
-
-	for (i = first; i <= last; i++)
-		cq_weight->limit[i] = weight; /* indexed by qid */
-
-	return 0;
-}
-
 /* override defaults with value(s) provided on command line */
 static void
 dlb2_init_port_cos(struct dlb2_eventdev *dlb2, int *port_cos)
@@ -397,7 +340,6 @@ set_max_enq_depth(const char *key __rte_unused,
 	return 0;
 }
 
-
 static int
 set_max_num_events(const char *key __rte_unused,
 		   const char *value,
@@ -667,6 +609,26 @@ set_default_ldb_port_allocation(const char *key __rte_unused,
 	return 0;
 }
 
+static int
+set_enable_cq_weight(const char *key __rte_unused,
+		      const char *value,
+		      void *opaque)
+{
+	bool *enable_cq_weight = opaque;
+
+	if (value == NULL || opaque == NULL) {
+		DLB2_LOG_ERR("NULL pointer\n");
+		return -EINVAL;
+	}
+
+	if ((*value == 'y') || (*value == 'Y'))
+		*enable_cq_weight = true;
+	else
+		*enable_cq_weight = false;
+
+	return 0;
+}
+
 static int
 set_qid_depth_thresh(const char *key __rte_unused,
 		     const char *value,
@@ -1644,26 +1606,20 @@ dlb2_hw_create_ldb_port(struct dlb2_eventdev *dlb2,
 
 	qm_port->id = qm_port_id;
 
-	if (dlb2->version == DLB2_HW_V2) {
-		qm_port->cached_ldb_credits = 0;
-		qm_port->cached_dir_credits = 0;
-		if (ev_port->cq_weight) {
-			struct dlb2_enable_cq_weight_args
-				cq_weight_args = { {0} };
-
-			cq_weight_args.port_id = qm_port->id;
-			cq_weight_args.limit = ev_port->cq_weight;
-			ret = dlb2_iface_enable_cq_weight(handle, &cq_weight_args);
-			if (ret < 0) {
-				DLB2_LOG_ERR("dlb2: dlb2_dir_port_create error, ret=%d (driver status: %s)\n",
+	if (dlb2->version == DLB2_HW_V2_5 && (dlb2->enable_cq_weight == true)) {
+		struct dlb2_enable_cq_weight_args cq_weight_args = { {0} };
+		cq_weight_args.port_id = qm_port->id;
+		cq_weight_args.limit = dequeue_depth;
+		ret = dlb2_iface_enable_cq_weight(handle, &cq_weight_args);
+
+		if (ret < 0) {
+			DLB2_LOG_ERR("dlb2: dlb2_dir_port_create error, ret=%d (driver status: %s)\n",
 					ret,
 					dlb2_error_strings[cfg.response.  status]);
-				goto error_exit;
-			}
+			goto error_exit;
 		}
-		qm_port->cq_weight = ev_port->cq_weight;
+		qm_port->cq_weight = dequeue_depth;
 	} else {
-		qm_port->cached_credits = 0;
 		qm_port->cq_weight = 0;
 	}
 
@@ -4621,6 +4577,7 @@ dlb2_primary_eventdev_probe(struct rte_eventdev *dev,
 	dlb2->hw_credit_quanta = dlb2_args->hw_credit_quanta;
 	dlb2->default_depth_thresh = dlb2_args->default_depth_thresh;
 	dlb2->vector_opts_enabled = dlb2_args->vector_opts_enabled;
+	dlb2->enable_cq_weight = dlb2_args->enable_cq_weight;
 
 
 	if (dlb2_args->max_cq_depth != 0)
@@ -4641,9 +4598,6 @@ dlb2_primary_eventdev_probe(struct rte_eventdev *dev,
 	dlb2_init_queue_depth_thresholds(dlb2,
 					 dlb2_args->qid_depth_thresholds.val);
 
-	dlb2_init_cq_weight(dlb2,
-			    dlb2_args->cq_weight.limit);
-
 	dlb2_init_port_cos(dlb2,
 			   dlb2_args->port_cos.cos_id);
 
@@ -4774,11 +4728,11 @@ dlb2_parse_params(const char *params,
 					     DLB2_VECTOR_OPTS_ENAB_ARG,
 					     DLB2_MAX_CQ_DEPTH,
 					     DLB2_MAX_ENQ_DEPTH,
-					     DLB2_CQ_WEIGHT,
 					     DLB2_PORT_COS,
 					     DLB2_COS_BW,
 					     DLB2_PRODUCER_COREMASK,
 					     DLB2_DEFAULT_LDB_PORT_ALLOCATION_ARG,
+						 DLB2_ENABLE_CQ_WEIGHT_ARG,
 					     NULL };
 
 	if (params != NULL && params[0] != '\0') {
@@ -4926,17 +4880,6 @@ dlb2_parse_params(const char *params,
 				return ret;
 			}
 
-			ret = rte_kvargs_process(kvlist,
-					DLB2_CQ_WEIGHT,
-					set_cq_weight,
-					&dlb2_args->cq_weight);
-			if (ret != 0) {
-				DLB2_LOG_ERR("%s: Error parsing cq weight on",
-					     name);
-				rte_kvargs_free(kvlist);
-				return ret;
-			}
-
 			ret = rte_kvargs_process(kvlist,
 					DLB2_PORT_COS,
 					set_port_cos,
@@ -4983,6 +4926,17 @@ dlb2_parse_params(const char *params,
 				return ret;
 			}
 
+			ret = rte_kvargs_process(kvlist,
+						 DLB2_ENABLE_CQ_WEIGHT_ARG,
+						 set_enable_cq_weight,
+						 &dlb2_args->enable_cq_weight);
+			if (ret != 0) {
+				DLB2_LOG_ERR("%s: Error parsing enable_cq_weight arg",
+					     name);
+				rte_kvargs_free(kvlist);
+				return ret;
+			}
+
 			rte_kvargs_free(kvlist);
 		}
 	}
diff --git a/drivers/event/dlb2/dlb2_priv.h b/drivers/event/dlb2/dlb2_priv.h
index f4b9e7f9ff..31a3beeb6c 100644
--- a/drivers/event/dlb2/dlb2_priv.h
+++ b/drivers/event/dlb2/dlb2_priv.h
@@ -47,11 +47,11 @@
 #define DLB2_VECTOR_OPTS_ENAB_ARG "vector_opts_enable"
 #define DLB2_MAX_CQ_DEPTH "max_cq_depth"
 #define DLB2_MAX_ENQ_DEPTH "max_enqueue_depth"
-#define DLB2_CQ_WEIGHT "cq_weight"
 #define DLB2_PORT_COS "port_cos"
 #define DLB2_COS_BW "cos_bw"
 #define DLB2_PRODUCER_COREMASK "producer_coremask"
 #define DLB2_DEFAULT_LDB_PORT_ALLOCATION_ARG "default_port_allocation"
+#define DLB2_ENABLE_CQ_WEIGHT_ARG "enable_cq_weight"
 
 /* Begin HW related defines and structs */
 
@@ -637,6 +637,7 @@ struct dlb2_eventdev {
 	uint32_t cos_ports[DLB2_COS_NUM_VALS]; /* total ldb ports in each class */
 	uint32_t cos_bw[DLB2_COS_NUM_VALS]; /* bandwidth per cos domain */
 	uint8_t max_cos_port; /* Max LDB port from any cos */
+	bool enable_cq_weight;
 };
 
 /* used for collecting and passing around the dev args */
@@ -674,6 +675,7 @@ struct dlb2_devargs {
 	struct dlb2_cos_bw cos_bw;
 	const char *producer_coremask;
 	bool default_ldb_port_allocation;
+	bool enable_cq_weight;
 };
 
 /* End Eventdev related defines and structs */
-- 
2.25.1