[RFC 0/3] Introduce event prefetching

[RFC 0/3] Introduce event prefetching

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Event prefetching improves scheduling performance by pre-scheduling events to
event ports when dequeues are issued. This series introduces various types and
levels of prefetching to the eventdev library.

Prefetching Types:
 * RTE_EVENT_DEV_PREFETCH_NONE: No prefetching.
 * RTE_EVENT_DEV_PREFETCH: Always issue a prefetch when dequeue is issued.
 * RTE_EVENT_DEV_PREFETCH_INTELLIGENT: Delay issuing prefetch until there
   are no forward progress constraints with the held flow contexts.

Prefetching Levels:
 * Event Device Level Prefetch: Prefetching can be enabled or disabled at the
   event device during configuration. Event devices can indicate prefetching
   capabilities using `RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and
  `RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event device info
  function `info.event_dev_cap`.
 * Event Port Level Prefetch: Prefetching can be selectively enabled or disabled
   at the event port during runtime. Event devices can indicate this capability
   using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PREFETCH` via the event device info
   function `info.event_port_cap`.
 * Application Controlled Prefetch Hint: Applications can provide hints to the
   event device to start prefetching events using the new API
   `rte_event_port_prefetch`. Event devices can indicate this capabilities using
   `RTE_EVENT_DEV_CAP_SW_PREFETCH` via the event device info function
   `info.event_dev_cap`.

The rte_event_dequeue_burst operation initiates the prefetch operation, which
completes in parallel without affecting the dequeued event flow context and
dequeue latency. On the next dequeue operation, the prefetched events are
dequeued, and prefetch is initiated again.

In the case of application-controlled prefetch hints, the currently held flow
contexts, if any, are not affected by the prefetch operation. On the next
dequeue operation, the prefetched events are returned, but prefetch is not
initiated again until the application provides the hint again. If prefetching
is already enabled at the event device level or event port level,
the hint is ignored.

Pavan Nikhilesh (3):
  eventdev: introduce event prefetching
  eventdev: allow event ports to modified prefetches
  eventdev: add SW event prefetch hint

 app/test/test_eventdev.c                    | 113 +++++++++++++++
 doc/guides/prog_guide/eventdev/eventdev.rst |  42 ++++++
 lib/eventdev/eventdev_pmd.h                 |   4 +
 lib/eventdev/eventdev_private.c             |  19 +++
 lib/eventdev/eventdev_trace_points.c        |   6 +
 lib/eventdev/rte_eventdev.h                 | 152 ++++++++++++++++++++
 lib/eventdev/rte_eventdev_core.h            |   9 ++
 lib/eventdev/rte_eventdev_trace_fp.h        |  19 ++-
 lib/eventdev/version.map                    |   6 +
 9 files changed, 369 insertions(+), 1 deletion(-)

--
2.46.0

[RFC 1/3] eventdev: introduce event prefetching

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Event prefetching enhances scheduling performance by pre-scheduling
events to event ports when dequeues are issued.
The dequeue operation initiates the prefetch operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.

Event devices can indicate prefetching capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and
`RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event device info
function `info.event_dev_cap`.

Applications can select the prefetch type and configure it via
`rte_event_dev_config.prefetch_type` in the `rte_event_dev_configure` API.

The supported prefetch types are:
 * `RTE_EVENT_DEV_PREFETCH_NONE` - No prefetching.
 * `RTE_EVENT_DEV_PREFETCH` - Always issue a prefetch on dequeue.
 * `RTE_EVENT_DEV_PREFETCH_INTELLIGENT` - Delay issuing prefetch until
   there are no forward progress constraints with the held flow contexts.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 app/test/test_eventdev.c                    | 62 +++++++++++++++++++++
 doc/guides/prog_guide/eventdev/eventdev.rst | 22 ++++++++
 lib/eventdev/rte_eventdev.h                 | 48 ++++++++++++++++
 3 files changed, 132 insertions(+)

diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index e4e234dc98..1fd3d1fa69 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1250,6 +1250,66 @@ test_eventdev_profile_switch(void)
 	return TEST_SUCCESS;
 }
 
+static int
+prefetch_test(rte_event_dev_prefetch_type_t prefetch_type, const char *prefetch_name)
+{
+#define NB_EVENTS     1024
+	uint64_t start, total;
+	struct rte_event ev;
+	int rc, cnt;
+
+	ev.event_type = RTE_EVENT_TYPE_CPU;
+	ev.queue_id = 0;
+	ev.op = RTE_EVENT_OP_NEW;
+	ev.u64 = 0xBADF00D0;
+
+	for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+		ev.flow_id = cnt;
+		rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+		TEST_ASSERT(rc == 1, "Failed to enqueue event");
+	}
+
+	RTE_SET_USED(prefetch_type);
+	total = 0;
+	while (cnt) {
+		start = rte_rdtsc_precise();
+		rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+		if (rc) {
+			total += rte_rdtsc_precise() - start;
+			cnt--;
+		}
+	}
+	printf("Prefetch type : %s, avg cycles %" PRIu64 "\n", prefetch_name, total / NB_EVENTS);
+
+	return TEST_SUCCESS;
+}
+
+static int
+test_eventdev_prefetch_configure(void)
+{
+	struct rte_event_dev_config dev_conf;
+	struct rte_event_dev_info info;
+	int rc;
+
+	rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+	if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PREFETCH) == 0)
+		return TEST_SKIPPED;
+
+	devconf_set_default_sane_values(&dev_conf, &info);
+	dev_conf.prefetch_type = RTE_EVENT_DEV_PREFETCH;
+	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+	rc = prefetch_test(RTE_EVENT_DEV_PREFETCH_NONE, "RTE_EVENT_DEV_PREFETCH_NONE");
+	rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH, "RTE_EVENT_DEV_PREFETCH");
+	if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH)
+		rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH_INTELLIGENT,
+				    "RTE_EVENT_DEV_PREFETCH_INTELLIGENT");
+
+	return rc;
+}
+
 static int
 test_eventdev_close(void)
 {
@@ -1310,6 +1370,8 @@ static struct unit_test_suite eventdev_common_testsuite  = {
 			test_eventdev_start_stop),
 		TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
 			test_eventdev_profile_switch),
+		TEST_CASE_ST(eventdev_configure_setup, NULL,
+			test_eventdev_prefetch_configure),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
 			test_eventdev_link),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index fb6dfce102..89064883b7 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,28 @@ Worker path:
        // Process the event received.
    }
 
+Event Prefetching
+~~~~~~~~~~~~~~~~~
+
+Event prefetching enhances scheduling performance by pre-scheduling events to event ports
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the prefetch operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the prefetched events are dequeued and prefetch is initiated
+again.
+
+An application can use event prefetching if the event device supports it at either device
+level or at a individual port level.
+The application can check prefetch capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PREFETCH`` set, if present prefetching can be enabled at device
+configuration time by setting appropriate prefetch type in ``rte_event_dev_config.prefetch``.
+
+Currently, the following prefetch types are supported:
+ * ``RTE_EVENT_DEV_PREFETCH_NONE`` - No prefetching.
+ * ``RTE_EVENT_DEV_PREFETCH`` - Always issue a prefetch when dequeue is issued.
+ * ``RTE_EVENT_DEV_PREFETCH_INTELLIGENT`` - Issue prefetch when dequeue is issued and there are
+   no forward progress constraints.
+
 Starting the EventDev
 ~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 08e5f9320b..59c323c8ee 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -446,6 +446,30 @@ struct rte_event;
  * @see RTE_SCHED_TYPE_PARALLEL
  */
 
+#define RTE_EVENT_DEV_CAP_EVENT_PREFETCH (1ULL << 16)
+/**< Event device supports event prefetching.
+ *
+ * When this capability is available, the application can enable event prefetching on the event
+ * device to prefetch/pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The prefetch process starts with the `rte_event_dequeue_burst()` call and the
+ * prefetched events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH (1ULL << 17)
+/**< Event device supports intelligent event prefetching.
+ *
+ * When this capability is available, the application can enable intelligent prefetching
+ * on the event device where the events are prefetched/pre-scheduled when
+ * there are no forward progress constraints with the currently held flow contexts.
+ * The prefetch process starts with the `rte_event_dequeue_burst()` call and the
+ * prefetched events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
 /* Event device priority levels */
 #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
 /**< Highest priority level for events and queues.
@@ -680,6 +704,25 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
  *  @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
  */
 
+typedef enum {
+	RTE_EVENT_DEV_PREFETCH_NONE = 0,
+	/* Disable prefetch across the event device or on a given event port.
+	 * @ref rte_event_dev_config.prefetch_type
+	 */
+	RTE_EVENT_DEV_PREFETCH,
+	/* Enable prefetch always across the event device or a given event port.
+	 * @ref rte_event_dev_config.prefetch_type
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PREFETCH
+	 */
+	RTE_EVENT_DEV_PREFETCH_INTELLIGENT,
+	/* Enable intelligent prefetch across the event device or a given event port.
+	 * Delay issuing prefetch until there are no forward progress constraints with
+	 * the held flow contexts.
+	 * @ref rte_event_dev_config.prefetch_type
+	 * @see RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH
+	 */
+} rte_event_dev_prefetch_type_t;
+
 /** Event device configuration structure */
 struct rte_event_dev_config {
 	uint32_t dequeue_timeout_ns;
@@ -752,6 +795,11 @@ struct rte_event_dev_config {
 	 * optimized for single-link usage, this field is a hint for how many
 	 * to allocate; otherwise, regular event ports and queues will be used.
 	 */
+	rte_event_dev_prefetch_type_t prefetch_type;
+	/**< Event prefetch type to use across the event device, if supported.
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PREFETCH
+	 * @see RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH
+	 */
 };
 
 /**
-- 
2.25.1

[RFC 2/3] eventdev: allow event ports to modified prefetches

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Some event devices allow prefetch types to be modified at
runtime on an event port.
Add `RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PREFETCH` capability
to indicate that the event device supports this feature.

Add `rte_event_port_prefetch_modify()` API to modify the
prefetch type at runtime.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 app/test/test_eventdev.c                    | 61 +++++++++++++++++++--
 doc/guides/prog_guide/eventdev/eventdev.rst | 12 ++++
 lib/eventdev/eventdev_pmd.h                 |  2 +
 lib/eventdev/eventdev_private.c             | 10 ++++
 lib/eventdev/eventdev_trace_points.c        |  3 +
 lib/eventdev/rte_eventdev.h                 | 55 +++++++++++++++++++
 lib/eventdev/rte_eventdev_core.h            |  5 ++
 lib/eventdev/rte_eventdev_trace_fp.h        | 11 +++-
 lib/eventdev/version.map                    |  4 ++
 9 files changed, 157 insertions(+), 6 deletions(-)

diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 1fd3d1fa69..af27a9f0ec 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,7 +1251,8 @@ test_eventdev_profile_switch(void)
 }
 
 static int
-prefetch_test(rte_event_dev_prefetch_type_t prefetch_type, const char *prefetch_name)
+prefetch_test(rte_event_dev_prefetch_type_t prefetch_type, const char *prefetch_name,
+	      uint8_t modify)
 {
 #define NB_EVENTS     1024
 	uint64_t start, total;
@@ -1269,7 +1270,11 @@ prefetch_test(rte_event_dev_prefetch_type_t prefetch_type, const char *prefetch_
 		TEST_ASSERT(rc == 1, "Failed to enqueue event");
 	}
 
-	RTE_SET_USED(prefetch_type);
+	if (modify) {
+		rc = rte_event_port_prefetch_modify(TEST_DEV_ID, 0, prefetch_type);
+		TEST_ASSERT_SUCCESS(rc, "Failed to modify prefetch type");
+	}
+
 	total = 0;
 	while (cnt) {
 		start = rte_rdtsc_precise();
@@ -1301,11 +1306,55 @@ test_eventdev_prefetch_configure(void)
 	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
 	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
 
-	rc = prefetch_test(RTE_EVENT_DEV_PREFETCH_NONE, "RTE_EVENT_DEV_PREFETCH_NONE");
-	rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH, "RTE_EVENT_DEV_PREFETCH");
+	rc = prefetch_test(RTE_EVENT_DEV_PREFETCH_NONE, "RTE_EVENT_DEV_PREFETCH_NONE", 0);
+	rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH, "RTE_EVENT_DEV_PREFETCH", 0);
 	if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH)
 		rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH_INTELLIGENT,
-				    "RTE_EVENT_DEV_PREFETCH_INTELLIGENT");
+				    "RTE_EVENT_DEV_PREFETCH_INTELLIGENT", 0);
+
+	return rc;
+}
+
+static int
+test_eventdev_prefetch_modify(void)
+{
+	struct rte_event_dev_config dev_conf;
+	struct rte_event_queue_conf qcfg;
+	struct rte_event_port_conf pcfg;
+	struct rte_event_dev_info info;
+	int rc;
+
+	rte_event_dev_info_get(TEST_DEV_ID, &info);
+	if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PREFETCH) == 0)
+		return TEST_SKIPPED;
+
+	devconf_set_default_sane_values(&dev_conf, &info);
+	dev_conf.prefetch_type = RTE_EVENT_DEV_PREFETCH_NONE;
+	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+	rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+	rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+	rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+	rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+	rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+	TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+		    TEST_DEV_ID);
+
+	rc = rte_event_dev_start(TEST_DEV_ID);
+	TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+	rc = prefetch_test(RTE_EVENT_DEV_PREFETCH_NONE, "RTE_EVENT_DEV_PREFETCH_NONE", 1);
+	rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH, "RTE_EVENT_DEV_PREFETCH", 1);
+	if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH)
+		rc |= prefetch_test(RTE_EVENT_DEV_PREFETCH_INTELLIGENT,
+				    "RTE_EVENT_DEV_PREFETCH_INTELLIGENT", 1);
 
 	return rc;
 }
@@ -1372,6 +1421,8 @@ static struct unit_test_suite eventdev_common_testsuite  = {
 			test_eventdev_profile_switch),
 		TEST_CASE_ST(eventdev_configure_setup, NULL,
 			test_eventdev_prefetch_configure),
+		TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
+			test_eventdev_prefetch_modify),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
 			test_eventdev_link),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index 89064883b7..8b6085512d 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -379,6 +379,18 @@ Currently, the following prefetch types are supported:
  * ``RTE_EVENT_DEV_PREFETCH_INTELLIGENT`` - Issue prefetch when dequeue is issued and there are
    no forward progress constraints.
 
+To enable or disable event prefetching at a given event port, the application can use
+``rte_event_port_prefetch_modify()`` API.
+
+.. code-block:: c
+
+   rte_event_port_prefetch_modify(dev_id, port_id, RTE_EVENT_DEV_PREFETCH);
+   // Dequeue events from the event port with normal dequeue() function.
+   rte_event_port_prefetch_modify(dev_id, port_id, RTE_EVENT_DEV_PREFETCH_NONE);
+   // Disable prefetching if thread is about to be scheduled out and issue dequeue() to drain
+   // pending events.
+
+
 Starting the EventDev
 ~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index 7a5699f14b..d93c11e9f1 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -184,6 +184,8 @@ struct __rte_cache_aligned rte_eventdev {
 	/**< Pointer to PMD DMA adapter enqueue function. */
 	event_profile_switch_t profile_switch;
 	/**< Pointer to PMD Event switch profile function. */
+	event_prefetch_modify_t prefetch_modify;
+	/**< Pointer to PMD Event port prefetch modify function.  */
 
 	uint64_t reserved_64s[3]; /**< Reserved for future fields */
 	void *reserved_ptrs[3];	  /**< Reserved for future fields */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 017f97ccab..6e8beb29ad 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -96,6 +96,14 @@ dummy_event_port_profile_switch(__rte_unused void *port, __rte_unused uint8_t pr
 	return -EINVAL;
 }
 
+static int
+dummy_event_port_prefetch_modify(__rte_unused void *port,
+				 __rte_unused rte_event_dev_prefetch_type_t prefetch)
+{
+	RTE_EDEV_LOG_ERR("modify prefetch requested for unconfigured event device");
+	return -EINVAL;
+}
+
 void
 event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 {
@@ -114,6 +122,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 		.ca_enqueue = dummy_event_crypto_adapter_enqueue,
 		.dma_enqueue = dummy_event_dma_adapter_enqueue,
 		.profile_switch = dummy_event_port_profile_switch,
+		.prefetch_modify = dummy_event_port_prefetch_modify,
 		.data = dummy_data,
 	};
 
@@ -136,5 +145,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
 	fp_op->ca_enqueue = dev->ca_enqueue;
 	fp_op->dma_enqueue = dev->dma_enqueue;
 	fp_op->profile_switch = dev->profile_switch;
+	fp_op->prefetch_modify = dev->prefetch_modify;
 	fp_op->data = dev->data->ports;
 }
diff --git a/lib/eventdev/eventdev_trace_points.c b/lib/eventdev/eventdev_trace_points.c
index 8024e07531..e0547d18c6 100644
--- a/lib/eventdev/eventdev_trace_points.c
+++ b/lib/eventdev/eventdev_trace_points.c
@@ -49,6 +49,9 @@ RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_maintain,
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_profile_switch,
 	lib.eventdev.port.profile.switch)
 
+RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_prefetch_modify,
+	lib.eventdev.port.prefetch.modify)
+
 /* Eventdev Rx adapter trace points */
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_eth_rx_adapter_create,
 	lib.eventdev.rx.adapter.create)
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 59c323c8ee..1bc6c48dd3 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -470,6 +470,16 @@ struct rte_event;
  * @see rte_event_dev_configure()
  */
 
+#define RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PREFETCH (1ULL << 18)
+/**< Event device supports event prefetching per event port.
+ *
+ * When this flag is set, the event device allows controlling the event
+ * prefetching/pre-scheduling at a event port granularity.
+ *
+ * @see rte_event_dev_configure()
+ * @see rte_event_port_prefetch_modify()
+ */
+
 /* Event device priority levels */
 #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
 /**< Highest priority level for events and queues.
@@ -708,18 +718,23 @@ typedef enum {
 	RTE_EVENT_DEV_PREFETCH_NONE = 0,
 	/* Disable prefetch across the event device or on a given event port.
 	 * @ref rte_event_dev_config.prefetch_type
+	 * @ref rte_event_port_prefetch_modify()
 	 */
 	RTE_EVENT_DEV_PREFETCH,
 	/* Enable prefetch always across the event device or a given event port.
 	 * @ref rte_event_dev_config.prefetch_type
+	 * @ref rte_event_port_prefetch_modify()
 	 * @see RTE_EVENT_DEV_CAP_EVENT_PREFETCH
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PREFETCH
 	 */
 	RTE_EVENT_DEV_PREFETCH_INTELLIGENT,
 	/* Enable intelligent prefetch across the event device or a given event port.
 	 * Delay issuing prefetch until there are no forward progress constraints with
 	 * the held flow contexts.
 	 * @ref rte_event_dev_config.prefetch_type
+	 * @ref rte_event_port_prefetch_modify()
 	 * @see RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PREFETCH
 	 */
 } rte_event_dev_prefetch_type_t;
 
@@ -2922,6 +2937,46 @@ rte_event_port_profile_switch(uint8_t dev_id, uint8_t port_id, uint8_t profile_i
 	return fp_ops->profile_switch(port, profile_id);
 }
 
+/**
+ * Change the prefetch type to use on an event port.
+ *
+ * This function is used to change the current prefetch type configured
+ * on an event port, the prefetch type can be set to none to disable prefetching.
+ * This effects the subsequent ``rte_event_dequeue_burst`` call.
+ * The event device should support RTE_EVENT_DEV_CAP_PER_PORT_PREFETCH capability.
+ *
+ * @param dev_id
+ *   The identifier of the device.
+ * @param port_id
+ *   The identifier of the event port.
+ * @param type
+ *   The prefetch type to use on the event port.
+ * @return
+ *  - 0 on success.
+ *  - -EINVAL if *dev_id*,  *port_id*, or *type* is invalid.
+ *  - -ENOTSUP if the device doesn't support prefetch or per port prefetch.
+ */
+static inline int
+rte_event_port_prefetch_modify(uint8_t dev_id, uint8_t port_id, rte_event_dev_prefetch_type_t type)
+{
+	const struct rte_event_fp_ops *fp_ops;
+	void *port;
+
+	fp_ops = &rte_event_fp_ops[dev_id];
+	port = fp_ops->data[port_id];
+
+#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
+	if (dev_id >= RTE_EVENT_MAX_DEVS || port_id >= RTE_EVENT_MAX_PORTS_PER_DEV)
+		return -EINVAL;
+
+	if (port == NULL)
+		return -EINVAL;
+#endif
+	rte_eventdev_trace_port_prefetch_modify(dev_id, port_id, type);
+
+	return fp_ops->prefetch_modify(port, type);
+}
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index fc8e1556ab..b185fe3654 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -49,6 +49,9 @@ typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[
 typedef int (*event_profile_switch_t)(void *port, uint8_t profile);
 /**< @internal Switch active link profile on the event port. */
 
+typedef int (*event_prefetch_modify_t)(void *port, rte_event_dev_prefetch_type_t prefetch_type);
+/**< @internal Modify prefetch type on the event port. */
+
 struct __rte_cache_aligned rte_event_fp_ops {
 	void **data;
 	/**< points to array of internal port data pointers */
@@ -76,6 +79,8 @@ struct __rte_cache_aligned rte_event_fp_ops {
 	/**< PMD DMA adapter enqueue function. */
 	event_profile_switch_t profile_switch;
 	/**< PMD Event switch profile function. */
+	event_prefetch_modify_t prefetch_modify;
+	/**< PMD Event port prefetch switch. */
 	uintptr_t reserved[4];
 };
 
diff --git a/lib/eventdev/rte_eventdev_trace_fp.h b/lib/eventdev/rte_eventdev_trace_fp.h
index 04d510ad00..06bfd48011 100644
--- a/lib/eventdev/rte_eventdev_trace_fp.h
+++ b/lib/eventdev/rte_eventdev_trace_fp.h
@@ -8,7 +8,7 @@
 /**
  * @file
  *
- * API for ethdev trace support
+ * API for eventdev trace support
  */
 
 #ifdef __cplusplus
@@ -54,6 +54,15 @@ RTE_TRACE_POINT_FP(
 	rte_trace_point_emit_u8(profile);
 )
 
+RTE_TRACE_POINT_FP(
+	rte_eventdev_trace_port_prefetch_modify,
+	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id,
+			     int type),
+	rte_trace_point_emit_u8(dev_id);
+	rte_trace_point_emit_u8(port_id);
+	rte_trace_point_emit_int(type);
+)
+
 RTE_TRACE_POINT_FP(
 	rte_eventdev_trace_eth_tx_adapter_enqueue,
 	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, void *ev_table,
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 4947bb4ec6..62c0563fb1 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -147,6 +147,10 @@ EXPERIMENTAL {
 	rte_event_port_profile_unlink;
 	rte_event_port_profile_links_get;
 	__rte_eventdev_trace_port_profile_switch;
+
+	# added in 24.11
+	rte_event_port_prefetch_modify;
+	__rte_eventdev_trace_port_prefetch_modify;
 };
 
 INTERNAL {
-- 
2.25.1

[RFC 3/3] eventdev: add SW event prefetch hint

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Add a new eventdev API to provide a hint to the eventdev PMD to
prefetch the next event into the event port, without releasing
the current flow context.
Event device that support this feature advertises the capability
using the RTE_EVENT_DEV_CAP_SW_PREFETCH capability flag.

Application can invoke `rte_event_port_prefetch` to hint the PMD.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 doc/guides/prog_guide/eventdev/eventdev.rst |  8 ++++
 lib/eventdev/eventdev_pmd.h                 |  2 +
 lib/eventdev/eventdev_private.c             |  9 ++++
 lib/eventdev/eventdev_trace_points.c        |  3 ++
 lib/eventdev/rte_eventdev.h                 | 49 +++++++++++++++++++++
 lib/eventdev/rte_eventdev_core.h            |  4 ++
 lib/eventdev/rte_eventdev_trace_fp.h        |  8 ++++
 lib/eventdev/version.map                    |  2 +
 8 files changed, 85 insertions(+)

diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index 8b6085512d..ea05de53f5 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -390,6 +390,14 @@ To enable or disable event prefetching at a given event port, the application ca
    // Disable prefetching if thread is about to be scheduled out and issue dequeue() to drain
    // pending events.
 
+Event Prefetch Hint can be used to provide a hint to the eventdev PMD to prefetch the next event
+without releasing the current flow context. Event device that support this feature advertises the
+capability using the ``RTE_EVENT_DEV_CAP_SW_PREFETCH`` capability flag.
+If prefetching is already enabled at a event device or event port level then the hint is ignored.
+
+.. code-block:: c
+
+   rte_event_port_prefetch(dev_id, port_id, RTE_EVENT_DEV_PREFETCH);
 
 Starting the EventDev
 ~~~~~~~~~~~~~~~~~~~~~
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index d93c11e9f1..698bdcc14f 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -186,6 +186,8 @@ struct __rte_cache_aligned rte_eventdev {
 	/**< Pointer to PMD Event switch profile function. */
 	event_prefetch_modify_t prefetch_modify;
 	/**< Pointer to PMD Event port prefetch modify function.  */
+	event_prefetch_t prefetch;
+	/**< Pointer to PMD Event port prefetch function. */
 
 	uint64_t reserved_64s[3]; /**< Reserved for future fields */
 	void *reserved_ptrs[3];	  /**< Reserved for future fields */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 6e8beb29ad..39d5234551 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -104,6 +104,13 @@ dummy_event_port_prefetch_modify(__rte_unused void *port,
 	return -EINVAL;
 }
 
+static void
+dummy_event_port_prefetch(__rte_unused void *port,
+			  __rte_unused rte_event_dev_prefetch_type_t prefetch)
+{
+	RTE_EDEV_LOG_ERR("prefetch requested for unconfigured event device");
+}
+
 void
 event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 {
@@ -123,6 +130,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 		.dma_enqueue = dummy_event_dma_adapter_enqueue,
 		.profile_switch = dummy_event_port_profile_switch,
 		.prefetch_modify = dummy_event_port_prefetch_modify,
+		.prefetch = dummy_event_port_prefetch,
 		.data = dummy_data,
 	};
 
@@ -146,5 +154,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
 	fp_op->dma_enqueue = dev->dma_enqueue;
 	fp_op->profile_switch = dev->profile_switch;
 	fp_op->prefetch_modify = dev->prefetch_modify;
+	fp_op->prefetch = dev->prefetch;
 	fp_op->data = dev->data->ports;
 }
diff --git a/lib/eventdev/eventdev_trace_points.c b/lib/eventdev/eventdev_trace_points.c
index e0547d18c6..199cfa742f 100644
--- a/lib/eventdev/eventdev_trace_points.c
+++ b/lib/eventdev/eventdev_trace_points.c
@@ -52,6 +52,9 @@ RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_profile_switch,
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_prefetch_modify,
 	lib.eventdev.port.prefetch.modify)
 
+RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_prefetch,
+	lib.eventdev.port.prefetch)
+
 /* Eventdev Rx adapter trace points */
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_eth_rx_adapter_create,
 	lib.eventdev.rx.adapter.create)
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 1bc6c48dd3..d487389a2c 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -480,6 +480,15 @@ struct rte_event;
  * @see rte_event_port_prefetch_modify()
  */
 
+#define RTE_EVENT_DEV_CAP_SW_PREFETCH (1ULL << 19)
+/**< Event device supports software prefetching.
+ *
+ * When this flag is set, the application can issue prefetch request on
+ * a event port.
+ *
+ * @see rte_event_port_prefetch()
+ */
+
 /* Event device priority levels */
 #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
 /**< Highest priority level for events and queues.
@@ -2977,6 +2986,46 @@ rte_event_port_prefetch_modify(uint8_t dev_id, uint8_t port_id, rte_event_dev_pr
 	return fp_ops->prefetch_modify(port, type);
 }
 
+/**
+ * Provide a hint to the event device to prefetch events to event port .
+ *
+ * Hint the event device to prefetch events to the event port.
+ * The call doesn't not guarantee that the events will be prefetched.
+ * The call doesn't release the flow context currently held by the event port.
+ * The event device should support RTE_EVENT_DEV_CAP_SW_PREFETCH capability.
+ *
+ * When prefetching is enabled at an event device or event port level, the hint
+ * is ignored.
+ *
+ * Subsequent calls to rte_event_dequeue_burst() will dequeue the prefetch events
+ * but prefetch operation is not issued again.
+ *
+ * @param dev_id
+ *   The identifier of the device.
+ * @param port_id
+ *   The identifier of the event port.
+ * @param type
+ *   The prefetch type to use on the event port.
+ */
+static inline void
+rte_event_port_prefetch(uint8_t dev_id, uint8_t port_id, rte_event_dev_prefetch_type_t type)
+{
+	const struct rte_event_fp_ops *fp_ops;
+	void *port;
+
+	fp_ops = &rte_event_fp_ops[dev_id];
+	port = fp_ops->data[port_id];
+
+#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
+	if (dev_id >= RTE_EVENT_MAX_DEVS || port_id >= RTE_EVENT_MAX_PORTS_PER_DEV)
+		return;
+	if (port == NULL)
+		return;
+#endif
+	rte_eventdev_trace_port_prefetch(dev_id, port_id, type);
+
+	fp_ops->prefetch(port, type);
+}
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index b185fe3654..66edb75649 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -52,6 +52,9 @@ typedef int (*event_profile_switch_t)(void *port, uint8_t profile);
 typedef int (*event_prefetch_modify_t)(void *port, rte_event_dev_prefetch_type_t prefetch_type);
 /**< @internal Modify prefetch type on the event port. */
 
+typedef void (*event_prefetch_t)(void *port, rte_event_dev_prefetch_type_t prefetch_type);
+/**< @internal Issue prefetch on an event port. */
+
 struct __rte_cache_aligned rte_event_fp_ops {
 	void **data;
 	/**< points to array of internal port data pointers */
@@ -81,6 +84,7 @@ struct __rte_cache_aligned rte_event_fp_ops {
 	/**< PMD Event switch profile function. */
 	event_prefetch_modify_t prefetch_modify;
 	/**< PMD Event port prefetch switch. */
+	event_prefetch_t prefetch;
 	uintptr_t reserved[4];
 };
 
diff --git a/lib/eventdev/rte_eventdev_trace_fp.h b/lib/eventdev/rte_eventdev_trace_fp.h
index 06bfd48011..01fa5cba7c 100644
--- a/lib/eventdev/rte_eventdev_trace_fp.h
+++ b/lib/eventdev/rte_eventdev_trace_fp.h
@@ -63,6 +63,14 @@ RTE_TRACE_POINT_FP(
 	rte_trace_point_emit_int(type);
 )
 
+RTE_TRACE_POINT_FP(
+	rte_eventdev_trace_port_prefetch,
+	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, int type),
+	rte_trace_point_emit_u8(dev_id);
+	rte_trace_point_emit_u8(port_id);
+	rte_trace_point_emit_int(type);
+)
+
 RTE_TRACE_POINT_FP(
 	rte_eventdev_trace_eth_tx_adapter_enqueue,
 	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, void *ev_table,
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 62c0563fb1..bb291f23e3 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -151,6 +151,8 @@ EXPERIMENTAL {
 	# added in 24.11
 	rte_event_port_prefetch_modify;
 	__rte_eventdev_trace_port_prefetch_modify;
+	rte_event_port_prefetch;
+	__rte_eventdev_trace_port_prefetch;
 };
 
 INTERNAL {
-- 
2.25.1

Re: [RFC 0/3] Introduce event prefetching

[Mattias Rönnblom]

On 2024-09-10 10:31, pbhagavatula@marvell.com wrote:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Event prefetching improves scheduling performance by pre-scheduling events to
> event ports when dequeues are issued. This series introduces various types and
> levels of prefetching to the eventdev library.
> 

I would avoid to use the term "prefetch", since that word is used for 
manual (software) prefetching (preloading in ARM speak). In particular, 
since having the event device prefetch future event-related data may 
potentially also be interesting to do (i.e., to have an option to 
prefetch user-configurable lines in the buffer pointed to by a future 
event).

I suggest RTE_EVENT_DEV_PRESCHEDULE*.

If you make prescheduling a hint, you don't need the capability flags, 
and no error handling. Much smaller API, less fuzz for the application.

> Prefetching Types:
>   * RTE_EVENT_DEV_PREFETCH_NONE: No prefetching.
>   * RTE_EVENT_DEV_PREFETCH: Always issue a prefetch when dequeue is issued.
>   * RTE_EVENT_DEV_PREFETCH_INTELLIGENT: Delay issuing prefetch until there
>     are no forward progress constraints with the held flow contexts.
> 

Maybe RTE_EVENT_DEV_PREFETCH_ADAPTIVE is better. I suggest that being 
the default. In that case, for event devices which can't preschedule or 
which always (by design) have to preschedule, can continue doing what 
they are doing and still calling this "adaptive".

RTE_EVENT_DEV_PREFETCH_AUTO would also work.

Setting aside the complexity issue, prescheduling hints or configuration 
seems like a valuable feature to include in the Eventdev API.

> Prefetching Levels:
>   * Event Device Level Prefetch: Prefetching can be enabled or disabled at the
>     event device during configuration. Event devices can indicate prefetching
>     capabilities using `RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and
>    `RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event device info
>    function `info.event_dev_cap`.
>   * Event Port Level Prefetch: Prefetching can be selectively enabled or disabled
>     at the event port during runtime. Event devices can indicate this capability
>     using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PREFETCH` via the event device info
>     function `info.event_port_cap`.
>   * Application Controlled Prefetch Hint: Applications can provide hints to the
>     event device to start prefetching events using the new API
>     `rte_event_port_prefetch`. Event devices can indicate this capabilities using
>     `RTE_EVENT_DEV_CAP_SW_PREFETCH` via the event device info function
>     `info.event_dev_cap`.
> 
> The rte_event_dequeue_burst operation initiates the prefetch operation, which
> completes in parallel without affecting the dequeued event flow context and
> dequeue latency. On the next dequeue operation, the prefetched events are
> dequeued, and prefetch is initiated again.
> 
> In the case of application-controlled prefetch hints, the currently held flow
> contexts, if any, are not affected by the prefetch operation. On the next
> dequeue operation, the prefetched events are returned, but prefetch is not
> initiated again until the application provides the hint again. If prefetching
> is already enabled at the event device level or event port level,
> the hint is ignored.
> 
> Pavan Nikhilesh (3):
>    eventdev: introduce event prefetching
>    eventdev: allow event ports to modified prefetches
>    eventdev: add SW event prefetch hint
> 
>   app/test/test_eventdev.c                    | 113 +++++++++++++++
>   doc/guides/prog_guide/eventdev/eventdev.rst |  42 ++++++
>   lib/eventdev/eventdev_pmd.h                 |   4 +
>   lib/eventdev/eventdev_private.c             |  19 +++
>   lib/eventdev/eventdev_trace_points.c        |   6 +
>   lib/eventdev/rte_eventdev.h                 | 152 ++++++++++++++++++++
>   lib/eventdev/rte_eventdev_core.h            |   9 ++
>   lib/eventdev/rte_eventdev_trace_fp.h        |  19 ++-
>   lib/eventdev/version.map                    |   6 +
>   9 files changed, 369 insertions(+), 1 deletion(-)
> 
> --
> 2.46.0

RE: [EXTERNAL] Re: [RFC 0/3] Introduce event prefetching

[Pavan Nikhilesh Bhagavatula]

> On 2024-09-10 10:31, pbhagavatula@marvell.com wrote:
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >
> > Event prefetching improves scheduling performance by pre-scheduling
> events to
> > event ports when dequeues are issued. This series introduces various types
> and
> > levels of prefetching to the eventdev library.
> >
> 
> I would avoid to use the term "prefetch", since that word is used for
> manual (software) prefetching (preloading in ARM speak). In particular,
> since having the event device prefetch future event-related data may
> potentially also be interesting to do (i.e., to have an option to
> prefetch user-configurable lines in the buffer pointed to by a future
> event).
> 
> I suggest RTE_EVENT_DEV_PRESCHEDULE*.

Yeah, preschedule makes more sense in this context, I was too far into the RFC 
to make the change. I will make the change in the next version.
In fact OCTEON 10 already supports event data prefetching, I have a RFC for that in
the pipeline.

> 
> If you make prescheduling a hint, you don't need the capability flags,
> and no error handling. Much smaller API, less fuzz for the application.
> 

The hint part is in 3/3, it still has a capability flag attached to it so that 
application don’t crash if driver doesn’t implement the API i.e., ops is NULL.

We could add a NULL check for ops but since its fast path I am not sure.

> > Prefetching Types:
> >   * RTE_EVENT_DEV_PREFETCH_NONE: No prefetching.
> >   * RTE_EVENT_DEV_PREFETCH: Always issue a prefetch when dequeue is
> issued.
> >   * RTE_EVENT_DEV_PREFETCH_INTELLIGENT: Delay issuing prefetch until
> there
> >     are no forward progress constraints with the held flow contexts.
> >
> 
> Maybe RTE_EVENT_DEV_PREFETCH_ADAPTIVE is better. I suggest that being
> the default. In that case, for event devices which can't preschedule or
> which always (by design) have to preschedule, can continue doing what
> they are doing and still calling this "adaptive".
> 

Adaptive sounds good. I will make the change next version.

I think the driver layer can fix up if application requests to disable a preschedule
type that It's already doing and can't be disabled.
But claiming prefetch capability when its not supported might not be a good idea.

In the end the idea is to give the application fine grained control over
prescheduling. As from what we have seen customers have wanted these 
fine grained controls over pre-scheduling.

> RTE_EVENT_DEV_PREFETCH_AUTO would also work.
> 
> Setting aside the complexity issue, prescheduling hints or configuration
> seems like a valuable feature to include in the Eventdev API.
> 
> > Prefetching Levels:
> >   * Event Device Level Prefetch: Prefetching can be enabled or disabled at the
> >     event device during configuration. Event devices can indicate prefetching
> >     capabilities using `RTE_EVENT_DEV_CAP_EVENT_PREFETCH` and
> >    `RTE_EVENT_DEV_CAP_EVENT_INTELLIGENT_PREFETCH` via the event
> device info
> >    function `info.event_dev_cap`.
> >   * Event Port Level Prefetch: Prefetching can be selectively enabled or
> disabled
> >     at the event port during runtime. Event devices can indicate this capability
> >     using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PREFETCH` via the
> event device info
> >     function `info.event_port_cap`.
> >   * Application Controlled Prefetch Hint: Applications can provide hints to the
> >     event device to start prefetching events using the new API
> >     `rte_event_port_prefetch`. Event devices can indicate this capabilities using
> >     `RTE_EVENT_DEV_CAP_SW_PREFETCH` via the event device info function
> >     `info.event_dev_cap`.
> >
> > The rte_event_dequeue_burst operation initiates the prefetch operation,
> which
> > completes in parallel without affecting the dequeued event flow context and
> > dequeue latency. On the next dequeue operation, the prefetched events are
> > dequeued, and prefetch is initiated again.
> >
> > In the case of application-controlled prefetch hints, the currently held flow
> > contexts, if any, are not affected by the prefetch operation. On the next
> > dequeue operation, the prefetched events are returned, but prefetch is not
> > initiated again until the application provides the hint again. If prefetching
> > is already enabled at the event device level or event port level,
> > the hint is ignored.
> >
> > Pavan Nikhilesh (3):
> >    eventdev: introduce event prefetching
> >    eventdev: allow event ports to modified prefetches
> >    eventdev: add SW event prefetch hint
> >
> >   app/test/test_eventdev.c                    | 113 +++++++++++++++
> >   doc/guides/prog_guide/eventdev/eventdev.rst |  42 ++++++
> >   lib/eventdev/eventdev_pmd.h                 |   4 +
> >   lib/eventdev/eventdev_private.c             |  19 +++
> >   lib/eventdev/eventdev_trace_points.c        |   6 +
> >   lib/eventdev/rte_eventdev.h                 | 152 ++++++++++++++++++++
> >   lib/eventdev/rte_eventdev_core.h            |   9 ++
> >   lib/eventdev/rte_eventdev_trace_fp.h        |  19 ++-
> >   lib/eventdev/version.map                    |   6 +
> >   9 files changed, 369 insertions(+), 1 deletion(-)
> >
> > --
> > 2.46.0

[PATCH v2 0/3] Introduce event pre-scheduling

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
This series introduces various types and levels of pre-scheduling to the
eventdev library.

pre-scheduling Types:
 * RTE_EVENT_DEV_PRESCHEDULE_NONE: No pre-scheduling.
 * RTE_EVENT_DEV_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
 * RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
   are no forward progress constraints with the held flow contexts.

pre-scheduling Levels:
 * Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
   event device during configuration. Event devices can indicate pre-scheduling
   capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
  `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
  function `info.event_dev_cap`.
 * Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
   at the event port during runtime. Event devices can indicate this capability
   using `RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE` via the event device info
   function `info.event_port_cap`.
 * Application Controlled Prefetch Hint: Applications can provide hints to the
   event device to start pre-scheduling events using the new API
   `rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
   `RTE_EVENT_DEV_CAP_SW_PRESCHEDULE` via the event device info function
   `info.event_dev_cap`.

The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
completes in parallel without affecting the flow context of the dequeued events and
dequeue latency. On the next dequeue operation, the pre-scheduleed events are
dequeued, and pre-schedule operation is initiated again.

In the case of application-controlled pre-schedule hints, the currently held flow
contexts, if any, are not affected by the pre-schedule operation. On the next
dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
initiated again until the application provides the hint again. If pre-scheduling
is already enabled at the event device level or event port level, the hint is ignored.

v2 Changes:
- s/prefetch/pre-schedule (Mattias)

Pavan Nikhilesh (3):
  eventdev: introduce event pre-scheduling
  eventdev: add event port pre-schedule modify
  eventdev: add SW event preschedule hint

 app/test/test_eventdev.c                    | 114 +++++++++++++++
 doc/guides/prog_guide/eventdev/eventdev.rst |  42 ++++++
 lib/eventdev/eventdev_pmd.h                 |   4 +
 lib/eventdev/eventdev_private.c             |  22 ++-
 lib/eventdev/eventdev_trace_points.c        |   6 +
 lib/eventdev/rte_eventdev.h                 | 152 ++++++++++++++++++++
 lib/eventdev/rte_eventdev_core.h            |  11 ++
 lib/eventdev/rte_eventdev_trace_fp.h        |  19 ++-
 lib/eventdev/version.map                    |   6 +
 9 files changed, 373 insertions(+), 3 deletions(-)

--
2.25.1

[PATCH v2 1/3] eventdev: introduce event pre-scheduling

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
The dequeue operation initiates the pre-schedule operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.

Event devices can indicate pre-scheduling capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.

Applications can select the pre-schedule type and configure it through
`rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.

The supported pre-schedule types are:
 * `RTE_EVENT_DEV_PRESCHEDULE_NONE` - No pre-scheduling.
 * `RTE_EVENT_DEV_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
 * `RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
   there are no forward progress constraints with the held flow contexts.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 app/test/test_eventdev.c                    | 63 +++++++++++++++++++++
 doc/guides/prog_guide/eventdev/eventdev.rst | 22 +++++++
 lib/eventdev/rte_eventdev.h                 | 48 ++++++++++++++++
 3 files changed, 133 insertions(+)

diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index e4e234dc98..cf496ee88d 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1250,6 +1250,67 @@ test_eventdev_profile_switch(void)
 	return TEST_SUCCESS;
 }

+static int
+preschedule_test(rte_event_dev_preschedule_type_t preschedule_type, const char *preschedule_name)
+{
+#define NB_EVENTS     1024
+	uint64_t start, total;
+	struct rte_event ev;
+	int rc, cnt;
+
+	ev.event_type = RTE_EVENT_TYPE_CPU;
+	ev.queue_id = 0;
+	ev.op = RTE_EVENT_OP_NEW;
+	ev.u64 = 0xBADF00D0;
+
+	for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+		ev.flow_id = cnt;
+		rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+		TEST_ASSERT(rc == 1, "Failed to enqueue event");
+	}
+
+	RTE_SET_USED(preschedule_type);
+	total = 0;
+	while (cnt) {
+		start = rte_rdtsc_precise();
+		rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+		if (rc) {
+			total += rte_rdtsc_precise() - start;
+			cnt--;
+		}
+	}
+	printf("Preschedule type : %s, avg cycles %" PRIu64 "\n", preschedule_name,
+	       total / NB_EVENTS);
+
+	return TEST_SUCCESS;
+}
+
+static int
+test_eventdev_preschedule_configure(void)
+{
+	struct rte_event_dev_config dev_conf;
+	struct rte_event_dev_info info;
+	int rc;
+
+	rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+	if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE) == 0)
+		return TEST_SKIPPED;
+
+	devconf_set_default_sane_values(&dev_conf, &info);
+	dev_conf.preschedule_type = RTE_EVENT_DEV_PRESCHEDULE;
+	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+	rc = preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_NONE, "RTE_EVENT_DEV_PRESCHEDULE_NONE");
+	rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE, "RTE_EVENT_DEV_PRESCHEDULE");
+	if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE)
+		rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
+				       "RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE");
+
+	return rc;
+}
+
 static int
 test_eventdev_close(void)
 {
@@ -1310,6 +1371,8 @@ static struct unit_test_suite eventdev_common_testsuite  = {
 			test_eventdev_start_stop),
 		TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
 			test_eventdev_profile_switch),
+		TEST_CASE_ST(eventdev_configure_setup, NULL,
+			test_eventdev_preschedule_configure),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
 			test_eventdev_link),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index fb6dfce102..341b9bb2c6 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,28 @@ Worker path:
        // Process the event received.
    }

+Event Pre-scheduling
+~~~~~~~~~~~~~~~~~~~~
+
+Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
+again.
+
+An application can use event pre-scheduling if the event device supports it at either device
+level or at a individual port level.
+The application can check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` set, if present pre-scheduling can be enabled at device
+configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
+
+Currently, the following pre-schedule types are supported:
+ * ``RTE_EVENT_DEV_PRESCHEDULE_NONE`` - No pre-scheduling.
+ * ``RTE_EVENT_DEV_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
+ * ``RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
+   no forward progress constraints.
+
 Starting the EventDev
 ~~~~~~~~~~~~~~~~~~~~~

diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 08e5f9320b..5ea7f5a07b 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -446,6 +446,30 @@ struct rte_event;
  * @see RTE_SCHED_TYPE_PARALLEL
  */

+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16)
+/**< Event device supports event pre-scheduling.
+ *
+ * When this capability is available, the application can enable event pre-scheduling on the event
+ * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 17)
+/**< Event device supports adaptive event pre-scheduling.
+ *
+ * When this capability is available, the application can enable adaptive pre-scheduling
+ * on the event device where the events are pre-scheduled when there are no forward
+ * progress constraints with the currently held flow contexts.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
 /* Event device priority levels */
 #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
 /**< Highest priority level for events and queues.
@@ -680,6 +704,25 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
  *  @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
  */

+typedef enum {
+	RTE_EVENT_DEV_PRESCHEDULE_NONE = 0,
+	/* Disable pre-schedule across the event device or on a given event port.
+	 * @ref rte_event_dev_config.preschedule_type
+	 */
+	RTE_EVENT_DEV_PRESCHEDULE,
+	/* Enable pre-schedule always across the event device or a given event port.
+	 * @ref rte_event_dev_config.preschedule_type
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+	 */
+	RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
+	/* Enable adaptive pre-schedule across the event device or a given event port.
+	 * Delay issuing pre-schedule until there are no forward progress constraints with
+	 * the held flow contexts.
+	 * @ref rte_event_dev_config.preschedule_type
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+	 */
+} rte_event_dev_preschedule_type_t;
+
 /** Event device configuration structure */
 struct rte_event_dev_config {
 	uint32_t dequeue_timeout_ns;
@@ -752,6 +795,11 @@ struct rte_event_dev_config {
 	 * optimized for single-link usage, this field is a hint for how many
 	 * to allocate; otherwise, regular event ports and queues will be used.
 	 */
+	rte_event_dev_preschedule_type_t preschedule_type;
+	/**< Event pre-schedule type to use across the event device, if supported.
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+	 */
 };

 /**
--
2.25.1

[PATCH v2 2/3] eventdev: add event port pre-schedule modify

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Some event devices allow pre-schedule types to be modified at
runtime on an event port.
Add `RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE` capability
to indicate that the event device supports this feature.

Add `rte_event_port_preschedule_modify()` API to modify the
pre-schedule type at runtime.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 app/test/test_eventdev.c                    | 61 +++++++++++++++++++--
 doc/guides/prog_guide/eventdev/eventdev.rst | 12 ++++
 lib/eventdev/eventdev_pmd.h                 |  2 +
 lib/eventdev/eventdev_private.c             | 10 ++++
 lib/eventdev/eventdev_trace_points.c        |  3 +
 lib/eventdev/rte_eventdev.h                 | 55 +++++++++++++++++++
 lib/eventdev/rte_eventdev_core.h            |  6 ++
 lib/eventdev/rte_eventdev_trace_fp.h        | 11 +++-
 lib/eventdev/version.map                    |  4 ++
 9 files changed, 158 insertions(+), 6 deletions(-)

diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index cf496ee88d..6c799e3d11 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,7 +1251,8 @@ test_eventdev_profile_switch(void)
 }
 
 static int
-preschedule_test(rte_event_dev_preschedule_type_t preschedule_type, const char *preschedule_name)
+preschedule_test(rte_event_dev_preschedule_type_t preschedule_type, const char *preschedule_name,
+	      uint8_t modify)
 {
 #define NB_EVENTS     1024
 	uint64_t start, total;
@@ -1269,7 +1270,11 @@ preschedule_test(rte_event_dev_preschedule_type_t preschedule_type, const char *
 		TEST_ASSERT(rc == 1, "Failed to enqueue event");
 	}
 
-	RTE_SET_USED(preschedule_type);
+	if (modify) {
+		rc = rte_event_port_preschedule_modify(TEST_DEV_ID, 0, preschedule_type);
+		TEST_ASSERT_SUCCESS(rc, "Failed to modify preschedule type");
+	}
+
 	total = 0;
 	while (cnt) {
 		start = rte_rdtsc_precise();
@@ -1302,11 +1307,55 @@ test_eventdev_preschedule_configure(void)
 	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
 	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
 
-	rc = preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_NONE, "RTE_EVENT_DEV_PRESCHEDULE_NONE");
-	rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE, "RTE_EVENT_DEV_PRESCHEDULE");
+	rc = preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_NONE, "RTE_EVENT_DEV_PRESCHEDULE_NONE", 0);
+	rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE, "RTE_EVENT_DEV_PRESCHEDULE", 0);
 	if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE)
 		rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
-				       "RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE");
+				    "RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE", 0);
+
+	return rc;
+}
+
+static int
+test_eventdev_preschedule_modify(void)
+{
+	struct rte_event_dev_config dev_conf;
+	struct rte_event_queue_conf qcfg;
+	struct rte_event_port_conf pcfg;
+	struct rte_event_dev_info info;
+	int rc;
+
+	rte_event_dev_info_get(TEST_DEV_ID, &info);
+	if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE) == 0)
+		return TEST_SKIPPED;
+
+	devconf_set_default_sane_values(&dev_conf, &info);
+	dev_conf.preschedule_type = RTE_EVENT_DEV_PRESCHEDULE_NONE;
+	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+	rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+	rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+	rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+	rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+	TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+	rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+	TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+		    TEST_DEV_ID);
+
+	rc = rte_event_dev_start(TEST_DEV_ID);
+	TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+	rc = preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_NONE, "RTE_EVENT_DEV_PRESCHEDULE_NONE", 1);
+	rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE, "RTE_EVENT_DEV_PRESCHEDULE", 1);
+	if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE)
+		rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
+				    "RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE", 1);
 
 	return rc;
 }
@@ -1373,6 +1422,8 @@ static struct unit_test_suite eventdev_common_testsuite  = {
 			test_eventdev_profile_switch),
 		TEST_CASE_ST(eventdev_configure_setup, NULL,
 			test_eventdev_preschedule_configure),
+		TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
+			test_eventdev_preschedule_modify),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
 			test_eventdev_link),
 		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index 341b9bb2c6..2deab0333e 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -379,6 +379,18 @@ Currently, the following pre-schedule types are supported:
  * ``RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
    no forward progress constraints.
 
+To enable or disable event pre-scheduling at a given event port, the application can use
+``rte_event_port_preschedule_modify()`` API.
+
+.. code-block:: c
+
+   rte_event_port_preschedule_modify(dev_id, port_id, RTE_EVENT_DEV_PRESCHEDULE);
+   // Dequeue events from the event port with normal dequeue() function.
+   rte_event_port_preschedule_modify(dev_id, port_id, RTE_EVENT_DEV_PRESCHEDULE_NONE);
+   // Disable pre-scheduling if thread is about to be scheduled out and issue dequeue() to drain
+   // pending events.
+
+
 Starting the EventDev
 ~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index 7a5699f14b..9ea23aa6cd 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -184,6 +184,8 @@ struct __rte_cache_aligned rte_eventdev {
 	/**< Pointer to PMD DMA adapter enqueue function. */
 	event_profile_switch_t profile_switch;
 	/**< Pointer to PMD Event switch profile function. */
+	event_preschedule_modify_t preschedule_modify;
+	/**< Pointer to PMD Event port pre-schedule type modify function.  */
 
 	uint64_t reserved_64s[3]; /**< Reserved for future fields */
 	void *reserved_ptrs[3];	  /**< Reserved for future fields */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index 017f97ccab..f0aae766ea 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -96,6 +96,14 @@ dummy_event_port_profile_switch(__rte_unused void *port, __rte_unused uint8_t pr
 	return -EINVAL;
 }
 
+static int
+dummy_event_port_preschedule_modify(__rte_unused void *port,
+				    __rte_unused rte_event_dev_preschedule_type_t preschedule)
+{
+	RTE_EDEV_LOG_ERR("modify pre-schedule requested for unconfigured event device");
+	return -EINVAL;
+}
+
 void
 event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 {
@@ -114,6 +122,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 		.ca_enqueue = dummy_event_crypto_adapter_enqueue,
 		.dma_enqueue = dummy_event_dma_adapter_enqueue,
 		.profile_switch = dummy_event_port_profile_switch,
+		.preschedule_modify = dummy_event_port_preschedule_modify,
 		.data = dummy_data,
 	};
 
@@ -136,5 +145,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
 	fp_op->ca_enqueue = dev->ca_enqueue;
 	fp_op->dma_enqueue = dev->dma_enqueue;
 	fp_op->profile_switch = dev->profile_switch;
+	fp_op->preschedule_modify = dev->preschedule_modify;
 	fp_op->data = dev->data->ports;
 }
diff --git a/lib/eventdev/eventdev_trace_points.c b/lib/eventdev/eventdev_trace_points.c
index 8024e07531..e41674123c 100644
--- a/lib/eventdev/eventdev_trace_points.c
+++ b/lib/eventdev/eventdev_trace_points.c
@@ -49,6 +49,9 @@ RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_maintain,
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_profile_switch,
 	lib.eventdev.port.profile.switch)
 
+RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_preschedule_modify,
+	lib.eventdev.port.preschedule.modify)
+
 /* Eventdev Rx adapter trace points */
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_eth_rx_adapter_create,
 	lib.eventdev.rx.adapter.create)
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 5ea7f5a07b..0add0093ac 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -470,6 +470,16 @@ struct rte_event;
  * @see rte_event_dev_configure()
  */
 
+#define RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE (1ULL << 18)
+/**< Event device supports event pre-scheduling per event port.
+ *
+ * When this flag is set, the event device allows controlling the event
+ * pre-scheduling at a event port granularity.
+ *
+ * @see rte_event_dev_configure()
+ * @see rte_event_port_preschedule_modify()
+ */
+
 /* Event device priority levels */
 #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
 /**< Highest priority level for events and queues.
@@ -708,18 +718,23 @@ typedef enum {
 	RTE_EVENT_DEV_PRESCHEDULE_NONE = 0,
 	/* Disable pre-schedule across the event device or on a given event port.
 	 * @ref rte_event_dev_config.preschedule_type
+	 * @ref rte_event_port_preschedule_modify()
 	 */
 	RTE_EVENT_DEV_PRESCHEDULE,
 	/* Enable pre-schedule always across the event device or a given event port.
 	 * @ref rte_event_dev_config.preschedule_type
+	 * @ref rte_event_port_preschedule_modify()
 	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE
 	 */
 	RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
 	/* Enable adaptive pre-schedule across the event device or a given event port.
 	 * Delay issuing pre-schedule until there are no forward progress constraints with
 	 * the held flow contexts.
 	 * @ref rte_event_dev_config.preschedule_type
+	 * @ref rte_event_port_preschedule_modify()
 	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+	 * @see RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE
 	 */
 } rte_event_dev_preschedule_type_t;
 
@@ -2922,6 +2937,46 @@ rte_event_port_profile_switch(uint8_t dev_id, uint8_t port_id, uint8_t profile_i
 	return fp_ops->profile_switch(port, profile_id);
 }
 
+/**
+ * Change the pre-schedule type to use on an event port.
+ *
+ * This function is used to change the current pre-schedule type configured
+ * on an event port, the pre-schedule type can be set to none to disable pre-scheduling.
+ * This effects the subsequent ``rte_event_dequeue_burst`` call.
+ * The event device should support RTE_EVENT_DEV_CAP_PER_PORT_PRESCHEDULE capability.
+ *
+ * @param dev_id
+ *   The identifier of the device.
+ * @param port_id
+ *   The identifier of the event port.
+ * @param type
+ *   The preschedule type to use on the event port.
+ * @return
+ *  - 0 on success.
+ *  - -EINVAL if *dev_id*,  *port_id*, or *type* is invalid.
+ */
+static inline int
+rte_event_port_preschedule_modify(uint8_t dev_id, uint8_t port_id,
+				  rte_event_dev_preschedule_type_t type)
+{
+	const struct rte_event_fp_ops *fp_ops;
+	void *port;
+
+	fp_ops = &rte_event_fp_ops[dev_id];
+	port = fp_ops->data[port_id];
+
+#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
+	if (dev_id >= RTE_EVENT_MAX_DEVS || port_id >= RTE_EVENT_MAX_PORTS_PER_DEV)
+		return -EINVAL;
+
+	if (port == NULL)
+		return -EINVAL;
+#endif
+	rte_eventdev_trace_port_preschedule_modify(dev_id, port_id, type);
+
+	return fp_ops->preschedule_modify(port, type);
+}
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index fc8e1556ab..2275888a6b 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -49,6 +49,10 @@ typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[
 typedef int (*event_profile_switch_t)(void *port, uint8_t profile);
 /**< @internal Switch active link profile on the event port. */
 
+typedef int (*event_preschedule_modify_t)(void *port,
+					  rte_event_dev_preschedule_type_t preschedule_type);
+/**< @internal Modify pre-schedule type on the event port. */
+
 struct __rte_cache_aligned rte_event_fp_ops {
 	void **data;
 	/**< points to array of internal port data pointers */
@@ -76,6 +80,8 @@ struct __rte_cache_aligned rte_event_fp_ops {
 	/**< PMD DMA adapter enqueue function. */
 	event_profile_switch_t profile_switch;
 	/**< PMD Event switch profile function. */
+	event_preschedule_modify_t preschedule_modify;
+	/**< PMD Event port pre-schedule switch. */
 	uintptr_t reserved[4];
 };
 
diff --git a/lib/eventdev/rte_eventdev_trace_fp.h b/lib/eventdev/rte_eventdev_trace_fp.h
index 04d510ad00..78baed94de 100644
--- a/lib/eventdev/rte_eventdev_trace_fp.h
+++ b/lib/eventdev/rte_eventdev_trace_fp.h
@@ -8,7 +8,7 @@
 /**
  * @file
  *
- * API for ethdev trace support
+ * API for eventdev trace support
  */
 
 #ifdef __cplusplus
@@ -54,6 +54,15 @@ RTE_TRACE_POINT_FP(
 	rte_trace_point_emit_u8(profile);
 )
 
+RTE_TRACE_POINT_FP(
+	rte_eventdev_trace_port_preschedule_modify,
+	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id,
+			     int type),
+	rte_trace_point_emit_u8(dev_id);
+	rte_trace_point_emit_u8(port_id);
+	rte_trace_point_emit_int(type);
+)
+
 RTE_TRACE_POINT_FP(
 	rte_eventdev_trace_eth_tx_adapter_enqueue,
 	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, void *ev_table,
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 4947bb4ec6..b6d63ba576 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -147,6 +147,10 @@ EXPERIMENTAL {
 	rte_event_port_profile_unlink;
 	rte_event_port_profile_links_get;
 	__rte_eventdev_trace_port_profile_switch;
+
+	# added in 24.11
+	rte_event_port_preschedule_modify;
+	__rte_eventdev_trace_port_preschedule_modify;
 };
 
 INTERNAL {
-- 
2.25.1

[PATCH v2 3/3] eventdev: add SW event preschedule hint

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Add a new eventdev API to provide a hint to the eventdev PMD to
pre-schedule the next event into the event port, without releasing
the current flow context.
Event device that support this feature advertises the capability
using the RTE_EVENT_DEV_CAP_SW_PRESCHEDULE capability flag.

Application can invoke `rte_event_port_preschedule` to hint the PMD.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 doc/guides/prog_guide/eventdev/eventdev.rst |  8 ++++
 lib/eventdev/eventdev_pmd.h                 |  2 +
 lib/eventdev/eventdev_private.c             | 12 ++++-
 lib/eventdev/eventdev_trace_points.c        |  3 ++
 lib/eventdev/rte_eventdev.h                 | 49 +++++++++++++++++++++
 lib/eventdev/rte_eventdev_core.h            |  5 +++
 lib/eventdev/rte_eventdev_trace_fp.h        |  8 ++++
 lib/eventdev/version.map                    |  2 +
 8 files changed, 87 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index 2deab0333e..1d8b86ab66 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -390,6 +390,14 @@ To enable or disable event pre-scheduling at a given event port, the application
    // Disable pre-scheduling if thread is about to be scheduled out and issue dequeue() to drain
    // pending events.
 
+Event Pre-schedule Hint can be used to provide a hint to the eventdev PMD to pre-schedule the next
+event without releasing the current flow context. Event device that support this feature advertises
+the capability using the ``RTE_EVENT_DEV_CAP_SW_PRESCHEDULE`` capability flag.
+If pre-scheduling is already enabled at a event device or event port level then the hint is ignored.
+
+.. code-block:: c
+
+   rte_event_port_preschedule(dev_id, port_id, RTE_EVENT_DEV_PRESCHEDULE);
 
 Starting the EventDev
 ~~~~~~~~~~~~~~~~~~~~~
diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
index 9ea23aa6cd..0bee2347ef 100644
--- a/lib/eventdev/eventdev_pmd.h
+++ b/lib/eventdev/eventdev_pmd.h
@@ -186,6 +186,8 @@ struct __rte_cache_aligned rte_eventdev {
 	/**< Pointer to PMD Event switch profile function. */
 	event_preschedule_modify_t preschedule_modify;
 	/**< Pointer to PMD Event port pre-schedule type modify function.  */
+	event_preschedule_t preschedule;
+	/**< Pointer to PMD Event port pre-schedule function. */
 
 	uint64_t reserved_64s[3]; /**< Reserved for future fields */
 	void *reserved_ptrs[3];	  /**< Reserved for future fields */
diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
index f0aae766ea..ed6e12dece 100644
--- a/lib/eventdev/eventdev_private.c
+++ b/lib/eventdev/eventdev_private.c
@@ -104,6 +104,13 @@ dummy_event_port_preschedule_modify(__rte_unused void *port,
 	return -EINVAL;
 }
 
+static void
+dummy_event_port_preschedule(__rte_unused void *port,
+			     __rte_unused rte_event_dev_preschedule_type_t preschedule)
+{
+	RTE_EDEV_LOG_ERR("pre-schedule requested for unconfigured event device");
+}
+
 void
 event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 {
@@ -117,12 +124,12 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
 		.dequeue_burst = dummy_event_dequeue_burst,
 		.maintain = dummy_event_maintain,
 		.txa_enqueue = dummy_event_tx_adapter_enqueue,
-		.txa_enqueue_same_dest =
-			dummy_event_tx_adapter_enqueue_same_dest,
+		.txa_enqueue_same_dest = dummy_event_tx_adapter_enqueue_same_dest,
 		.ca_enqueue = dummy_event_crypto_adapter_enqueue,
 		.dma_enqueue = dummy_event_dma_adapter_enqueue,
 		.profile_switch = dummy_event_port_profile_switch,
 		.preschedule_modify = dummy_event_port_preschedule_modify,
+		.preschedule = dummy_event_port_preschedule,
 		.data = dummy_data,
 	};
 
@@ -146,5 +153,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
 	fp_op->dma_enqueue = dev->dma_enqueue;
 	fp_op->profile_switch = dev->profile_switch;
 	fp_op->preschedule_modify = dev->preschedule_modify;
+	fp_op->preschedule = dev->preschedule;
 	fp_op->data = dev->data->ports;
 }
diff --git a/lib/eventdev/eventdev_trace_points.c b/lib/eventdev/eventdev_trace_points.c
index e41674123c..e7af1591f7 100644
--- a/lib/eventdev/eventdev_trace_points.c
+++ b/lib/eventdev/eventdev_trace_points.c
@@ -52,6 +52,9 @@ RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_profile_switch,
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_preschedule_modify,
 	lib.eventdev.port.preschedule.modify)
 
+RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_preschedule,
+	lib.eventdev.port.preschedule)
+
 /* Eventdev Rx adapter trace points */
 RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_eth_rx_adapter_create,
 	lib.eventdev.rx.adapter.create)
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 0add0093ac..8df6a8bee1 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -480,6 +480,15 @@ struct rte_event;
  * @see rte_event_port_preschedule_modify()
  */
 
+#define RTE_EVENT_DEV_CAP_SW_PRESCHEDULE (1ULL << 19)
+/**< Event device supports software prescheduling.
+ *
+ * When this flag is set, the application can issue preschedule request on
+ * a event port.
+ *
+ * @see rte_event_port_preschedule()
+ */
+
 /* Event device priority levels */
 #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
 /**< Highest priority level for events and queues.
@@ -2977,6 +2986,46 @@ rte_event_port_preschedule_modify(uint8_t dev_id, uint8_t port_id,
 	return fp_ops->preschedule_modify(port, type);
 }
 
+/**
+ * Provide a hint to the event device to pre-schedule events to event port .
+ *
+ * Hint the event device to pre-schedule events to the event port.
+ * The call doesn't not guarantee that the events will be pre-scheduleed.
+ * The call doesn't release the flow context currently held by the event port.
+ * The event device should support RTE_EVENT_DEV_CAP_SW_PRESCHEDULE capability.
+ *
+ * When pre-scheduling is enabled at an event device or event port level, the
+ * hint is ignored.
+ *
+ * Subsequent calls to rte_event_dequeue_burst() will dequeue the pre-schedule
+ * events but pre-schedule operation is not issued again.
+ *
+ * @param dev_id
+ *   The identifier of the device.
+ * @param port_id
+ *   The identifier of the event port.
+ * @param type
+ *   The pre-schedule type to use on the event port.
+ */
+static inline void
+rte_event_port_preschedule(uint8_t dev_id, uint8_t port_id, rte_event_dev_preschedule_type_t type)
+{
+	const struct rte_event_fp_ops *fp_ops;
+	void *port;
+
+	fp_ops = &rte_event_fp_ops[dev_id];
+	port = fp_ops->data[port_id];
+
+#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
+	if (dev_id >= RTE_EVENT_MAX_DEVS || port_id >= RTE_EVENT_MAX_PORTS_PER_DEV)
+		return;
+	if (port == NULL)
+		return;
+#endif
+	rte_eventdev_trace_port_preschedule(dev_id, port_id, type);
+
+	fp_ops->preschedule(port, type);
+}
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
index 2275888a6b..21988abb4f 100644
--- a/lib/eventdev/rte_eventdev_core.h
+++ b/lib/eventdev/rte_eventdev_core.h
@@ -53,6 +53,9 @@ typedef int (*event_preschedule_modify_t)(void *port,
 					  rte_event_dev_preschedule_type_t preschedule_type);
 /**< @internal Modify pre-schedule type on the event port. */
 
+typedef void (*event_preschedule_t)(void *port, rte_event_dev_preschedule_type_t preschedule_type);
+/**< @internal Issue pre-schedule on an event port. */
+
 struct __rte_cache_aligned rte_event_fp_ops {
 	void **data;
 	/**< points to array of internal port data pointers */
@@ -82,6 +85,8 @@ struct __rte_cache_aligned rte_event_fp_ops {
 	/**< PMD Event switch profile function. */
 	event_preschedule_modify_t preschedule_modify;
 	/**< PMD Event port pre-schedule switch. */
+	event_preschedule_t preschedule;
+	/**< PMD Event port pre-schedule. */
 	uintptr_t reserved[4];
 };
 
diff --git a/lib/eventdev/rte_eventdev_trace_fp.h b/lib/eventdev/rte_eventdev_trace_fp.h
index 78baed94de..8290f8a248 100644
--- a/lib/eventdev/rte_eventdev_trace_fp.h
+++ b/lib/eventdev/rte_eventdev_trace_fp.h
@@ -63,6 +63,14 @@ RTE_TRACE_POINT_FP(
 	rte_trace_point_emit_int(type);
 )
 
+RTE_TRACE_POINT_FP(
+	rte_eventdev_trace_port_preschedule,
+	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, int type),
+	rte_trace_point_emit_u8(dev_id);
+	rte_trace_point_emit_u8(port_id);
+	rte_trace_point_emit_int(type);
+)
+
 RTE_TRACE_POINT_FP(
 	rte_eventdev_trace_eth_tx_adapter_enqueue,
 	RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, void *ev_table,
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index b6d63ba576..42a5867aba 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -151,6 +151,8 @@ EXPERIMENTAL {
 	# added in 24.11
 	rte_event_port_preschedule_modify;
 	__rte_eventdev_trace_port_preschedule_modify;
+	rte_event_port_preschedule;
+	__rte_eventdev_trace_port_preschedule;
 };
 
 INTERNAL {
-- 
2.25.1

RE: [PATCH v2 1/3] eventdev: introduce event pre-scheduling

[Pathak, Pravin]

> -----Original Message-----
> From: pbhagavatula@marvell.com <pbhagavatula@marvell.com>
> Sent: Tuesday, September 17, 2024 3:11 AM
> To: jerinj@marvell.com; sthotton@marvell.com; Sevincer, Abdullah
> <abdullah.sevincer@intel.com>; hemant.agrawal@nxp.com;
> sachin.saxena@oss.nxp.com; Van Haaren, Harry <harry.van.haaren@intel.com>;
> mattias.ronnblom@ericsson.com; liangma@liangbit.com; Mccarthy, Peter
> <peter.mccarthy@intel.com>
> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
> Subject: [PATCH v2 1/3] eventdev: introduce event pre-scheduling
> 
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Event pre-scheduling improves scheduling performance by assigning events to
> event ports in advance when dequeues are issued.
> The dequeue operation initiates the pre-schedule operation, which completes in
> parallel without affecting the dequeued event flow contexts and dequeue
> latency.
> 
Is the prescheduling done to get the event more quickly in the next dequeue?
The first dequeue executes pre-schedule to make events available for the next dequeue.
Is this how it is supposed to work?

> Event devices can indicate pre-scheduling capabilities using
> `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
> `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device
> info function `info.event_dev_cap`.
> 
> Applications can select the pre-schedule type and configure it through
> `rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
> 
> The supported pre-schedule types are:
>  * `RTE_EVENT_DEV_PRESCHEDULE_NONE` - No pre-scheduling.
>  * `RTE_EVENT_DEV_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
>  * `RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule
> until
>    there are no forward progress constraints with the held flow contexts.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> ---
>  app/test/test_eventdev.c                    | 63 +++++++++++++++++++++
>  doc/guides/prog_guide/eventdev/eventdev.rst | 22 +++++++
>  lib/eventdev/rte_eventdev.h                 | 48 ++++++++++++++++
>  3 files changed, 133 insertions(+)
> 
> diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c index
> e4e234dc98..cf496ee88d 100644
> --- a/app/test/test_eventdev.c
> +++ b/app/test/test_eventdev.c
> @@ -1250,6 +1250,67 @@ test_eventdev_profile_switch(void)
>  	return TEST_SUCCESS;
>  }
> 
> +static int
> +preschedule_test(rte_event_dev_preschedule_type_t preschedule_type,
> +const char *preschedule_name) {
> +#define NB_EVENTS     1024
> +	uint64_t start, total;
> +	struct rte_event ev;
> +	int rc, cnt;
> +
> +	ev.event_type = RTE_EVENT_TYPE_CPU;
> +	ev.queue_id = 0;
> +	ev.op = RTE_EVENT_OP_NEW;
> +	ev.u64 = 0xBADF00D0;
> +
> +	for (cnt = 0; cnt < NB_EVENTS; cnt++) {
> +		ev.flow_id = cnt;
> +		rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
> +		TEST_ASSERT(rc == 1, "Failed to enqueue event");
> +	}
> +
> +	RTE_SET_USED(preschedule_type);
> +	total = 0;
> +	while (cnt) {
> +		start = rte_rdtsc_precise();
> +		rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
> +		if (rc) {
> +			total += rte_rdtsc_precise() - start;
> +			cnt--;
> +		}
> +	}
> +	printf("Preschedule type : %s, avg cycles %" PRIu64 "\n",
> preschedule_name,
> +	       total / NB_EVENTS);
> +
> +	return TEST_SUCCESS;
> +}
> +
> +static int
> +test_eventdev_preschedule_configure(void)
> +{
> +	struct rte_event_dev_config dev_conf;
> +	struct rte_event_dev_info info;
> +	int rc;
> +
> +	rte_event_dev_info_get(TEST_DEV_ID, &info);
> +
> +	if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE)
> == 0)
> +		return TEST_SKIPPED;
> +
> +	devconf_set_default_sane_values(&dev_conf, &info);
> +	dev_conf.preschedule_type = RTE_EVENT_DEV_PRESCHEDULE;
> +	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
> +	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
> +
> +	rc = preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_NONE,
> "RTE_EVENT_DEV_PRESCHEDULE_NONE");
> +	rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE,
> "RTE_EVENT_DEV_PRESCHEDULE");
> +	if (info.event_dev_cap &
> RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE)
> +		rc |=
> preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
> +
> "RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE");
> +
> +	return rc;
> +}
> +
>  static int
>  test_eventdev_close(void)
>  {
> @@ -1310,6 +1371,8 @@ static struct unit_test_suite
> eventdev_common_testsuite  = {
>  			test_eventdev_start_stop),
>  		TEST_CASE_ST(eventdev_configure_setup,
> eventdev_stop_device,
>  			test_eventdev_profile_switch),
> +		TEST_CASE_ST(eventdev_configure_setup, NULL,
> +			test_eventdev_preschedule_configure),
>  		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
>  			test_eventdev_link),
>  		TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
> diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst
> b/doc/guides/prog_guide/eventdev/eventdev.rst
> index fb6dfce102..341b9bb2c6 100644
> --- a/doc/guides/prog_guide/eventdev/eventdev.rst
> +++ b/doc/guides/prog_guide/eventdev/eventdev.rst
> @@ -357,6 +357,28 @@ Worker path:
>         // Process the event received.
>     }
> 
> +Event Pre-scheduling
> +~~~~~~~~~~~~~~~~~~~~
> +
> +Event pre-scheduling improves scheduling performance by assigning
> +events to event ports in advance when dequeues are issued.
> +The `rte_event_dequeue_burst` operation initiates the pre-schedule
> +operation, which completes in parallel without affecting the dequeued event
> flow contexts and dequeue latency.
> +On the next dequeue operation, the pre-scheduled events are dequeued
> +and pre-schedule is initiated again.
> +
> +An application can use event pre-scheduling if the event device
> +supports it at either device level or at a individual port level.
> +The application can check pre-schedule capability by checking if
> +``rte_event_dev_info.event_dev_cap``
> +has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` set, if present
> +pre-scheduling can be enabled at device configuration time by setting
> appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
> +
> +Currently, the following pre-schedule types are supported:
> + * ``RTE_EVENT_DEV_PRESCHEDULE_NONE`` - No pre-scheduling.
> + * ``RTE_EVENT_DEV_PRESCHEDULE`` - Always issue a pre-schedule when
> dequeue is issued.
> + * ``RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when
> dequeue is issued and there are
> +   no forward progress constraints.
> +
>  Starting the EventDev
>  ~~~~~~~~~~~~~~~~~~~~~
> 
> diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h index
> 08e5f9320b..5ea7f5a07b 100644
> --- a/lib/eventdev/rte_eventdev.h
> +++ b/lib/eventdev/rte_eventdev.h
> @@ -446,6 +446,30 @@ struct rte_event;
>   * @see RTE_SCHED_TYPE_PARALLEL
>   */
> 
> +#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16) /**< Event
> +device supports event pre-scheduling.
> + *
> + * When this capability is available, the application can enable event
> +pre-scheduling on the event
> + * device to pre-schedule events to a event port when
> +`rte_event_dequeue_burst()`
> + * is issued.
> + * The pre-schedule process starts with the `rte_event_dequeue_burst()`
> +call and the
> + * pre-scheduled events are returned on the next `rte_event_dequeue_burst()`
> call.
> + *
> + * @see rte_event_dev_configure()
> + */
> +
> +#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 17)
> /**<
> +Event device supports adaptive event pre-scheduling.
> + *
> + * When this capability is available, the application can enable
> +adaptive pre-scheduling
> + * on the event device where the events are pre-scheduled when there
> +are no forward
> + * progress constraints with the currently held flow contexts.
> + * The pre-schedule process starts with the `rte_event_dequeue_burst()`
> +call and the
> + * pre-scheduled events are returned on the next `rte_event_dequeue_burst()`
> call.
> + *
> + * @see rte_event_dev_configure()
> + */
> +
>  /* Event device priority levels */
>  #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
>  /**< Highest priority level for events and queues.
> @@ -680,6 +704,25 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t
> attr_id,
>   *  @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
>   */
> 
> +typedef enum {
> +	RTE_EVENT_DEV_PRESCHEDULE_NONE = 0,
> +	/* Disable pre-schedule across the event device or on a given event port.
> +	 * @ref rte_event_dev_config.preschedule_type
> +	 */
> +	RTE_EVENT_DEV_PRESCHEDULE,
> +	/* Enable pre-schedule always across the event device or a given event
> port.
> +	 * @ref rte_event_dev_config.preschedule_type
> +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> +	 */
> +	RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
> +	/* Enable adaptive pre-schedule across the event device or a given event
> port.
> +	 * Delay issuing pre-schedule until there are no forward progress
> constraints with
> +	 * the held flow contexts.
> +	 * @ref rte_event_dev_config.preschedule_type
> +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> +	 */
> +} rte_event_dev_preschedule_type_t;
> +
>  /** Event device configuration structure */  struct rte_event_dev_config {
>  	uint32_t dequeue_timeout_ns;
> @@ -752,6 +795,11 @@ struct rte_event_dev_config {
>  	 * optimized for single-link usage, this field is a hint for how many
>  	 * to allocate; otherwise, regular event ports and queues will be used.
>  	 */
> +	rte_event_dev_preschedule_type_t preschedule_type;
> +	/**< Event pre-schedule type to use across the event device, if
> supported.
> +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> +	 */
>  };
> 
>  /**
> --
> 2.25.1

RE: [PATCH v2 1/3] eventdev: introduce event pre-scheduling

[Pavan Nikhilesh Bhagavatula]

> > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com>
> > Sent: Tuesday, September 17, 2024 3:11 AM
> > To: jerinj@marvell.com; sthotton@marvell.com; Sevincer, Abdullah
> > <abdullah.sevincer@intel.com>; hemant.agrawal@nxp.com;
> > sachin.saxena@oss.nxp.com; Van Haaren, Harry
> <harry.van.haaren@intel.com>;
> > mattias.ronnblom@ericsson.com; liangma@liangbit.com; Mccarthy, Peter
> > <peter.mccarthy@intel.com>
> > Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
> > Subject: [PATCH v2 1/3] eventdev: introduce event pre-scheduling
> >
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >
> > Event pre-scheduling improves scheduling performance by assigning events
> to
> > event ports in advance when dequeues are issued.
> > The dequeue operation initiates the pre-schedule operation, which
> completes in
> > parallel without affecting the dequeued event flow contexts and dequeue
> > latency.
> >
> Is the prescheduling done to get the event more quickly in the next dequeue?
> The first dequeue executes pre-schedule to make events available for the next
> dequeue.
> Is this how it is supposed to work?
> 

Yes, that is correct.

> > Event devices can indicate pre-scheduling capabilities using
> > `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
> > `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event
> device
> > info function `info.event_dev_cap`.
> >
> > Applications can select the pre-schedule type and configure it through
> > `rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
> >
> > The supported pre-schedule types are:
> >  * `RTE_EVENT_DEV_PRESCHEDULE_NONE` - No pre-scheduling.
> >  * `RTE_EVENT_DEV_PRESCHEDULE` - Always issue a pre-schedule on
> dequeue.
> >  * `RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule
> > until
> >    there are no forward progress constraints with the held flow contexts.
> >
> > Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > ---
> >  app/test/test_eventdev.c                    | 63 +++++++++++++++++++++
> >  doc/guides/prog_guide/eventdev/eventdev.rst | 22 +++++++
> >  lib/eventdev/rte_eventdev.h                 | 48 ++++++++++++++++
> >  3 files changed, 133 insertions(+)
> >
> > diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c index
> > e4e234dc98..cf496ee88d 100644
> > --- a/app/test/test_eventdev.c
> > +++ b/app/test/test_eventdev.c
> > @@ -1250,6 +1250,67 @@ test_eventdev_profile_switch(void)
> >  	return TEST_SUCCESS;
> >  }
> >
> > +static int
> > +preschedule_test(rte_event_dev_preschedule_type_t preschedule_type,
> > +const char *preschedule_name) {
> > +#define NB_EVENTS     1024
> > +	uint64_t start, total;
> > +	struct rte_event ev;
> > +	int rc, cnt;
> > +
> > +	ev.event_type = RTE_EVENT_TYPE_CPU;
> > +	ev.queue_id = 0;
> > +	ev.op = RTE_EVENT_OP_NEW;
> > +	ev.u64 = 0xBADF00D0;
> > +
> > +	for (cnt = 0; cnt < NB_EVENTS; cnt++) {
> > +		ev.flow_id = cnt;
> > +		rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
> > +		TEST_ASSERT(rc == 1, "Failed to enqueue event");
> > +	}
> > +
> > +	RTE_SET_USED(preschedule_type);
> > +	total = 0;
> > +	while (cnt) {
> > +		start = rte_rdtsc_precise();
> > +		rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
> > +		if (rc) {
> > +			total += rte_rdtsc_precise() - start;
> > +			cnt--;
> > +		}
> > +	}
> > +	printf("Preschedule type : %s, avg cycles %" PRIu64 "\n",
> > preschedule_name,
> > +	       total / NB_EVENTS);
> > +
> > +	return TEST_SUCCESS;
> > +}
> > +
> > +static int
> > +test_eventdev_preschedule_configure(void)
> > +{
> > +	struct rte_event_dev_config dev_conf;
> > +	struct rte_event_dev_info info;
> > +	int rc;
> > +
> > +	rte_event_dev_info_get(TEST_DEV_ID, &info);
> > +
> > +	if ((info.event_dev_cap &
> RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE)
> > == 0)
> > +		return TEST_SKIPPED;
> > +
> > +	devconf_set_default_sane_values(&dev_conf, &info);
> > +	dev_conf.preschedule_type = RTE_EVENT_DEV_PRESCHEDULE;
> > +	rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
> > +	TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
> > +
> > +	rc = preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_NONE,
> > "RTE_EVENT_DEV_PRESCHEDULE_NONE");
> > +	rc |= preschedule_test(RTE_EVENT_DEV_PRESCHEDULE,
> > "RTE_EVENT_DEV_PRESCHEDULE");
> > +	if (info.event_dev_cap &
> > RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE)
> > +		rc |=
> > preschedule_test(RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
> > +
> > "RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE");
> > +
> > +	return rc;
> > +}
> > +
> >  static int
> >  test_eventdev_close(void)
> >  {
> > @@ -1310,6 +1371,8 @@ static struct unit_test_suite
> > eventdev_common_testsuite  = {
> >  			test_eventdev_start_stop),
> >  		TEST_CASE_ST(eventdev_configure_setup,
> > eventdev_stop_device,
> >  			test_eventdev_profile_switch),
> > +		TEST_CASE_ST(eventdev_configure_setup, NULL,
> > +			test_eventdev_preschedule_configure),
> >  		TEST_CASE_ST(eventdev_setup_device,
> eventdev_stop_device,
> >  			test_eventdev_link),
> >  		TEST_CASE_ST(eventdev_setup_device,
> eventdev_stop_device,
> > diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst
> > b/doc/guides/prog_guide/eventdev/eventdev.rst
> > index fb6dfce102..341b9bb2c6 100644
> > --- a/doc/guides/prog_guide/eventdev/eventdev.rst
> > +++ b/doc/guides/prog_guide/eventdev/eventdev.rst
> > @@ -357,6 +357,28 @@ Worker path:
> >         // Process the event received.
> >     }
> >
> > +Event Pre-scheduling
> > +~~~~~~~~~~~~~~~~~~~~
> > +
> > +Event pre-scheduling improves scheduling performance by assigning
> > +events to event ports in advance when dequeues are issued.
> > +The `rte_event_dequeue_burst` operation initiates the pre-schedule
> > +operation, which completes in parallel without affecting the dequeued
> event
> > flow contexts and dequeue latency.
> > +On the next dequeue operation, the pre-scheduled events are dequeued
> > +and pre-schedule is initiated again.
> > +
> > +An application can use event pre-scheduling if the event device
> > +supports it at either device level or at a individual port level.
> > +The application can check pre-schedule capability by checking if
> > +``rte_event_dev_info.event_dev_cap``
> > +has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` set, if present
> > +pre-scheduling can be enabled at device configuration time by setting
> > appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
> > +
> > +Currently, the following pre-schedule types are supported:
> > + * ``RTE_EVENT_DEV_PRESCHEDULE_NONE`` - No pre-scheduling.
> > + * ``RTE_EVENT_DEV_PRESCHEDULE`` - Always issue a pre-schedule when
> > dequeue is issued.
> > + * ``RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when
> > dequeue is issued and there are
> > +   no forward progress constraints.
> > +
> >  Starting the EventDev
> >  ~~~~~~~~~~~~~~~~~~~~~
> >
> > diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
> index
> > 08e5f9320b..5ea7f5a07b 100644
> > --- a/lib/eventdev/rte_eventdev.h
> > +++ b/lib/eventdev/rte_eventdev.h
> > @@ -446,6 +446,30 @@ struct rte_event;
> >   * @see RTE_SCHED_TYPE_PARALLEL
> >   */
> >
> > +#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16) /**<
> Event
> > +device supports event pre-scheduling.
> > + *
> > + * When this capability is available, the application can enable event
> > +pre-scheduling on the event
> > + * device to pre-schedule events to a event port when
> > +`rte_event_dequeue_burst()`
> > + * is issued.
> > + * The pre-schedule process starts with the `rte_event_dequeue_burst()`
> > +call and the
> > + * pre-scheduled events are returned on the next
> `rte_event_dequeue_burst()`
> > call.
> > + *
> > + * @see rte_event_dev_configure()
> > + */
> > +
> > +#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL <<
> 17)
> > /**<
> > +Event device supports adaptive event pre-scheduling.
> > + *
> > + * When this capability is available, the application can enable
> > +adaptive pre-scheduling
> > + * on the event device where the events are pre-scheduled when there
> > +are no forward
> > + * progress constraints with the currently held flow contexts.
> > + * The pre-schedule process starts with the `rte_event_dequeue_burst()`
> > +call and the
> > + * pre-scheduled events are returned on the next
> `rte_event_dequeue_burst()`
> > call.
> > + *
> > + * @see rte_event_dev_configure()
> > + */
> > +
> >  /* Event device priority levels */
> >  #define RTE_EVENT_DEV_PRIORITY_HIGHEST   0
> >  /**< Highest priority level for events and queues.
> > @@ -680,6 +704,25 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t
> > attr_id,
> >   *  @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
> >   */
> >
> > +typedef enum {
> > +	RTE_EVENT_DEV_PRESCHEDULE_NONE = 0,
> > +	/* Disable pre-schedule across the event device or on a given event
> port.
> > +	 * @ref rte_event_dev_config.preschedule_type
> > +	 */
> > +	RTE_EVENT_DEV_PRESCHEDULE,
> > +	/* Enable pre-schedule always across the event device or a given event
> > port.
> > +	 * @ref rte_event_dev_config.preschedule_type
> > +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> > +	 */
> > +	RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
> > +	/* Enable adaptive pre-schedule across the event device or a given
> event
> > port.
> > +	 * Delay issuing pre-schedule until there are no forward progress
> > constraints with
> > +	 * the held flow contexts.
> > +	 * @ref rte_event_dev_config.preschedule_type
> > +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> > +	 */
> > +} rte_event_dev_preschedule_type_t;
> > +
> >  /** Event device configuration structure */  struct rte_event_dev_config {
> >  	uint32_t dequeue_timeout_ns;
> > @@ -752,6 +795,11 @@ struct rte_event_dev_config {
> >  	 * optimized for single-link usage, this field is a hint for how many
> >  	 * to allocate; otherwise, regular event ports and queues will be used.
> >  	 */
> > +	rte_event_dev_preschedule_type_t preschedule_type;
> > +	/**< Event pre-schedule type to use across the event device, if
> > supported.
> > +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> > +	 * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> > +	 */
> >  };
> >
> >  /**
> > --
> > 2.25.1