DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets
@ 2021-09-09  5:57 Xuan Ding
  2021-09-15  6:49 ` Xia, Chenbo
                   ` (5 more replies)
  0 siblings, 6 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-09  5:57 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia
  Cc: jiayu.hu, cheng.jiang, bruce.richardson, sunil.pai.g, Xuan Ding

In async data path, when vring state changes, it is necessary to
know the number of inflight packets in DMA engine. This patch
provides a thread unsafe API to return the number of inflight
packets without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 25 +++++++++++++++++++++++++
 5 files changed, 52 insertions(+)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 8874033165..b4b1134f54 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+    ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 675b573834..db080e9490 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
index b25ff446f7..0af414bf78 100644
--- a/lib/vhost/rte_vhost_async.h
+++ b/lib/vhost/rte_vhost_async.h
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index c92a9d4962..b150dc408d 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
index 355ff37651..df96f84873 100644
--- a/lib/vhost/vhost.c
+++ b/lib/vhost/vhost.c
@@ -1886,5 +1886,30 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets
  2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
@ 2021-09-15  6:49 ` Xia, Chenbo
  2021-09-15  7:10   ` Ding, Xuan
  2021-09-15  8:52 ` [dpdk-dev] [PATCH v2 v2] " Xuan Ding
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 19+ messages in thread
From: Xia, Chenbo @ 2021-09-15  6:49 UTC (permalink / raw)
  To: Ding, Xuan, dev, maxime.coquelin
  Cc: Hu, Jiayu, cheng.jiang, Richardson, Bruce, Pai G, Sunil

Hi Xuan,

> -----Original Message-----
> From: Ding, Xuan <xuan.ding@intel.com>
> Sent: Thursday, September 9, 2021 1:58 PM
> To: dev@dpdk.org; maxime.coquelin@redhat.com; Xia, Chenbo
> <chenbo.xia@intel.com>
> Cc: Hu, Jiayu <jiayu.hu@intel.com>; cheng.jiang@intel.com; Richardson, Bruce
> <bruce.richardson@intel.com>; Pai G, Sunil <sunil.pai.g@intel.com>; Ding, Xuan
> <xuan.ding@intel.com>
> Subject: [PATCH] vhost: add unsafe API to check inflight packets
> 
> In async data path, when vring state changes, it is necessary to
> know the number of inflight packets in DMA engine. This patch
> provides a thread unsafe API to return the number of inflight
> packets without using any lock.
> 
> Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> ---
>  doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
>  doc/guides/rel_notes/release_21_11.rst |  5 +++++
>  lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
>  lib/vhost/version.map                  |  3 +++
>  lib/vhost/vhost.c                      | 25 +++++++++++++++++++++++++
>  5 files changed, 52 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/vhost_lib.rst
> b/doc/guides/prog_guide/vhost_lib.rst
> index 8874033165..b4b1134f54 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -305,6 +305,11 @@ The following is an overview of some key Vhost API
> functions:
>    This function returns the amount of in-flight packets for the vhost
>    queue using async acceleration.
> 
> +    ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
> +
> +  Get the number of inflight packets for a vhost queue without
> +  performing any locking.
> +
>  * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``

This does not align with others. Please check.

> 
>    Clear inflight packets which are submitted to DMA engine in vhost async
> data
> diff --git a/doc/guides/rel_notes/release_21_11.rst
> b/doc/guides/rel_notes/release_21_11.rst
> index 675b573834..db080e9490 100644
> --- a/doc/guides/rel_notes/release_21_11.rst
> +++ b/doc/guides/rel_notes/release_21_11.rst
> @@ -55,6 +55,11 @@ New Features
>       Also, make sure to start the actual text at the margin.
>       =======================================================
> 
> +* **Added vhost API to get the number of inflight packets.**
> +
> +  Added an API which can get the number of inflight packets in
> +  vhost async data path.
> +

Please add 'without lock' or something similar as we already have a lock version.

>  * **Enabled new devargs parser.**
> 
>    * Enabled devargs syntax
> diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
> index b25ff446f7..0af414bf78 100644
> --- a/lib/vhost/rte_vhost_async.h
> +++ b/lib/vhost/rte_vhost_async.h
> @@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid,
> uint16_t queue_id,
>  __rte_experimental
>  int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
> 
> +/**
> + * This function is lock-free version to return the amount of in-flight
> + * packets for the vhost queue which uses async channel acceleration.
> + *
> + * @param vid
> + *  id of vhost device to enqueue data
> + * @param queue_id
> + *  queue id to enqueue data

You can also check dequeue inflight packets, right?

> + * @return
> + *  the amount of in-flight packets on success; -1 on failure
> + */
> +__rte_experimental
> +int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
> +
>  /**
>   * This function checks async completion status and clear packets for
>   * a specific vhost device queue. Packets which are inflight will be
> diff --git a/lib/vhost/version.map b/lib/vhost/version.map
> index c92a9d4962..b150dc408d 100644
> --- a/lib/vhost/version.map
> +++ b/lib/vhost/version.map
> @@ -85,4 +85,7 @@ EXPERIMENTAL {
>  	rte_vhost_async_channel_register_thread_unsafe;
>  	rte_vhost_async_channel_unregister_thread_unsafe;
>  	rte_vhost_clear_queue_thread_unsafe;
> +
> +	#added in 21.11
> +	rte_vhost_async_get_inflight_thread_unsafe;
>  };
> diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
> index 355ff37651..df96f84873 100644
> --- a/lib/vhost/vhost.c
> +++ b/lib/vhost/vhost.c
> @@ -1886,5 +1886,30 @@ int rte_vhost_async_get_inflight(int vid, uint16_t
> queue_id)
>  	return ret;
>  }
> 
> +int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)

According to DPDK coding style, return type and func name should be in different
lines. I notice there are also some left to clean. May clean in another patch.

Thanks,
Chenbo 

> +{
> +	struct vhost_virtqueue *vq;
> +	struct virtio_net *dev = get_device(vid);
> +	int ret = -1;
> +
> +	if (dev == NULL)
> +		return ret;
> +
> +	if (queue_id >= VHOST_MAX_VRING)
> +		return ret;
> +
> +	vq = dev->virtqueue[queue_id];
> +
> +	if (vq == NULL)
> +		return ret;
> +
> +	if (!vq->async_registered)
> +		return ret;
> +
> +	ret = vq->async_pkts_inflight_n;
> +
> +	return ret;
> +}
> +
>  RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
>  RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
> --
> 2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets
  2021-09-15  6:49 ` Xia, Chenbo
@ 2021-09-15  7:10   ` Ding, Xuan
  0 siblings, 0 replies; 19+ messages in thread
From: Ding, Xuan @ 2021-09-15  7:10 UTC (permalink / raw)
  To: Xia, Chenbo, dev, maxime.coquelin
  Cc: Hu, Jiayu, Jiang, Cheng1, Richardson, Bruce, Pai G, Sunil

Hi Chenbo,

> -----Original Message-----
> From: Xia, Chenbo <chenbo.xia@intel.com>
> Sent: Wednesday, September 15, 2021 2:49 PM
> To: Ding, Xuan <xuan.ding@intel.com>; dev@dpdk.org;
> maxime.coquelin@redhat.com
> Cc: Hu, Jiayu <jiayu.hu@intel.com>; cheng.jiang@intel.com; Richardson, Bruce
> <bruce.richardson@intel.com>; Pai G, Sunil <sunil.pai.g@intel.com>
> Subject: RE: [PATCH] vhost: add unsafe API to check inflight packets
> 
> Hi Xuan,
> 
> > -----Original Message-----
> > From: Ding, Xuan <xuan.ding@intel.com>
> > Sent: Thursday, September 9, 2021 1:58 PM
> > To: dev@dpdk.org; maxime.coquelin@redhat.com; Xia, Chenbo
> > <chenbo.xia@intel.com>
> > Cc: Hu, Jiayu <jiayu.hu@intel.com>; cheng.jiang@intel.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; Pai G, Sunil <sunil.pai.g@intel.com>; Ding,
> Xuan
> > <xuan.ding@intel.com>
> > Subject: [PATCH] vhost: add unsafe API to check inflight packets
> >
> > In async data path, when vring state changes, it is necessary to
> > know the number of inflight packets in DMA engine. This patch
> > provides a thread unsafe API to return the number of inflight
> > packets without using any lock.
> >
> > Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> > ---
> >  doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
> >  doc/guides/rel_notes/release_21_11.rst |  5 +++++
> >  lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
> >  lib/vhost/version.map                  |  3 +++
> >  lib/vhost/vhost.c                      | 25 +++++++++++++++++++++++++
> >  5 files changed, 52 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/vhost_lib.rst
> > b/doc/guides/prog_guide/vhost_lib.rst
> > index 8874033165..b4b1134f54 100644
> > --- a/doc/guides/prog_guide/vhost_lib.rst
> > +++ b/doc/guides/prog_guide/vhost_lib.rst
> > @@ -305,6 +305,11 @@ The following is an overview of some key Vhost API
> > functions:
> >    This function returns the amount of in-flight packets for the vhost
> >    queue using async acceleration.
> >
> > +    ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
> > +
> > +  Get the number of inflight packets for a vhost queue without
> > +  performing any locking.
> > +
> >  * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
> 
> This does not align with others. Please check.

Thanks, will fix it in next version.

> 
> >
> >    Clear inflight packets which are submitted to DMA engine in vhost async
> > data
> > diff --git a/doc/guides/rel_notes/release_21_11.rst
> > b/doc/guides/rel_notes/release_21_11.rst
> > index 675b573834..db080e9490 100644
> > --- a/doc/guides/rel_notes/release_21_11.rst
> > +++ b/doc/guides/rel_notes/release_21_11.rst
> > @@ -55,6 +55,11 @@ New Features
> >       Also, make sure to start the actual text at the margin.
> >       =======================================================
> >
> > +* **Added vhost API to get the number of inflight packets.**
> > +
> > +  Added an API which can get the number of inflight packets in
> > +  vhost async data path.
> > +
> 
> Please add 'without lock' or something similar as we already have a lock version.

You are right, add "without lock" is more accuracy.

> 
> >  * **Enabled new devargs parser.**
> >
> >    * Enabled devargs syntax
> > diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
> > index b25ff446f7..0af414bf78 100644
> > --- a/lib/vhost/rte_vhost_async.h
> > +++ b/lib/vhost/rte_vhost_async.h
> > @@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid,
> > uint16_t queue_id,
> >  __rte_experimental
> >  int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
> >
> > +/**
> > + * This function is lock-free version to return the amount of in-flight
> > + * packets for the vhost queue which uses async channel acceleration.
> > + *
> > + * @param vid
> > + *  id of vhost device to enqueue data
> > + * @param queue_id
> > + *  queue id to enqueue data
> 
> You can also check dequeue inflight packets, right?

Yes, this API applies to both enqueue and dequeue directions.

> 
> > + * @return
> > + *  the amount of in-flight packets on success; -1 on failure
> > + */
> > +__rte_experimental
> > +int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
> > +
> >  /**
> >   * This function checks async completion status and clear packets for
> >   * a specific vhost device queue. Packets which are inflight will be
> > diff --git a/lib/vhost/version.map b/lib/vhost/version.map
> > index c92a9d4962..b150dc408d 100644
> > --- a/lib/vhost/version.map
> > +++ b/lib/vhost/version.map
> > @@ -85,4 +85,7 @@ EXPERIMENTAL {
> >  	rte_vhost_async_channel_register_thread_unsafe;
> >  	rte_vhost_async_channel_unregister_thread_unsafe;
> >  	rte_vhost_clear_queue_thread_unsafe;
> > +
> > +	#added in 21.11
> > +	rte_vhost_async_get_inflight_thread_unsafe;
> >  };
> > diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
> > index 355ff37651..df96f84873 100644
> > --- a/lib/vhost/vhost.c
> > +++ b/lib/vhost/vhost.c
> > @@ -1886,5 +1886,30 @@ int rte_vhost_async_get_inflight(int vid, uint16_t
> > queue_id)
> >  	return ret;
> >  }
> >
> > +int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
> 
> According to DPDK coding style, return type and func name should be in
> different
> lines. I notice there are also some left to clean. May clean in another patch.

Thanks, notice the rte_vhost_async_get_inflight() API also has this problem.
Will fix both in next version.

Regards,
Xuan

> 
> Thanks,
> Chenbo
> 
> > +{
> > +	struct vhost_virtqueue *vq;
> > +	struct virtio_net *dev = get_device(vid);
> > +	int ret = -1;
> > +
> > +	if (dev == NULL)
> > +		return ret;
> > +
> > +	if (queue_id >= VHOST_MAX_VRING)
> > +		return ret;
> > +
> > +	vq = dev->virtqueue[queue_id];
> > +
> > +	if (vq == NULL)
> > +		return ret;
> > +
> > +	if (!vq->async_registered)
> > +		return ret;
> > +
> > +	ret = vq->async_pkts_inflight_n;
> > +
> > +	return ret;
> > +}
> > +
> >  RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
> >  RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
> > --
> > 2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v2 v2] vhost: add unsafe API to check inflight packets
  2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
  2021-09-15  6:49 ` Xia, Chenbo
@ 2021-09-15  8:52 ` Xuan Ding
  2021-09-16  2:58 ` [dpdk-dev] [PATCH v3] " Xuan Ding
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-15  8:52 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	YvonneX.Yang, Xuan Ding

In async data path, when vring state changes, it is necessary to
know the number of inflight packets in DMA engine. This patch
provides a thread unsafe API to return the number of inflight
packets without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---

v2:
* Fixed some format issues.
---
 doc/guides/prog_guide/vhost_lib.rst    |  5 ++++
 doc/guides/rel_notes/release_21_11.rst |  5 ++++
 lib/vhost/rte_vhost_async.h            | 14 +++++++++
 lib/vhost/version.map                  |  3 ++
 lib/vhost/vhost.c                      | 41 ++++++++++++++++++++++----
 5 files changed, 63 insertions(+), 5 deletions(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 8874033165..0c4fb9ea91 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 675b573834..c08e2b80c4 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
index b25ff446f7..0af414bf78 100644
--- a/lib/vhost/rte_vhost_async.h
+++ b/lib/vhost/rte_vhost_async.h
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index c92a9d4962..b150dc408d 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
index 355ff37651..69e9d229af 100644
--- a/lib/vhost/vhost.c
+++ b/lib/vhost/vhost.c
@@ -1500,7 +1500,8 @@ rte_vhost_get_vdpa_device(int vid)
 	return dev->vdpa_dev;
 }
 
-int rte_vhost_get_log_base(int vid, uint64_t *log_base,
+int
+rte_vhost_get_log_base(int vid, uint64_t *log_base,
 		uint64_t *log_size)
 {
 	struct virtio_net *dev = get_device(vid);
@@ -1514,7 +1515,8 @@ int rte_vhost_get_log_base(int vid, uint64_t *log_base,
 	return 0;
 }
 
-int rte_vhost_get_vring_base(int vid, uint16_t queue_id,
+int
+rte_vhost_get_vring_base(int vid, uint16_t queue_id,
 		uint16_t *last_avail_idx, uint16_t *last_used_idx)
 {
 	struct vhost_virtqueue *vq;
@@ -1543,7 +1545,8 @@ int rte_vhost_get_vring_base(int vid, uint16_t queue_id,
 	return 0;
 }
 
-int rte_vhost_set_vring_base(int vid, uint16_t queue_id,
+int
+rte_vhost_set_vring_base(int vid, uint16_t queue_id,
 		uint16_t last_avail_idx, uint16_t last_used_idx)
 {
 	struct vhost_virtqueue *vq;
@@ -1606,7 +1609,8 @@ rte_vhost_get_vring_base_from_inflight(int vid,
 	return 0;
 }
 
-int rte_vhost_extern_callback_register(int vid,
+int
+rte_vhost_extern_callback_register(int vid,
 		struct rte_vhost_user_extern_ops const * const ops, void *ctx)
 {
 	struct virtio_net *dev = get_device(vid);
@@ -1854,7 +1858,8 @@ rte_vhost_async_channel_unregister_thread_unsafe(int vid, uint16_t queue_id)
 	return 0;
 }
 
-int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
+int
+rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 {
 	struct vhost_virtqueue *vq;
 	struct virtio_net *dev = get_device(vid);
@@ -1886,5 +1891,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v3] vhost: add unsafe API to check inflight packets
  2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
  2021-09-15  6:49 ` Xia, Chenbo
  2021-09-15  8:52 ` [dpdk-dev] [PATCH v2 v2] " Xuan Ding
@ 2021-09-16  2:58 ` Xuan Ding
  2021-09-21 17:27   ` Kevin Traynor
  2021-09-27  1:42 ` [dpdk-dev] [PATCH v4 0/2] add unsafe API to get " Xuan Ding
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 19+ messages in thread
From: Xuan Ding @ 2021-09-16  2:58 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	YvonneX.Yang, Xuan Ding

In async data path, when vring state changes, it is necessary to
know the number of inflight packets in DMA engine. This patch
provides a thread unsafe API to return the number of inflight
packets without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.
---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 8874033165..0c4fb9ea91 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 675b573834..c9814c68df 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
index b25ff446f7..0af414bf78 100644
--- a/lib/vhost/rte_vhost_async.h
+++ b/lib/vhost/rte_vhost_async.h
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index c92a9d4962..b150dc408d 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
index 355ff37651..24ae1025ea 100644
--- a/lib/vhost/vhost.c
+++ b/lib/vhost/vhost.c
@@ -1886,5 +1886,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: add unsafe API to check inflight packets
  2021-09-16  2:58 ` [dpdk-dev] [PATCH v3] " Xuan Ding
@ 2021-09-21 17:27   ` Kevin Traynor
  2021-09-23  2:40     ` Ding, Xuan
  0 siblings, 1 reply; 19+ messages in thread
From: Kevin Traynor @ 2021-09-21 17:27 UTC (permalink / raw)
  To: Xuan Ding, dev, maxime.coquelin, chenbo.xia
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g, YvonneX.Yang

On 16/09/2021 03:58, Xuan Ding wrote:
> In async data path, when vring state changes, it is necessary to
> know the number of inflight packets in DMA engine. This patch
> provides a thread unsafe API to return the number of inflight
> packets without using any lock.
> 
> Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> ---
> v3:
> * Fixed one typo.
> * Revised the doc to be more accuracy.
> 
> v2:
> * Fixed some format issues.
> ---
>   doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
>   doc/guides/rel_notes/release_21_11.rst |  5 +++++
>   lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
>   lib/vhost/version.map                  |  3 +++
>   lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
>   5 files changed, 53 insertions(+)
> 

<snip>

Should there be a change to the vring_state_changed() in vhost example 
app to go along with this patch? It would help to understand the 
operation if this API was used.


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: add unsafe API to check inflight packets
  2021-09-21 17:27   ` Kevin Traynor
@ 2021-09-23  2:40     ` Ding, Xuan
  0 siblings, 0 replies; 19+ messages in thread
From: Ding, Xuan @ 2021-09-23  2:40 UTC (permalink / raw)
  To: Kevin Traynor, dev, maxime.coquelin, Xia, Chenbo
  Cc: Hu, Jiayu, Jiang, Cheng1, Richardson, Bruce, Pai G, Sunil, Yang, YvonneX

Hi,

> -----Original Message-----
> From: Kevin Traynor <ktraynor@redhat.com>
> Sent: Wednesday, September 22, 2021 1:28 AM
> To: Ding, Xuan <xuan.ding@intel.com>; dev@dpdk.org;
> maxime.coquelin@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
> Cc: Hu, Jiayu <jiayu.hu@intel.com>; Jiang, Cheng1 <cheng1.jiang@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Pai G, Sunil
> <sunil.pai.g@intel.com>; Yang, YvonneX <yvonnex.yang@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v3] vhost: add unsafe API to check inflight
> packets
> 
> On 16/09/2021 03:58, Xuan Ding wrote:
> > In async data path, when vring state changes, it is necessary to
> > know the number of inflight packets in DMA engine. This patch
> > provides a thread unsafe API to return the number of inflight
> > packets without using any lock.
> >
> > Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> > ---
> > v3:
> > * Fixed one typo.
> > * Revised the doc to be more accuracy.
> >
> > v2:
> > * Fixed some format issues.
> > ---
> >   doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
> >   doc/guides/rel_notes/release_21_11.rst |  5 +++++
> >   lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
> >   lib/vhost/version.map                  |  3 +++
> >   lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
> >   5 files changed, 53 insertions(+)
> >
> 
> <snip>
> 
> Should there be a change to the vring_state_changed() in vhost example
> app to go along with this patch? It would help to understand the
> operation if this API was used.

Thanks Kevin, will add it in vhost example in next version.

Regards,
Xuan


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v4 0/2] add unsafe API to get inflight packets
  2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
                   ` (2 preceding siblings ...)
  2021-09-16  2:58 ` [dpdk-dev] [PATCH v3] " Xuan Ding
@ 2021-09-27  1:42 ` Xuan Ding
  2021-09-27  1:42   ` [dpdk-dev] [PATCH v4 1/2] vhost: add unsafe API to check " Xuan Ding
  2021-09-27  1:42   ` [dpdk-dev] [PATCH v4 2/2] examples/vhost: use " Xuan Ding
  2021-09-28  6:24 ` [dpdk-dev] [PATCH v5 0/2] add unsafe API to get " Xuan Ding
  2021-09-28 15:55 ` [dpdk-dev] [PATCH v6 0/2] add unsafe API to get " Xuan Ding
  5 siblings, 2 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-27  1:42 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

This patchset introduces an unsafe API to get the number of inflight
packets in DMA engine in some situations. Like vring state changes or
device is destroyed. Compared with rte_vhost_async_get_inflight(),
this is a lock free version.

v4:
* Added use case for API in vhost example.

v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Xuan Ding (2):
  vhost: add unsafe API to check inflight packets
  examples/vhost: use API to check inflight packets

 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 examples/vhost/main.c                  | 26 +++++++++++---------------
 examples/vhost/main.h                  |  1 -
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 7 files changed, 64 insertions(+), 16 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v4 1/2] vhost: add unsafe API to check inflight packets
  2021-09-27  1:42 ` [dpdk-dev] [PATCH v4 0/2] add unsafe API to get " Xuan Ding
@ 2021-09-27  1:42   ` Xuan Ding
  2021-09-27  1:42   ` [dpdk-dev] [PATCH v4 2/2] examples/vhost: use " Xuan Ding
  1 sibling, 0 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-27  1:42 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

In async data path, when vring state changes or device is destroyed,
it is necessary to know the number of inflight packets in DMA engine.
This patch provides a thread unsafe API to return the number of
inflight packets for a vhost queue without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 8874033165..0c4fb9ea91 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -305,6 +305,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 675b573834..c9814c68df 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
index b25ff446f7..0af414bf78 100644
--- a/lib/vhost/rte_vhost_async.h
+++ b/lib/vhost/rte_vhost_async.h
@@ -246,6 +246,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index c92a9d4962..b150dc408d 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
index 355ff37651..24ae1025ea 100644
--- a/lib/vhost/vhost.c
+++ b/lib/vhost/vhost.c
@@ -1886,5 +1886,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v4 2/2] examples/vhost: use API to check inflight packets
  2021-09-27  1:42 ` [dpdk-dev] [PATCH v4 0/2] add unsafe API to get " Xuan Ding
  2021-09-27  1:42   ` [dpdk-dev] [PATCH v4 1/2] vhost: add unsafe API to check " Xuan Ding
@ 2021-09-27  1:42   ` Xuan Ding
  1 sibling, 0 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-27  1:42 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe()
API to directly return the number of inflight packets instead of
maintaining a local variable.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 examples/vhost/main.c | 26 +++++++++++---------------
 examples/vhost/main.h |  1 -
 2 files changed, 11 insertions(+), 16 deletions(-)

diff --git a/examples/vhost/main.c b/examples/vhost/main.c
index bc3d71c898..f0b74b5086 100644
--- a/examples/vhost/main.c
+++ b/examples/vhost/main.c
@@ -851,11 +851,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
 	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
 					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
 		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
 }
 
 static __rte_always_inline void
@@ -898,7 +895,6 @@ drain_vhost(struct vhost_dev *vdev)
 		complete_async_pkts(vdev);
 		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ,
 					m, nr_xmit, m_cpu_cpl, &cpu_cpl_nr);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret - cpu_cpl_nr, __ATOMIC_SEQ_CST);
 
 		if (cpu_cpl_nr)
 			free_pkts(m_cpu_cpl, cpu_cpl_nr);
@@ -1230,8 +1226,6 @@ drain_eth_rx(struct vhost_dev *vdev)
 		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
 					VIRTIO_RXQ, pkts, rx_count,
 					m_cpu_cpl, &cpu_cpl_nr);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count - cpu_cpl_nr,
-					__ATOMIC_SEQ_CST);
 
 		if (cpu_cpl_nr)
 			free_pkts(m_cpu_cpl, cpu_cpl_nr);
@@ -1360,6 +1354,7 @@ destroy_device(int vid)
 	struct vhost_dev *vdev = NULL;
 	int lcore;
 	uint16_t i;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1406,13 +1401,13 @@ destroy_device(int vid)
 
 	if (async_vhost_driver) {
 		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
 			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
 			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 		}
 
 		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1509,6 +1504,7 @@ static int
 vring_state_changed(int vid, uint16_t queue_id, int enable)
 {
 	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1523,13 +1519,13 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
 	if (async_vhost_driver) {
 		if (!enable) {
 			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
 				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
 				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 			}
 		}
 	}
diff --git a/examples/vhost/main.h b/examples/vhost/main.h
index e7b1ac60a6..0ccdce4b4a 100644
--- a/examples/vhost/main.h
+++ b/examples/vhost/main.h
@@ -51,7 +51,6 @@ struct vhost_dev {
 	uint64_t features;
 	size_t hdr_len;
 	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
 	struct rte_vhost_memory *mem;
 	struct device_statistics stats;
 	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v5 0/2] add unsafe API to get inflight packets
  2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
                   ` (3 preceding siblings ...)
  2021-09-27  1:42 ` [dpdk-dev] [PATCH v4 0/2] add unsafe API to get " Xuan Ding
@ 2021-09-28  6:24 ` Xuan Ding
  2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 1/2] vhost: add unsafe API to check " Xuan Ding
  2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 2/2] examples/vhost: use " Xuan Ding
  2021-09-28 15:55 ` [dpdk-dev] [PATCH v6 0/2] add unsafe API to get " Xuan Ding
  5 siblings, 2 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-28  6:24 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

This patchset introduces an unsafe API to get the number of inflight
packets in DMA engine in some situations. Like vring state changes or
device is destroyed. Compared with rte_vhost_async_get_inflight(),
this is a lock free version.

v5:
* Rebased to the lastest branch.

v4:
* Added use case for API in vhost example.

v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Xuan Ding (2):
  vhost: add unsafe API to check inflight packets
  examples/vhost: use API to check inflight packets

 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 examples/vhost/main.c                  | 25 +++++++++++--------------
 examples/vhost/main.h                  |  1 -
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 7 files changed, 64 insertions(+), 15 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v5 1/2] vhost: add unsafe API to check inflight packets
  2021-09-28  6:24 ` [dpdk-dev] [PATCH v5 0/2] add unsafe API to get " Xuan Ding
@ 2021-09-28  6:24   ` Xuan Ding
  2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 2/2] examples/vhost: use " Xuan Ding
  1 sibling, 0 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-28  6:24 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

In async data path, when vring state changes or device is destroyed,
it is necessary to know the number of inflight packets in DMA engine.
This patch provides a thread unsafe API to return the number of
inflight packets for a vhost queue without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 171e0096f6..73a4e24640 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -298,6 +298,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index ad7c1afec0..fd821b29d3 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
index ad71555a7f..1f0708378b 100644
--- a/lib/vhost/rte_vhost_async.h
+++ b/lib/vhost/rte_vhost_async.h
@@ -234,6 +234,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index 8ebde3f694..ed6cb53a99 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
index 996287c3c2..c16c6bc2bb 100644
--- a/lib/vhost/vhost.c
+++ b/lib/vhost/vhost.c
@@ -1884,5 +1884,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets
  2021-09-28  6:24 ` [dpdk-dev] [PATCH v5 0/2] add unsafe API to get " Xuan Ding
  2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 1/2] vhost: add unsafe API to check " Xuan Ding
@ 2021-09-28  6:24   ` Xuan Ding
  2021-09-28  9:17     ` Kevin Traynor
  1 sibling, 1 reply; 19+ messages in thread
From: Xuan Ding @ 2021-09-28  6:24 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe()
API to directly return the number of inflight packets instead of
maintaining a local variable.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 examples/vhost/main.c | 25 +++++++++++--------------
 examples/vhost/main.h |  1 -
 2 files changed, 11 insertions(+), 15 deletions(-)

diff --git a/examples/vhost/main.c b/examples/vhost/main.c
index d0bf1f31e3..3faac6d053 100644
--- a/examples/vhost/main.c
+++ b/examples/vhost/main.c
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
 	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
 					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
 		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
 }
 
 static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
 		complete_async_pkts(vdev);
 		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = nr_xmit - ret;
 		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
 		complete_async_pkts(vdev);
 		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
 					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = rx_count - enqueue_count;
 		if (enqueue_fail)
@@ -1338,6 +1333,7 @@ destroy_device(int vid)
 	struct vhost_dev *vdev = NULL;
 	int lcore;
 	uint16_t i;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1384,13 +1380,13 @@ destroy_device(int vid)
 
 	if (async_vhost_driver) {
 		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
 
-		while (vdev->pkts_inflight) {
+		while (pkts_inflight) {
 			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
 			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 		}
 
 		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1486,6 +1482,7 @@ static int
 vring_state_changed(int vid, uint16_t queue_id, int enable)
 {
 	struct vhost_dev *vdev = NULL;
+	int pkts_inflight;
 
 	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
 		if (vdev->vid == vid)
@@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
 	if (async_vhost_driver) {
 		if (!enable) {
 			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
 				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
 				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
 			}
 		}
 	}
diff --git a/examples/vhost/main.h b/examples/vhost/main.h
index e7b1ac60a6..0ccdce4b4a 100644
--- a/examples/vhost/main.h
+++ b/examples/vhost/main.h
@@ -51,7 +51,6 @@ struct vhost_dev {
 	uint64_t features;
 	size_t hdr_len;
 	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
 	struct rte_vhost_memory *mem;
 	struct device_statistics stats;
 	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets
  2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 2/2] examples/vhost: use " Xuan Ding
@ 2021-09-28  9:17     ` Kevin Traynor
  2021-09-28 11:50       ` Ding, Xuan
  0 siblings, 1 reply; 19+ messages in thread
From: Kevin Traynor @ 2021-09-28  9:17 UTC (permalink / raw)
  To: Xuan Ding, dev, maxime.coquelin, chenbo.xia
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang

On 28/09/2021 07:24, Xuan Ding wrote:
> In async data path, call rte_vhost_async_get_inflight_thread_unsafe()
> API to directly return the number of inflight packets instead of
> maintaining a local variable.
> 
> Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> ---
>   examples/vhost/main.c | 25 +++++++++++--------------
>   examples/vhost/main.h |  1 -
>   2 files changed, 11 insertions(+), 15 deletions(-)
> 
> diff --git a/examples/vhost/main.c b/examples/vhost/main.c
> index d0bf1f31e3..3faac6d053 100644
> --- a/examples/vhost/main.c
> +++ b/examples/vhost/main.c
> @@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
>   
>   	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
>   					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
> -	if (complete_count) {
> +	if (complete_count)
>   		free_pkts(p_cpl, complete_count);
> -		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
> -	}
> -
>   }
>   
>   static __rte_always_inline void
> @@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
>   
>   		complete_async_pkts(vdev);
>   		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit);
> -		__atomic_add_fetch(&vdev->pkts_inflight, ret, __ATOMIC_SEQ_CST);
>   
>   		enqueue_fail = nr_xmit - ret;
>   		if (enqueue_fail)
> @@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
>   		complete_async_pkts(vdev);
>   		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
>   					VIRTIO_RXQ, pkts, rx_count);
> -		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count, __ATOMIC_SEQ_CST);
>   
>   		enqueue_fail = rx_count - enqueue_count;
>   		if (enqueue_fail)
> @@ -1338,6 +1333,7 @@ destroy_device(int vid)
>   	struct vhost_dev *vdev = NULL;
>   	int lcore;
>   	uint16_t i;

> +	int pkts_inflight;

You can move this down to the block it is used in

>   
>   	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
>   		if (vdev->vid == vid)
> @@ -1384,13 +1380,13 @@ destroy_device(int vid)
>   
>   	if (async_vhost_driver) {
>   		uint16_t n_pkt = 0;
> -		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
> +		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
> +		struct rte_mbuf *m_cpl[pkts_inflight];
>   
> -		while (vdev->pkts_inflight) {
> +		while (pkts_inflight) {
>   			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
> -						m_cpl, vdev->pkts_inflight);
> +						m_cpl, pkts_inflight);
>   			free_pkts(m_cpl, n_pkt);
> -			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);

This is an infinite loop if there are pkts_inflight, need to recheck 
pkts_inflight in the loop.

>   		}
>   
>   		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
> @@ -1486,6 +1482,7 @@ static int
>   vring_state_changed(int vid, uint16_t queue_id, int enable)
>   {
>   	struct vhost_dev *vdev = NULL;
> +	int pkts_inflight;
>   
>   	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
>   		if (vdev->vid == vid)
> @@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
>   	if (async_vhost_driver) {
>   		if (!enable) {
>   			uint16_t n_pkt = 0;
> -			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
> +			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
> +			struct rte_mbuf *m_cpl[pkts_inflight];
>   
> -			while (vdev->pkts_inflight) {
> +			while (pkts_inflight) {
>   				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
> -							m_cpl, vdev->pkts_inflight);
> +							m_cpl, pkts_inflight);
>   				free_pkts(m_cpl, n_pkt);
> -				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);

Same comments as destroy_device

>   			}
>   		}
>   	}
> diff --git a/examples/vhost/main.h b/examples/vhost/main.h
> index e7b1ac60a6..0ccdce4b4a 100644
> --- a/examples/vhost/main.h
> +++ b/examples/vhost/main.h
> @@ -51,7 +51,6 @@ struct vhost_dev {
>   	uint64_t features;
>   	size_t hdr_len;
>   	uint16_t nr_vrings;
> -	uint16_t pkts_inflight;
>   	struct rte_vhost_memory *mem;
>   	struct device_statistics stats;
>   	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets
  2021-09-28  9:17     ` Kevin Traynor
@ 2021-09-28 11:50       ` Ding, Xuan
  2021-09-28 11:59         ` Ding, Xuan
  0 siblings, 1 reply; 19+ messages in thread
From: Ding, Xuan @ 2021-09-28 11:50 UTC (permalink / raw)
  To: Kevin Traynor, dev, maxime.coquelin, Xia, Chenbo
  Cc: Hu, Jiayu, Jiang, Cheng1, Richardson, Bruce, Pai G, Sunil, Wang,
	Yinan, Yang, YvonneX

Hi Kevin,

> -----Original Message-----
> From: Kevin Traynor <ktraynor@redhat.com>
> Sent: Tuesday, September 28, 2021 5:18 PM
> To: Ding, Xuan <xuan.ding@intel.com>; dev@dpdk.org;
> maxime.coquelin@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
> Cc: Hu, Jiayu <jiayu.hu@intel.com>; Jiang, Cheng1 <cheng1.jiang@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Pai G, Sunil
> <sunil.pai.g@intel.com>; Wang, Yinan <yinan.wang@intel.com>; Yang, YvonneX
> <yvonnex.yang@intel.com>
> Subject: Re: [PATCH v5 2/2] examples/vhost: use API to check inflight packets
> 
> On 28/09/2021 07:24, Xuan Ding wrote:
> > In async data path, call rte_vhost_async_get_inflight_thread_unsafe()
> > API to directly return the number of inflight packets instead of
> > maintaining a local variable.
> >
> > Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> > ---
> >   examples/vhost/main.c | 25 +++++++++++--------------
> >   examples/vhost/main.h |  1 -
> >   2 files changed, 11 insertions(+), 15 deletions(-)
> >
> > diff --git a/examples/vhost/main.c b/examples/vhost/main.c
> > index d0bf1f31e3..3faac6d053 100644
> > --- a/examples/vhost/main.c
> > +++ b/examples/vhost/main.c
> > @@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
> >
> >   	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
> >   					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
> > -	if (complete_count) {
> > +	if (complete_count)
> >   		free_pkts(p_cpl, complete_count);
> > -		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count,
> __ATOMIC_SEQ_CST);
> > -	}
> > -
> >   }
> >
> >   static __rte_always_inline void
> > @@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
> >
> >   		complete_async_pkts(vdev);
> >   		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ,
> m, nr_xmit);
> > -		__atomic_add_fetch(&vdev->pkts_inflight, ret,
> __ATOMIC_SEQ_CST);
> >
> >   		enqueue_fail = nr_xmit - ret;
> >   		if (enqueue_fail)
> > @@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
> >   		complete_async_pkts(vdev);
> >   		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
> >   					VIRTIO_RXQ, pkts, rx_count);
> > -		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count,
> __ATOMIC_SEQ_CST);
> >
> >   		enqueue_fail = rx_count - enqueue_count;
> >   		if (enqueue_fail)
> > @@ -1338,6 +1333,7 @@ destroy_device(int vid)
> >   	struct vhost_dev *vdev = NULL;
> >   	int lcore;
> >   	uint16_t i;
> 
> > +	int pkts_inflight;
> 
> You can move this down to the block it is used in

Thanks for the suggestion.
I consider calling the unsafe API in while (condition), and there is no need to define this variable.

> 
> >
> >   	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
> >   		if (vdev->vid == vid)
> > @@ -1384,13 +1380,13 @@ destroy_device(int vid)
> >
> >   	if (async_vhost_driver) {
> >   		uint16_t n_pkt = 0;
> > -		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
> > +		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,
> VIRTIO_RXQ);
> > +		struct rte_mbuf *m_cpl[pkts_inflight];
> >
> > -		while (vdev->pkts_inflight) {
> > +		while (pkts_inflight) {
> >   			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid,
> VIRTIO_RXQ,
> > -						m_cpl, vdev->pkts_inflight);
> > +						m_cpl, pkts_inflight);
> >   			free_pkts(m_cpl, n_pkt);
> > -			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt,
> __ATOMIC_SEQ_CST);
> 
> This is an infinite loop if there are pkts_inflight, need to recheck
> pkts_inflight in the loop.

Thanks for the catch, will call the unsafe API directly in the while (condition).

> 
> >   		}
> >
> >   		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
> > @@ -1486,6 +1482,7 @@ static int
> >   vring_state_changed(int vid, uint16_t queue_id, int enable)
> >   {
> >   	struct vhost_dev *vdev = NULL;
> > +	int pkts_inflight;
> >
> >   	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
> >   		if (vdev->vid == vid)
> > @@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id,
> int enable)
> >   	if (async_vhost_driver) {
> >   		if (!enable) {
> >   			uint16_t n_pkt = 0;
> > -			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
> > +			pkts_inflight =
> rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
> > +			struct rte_mbuf *m_cpl[pkts_inflight];
> >
> > -			while (vdev->pkts_inflight) {
> > +			while (pkts_inflight) {
> >   				n_pkt =
> rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
> > -							m_cpl, vdev-
> >pkts_inflight);
> > +							m_cpl, pkts_inflight);
> >   				free_pkts(m_cpl, n_pkt);
> > -				__atomic_sub_fetch(&vdev->pkts_inflight,
> n_pkt, __ATOMIC_SEQ_CST);
> 
> Same comments as destroy_device

Same as vring_state_changed.

Regards,
Xuan

> 
> >   			}
> >   		}
> >   	}
> > diff --git a/examples/vhost/main.h b/examples/vhost/main.h
> > index e7b1ac60a6..0ccdce4b4a 100644
> > --- a/examples/vhost/main.h
> > +++ b/examples/vhost/main.h
> > @@ -51,7 +51,6 @@ struct vhost_dev {
> >   	uint64_t features;
> >   	size_t hdr_len;
> >   	uint16_t nr_vrings;
> > -	uint16_t pkts_inflight;
> >   	struct rte_vhost_memory *mem;
> >   	struct device_statistics stats;
> >   	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
> >


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [dpdk-dev] [PATCH v5 2/2] examples/vhost: use API to check inflight packets
  2021-09-28 11:50       ` Ding, Xuan
@ 2021-09-28 11:59         ` Ding, Xuan
  0 siblings, 0 replies; 19+ messages in thread
From: Ding, Xuan @ 2021-09-28 11:59 UTC (permalink / raw)
  To: Kevin Traynor, dev, maxime.coquelin, Xia, Chenbo
  Cc: Hu, Jiayu, Jiang, Cheng1, Richardson, Bruce, Pai G, Sunil, Wang,
	Yinan, Yang, YvonneX

Hi,

> -----Original Message-----
> From: Ding, Xuan
> Sent: Tuesday, September 28, 2021 7:51 PM
> To: Kevin Traynor <ktraynor@redhat.com>; dev@dpdk.org;
> maxime.coquelin@redhat.com; Xia, Chenbo <Chenbo.Xia@intel.com>
> Cc: Hu, Jiayu <Jiayu.Hu@intel.com>; Jiang, Cheng1 <Cheng1.Jiang@intel.com>;
> Richardson, Bruce <bruce.richardson@intel.com>; Pai G, Sunil
> <Sunil.Pai.G@intel.com>; Wang, Yinan <yinan.wang@intel.com>; Yang,
> YvonneX <YvonneX.Yang@intel.com>
> Subject: RE: [PATCH v5 2/2] examples/vhost: use API to check inflight packets
> 
> Hi Kevin,
> 
> > -----Original Message-----
> > From: Kevin Traynor <ktraynor@redhat.com>
> > Sent: Tuesday, September 28, 2021 5:18 PM
> > To: Ding, Xuan <xuan.ding@intel.com>; dev@dpdk.org;
> > maxime.coquelin@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
> > Cc: Hu, Jiayu <jiayu.hu@intel.com>; Jiang, Cheng1 <cheng1.jiang@intel.com>;
> > Richardson, Bruce <bruce.richardson@intel.com>; Pai G, Sunil
> > <sunil.pai.g@intel.com>; Wang, Yinan <yinan.wang@intel.com>; Yang,
> YvonneX
> > <yvonnex.yang@intel.com>
> > Subject: Re: [PATCH v5 2/2] examples/vhost: use API to check inflight packets
> >
> > On 28/09/2021 07:24, Xuan Ding wrote:
> > > In async data path, call rte_vhost_async_get_inflight_thread_unsafe()
> > > API to directly return the number of inflight packets instead of
> > > maintaining a local variable.
> > >
> > > Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> > > ---
> > >   examples/vhost/main.c | 25 +++++++++++--------------
> > >   examples/vhost/main.h |  1 -
> > >   2 files changed, 11 insertions(+), 15 deletions(-)
> > >
> > > diff --git a/examples/vhost/main.c b/examples/vhost/main.c
> > > index d0bf1f31e3..3faac6d053 100644
> > > --- a/examples/vhost/main.c
> > > +++ b/examples/vhost/main.c
> > > @@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
> > >
> > >   	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
> > >   					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
> > > -	if (complete_count) {
> > > +	if (complete_count)
> > >   		free_pkts(p_cpl, complete_count);
> > > -		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count,
> > __ATOMIC_SEQ_CST);
> > > -	}
> > > -
> > >   }
> > >
> > >   static __rte_always_inline void
> > > @@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
> > >
> > >   		complete_async_pkts(vdev);
> > >   		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ,
> > m, nr_xmit);
> > > -		__atomic_add_fetch(&vdev->pkts_inflight, ret,
> > __ATOMIC_SEQ_CST);
> > >
> > >   		enqueue_fail = nr_xmit - ret;
> > >   		if (enqueue_fail)
> > > @@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
> > >   		complete_async_pkts(vdev);
> > >   		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
> > >   					VIRTIO_RXQ, pkts, rx_count);
> > > -		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count,
> > __ATOMIC_SEQ_CST);
> > >
> > >   		enqueue_fail = rx_count - enqueue_count;
> > >   		if (enqueue_fail)
> > > @@ -1338,6 +1333,7 @@ destroy_device(int vid)
> > >   	struct vhost_dev *vdev = NULL;
> > >   	int lcore;
> > >   	uint16_t i;
> >
> > > +	int pkts_inflight;
> >
> > You can move this down to the block it is used in
> 
> Thanks for the suggestion.
> I consider calling the unsafe API in while (condition), and there is no need to
> define this variable.
> 
> >
> > >
> > >   	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
> > >   		if (vdev->vid == vid)
> > > @@ -1384,13 +1380,13 @@ destroy_device(int vid)
> > >
> > >   	if (async_vhost_driver) {
> > >   		uint16_t n_pkt = 0;
> > > -		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
> > > +		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,
> > VIRTIO_RXQ);
> > > +		struct rte_mbuf *m_cpl[pkts_inflight];
> > >
> > > -		while (vdev->pkts_inflight) {
> > > +		while (pkts_inflight) {
> > >   			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid,
> > VIRTIO_RXQ,
> > > -						m_cpl, vdev->pkts_inflight);
> > > +						m_cpl, pkts_inflight);
> > >   			free_pkts(m_cpl, n_pkt);
> > > -			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt,
> > __ATOMIC_SEQ_CST);
> >
> > This is an infinite loop if there are pkts_inflight, need to recheck
> > pkts_inflight in the loop.
> 
> Thanks for the catch, will call the unsafe API directly in the while (condition).

Sorry for replying myself, as rte_mbuf *m_cpl also needs pkts_inflight here.
Will follow your suggestion, see next version, thanks!

Regards,
Xuan

> 
> >
> > >   		}
> > >
> > >   		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
> > > @@ -1486,6 +1482,7 @@ static int
> > >   vring_state_changed(int vid, uint16_t queue_id, int enable)
> > >   {
> > >   	struct vhost_dev *vdev = NULL;
> > > +	int pkts_inflight;
> > >
> > >   	TAILQ_FOREACH(vdev, &vhost_dev_list, global_vdev_entry) {
> > >   		if (vdev->vid == vid)
> > > @@ -1500,13 +1497,13 @@ vring_state_changed(int vid, uint16_t queue_id,
> > int enable)
> > >   	if (async_vhost_driver) {
> > >   		if (!enable) {
> > >   			uint16_t n_pkt = 0;
> > > -			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
> > > +			pkts_inflight =
> > rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
> > > +			struct rte_mbuf *m_cpl[pkts_inflight];
> > >
> > > -			while (vdev->pkts_inflight) {
> > > +			while (pkts_inflight) {
> > >   				n_pkt =
> > rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
> > > -							m_cpl, vdev-
> > >pkts_inflight);
> > > +							m_cpl, pkts_inflight);
> > >   				free_pkts(m_cpl, n_pkt);
> > > -				__atomic_sub_fetch(&vdev->pkts_inflight,
> > n_pkt, __ATOMIC_SEQ_CST);
> >
> > Same comments as destroy_device
> 
> Same as vring_state_changed.
> 
> Regards,
> Xuan
> 
> >
> > >   			}
> > >   		}
> > >   	}
> > > diff --git a/examples/vhost/main.h b/examples/vhost/main.h
> > > index e7b1ac60a6..0ccdce4b4a 100644
> > > --- a/examples/vhost/main.h
> > > +++ b/examples/vhost/main.h
> > > @@ -51,7 +51,6 @@ struct vhost_dev {
> > >   	uint64_t features;
> > >   	size_t hdr_len;
> > >   	uint16_t nr_vrings;
> > > -	uint16_t pkts_inflight;
> > >   	struct rte_vhost_memory *mem;
> > >   	struct device_statistics stats;
> > >   	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
> > >


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v6 0/2] add unsafe API to get inflight packets
  2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
                   ` (4 preceding siblings ...)
  2021-09-28  6:24 ` [dpdk-dev] [PATCH v5 0/2] add unsafe API to get " Xuan Ding
@ 2021-09-28 15:55 ` Xuan Ding
  2021-09-28 15:55   ` [dpdk-dev] [PATCH v6 1/2] vhost: add unsafe API to check " Xuan Ding
  2021-09-28 15:55   ` [dpdk-dev] [PATCH v6 2/2] examples/vhost: use " Xuan Ding
  5 siblings, 2 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-28 15:55 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

This patchset introduces an unsafe API to get the number of inflight
packets in DMA engine in some situations. Like vring state changes or
device is destroyed. Compared with rte_vhost_async_get_inflight(),
this is a lock free version.

v6:
* Fixed an issue of infinite loop.

v5:
* Rebased to the lastest branch.

v4:
* Added use case for API in vhost example.

v3:
* Fixed one typo.
* Revised the doc to be more accuracy.

v2:
* Fixed some format issues.

Xuan Ding (2):
  vhost: add unsafe API to check inflight packets
  examples/vhost: use API to check inflight packets

 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 examples/vhost/main.c                  | 31 ++++++++++++++------------
 examples/vhost/main.h                  |  1 -
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 +++++++++++++++++++++
 7 files changed, 70 insertions(+), 15 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v6 1/2] vhost: add unsafe API to check inflight packets
  2021-09-28 15:55 ` [dpdk-dev] [PATCH v6 0/2] add unsafe API to get " Xuan Ding
@ 2021-09-28 15:55   ` Xuan Ding
  2021-09-28 15:55   ` [dpdk-dev] [PATCH v6 2/2] examples/vhost: use " Xuan Ding
  1 sibling, 0 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-28 15:55 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

In async data path, when vring state changes or device is destroyed,
it is necessary to know the number of inflight packets in DMA engine.
This patch provides a thread unsafe API to return the number of
inflight packets for a vhost queue without using any lock.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 doc/guides/prog_guide/vhost_lib.rst    |  5 +++++
 doc/guides/rel_notes/release_21_11.rst |  5 +++++
 lib/vhost/rte_vhost_async.h            | 14 ++++++++++++++
 lib/vhost/version.map                  |  3 +++
 lib/vhost/vhost.c                      | 26 ++++++++++++++++++++++++++
 5 files changed, 53 insertions(+)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 171e0096f6..73a4e24640 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -298,6 +298,11 @@ The following is an overview of some key Vhost API functions:
   This function returns the amount of in-flight packets for the vhost
   queue using async acceleration.
 
+* ``rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id)``
+
+  Get the number of inflight packets for a vhost queue without
+  performing any locking.
+
 * ``rte_vhost_clear_queue_thread_unsafe(vid, queue_id, **pkts, count)``
 
   Clear inflight packets which are submitted to DMA engine in vhost async data
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index ad7c1afec0..fd821b29d3 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Added vhost API to get the number of inflight packets.**
+
+  Added an API which can get the number of inflight packets in
+  vhost async data path without using lock.
+
 * **Enabled new devargs parser.**
 
   * Enabled devargs syntax
diff --git a/lib/vhost/rte_vhost_async.h b/lib/vhost/rte_vhost_async.h
index ad71555a7f..1f0708378b 100644
--- a/lib/vhost/rte_vhost_async.h
+++ b/lib/vhost/rte_vhost_async.h
@@ -234,6 +234,20 @@ uint16_t rte_vhost_poll_enqueue_completed(int vid, uint16_t queue_id,
 __rte_experimental
 int rte_vhost_async_get_inflight(int vid, uint16_t queue_id);
 
+/**
+ * This function is lock-free version to return the amount of in-flight
+ * packets for the vhost queue which uses async channel acceleration.
+ *
+ * @param vid
+ *  id of vhost device to enqueue data
+ * @param queue_id
+ *  queue id to enqueue data
+ * @return
+ *  the amount of in-flight packets on success; -1 on failure
+ */
+__rte_experimental
+int rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id);
+
 /**
  * This function checks async completion status and clear packets for
  * a specific vhost device queue. Packets which are inflight will be
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index 8ebde3f694..ed6cb53a99 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -85,4 +85,7 @@ EXPERIMENTAL {
 	rte_vhost_async_channel_register_thread_unsafe;
 	rte_vhost_async_channel_unregister_thread_unsafe;
 	rte_vhost_clear_queue_thread_unsafe;
+
+	#added in 21.11
+	rte_vhost_async_get_inflight_thread_unsafe;
 };
diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c
index 996287c3c2..c16c6bc2bb 100644
--- a/lib/vhost/vhost.c
+++ b/lib/vhost/vhost.c
@@ -1884,5 +1884,31 @@ int rte_vhost_async_get_inflight(int vid, uint16_t queue_id)
 	return ret;
 }
 
+int
+rte_vhost_async_get_inflight_thread_unsafe(int vid, uint16_t queue_id)
+{
+	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = get_device(vid);
+	int ret = -1;
+
+	if (dev == NULL)
+		return ret;
+
+	if (queue_id >= VHOST_MAX_VRING)
+		return ret;
+
+	vq = dev->virtqueue[queue_id];
+
+	if (vq == NULL)
+		return ret;
+
+	if (!vq->async_registered)
+		return ret;
+
+	ret = vq->async_pkts_inflight_n;
+
+	return ret;
+}
+
 RTE_LOG_REGISTER_SUFFIX(vhost_config_log_level, config, INFO);
 RTE_LOG_REGISTER_SUFFIX(vhost_data_log_level, data, WARNING);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [dpdk-dev] [PATCH v6 2/2] examples/vhost: use API to check inflight packets
  2021-09-28 15:55 ` [dpdk-dev] [PATCH v6 0/2] add unsafe API to get " Xuan Ding
  2021-09-28 15:55   ` [dpdk-dev] [PATCH v6 1/2] vhost: add unsafe API to check " Xuan Ding
@ 2021-09-28 15:55   ` Xuan Ding
  1 sibling, 0 replies; 19+ messages in thread
From: Xuan Ding @ 2021-09-28 15:55 UTC (permalink / raw)
  To: dev, maxime.coquelin, chenbo.xia, ktraynor
  Cc: jiayu.hu, cheng1.jiang, bruce.richardson, sunil.pai.g,
	yinan.wang, YvonneX.Yang, Xuan Ding

In async data path, call rte_vhost_async_get_inflight_thread_unsafe()
API to directly return the number of inflight packets instead of
maintaining a local variable.

Signed-off-by: Xuan Ding <xuan.ding@intel.com>
---
 examples/vhost/main.c | 31 +++++++++++++++++--------------
 examples/vhost/main.h |  1 -
 2 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/examples/vhost/main.c b/examples/vhost/main.c
index d0bf1f31e3..7a5d97cef6 100644
--- a/examples/vhost/main.c
+++ b/examples/vhost/main.c
@@ -842,11 +842,8 @@ complete_async_pkts(struct vhost_dev *vdev)
 
 	complete_count = rte_vhost_poll_enqueue_completed(vdev->vid,
 					VIRTIO_RXQ, p_cpl, MAX_PKT_BURST);
-	if (complete_count) {
+	if (complete_count)
 		free_pkts(p_cpl, complete_count);
-		__atomic_sub_fetch(&vdev->pkts_inflight, complete_count, __ATOMIC_SEQ_CST);
-	}
-
 }
 
 static __rte_always_inline void
@@ -886,7 +883,6 @@ drain_vhost(struct vhost_dev *vdev)
 
 		complete_async_pkts(vdev);
 		ret = rte_vhost_submit_enqueue_burst(vdev->vid, VIRTIO_RXQ, m, nr_xmit);
-		__atomic_add_fetch(&vdev->pkts_inflight, ret, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = nr_xmit - ret;
 		if (enqueue_fail)
@@ -1212,7 +1208,6 @@ drain_eth_rx(struct vhost_dev *vdev)
 		complete_async_pkts(vdev);
 		enqueue_count = rte_vhost_submit_enqueue_burst(vdev->vid,
 					VIRTIO_RXQ, pkts, rx_count);
-		__atomic_add_fetch(&vdev->pkts_inflight, enqueue_count, __ATOMIC_SEQ_CST);
 
 		enqueue_fail = rx_count - enqueue_count;
 		if (enqueue_fail)
@@ -1384,13 +1379,17 @@ destroy_device(int vid)
 
 	if (async_vhost_driver) {
 		uint16_t n_pkt = 0;
-		struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+		int pkts_inflight;
 
-		while (vdev->pkts_inflight) {
+		pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, VIRTIO_RXQ);
+		struct rte_mbuf *m_cpl[pkts_inflight];
+
+		while (pkts_inflight) {
 			n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, VIRTIO_RXQ,
-						m_cpl, vdev->pkts_inflight);
+						m_cpl, pkts_inflight);
 			free_pkts(m_cpl, n_pkt);
-			__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,
+									VIRTIO_RXQ);
 		}
 
 		rte_vhost_async_channel_unregister(vid, VIRTIO_RXQ);
@@ -1500,13 +1499,17 @@ vring_state_changed(int vid, uint16_t queue_id, int enable)
 	if (async_vhost_driver) {
 		if (!enable) {
 			uint16_t n_pkt = 0;
-			struct rte_mbuf *m_cpl[vdev->pkts_inflight];
+			int pkts_inflight;
+
+			pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid, queue_id);
+			struct rte_mbuf *m_cpl[pkts_inflight];
 
-			while (vdev->pkts_inflight) {
+			while (pkts_inflight) {
 				n_pkt = rte_vhost_clear_queue_thread_unsafe(vid, queue_id,
-							m_cpl, vdev->pkts_inflight);
+							m_cpl, pkts_inflight);
 				free_pkts(m_cpl, n_pkt);
-				__atomic_sub_fetch(&vdev->pkts_inflight, n_pkt, __ATOMIC_SEQ_CST);
+				pkts_inflight = rte_vhost_async_get_inflight_thread_unsafe(vid,
+										queue_id);
 			}
 		}
 	}
diff --git a/examples/vhost/main.h b/examples/vhost/main.h
index e7b1ac60a6..0ccdce4b4a 100644
--- a/examples/vhost/main.h
+++ b/examples/vhost/main.h
@@ -51,7 +51,6 @@ struct vhost_dev {
 	uint64_t features;
 	size_t hdr_len;
 	uint16_t nr_vrings;
-	uint16_t pkts_inflight;
 	struct rte_vhost_memory *mem;
 	struct device_statistics stats;
 	TAILQ_ENTRY(vhost_dev) global_vdev_entry;
-- 
2.17.1


^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2021-09-28 16:06 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-09  5:57 [dpdk-dev] [PATCH] vhost: add unsafe API to check inflight packets Xuan Ding
2021-09-15  6:49 ` Xia, Chenbo
2021-09-15  7:10   ` Ding, Xuan
2021-09-15  8:52 ` [dpdk-dev] [PATCH v2 v2] " Xuan Ding
2021-09-16  2:58 ` [dpdk-dev] [PATCH v3] " Xuan Ding
2021-09-21 17:27   ` Kevin Traynor
2021-09-23  2:40     ` Ding, Xuan
2021-09-27  1:42 ` [dpdk-dev] [PATCH v4 0/2] add unsafe API to get " Xuan Ding
2021-09-27  1:42   ` [dpdk-dev] [PATCH v4 1/2] vhost: add unsafe API to check " Xuan Ding
2021-09-27  1:42   ` [dpdk-dev] [PATCH v4 2/2] examples/vhost: use " Xuan Ding
2021-09-28  6:24 ` [dpdk-dev] [PATCH v5 0/2] add unsafe API to get " Xuan Ding
2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 1/2] vhost: add unsafe API to check " Xuan Ding
2021-09-28  6:24   ` [dpdk-dev] [PATCH v5 2/2] examples/vhost: use " Xuan Ding
2021-09-28  9:17     ` Kevin Traynor
2021-09-28 11:50       ` Ding, Xuan
2021-09-28 11:59         ` Ding, Xuan
2021-09-28 15:55 ` [dpdk-dev] [PATCH v6 0/2] add unsafe API to get " Xuan Ding
2021-09-28 15:55   ` [dpdk-dev] [PATCH v6 1/2] vhost: add unsafe API to check " Xuan Ding
2021-09-28 15:55   ` [dpdk-dev] [PATCH v6 2/2] examples/vhost: use " Xuan Ding

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).