DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Hu, Jiayu" <jiayu.hu@intel.com>
To: Maxime Coquelin <maxime.coquelin@redhat.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"Xia, Chenbo" <chenbo.xia@intel.com>,
	"Wang, YuanX" <yuanx.wang@intel.com>,
	"Ma, WenwuX" <wenwux.ma@intel.com>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"Mcnamara, John" <john.mcnamara@intel.com>,
	"david.marchand@redhat.com" <david.marchand@redhat.com>
Subject: Re: [dpdk-dev] [PATCH v1 08/14] vhost: improve IO vector logic
Date: Mon, 25 Oct 2021 07:22:39 +0000	[thread overview]
Message-ID: <9239770ed6c74f67975524bd081af39a@intel.com> (raw)
In-Reply-To: <20211018130229.308694-9-maxime.coquelin@redhat.com>

Hi Maxime,

> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, October 18, 2021 9:02 PM
> To: dev@dpdk.org; Xia, Chenbo <chenbo.xia@intel.com>; Hu, Jiayu
> <jiayu.hu@intel.com>; Wang, YuanX <yuanx.wang@intel.com>; Ma,
> WenwuX <wenwux.ma@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; david.marchand@redhat.com
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH v1 08/14] vhost: improve IO vector logic
> 
> IO vectors and their iterators arrays were part of the async metadata but not
> their indexes.
> 
> In order to makes this more consistent, the patch adds the indexes to the
> async metadata. Doing that, we can avoid triggering DMA transfer within the
> loop as it IO vector index overflow is now prevented in the
> async_mbuf_to_desc() function.
> 
> Note that previous detection mechanism was broken since the overflow
> already happened when detected, so OOB memory access would already
> have happened.
> 
> With this changes done, virtio_dev_rx_async_submit_split()
> and virtio_dev_rx_async_submit_packed() can be further simplified.
> 
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
>  lib/vhost/vhost.h      |   2 +
>  lib/vhost/virtio_net.c | 291 ++++++++++++++++++-----------------------
>  2 files changed, 131 insertions(+), 162 deletions(-)
> 
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h index
> dae9a1ac2d..812d4c55a5 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -134,6 +134,8 @@ struct vhost_async {
> 
>  	struct rte_vhost_iov_iter iov_iter[VHOST_MAX_ASYNC_IT];
>  	struct rte_vhost_iovec iovec[VHOST_MAX_ASYNC_VEC];
> +	uint16_t iter_idx;
> +	uint16_t iovec_idx;
> 
>  	/* data transfer status */
>  	struct async_inflight_info *pkts_info; diff --git a/lib/vhost/virtio_net.c
> b/lib/vhost/virtio_net.c index ae7dded979..c80823a8de 100644
> --- a/lib/vhost/virtio_net.c
> +++ b/lib/vhost/virtio_net.c
> @@ -924,33 +924,86 @@ copy_mbuf_to_desc(struct virtio_net *dev, struct
> vhost_virtqueue *vq,
>  	return error;
>  }
> 
> +static __rte_always_inline int
> +async_iter_initialize(struct vhost_async *async) {
> +	struct rte_vhost_iov_iter *iter;
> +
> +	if (unlikely(async->iovec_idx >= VHOST_MAX_ASYNC_VEC)) {
> +		VHOST_LOG_DATA(ERR, "no more async iovec available\n");
> +		return -1;
> +	}
> +
> +	iter = async->iov_iter + async->iter_idx;
> +	iter->iov = async->iovec + async->iovec_idx;
> +	iter->nr_segs = 0;
> +
> +	return 0;
> +}
> +
> +static __rte_always_inline int
> +async_iter_add_iovec(struct vhost_async *async, void *src, void *dst,
> +size_t len) {
> +	struct rte_vhost_iov_iter *iter;
> +	struct rte_vhost_iovec *iovec;
> +
> +	if (unlikely(async->iovec_idx >= VHOST_MAX_ASYNC_VEC)) {
> +		VHOST_LOG_DATA(ERR, "no more async iovec available\n");
> +		return -1;
> +	}

For large packets, like 64KB in iperf test, async_iter_add_iovec() frequently
reports the log above, as we run out of iovecs. I think it's better to change
the log from ERR to DEBUG.

In addition, the size of iovec is too small. For burst 32 and 64KB pkts, it's
easy to run out of iovecs and we will drop the pkts to enqueue if it happens,
which hurts performance. Enlarging the array is a choice to mitigate the
issue, but another solution is to reallocate iovec once we run out of it. How do
you think?

Thanks,
Jiayu
> +
> +	iter = async->iov_iter + async->iter_idx;
> +	iovec = async->iovec + async->iovec_idx;
> +
> +	iovec->src_addr = src;
> +	iovec->dst_addr = dst;
> +	iovec->len = len;
> +
> +	iter->nr_segs++;
> +	async->iovec_idx++;
> +
> +	return 0;
> +}

  reply	other threads:[~2021-10-25  7:22 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-18 13:02 [dpdk-dev] [PATCH v1 00/14] vhost: clean-up and simplify async implementation Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 01/14] vhost: move async data in a dedicated structure Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 02/14] vhost: hide inflight async structure Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 03/14] vhost: simplify async IO vectors Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 04/14] vhost: simplify async IO vectors iterators Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 05/14] vhost: remove async batch threshold Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 06/14] vhost: introduce specific iovec structure Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 07/14] vhost: remove useless fields in async iterator struct Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 08/14] vhost: improve IO vector logic Maxime Coquelin
2021-10-25  7:22   ` Hu, Jiayu [this message]
2021-10-25 10:02     ` Maxime Coquelin
2021-10-26  7:07       ` Hu, Jiayu
2021-10-26  7:27         ` Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 09/14] vhost: remove notion of async descriptor Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 10/14] vhost: simplify async enqueue completion Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 11/14] vhost: simplify getting the first in-flight index Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 12/14] vhost: prepare async for mbuf to desc refactoring Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 13/14] vhost: prepare sync " Maxime Coquelin
2021-10-18 13:02 ` [dpdk-dev] [PATCH v1 14/14] vhost: merge sync and async mbuf to desc filling Maxime Coquelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9239770ed6c74f67975524bd081af39a@intel.com \
    --to=jiayu.hu@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=chenbo.xia@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=john.mcnamara@intel.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=wenwux.ma@intel.com \
    --cc=yuanx.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).