From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: Yuanhan Liu <yuanhan.liu@linux.intel.com>, dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v2 4/7] vhost: add dequeue zero copy
Date: Mon, 26 Sep 2016 22:45:46 +0200 [thread overview]
Message-ID: <038cdf17-511f-c582-ef8f-46c81f51d161@redhat.com> (raw)
In-Reply-To: <1474604007-5221-5-git-send-email-yuanhan.liu@linux.intel.com>
On 09/23/2016 06:13 AM, Yuanhan Liu wrote:
> The basic idea of dequeue zero copy is, instead of copying data from
> the desc buf, here we let the mbuf reference the desc buf addr directly.
>
> Doing so, however, has one major issue: we can't update the used ring
> at the end of rte_vhost_dequeue_burst. Because we don't do the copy
> here, an update of the used ring would let the driver to reclaim the
> desc buf. As a result, DPDK might reference a stale memory region.
>
> To update the used ring properly, this patch does several tricks:
>
> - when mbuf references a desc buf, refcnt is added by 1.
>
> This is to pin lock the mbuf, so that a mbuf free from the DPDK
> won't actually free it, instead, refcnt is subtracted by 1.
>
> - We chain all those mbuf together (by tailq)
>
> And we check it every time on the rte_vhost_dequeue_burst entrance,
> to see if the mbuf is freed (when refcnt equals to 1). If that
> happens, it means we are the last user of this mbuf and we are
> safe to update the used ring.
>
> - "struct zcopy_mbuf" is introduced, to associate an mbuf with the
> right desc idx.
>
> Dequeue zero copy is introduced for performance reason, and some rough
> tests show about 50% perfomance boost for packet size 1500B. For small
> packets, (e.g. 64B), it actually slows a bit down (well, it could up to
> 15%). That is expected because this patch introduces some extra works,
> and it outweighs the benefit from saving few bytes copy.
>
> Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> ---
>
> v2: - use unlikely/likely for dequeue_zero_copy check, as it's not enabled
> by default, as well as it has some limitations in vm2nic case.
>
> - handle the case that a desc buf might across 2 host phys pages
>
> - reset nr_zmbuf to 0 at set_vring_num
>
> - set the zmbuf_size to vq->size, but not the double of it.
> ---
> lib/librte_vhost/vhost.c | 2 +
> lib/librte_vhost/vhost.h | 22 +++++-
> lib/librte_vhost/vhost_user.c | 42 +++++++++-
> lib/librte_vhost/virtio_net.c | 173 +++++++++++++++++++++++++++++++++++++-----
> 4 files changed, 219 insertions(+), 20 deletions(-)
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Thanks,
Maxime
next prev parent reply other threads:[~2016-09-26 20:45 UTC|newest]
Thread overview: 75+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-08-23 8:10 [dpdk-dev] [PATCH 0/6] vhost: add Tx zero copy support Yuanhan Liu
2016-08-23 8:10 ` [dpdk-dev] [PATCH 1/6] vhost: simplify memory regions handling Yuanhan Liu
2016-08-23 9:17 ` Maxime Coquelin
2016-08-24 7:26 ` Xu, Qian Q
2016-08-24 7:40 ` Yuanhan Liu
2016-08-24 7:36 ` Xu, Qian Q
2016-08-23 8:10 ` [dpdk-dev] [PATCH 2/6] vhost: get guest/host physical address mappings Yuanhan Liu
2016-08-23 9:58 ` Maxime Coquelin
2016-08-23 12:32 ` Yuanhan Liu
2016-08-23 13:25 ` Maxime Coquelin
2016-08-23 13:49 ` Yuanhan Liu
2016-08-23 14:05 ` Maxime Coquelin
2016-08-23 8:10 ` [dpdk-dev] [PATCH 3/6] vhost: introduce last avail idx for Tx Yuanhan Liu
2016-08-23 12:27 ` Maxime Coquelin
2016-08-23 8:10 ` [dpdk-dev] [PATCH 4/6] vhost: add Tx zero copy Yuanhan Liu
2016-08-23 14:04 ` Maxime Coquelin
2016-08-23 14:31 ` Yuanhan Liu
2016-08-23 15:40 ` Maxime Coquelin
2016-08-23 8:10 ` [dpdk-dev] [PATCH 5/6] vhost: add a flag to enable " Yuanhan Liu
2016-09-06 9:00 ` Xu, Qian Q
2016-09-06 9:42 ` Xu, Qian Q
2016-09-06 10:02 ` Yuanhan Liu
2016-09-07 2:43 ` Xu, Qian Q
2016-09-06 9:55 ` Yuanhan Liu
2016-09-07 16:00 ` Thomas Monjalon
2016-09-08 7:21 ` Yuanhan Liu
2016-09-08 7:57 ` Thomas Monjalon
2016-08-23 8:10 ` [dpdk-dev] [PATCH 6/6] examples/vhost: add an option " Yuanhan Liu
2016-08-23 9:31 ` Thomas Monjalon
2016-08-23 12:33 ` Yuanhan Liu
2016-08-23 14:14 ` Maxime Coquelin
2016-08-23 14:45 ` Yuanhan Liu
2016-08-23 14:18 ` [dpdk-dev] [PATCH 0/6] vhost: add Tx zero copy support Maxime Coquelin
2016-08-23 14:42 ` Yuanhan Liu
2016-08-23 14:53 ` Yuanhan Liu
2016-08-23 16:41 ` Maxime Coquelin
2016-08-29 8:32 ` Xu, Qian Q
2016-08-29 8:57 ` Xu, Qian Q
2016-09-23 4:11 ` Yuanhan Liu
2016-10-09 15:20 ` Yuanhan Liu
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 0/7] vhost: add dequeue " Yuanhan Liu
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 1/7] vhost: simplify memory regions handling Yuanhan Liu
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 2/7] vhost: get guest/host physical address mappings Yuanhan Liu
2016-09-26 20:17 ` Maxime Coquelin
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 3/7] vhost: introduce last avail idx for dequeue Yuanhan Liu
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 4/7] vhost: add dequeue zero copy Yuanhan Liu
2016-09-26 20:45 ` Maxime Coquelin [this message]
2016-10-06 14:37 ` Xu, Qian Q
2016-10-09 2:03 ` Yuanhan Liu
2016-10-10 10:12 ` Xu, Qian Q
2016-10-10 10:14 ` Maxime Coquelin
2016-10-10 10:22 ` Xu, Qian Q
2016-10-10 10:40 ` Xu, Qian Q
2016-10-10 11:48 ` Maxime Coquelin
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 5/7] vhost: add a flag to enable " Yuanhan Liu
2016-09-26 20:57 ` Maxime Coquelin
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 6/7] examples/vhost: add an option " Yuanhan Liu
2016-09-26 21:05 ` Maxime Coquelin
2016-09-23 4:13 ` [dpdk-dev] [PATCH v2 7/7] net/vhost: " Yuanhan Liu
2016-09-26 21:05 ` Maxime Coquelin
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 0/7] vhost: add dequeue zero copy support Yuanhan Liu
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 1/7] vhost: simplify memory regions handling Yuanhan Liu
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 2/7] vhost: get guest/host physical address mappings Yuanhan Liu
2016-11-29 3:10 ` linhaifeng
2016-11-29 13:14 ` linhaifeng
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 3/7] vhost: introduce last avail idx for dequeue Yuanhan Liu
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 4/7] vhost: add dequeue zero copy Yuanhan Liu
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 5/7] vhost: add a flag to enable " Yuanhan Liu
2016-10-09 7:27 ` [dpdk-dev] [PATCH v3 6/7] examples/vhost: add an option " Yuanhan Liu
2016-10-09 7:28 ` [dpdk-dev] [PATCH v3 7/7] net/vhost: " Yuanhan Liu
2016-10-11 13:04 ` [dpdk-dev] [PATCH v3 0/7] vhost: add dequeue zero copy support Xu, Qian Q
2016-10-12 7:48 ` Yuanhan Liu
2016-10-09 10:46 ` [dpdk-dev] [PATCH 0/6] vhost: add Tx " linhaifeng
2016-10-10 8:03 ` Yuanhan Liu
2016-10-14 7:30 ` linhaifeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=038cdf17-511f-c582-ef8f-46c81f51d161@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=dev@dpdk.org \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).