From: "Tan, Jianfeng" <jianfeng.tan@intel.com>
To: Maxime Coquelin <maxime.coquelin@redhat.com>,
Victor Kaplansky <vkaplans@redhat.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"stable@dpdk.org" <stable@dpdk.org>,
"Yang, Yi Y" <yi.y.yang@intel.com>,
"Harris, James R" <james.r.harris@intel.com>,
"Yang, Ziye" <ziye.yang@intel.com>,
"Liu, Changpeng" <changpeng.liu@intel.com>,
"Wodkowski, PawelX" <pawelx.wodkowski@intel.com>,
"Stojaczyk, DariuszX" <dariuszx.stojaczyk@intel.com>,
Yuanhan Liu <yliu@fridaylinux.org>
Subject: Re: [dpdk-stable] [PATCH] vhost: fix segfault as handle set_mem_table message
Date: Thu, 29 Mar 2018 07:01:56 +0000 [thread overview]
Message-ID: <ED26CBA2FAD1BF48A8719AEF02201E36514AD030@SHSMSX103.ccr.corp.intel.com> (raw)
In-Reply-To: <cd734b70-1532-b591-63d4-63e2d6b1ec97@redhat.com>
Hi Maxime and Victor,
> -----Original Message-----
> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Tuesday, December 5, 2017 10:28 PM
> To: Yuanhan Liu; Tan, Jianfeng; Victor Kaplansky
> Cc: dev@dpdk.org; stable@dpdk.org; Yang, Yi Y
> Subject: Re: [PATCH] vhost: fix segfault as handle set_mem_table message
>
>
>
> On 12/05/2017 03:19 PM, Yuanhan Liu wrote:
> > On Tue, Nov 28, 2017 at 01:09:29PM +0100, Maxime Coquelin wrote:
> >>
> >>
> >> On 11/15/2017 12:41 PM, Jianfeng Tan wrote:
> >>> In a running VM, operations (like device attach/detach) will
> >>> trigger the QEMU to resend set_mem_table to vhost-user backend.
> >>>
> >>> DPDK vhost-user handles this message rudely by unmap all existing
> >>> regions and map new ones. This might lead to segfault if there
> >>> is pmd thread just trying to touch those unmapped memory regions.
> >>>
> >>> But for most cases, except VM memory hotplug,
>
> FYI, Victor is working on implementing a lock-less protection mechanism
> to prevent crashes in such cases. It is intended first to protect
> log_base in case of multiqueue + live-migration, but would solve thi
> issue too.
Bring this issue back for discussion.
Reported by SPDK guys, even with per-queue lock, they could still run into crash as of memory hot plug or unplug.
In detail, you add the lock in an internal struct, vhost_queue which is, unfortunately, not visible to the external datapath, like vhost-scsi in SPDK.
The memory hot plug and unplug would be the main issue from SPDK side so far. For this specific issue, I think we can continue this patch to filter out the changed regions, and keep unchanged regions not remapped.
But I know that the per-vq lock is not only to resolve the memory table issue, some other vhost-user messages could also lead to problems? If yes, shall we take a step back, to think about how to solve this kind of issues for backends, like vhost-scsi.
Thoughts?
Thanks,
Jianfeng
>
> >>>> QEMU still sends the
> >>> set_mem_table message even the memory regions are not changed as
> >>> QEMU vhost-user filters out those not backed by file (fd > 0).
> >>>
> >>> To fix this case, we add a check in the handler to see if the
> >>> memory regions are really changed; if not, we just keep old memory
> >>> regions.
> >>>
> >>> Fixes: 8f972312b8f4 ("vhost: support vhost-user")
> >>>
> >>> CC: stable@dpdk.org
> >>>
> >>> CC: Yuanhan Liu <yliu@fridaylinux.org>
> >>> CC: Maxime Coquelin <maxime.coquelin@redhat.com>
> >>>
> >>> Reported-by: Yang Zhang <zy107165@alibaba-inc.com>
> >>> Reported-by: Xin Long <longxin.xl@alibaba-inc.com>
> >>> Signed-off-by: Yi Yang <yi.y.yang@intel.com>
> >>> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> >>> ---
> >>> lib/librte_vhost/vhost_user.c | 33
> +++++++++++++++++++++++++++++++++
> >>> 1 file changed, 33 insertions(+)
> >>
> >> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> >
> > Applied to dpdk-next-virtio.
> >
> > Thanks.
> >
> > --yliu
> >
>
> Maxime
next prev parent reply other threads:[~2018-03-29 7:02 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-15 11:41 Jianfeng Tan
2017-11-28 12:09 ` Maxime Coquelin
2017-12-05 14:19 ` Yuanhan Liu
2017-12-05 14:28 ` Maxime Coquelin
2018-03-29 7:01 ` Tan, Jianfeng [this message]
2018-03-29 7:35 ` Maxime Coquelin
2018-03-29 12:57 ` Wodkowski, PawelX
2018-03-29 13:20 ` Tan, Jianfeng
2018-03-29 16:37 ` Maxime Coquelin
2018-03-29 18:09 ` Wodkowski, PawelX
2018-03-29 12:59 ` Tan, Jianfeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ED26CBA2FAD1BF48A8719AEF02201E36514AD030@SHSMSX103.ccr.corp.intel.com \
--to=jianfeng.tan@intel.com \
--cc=changpeng.liu@intel.com \
--cc=dariuszx.stojaczyk@intel.com \
--cc=dev@dpdk.org \
--cc=james.r.harris@intel.com \
--cc=maxime.coquelin@redhat.com \
--cc=pawelx.wodkowski@intel.com \
--cc=stable@dpdk.org \
--cc=vkaplans@redhat.com \
--cc=yi.y.yang@intel.com \
--cc=yliu@fridaylinux.org \
--cc=ziye.yang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).