From: Linhaifeng <haifeng.lin@huawei.com>
To: Tetsuya Mukawa <mukawa@igel.co.jp>,
"Xie, Huawei" <huawei.xie@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] vhost-user technical isssues
Date: Thu, 13 Nov 2014 14:30:31 +0800 [thread overview]
Message-ID: <54645007.3010301@huawei.com> (raw)
In-Reply-To: <5462DE39.1070006@igel.co.jp>
On 2014/11/12 12:12, Tetsuya Mukawa wrote:
> Hi Xie,
>
> (2014/11/12 6:37), Xie, Huawei wrote:
>> Hi Tetsuya:
>> There are two major technical issues in my mind for vhost-user implementation.
>>
>> 1) memory region map
>> Vhost-user passes us file fd and offset for each memory region. Unfortunately the mmap offset is "very" wrong. I discovered this issue long time ago, and also found
>> that I couldn't mmap the huge page file even with correct offset(need double check).
>> Just now I find that people reported this issue on Nov 3.
>> [Qemu-devel] [PULL 27/29] vhost-user: fix mmap offset calculation
>> Anyway, I turned to the same idea used in our DPDK vhost-cuse: only use the fd for region(0) to map the whole file.
>> I think we should use this way temporarily to support qemu-2.1 as it has that bug.
> I agree with you.
> Also we may have an issue about un-mapping file on hugetlbfs of linux.
> When I check munmap(), it seems 'size' need to be aligned by hugepage size.
> (I guess it may be a kernel bug. Might be fixed already.)
> Please add return value checking code for munmap().
> Still munmap() might be failed.
>
are you munmmap the region 0? region 0 is not need to mmap so not need to munmap too.
I can munmap success with the other regions.
>>
>> 2) what message is the indicator for vhost start/release?
>> Previously for vhost-cuse, it has SET_BACKEND message.
>> What we should do for vhost-user?
>> SET_VRING_KICK for start?
> I think so.
>
>> What about for release?
>> Unlike the kernel virtio, the DPDK virtio in guest could be restarted.
>>
>> Thoughts?
> I guess we need to consider 2 types of restarting.
> One is virtio-net driver restarting, the other is vhost-user backend
> restarting.
> But, so far, it's nice to start to think about virtio-net driver
> restarting first.
>
> Probably we need to implement a way to let vhost-user backend know
> virtio-net driver is restarted.
> I am not sure what is good way to let vhost-user backend know it.
> But how about followings RFC?
>
> - When unix domain socket is closed, vhost-user backend should treat it
> as "release".
> It is useful when QEMU itself is gone suddenly.
>
> - Also, implementing new ioctl command like VHOST_RESET_BACKEND.
> This command should be sent from virtio-net device of QEMU when
> VIRTIO_CONFIG_STATUS_RESET register of virtio-net device is set by
> vrtio-net driver.
> (Usually this register is set when virtio-net driver is initialized or
> stopped.)
> It means we need to change QEMU. ;)
> It seems virtio-net PMD already sets this register when PMD is
> initialized or stopped.
> So this framework should work, and can let vhost-user backend know
> driver resetting.
> (And I guess we can say same things for virtio-net kernel driver.)
> It might be enough to close an unix domain socket, instead of
> implementing new command.
> But in the case, we may need auto reconnection mechanism.
>
> - We also need to consider DPDK application is gone suddenly without
> setting reset register.
> In the case, vhost-user backend cannot know it. Only user (or some kind
> of watchdog
> applications on guest) knows it.
> Because of this, user(or app.) should have responsibility to solve this
> situation.
> To be more precise, user should let vhost-user backend know device
> releasing.
> If user starts an other DPDK application without solving the issue, the
> new DPDK application may
> access memory that vhost-user backend is also accessing.
> I guess user can solve the issue using "dpdk_nic_bind.py".
> The script can move virtio-net device to kernel virtio-net driver, and
> return it to igb_uio.
> While those steps, virtio-net device is initialized by virtio-net
> kernel driver.
> So vhost-user backend can know device releasing.
>
> Tetsuya
>
>>
>> -huawei
>
>
>
>
--
Regards,
Haifeng
next prev parent reply other threads:[~2014-11-13 6:20 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-11 21:37 Xie, Huawei
2014-11-12 4:12 ` Tetsuya Mukawa
2014-11-13 6:30 ` Linhaifeng [this message]
2014-11-14 2:30 ` Tetsuya Mukawa
2014-11-14 3:13 ` Linhaifeng
2014-11-14 3:40 ` Tetsuya Mukawa
2014-11-14 4:05 ` Tetsuya Mukawa
2014-11-14 4:42 ` Linhaifeng
2014-11-14 5:12 ` Tetsuya Mukawa
2014-11-14 5:30 ` Linhaifeng
2014-11-14 6:57 ` Tetsuya Mukawa
2014-11-14 10:59 ` Xie, Huawei
2014-11-17 6:14 ` Tetsuya Mukawa
2014-11-14 0:22 ` Xie, Huawei
2014-11-14 2:52 ` Tetsuya Mukawa
2014-11-15 1:42 ` Xie, Huawei
2014-11-13 6:12 ` Linhaifeng
2014-11-13 6:27 ` Linhaifeng
2014-11-14 1:28 ` Xie, Huawei
2014-11-14 2:24 ` Linhaifeng
2014-11-14 2:35 ` Tetsuya Mukawa
2014-11-14 6:24 ` Xie, Huawei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54645007.3010301@huawei.com \
--to=haifeng.lin@huawei.com \
--cc=dev@dpdk.org \
--cc=huawei.xie@intel.com \
--cc=mukawa@igel.co.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).