From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [119.145.14.65]) by dpdk.org (Postfix) with ESMTP id 25247595B for ; Thu, 13 Nov 2014 07:20:39 +0100 (CET) Received: from 172.24.2.119 (EHLO SZXEML455-HUB.china.huawei.com) ([172.24.2.119]) by szxrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id CCH20231; Thu, 13 Nov 2014 14:30:33 +0800 (CST) Received: from [127.0.0.1] (10.177.19.115) by SZXEML455-HUB.china.huawei.com (10.82.67.198) with Microsoft SMTP Server id 14.3.158.1; Thu, 13 Nov 2014 14:30:32 +0800 Message-ID: <54645007.3010301@huawei.com> Date: Thu, 13 Nov 2014 14:30:31 +0800 From: Linhaifeng User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Tetsuya Mukawa , "Xie, Huawei" , "dev@dpdk.org" References: <5462DE39.1070006@igel.co.jp> In-Reply-To: <5462DE39.1070006@igel.co.jp> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.19.115] X-CFilter-Loop: Reflected Subject: Re: [dpdk-dev] vhost-user technical isssues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Nov 2014 06:20:42 -0000 On 2014/11/12 12:12, Tetsuya Mukawa wrote: > Hi Xie, > > (2014/11/12 6:37), Xie, Huawei wrote: >> Hi Tetsuya: >> There are two major technical issues in my mind for vhost-user implementation. >> >> 1) memory region map >> Vhost-user passes us file fd and offset for each memory region. Unfortunately the mmap offset is "very" wrong. I discovered this issue long time ago, and also found >> that I couldn't mmap the huge page file even with correct offset(need double check). >> Just now I find that people reported this issue on Nov 3. >> [Qemu-devel] [PULL 27/29] vhost-user: fix mmap offset calculation >> Anyway, I turned to the same idea used in our DPDK vhost-cuse: only use the fd for region(0) to map the whole file. >> I think we should use this way temporarily to support qemu-2.1 as it has that bug. > I agree with you. > Also we may have an issue about un-mapping file on hugetlbfs of linux. > When I check munmap(), it seems 'size' need to be aligned by hugepage size. > (I guess it may be a kernel bug. Might be fixed already.) > Please add return value checking code for munmap(). > Still munmap() might be failed. > are you munmmap the region 0? region 0 is not need to mmap so not need to munmap too. I can munmap success with the other regions. >> >> 2) what message is the indicator for vhost start/release? >> Previously for vhost-cuse, it has SET_BACKEND message. >> What we should do for vhost-user? >> SET_VRING_KICK for start? > I think so. > >> What about for release? >> Unlike the kernel virtio, the DPDK virtio in guest could be restarted. >> >> Thoughts? > I guess we need to consider 2 types of restarting. > One is virtio-net driver restarting, the other is vhost-user backend > restarting. > But, so far, it's nice to start to think about virtio-net driver > restarting first. > > Probably we need to implement a way to let vhost-user backend know > virtio-net driver is restarted. > I am not sure what is good way to let vhost-user backend know it. > But how about followings RFC? > > - When unix domain socket is closed, vhost-user backend should treat it > as "release". > It is useful when QEMU itself is gone suddenly. > > - Also, implementing new ioctl command like VHOST_RESET_BACKEND. > This command should be sent from virtio-net device of QEMU when > VIRTIO_CONFIG_STATUS_RESET register of virtio-net device is set by > vrtio-net driver. > (Usually this register is set when virtio-net driver is initialized or > stopped.) > It means we need to change QEMU. ;) > It seems virtio-net PMD already sets this register when PMD is > initialized or stopped. > So this framework should work, and can let vhost-user backend know > driver resetting. > (And I guess we can say same things for virtio-net kernel driver.) > It might be enough to close an unix domain socket, instead of > implementing new command. > But in the case, we may need auto reconnection mechanism. > > - We also need to consider DPDK application is gone suddenly without > setting reset register. > In the case, vhost-user backend cannot know it. Only user (or some kind > of watchdog > applications on guest) knows it. > Because of this, user(or app.) should have responsibility to solve this > situation. > To be more precise, user should let vhost-user backend know device > releasing. > If user starts an other DPDK application without solving the issue, the > new DPDK application may > access memory that vhost-user backend is also accessing. > I guess user can solve the issue using "dpdk_nic_bind.py". > The script can move virtio-net device to kernel virtio-net driver, and > return it to igb_uio. > While those steps, virtio-net device is initialized by virtio-net > kernel driver. > So vhost-user backend can know device releasing. > > Tetsuya > >> >> -huawei > > > > -- Regards, Haifeng