From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f175.google.com (mail-pd0-f175.google.com [209.85.192.175]) by dpdk.org (Postfix) with ESMTP id 1DC517E75 for ; Wed, 12 Nov 2014 05:02:53 +0100 (CET) Received: by mail-pd0-f175.google.com with SMTP id y13so11467433pdi.34 for ; Tue, 11 Nov 2014 20:12:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=jqErlCdUR4LmXTFXUJz8RSS6BcTJUPQNSOY1lqqplKs=; b=hNF6gGUr5WRFqA03NRk4SYw47jb0QYR2Z2EAK5hjfZmI7xRwsuRKIxC0Qxyw6IMFGA k7Of46o2t+MJ2uDkjhwoRrUaqamA/Kbkye/H9OW0yB6z0P8vcqcFH6LaGixYYWTXTqhZ ebKuTfj7gc0MfJTnzekBKbOyFrDxT0rUAX+8kYY1FscZIu6pEg2nHriCLUmeCu75dIId q12oGeCw/iBHuusFU0cn5pmv5DD4AGZinuxwYqtowm1Jd8wQZx7hOBXJZjQPYHaYS1pj nlt2inGrQXUdTg7QduKeEy8LMBK5OcVArOu47VpOcx0ZRrzWrxtoa38sLfmw8BPL4mZO dD+g== X-Gm-Message-State: ALoCoQl7gGRDz0nZuv/bbYVgp4hHqqiAZge2x5oPydDIEGBjG+EIy/JN+aA8hX/12p+7nLP8ws3r X-Received: by 10.68.225.131 with SMTP id rk3mr15355164pbc.157.1415765564845; Tue, 11 Nov 2014 20:12:44 -0800 (PST) Received: from [10.16.129.101] (napt.igel.co.jp. [219.106.231.132]) by mx.google.com with ESMTPSA id dt11sm1477100pdb.34.2014.11.11.20.12.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 11 Nov 2014 20:12:43 -0800 (PST) Message-ID: <5462DE39.1070006@igel.co.jp> Date: Wed, 12 Nov 2014 13:12:41 +0900 From: Tetsuya Mukawa User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Xie, Huawei" , "dev@dpdk.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] vhost-user technical isssues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Nov 2014 04:02:53 -0000 X-List-Received-Date: Wed, 12 Nov 2014 04:02:53 -0000 Hi Xie, (2014/11/12 6:37), Xie, Huawei wrote: > Hi Tetsuya: > There are two major technical issues in my mind for vhost-user implemen= tation. > > 1) memory region map > Vhost-user passes us file fd and offset for each memory region. Unfortu= nately the mmap offset is "very" wrong. I discovered this issue long time= ago, and also found > that I couldn't mmap the huge page file even with correct offset(need d= ouble check). > Just now I find that people reported this issue on Nov 3. > [Qemu-devel] [PULL 27/29] vhost-user: fix mmap offset calculation > Anyway, I turned to the same idea used in our DPDK vhost-cuse: only use= the fd for region(0) to map the whole file. > I think we should use this way temporarily to support qemu-2.1 as it ha= s that bug. I agree with you. Also we may have an issue about un-mapping file on hugetlbfs of linux. When I check munmap(), it seems 'size' need to be aligned by hugepage siz= e. (I guess it may be a kernel bug. Might be fixed already.) Please add return value checking code for munmap(). Still munmap() might be failed. > > 2) what message is the indicator for vhost start/release? > Previously for vhost-cuse, it has SET_BACKEND message. > What we should do for vhost-user? > SET_VRING_KICK for start? I think so. > What about for release? > Unlike the kernel virtio, the DPDK virtio in guest could be restarted. = > > Thoughts? I guess we need to consider 2 types of restarting. One is virtio-net driver restarting, the other is vhost-user backend restarting. But, so far, it's nice to start to think about virtio-net driver restarting first. Probably we need to implement a way to let vhost-user backend know virtio-net driver is restarted. I am not sure what is good way to let vhost-user backend know it. But how about followings RFC? - When unix domain socket is closed, vhost-user backend should treat it as "release". It is useful when QEMU itself is gone suddenly. - Also, implementing new ioctl command like VHOST_RESET_BACKEND. This command should be sent from virtio-net device of QEMU when VIRTIO_CONFIG_STATUS_RESET register of virtio-net device is set by vrtio-net driver. (Usually this register is set when virtio-net driver is initialized or stopped.) It means we need to change QEMU. ;) It seems virtio-net PMD already sets this register when PMD is initialized or stopped. So this framework should work, and can let vhost-user backend know driver resetting. (And I guess we can say same things for virtio-net kernel driver.) It might be enough to close an unix domain socket, instead of implementing new command. But in the case, we may need auto reconnection mechanism. - We also need to consider DPDK application is gone suddenly without setting reset register. In the case, vhost-user backend cannot know it. Only user (or some kind of watchdog applications on guest) knows it. Because of this, user(or app.) should have responsibility to solve this situation. To be more precise, user should let vhost-user backend know device releasing. If user starts an other DPDK application without solving the issue, the new DPDK application may access memory that vhost-user backend is also accessing. I guess user can solve the issue using "dpdk_nic_bind.py". The script can move virtio-net device to kernel virtio-net driver, and return it to igb_uio. While those steps, virtio-net device is initialized by virtio-net kernel driver. So vhost-user backend can know device releasing. Tetsuya > > -huawei