From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yk0-f174.google.com (mail-yk0-f174.google.com [209.85.160.174]) by dpdk.org (Postfix) with ESMTP id 571B65A6C for ; Thu, 19 Nov 2015 19:16:25 +0100 (CET) Received: by ykdv3 with SMTP id v3so119291309ykd.0 for ; Thu, 19 Nov 2015 10:16:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bigswitch-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ieNcZFO4lle36ewsxp/ZoKzU84KP/Bi9oErEptqvSoI=; b=Er4I181IuD+6l24ZQ0QuzFvLBVJMD6lEARraKAmv1Q8dUkoCxqEzeoj7YKO0zAuexW ftlPrmeKqzFZvDP6dO+eBaR46/ChwPXbebTWbqMHw4Fn1/99vF+Gq0iWlfTCUI65Qfp3 sLywds3n6kZHrWKgCGtqrz5MEAPGD9kL+9Y1nojag1u5PCfY21qZHCYfq1gkTGZYyzfv twetimJmxP0rfyXqo643c0WD9j7+XRG+/DGPpi+O6ITrOXVjcaILdYPC8k6V0s3Jee+9 WC4zzbFAw8Iy6FHy4a4ZxgeZeVD6rw3H9IyJQYjIdUCYrF4Ruy+SE6qCKsCHKdmHbS0T Peuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=ieNcZFO4lle36ewsxp/ZoKzU84KP/Bi9oErEptqvSoI=; b=gopUsib9eT1+OMpkfYlZqlvuIss1WwvzIcz+jT+eUPyu8tXnvdw9bo8YTlClwNrWbx j8sG50qp00Eujt81a3De70eCoNnjj3uEzLkcpReRPPDHd7OW7H5Vdn/2O9w9TkHZ5Kv0 irpj1OxnKYgJ362xSB0/vy2+IRu0KTWDqfRXtUdeOlQ52Pz+97oSLXkMY4xkjOEL4z+y ruKxKFqE2gETgFhGnEZcGnFJJrcgah0N2B545+0SzbZTsMt85jowsuTygNv0hjyUVHWY wujtEP3N64kTC3VQCRzFj1Gl6wu1YqEoaih/5SsHEdB6VHiMOdUSGyUEnI8pQfiVy7Bx 04kw== X-Gm-Message-State: ALoCoQlLFb3pzzC5svya0ojRdFGBbSWWq77GLOsBmQv/UN798KS4VFUy0VlmrxPcTSUXPYrxCGNP MIME-Version: 1.0 X-Received: by 10.13.216.75 with SMTP id a72mr7789956ywe.312.1447956984722; Thu, 19 Nov 2015 10:16:24 -0800 (PST) Received: by 10.31.3.170 with HTTP; Thu, 19 Nov 2015 10:16:24 -0800 (PST) In-Reply-To: <1447930650-26023-1-git-send-email-mukawa@igel.co.jp> References: <1447930650-26023-1-git-send-email-mukawa@igel.co.jp> Date: Thu, 19 Nov 2015 10:16:24 -0800 Message-ID: From: Rich Lane To: Tetsuya Mukawa Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: nakajima.yoshihiro@lab.ntt.co.jp, zhbzg@huawei.com, mst@redhat.com, dev@dpdk.org, oscar.zhangbo@huawei.com, gaoxiaoqiu@huawei.com, ann.zhuangyanying@huawei.com, zhoujingbin@huawei.com, guohongzhen@huawei.com Subject: Re: [dpdk-dev] [RFC PATCH 0/2] Virtio-net PMD Extension to work on host. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 18:16:25 -0000 What's the reason for using qemu as a middleman? Couldn't the new PMD itself open /dev/vhost-net or the vhost-user socket and send the commands to set up virtqueues? That was the approach taken by Jianfeng's earlier RFC. On Thu, Nov 19, 2015 at 2:57 AM, Tetsuya Mukawa wrote: > THIS IS A PoC IMPLEMENATION. > > [Abstraction] > > Normally, virtio-net PMD only works on VM, because there is no virtio-net > device on host. > This RFC patch extends virtio-net PMD to be able to work on host as > virtual PMD. > But we didn't implement virtio-net device as a part of virtio-net PMD. > To prepare virtio-net device for the PMD, start QEMU process with special > QTest mode, then connect it from virtio-net PMD through unix domain socket. > > The PMD can connect to anywhere QEMU virtio-net device can. > For example, the PMD can connects to vhost-net kernel module and > vhost-user backend application. > Similar to virtio-net PMD on QEMU, application memory that uses virtio-net > PMD will be shared between vhost backend application. > But vhost backend application memory will not be shared. > > Main target of this PMD is container like docker, rkt, lxc and etc. > We can isolate related processes(virtio-net PMD process, QEMU and > vhost-user backend process) by container. > But, to communicate through unix domain socket, shared directory will be > needed. > > > [How to use] > > So far, we need QEMU patch to connect to vhost-user backend. > Please check known issue in later section. > Because of this, I will describe example of using vhost-net kernel module. > > - Compile > Set "CONFIG_RTE_VIRTIO_VDEV=y" in config/common_linux. > Then compile it. > > - Start QEMU like below. > $ sudo qemu-system-x86_64 -qtest unix:/tmp/qtest0,server -machine > accel=qtest \ > -display none -qtest-log /dev/null \ > -netdev > type=tap,script=/etc/qemu-ifup,id=net0,vhost=on \ > -device virtio-net-pci,netdev=net0 \ > -chardev > socket,id=chr1,path=/tmp/ivshmem0,server \ > -device ivshmem,size=1G,chardev=chr1,vectors=1 > > - Start DPDK application like below > $ sudo ./testpmd -c f -n 1 -m 1024 --shm \ > > --vdev="eth_cvio0,qtest=/tmp/qtest0,ivshmem=/tmp/ivshmem0" -- \ > --disable-hw-vlan --txqflags=0xf00 -i > > - Check created tap device. > > (*1) Please Specify same memory size in QEMU and DPDK command line. > > > [Detailed Description] > > - virtio-net device implementation > The PMD uses QEMU virtio-net device. To do that, QEMU QTest functionality > is used. > QTest is a test framework of QEMU devices. It allows us to implement a > device driver outside of QEMU. > With QTest, we can implement DPDK application and virtio-net PMD as > standalone process on host. > When QEMU is invoked as QTest mode, any guest code will not run. > To know more about QTest, see below. > http://wiki.qemu.org/Features/QTest > > - probing devices > QTest provides a unix domain socket. Through this socket, driver process > can access to I/O port and memory of QEMU virtual machine. > The PMD will send I/O port accesses to probe pci devices. > If we can find virtio-net and ivshmem device, initialize the devices. > Also, I/O port accesses of virtio-net PMD will be sent through socket, and > virtio-net PMD can initialize vitio-net device on QEMU correctly. > > - ivshmem device to share memory > To share memory that virtio-net PMD process uses, ivshmem device will be > used. > Because ivshmem device can only handle one file descriptor, shared memory > should be consist of one file. > To allocate such a memory, EAL has new option called "--shm". > If the option is specified, EAL will open a file and allocate memory from > hugepages. > While initializing ivshmem device, we can set BAR(Base Address Register). > It represents which memory QEMU vcpu can access to this shared memory. > We will specify host physical address of shared memory as this address. > It is very useful because we don't need to apply patch to QEMU to > calculate address offset. > (For example, if virtio-net PMD process will allocate memory from shared > memory, then specify the physical address of it to virtio-net register, > QEMU virtio-net device can understand it without calculating address > offset.) > > - Known limitation > So far, the PMD doesn't handle interrupts from QEMU devices. > Because of this, VIRTIO_NET_F_STATUS functionality is dropped. > But without it, we can use all virtio-net functions. > > - Known issues > So far, to use vhost-user, we need to apply vhost-user patch to QEMU and > DPDK vhost library. > This is because, QEMU will not send memory information and file descriptor > of ivshmem device to vhost-user backend. > (Anyway, vhost-net kernel module can receive the information. So > vhost-user behavior will not be correct. I will submit the patch to QEMU > soon) > Also, we may have an issue in DPDK vhost library to handle kickfd and > callfd. The patch for it is needed. > (Let me check it more) > If someone wants to check vhost-user behavior, I will describe it more in > later email. > > > [Addition] > > We can apply same manner to handle any kind of QEMU devices from DPDK > application. > So far, I don't have any ideas except for virtio-net device. But someone > would have. > > > Tetsuya Mukawa (2): > EAL: Add new EAL "--shm" option. > virtio: Extend virtio-net PMD to support container environment > > config/common_linuxapp | 5 + > drivers/net/virtio/Makefile | 4 + > drivers/net/virtio/qtest.c | 590 > +++++++++++++++++++++++++++++ > drivers/net/virtio/virtio_ethdev.c | 214 ++++++++++- > drivers/net/virtio/virtio_ethdev.h | 16 + > drivers/net/virtio/virtio_pci.h | 25 ++ > lib/librte_eal/common/eal_common_options.c | 5 + > lib/librte_eal/common/eal_internal_cfg.h | 1 + > lib/librte_eal/common/eal_options.h | 2 + > lib/librte_eal/common/include/rte_memory.h | 5 + > lib/librte_eal/linuxapp/eal/eal_memory.c | 71 ++++ > 11 files changed, 917 insertions(+), 21 deletions(-) > create mode 100644 drivers/net/virtio/qtest.c > > -- > 2.1.4 > >