From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f176.google.com (mail-pf0-f176.google.com [209.85.192.176]) by dpdk.org (Postfix) with ESMTP id 961C229D1 for ; Thu, 2 Jun 2016 05:29:57 +0200 (CEST) Received: by mail-pf0-f176.google.com with SMTP id 62so25648011pfd.1 for ; Wed, 01 Jun 2016 20:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bbIkw2wM/3i42YD224E+OWURuM4G62xBhCGcyAteFgU=; b=bl1pB92V5OOLQ2hSjcRMQpF0uI840o7UCYZmG5BZWeEbFPgQZ1lJTnIEWqS72pGNPL FUJZyQ2bJcevYRQ0mr+UTXXGa0RsVDN4BpxXXU6vMqfuIAMeUBnZ4UVSHfFbHpolkPAf WTD3YZkXCHyUFN6WL0CooBDleYhm70/UPmUCbLJpW//umRA3QlD1BJ+xGcNyYVf9av96 p2Hpe7ymoXW1jy9hwt6Ed5ROKPlpERwuxDbjwzKfZTkMBiY5LYftimWGCBTjipz7h9R6 ME2uYKSZ+pcmB3S9vKopzLxCHKUK3XBUMhaOGE3k1mWtyoxBK8loRLafcS2lF0/5XFf6 QFJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bbIkw2wM/3i42YD224E+OWURuM4G62xBhCGcyAteFgU=; b=XHULN3PGR4Y/RYvdldmHKuFLO3ZEs5OEKhq4caP2mPxC5fIu+j0cbNPx1Ag4JpwsFp wFPqHIhCPLOvghtJoernUUkg23h7l3KzoScYtiNc2kvuEPB/+hqCLB98AmtxeixfTVZl GPIl4tcvNLl9w2FZGd/c2te5Av/4HUAUi26q6LuU9d703F1EKaIrf6VtyO2tfGt3wmkz +yOMGk0Ay4bW8CLRxh76Je5QLGdezRCEufHzOK1xvbakTIy5TD3eyxGexJBRe7hMG/0H E/UNkD2YfIHf8lluqIFk7YfH+XJ2P4DQ7gQdHIsmiPJVi3qU4SvKDcKe5nXbZrzAdsMg gS3g== X-Gm-Message-State: ALyK8tJM0AmqSNENup7JT4ojXE9cvsHzUuI2SQhLgRzY+v8ioupx12F1pwD4DlXQewr5XQ== X-Received: by 10.98.69.202 with SMTP id n71mr402827pfi.82.1464838196666; Wed, 01 Jun 2016 20:29:56 -0700 (PDT) Received: from localhost.localdomain (napt.igel.co.jp. [219.106.231.132]) by smtp.gmail.com with ESMTPSA id th5sm59830828pab.31.2016.06.01.20.29.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 01 Jun 2016 20:29:56 -0700 (PDT) From: Tetsuya Mukawa To: dev@dpdk.org Cc: yuanhan.liu@linux.intel.com, jianfeng.tan@intel.com, huawei.xie@intel.com, Tetsuya Mukawa Date: Thu, 2 Jun 2016 12:29:39 +0900 Message-Id: <1464838185-21751-1-git-send-email-mukawa@igel.co.jp> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1457512409-24403-12-git-send-email-mukawa@igel.co.jp> References: <1457512409-24403-12-git-send-email-mukawa@igel.co.jp> Subject: [dpdk-dev] [PATCH v5 0/6] Virtio-net PMD: QEMU QTest extension for container X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jun 2016 03:29:58 -0000 The patches will work on below patch series. - [PATCH v5 0/8] virtio support for container It seems his implementation will be changed a bit. So, this patch series are also going to be changed to follow his implementation. [Changes] v5 changes: - Rebase on latest dpdk-next-virtio. - Follow Jianfeng's implementation to support virtual virtio-net device. - Split the patch series like followings. - This patch series. Only support basic functions. The functions to handle LSC interrupt and '--range-virtaddr' was removed from this patch series. This patch needs EAL memory mapped between (1<<31) to (1<<44). To allocate such a memory, just assume the user will use '--base-virtaddr'. If appropriate memory cannot be allocated, this PMD will exit as error. Then the users can try other values. - Supplement patches to support link status interrupt. - Supplement patches to support '--range-virtaddr'. This EAL option will help to allocate memory mapped between (1<<31) to (1<<44). v4 changes: - Rebase on latest master. - Split patches. - To abstract qtest code more, change interface between current virtio code and qtest code. - Rename qtest.c to qtest_utils.c - Change implementation like below. - Set pci device information out of qtest abstraction, then pass it to qtest to initialize devices. - Remove redundant condition checking from qtest_raw_send/recv(). - Fix return value of qtest_raw_send(). v3 changes: - Rebase on latest master. - remove "-qtest-virtio" option, then add "--range-virtaddr" and "--align-memsize" options. - Fix typos in qtest.c v2 changes: - Rebase on above patch seiries. - Rebase on master - Add "--qtest-virtio" EAL option. - Fixes in qtest.c - Fix error handling for the case qtest connection is closed. - Use eventfd for interrupt messaging. - Use linux header for PCI register definitions. - Fix qtest_raw_send/recv to handle error correctly. - Fix bit mask of PCI_CONFIG_ADDR. - Describe memory and ioport usage of qtest guest in qtest.c - Remove loop that is for finding PCI devices. [Abstraction] Normally, virtio-net PMD only works on VM, because there is no virtio-net device on host. This patches extend virtio-net PMD to be able to work on host as virtual PMD. But we didn't implement virtio-net device as a part of virtio-net PMD. To prepare virtio-net device for the PMD, start QEMU process with special QTest mode, then connect it from virtio-net PMD through unix domain socket. The PMD can connect to anywhere QEMU virtio-net device can. For example, the PMD can connects to vhost-net kernel module and vhost-user backend application. Similar to virtio-net PMD on QEMU, application memory that uses virtio-net PMD will be shared between vhost backend application. But vhost backend application memory will not be shared. Main target of this PMD is container like docker, rkt, lxc and etc. We can isolate related processes(virtio-net PMD process, QEMU and vhost-user backend process) by container. But, to communicate through unix domain socket, shared directory will be needed. [How to use] Please use QEMU-2.5.1, or above. (So far, QEMU-2.5.1 hasn't been released yet, so please checkout master from QEMU repository) - Compile Set "CONFIG_RTE_VIRTIO_VDEV_QTEST=y" in config/common_linux. Then compile it. - Start QEMU like below. $ qemu-system-x86_64 \ -machine pc-i440fx-1.4,accel=qtest \ -display none -qtest-log /dev/null \ -qtest unix:/tmp/socket,server \ -netdev type=tap,script=/etc/qemu-ifup,id=net0,queues=1 \ -device virtio-net-pci,netdev=net0,mq=on,disable-modern=false,addr=3 \ -chardev socket,id=chr1,path=/tmp/ivshmem,server \ -device ivshmem,size=1G,chardev=chr1,vectors=1,addr=4 - Start DPDK application like below $ testpmd -c f -n 1 -m 1024 --no-pci --base-virtaddr=0x400000000 \ --vdev="eth_virtio_qtest0,qtest=/tmp/socket,ivshmem=/tmp/ivshmem"\ -- --disable-hw-vlan --txqflags=0xf00 -i (*1) Please Specify same memory size in QEMU and DPDK command line. (*2) Should use qemu-2.5.1, or above. (*3) QEMU process is needed per port. (*4) virtio-1.0 device are only supported. (*5) The vhost backends like vhost-net and vhost-user can be specified. (*6) In most cases, just using above command is enough, but you can also specify other QEMU virtio-net options. (*7) Only checked "pc-i440fx-1.4" machine, but may work with other machines. It depends on a machine has piix3 south bridge. If the machine doesn't have, virtio-net PMD cannot receive status changed interrupts. (*8) Should not add "--enable-kvm" to QEMU command line. [Detailed Description] - virtio-net device implementation The PMD uses QEMU virtio-net device. To do that, QEMU QTest functionality is used. QTest is a test framework of QEMU devices. It allows us to implement a device driver outside of QEMU. With QTest, we can implement DPDK application and virtio-net PMD as standalone process on host. When QEMU is invoked as QTest mode, any guest code will not run. To know more about QTest, see below. http://wiki.qemu.org/Features/QTest - probing devices QTest provides a unix domain socket. Through this socket, driver process can access to I/O port and memory of QEMU virtual machine. The PMD will send I/O port accesses to probe pci devices. If we can find virtio-net and ivshmem device, initialize the devices. Also, I/O port accesses of virtio-net PMD will be sent through socket, and virtio-net PMD can initialize vitio-net device on QEMU correctly. - ivshmem device to share memory To share memory that virtio-net PMD process uses, ivshmem device will be used. Because ivshmem device can only handle one file descriptor, shared memory should be consist of one file. To allocate such a memory, EAL has new option called "--single-file". Also, the hugepages should be mapped between "1 << 31" to "1 << 44". To map like above, use '--base-virtaddr' option. While initializing ivshmem device, we can set BAR(Base Address Register). It represents which memory QEMU vcpu can access to this shared memory. We will specify host virtual address of shared memory as this address. It is very useful because we don't need to apply patch to QEMU to calculate address offset. (For example, if virtio-net PMD process will allocate memory from shared memory, then specify the virtual address of it to virtio-net register, QEMU virtio-net device can understand it without calculating address offset.) Tetsuya Mukawa (6): virtio, qtest: Add QTest utility basic functions virtio, qtest: Add pci device initialization function to qtest utils virtio, qtest: Add functionality to share memory between QTest guest virtio, qtest: Add misc functions to handle pci information virtio: Add QTest support to vtpci abstraction virtio: Add QTest support for virtio-net PMD config/common_linuxapp | 2 + drivers/net/virtio/Makefile | 6 + drivers/net/virtio/virtio_ethdev.c | 3 +- drivers/net/virtio/virtio_ethdev.h | 1 + drivers/net/virtio/virtio_qtest/qtest.h | 95 ++ drivers/net/virtio/virtio_qtest/qtest_utils.c | 1087 ++++++++++++++++++++ drivers/net/virtio/virtio_qtest/qtest_utils.h | 289 ++++++ drivers/net/virtio/virtio_qtest/virtio_qtest_dev.c | 393 +++++++ drivers/net/virtio/virtio_qtest/virtio_qtest_dev.h | 42 + drivers/net/virtio/virtio_qtest/virtio_qtest_pci.c | 407 ++++++++ drivers/net/virtio/virtio_qtest/virtio_qtest_pci.h | 39 + drivers/net/virtio/virtqueue.h | 6 +- 12 files changed, 2365 insertions(+), 5 deletions(-) create mode 100644 drivers/net/virtio/virtio_qtest/qtest.h create mode 100644 drivers/net/virtio/virtio_qtest/qtest_utils.c create mode 100644 drivers/net/virtio/virtio_qtest/qtest_utils.h create mode 100644 drivers/net/virtio/virtio_qtest/virtio_qtest_dev.c create mode 100644 drivers/net/virtio/virtio_qtest/virtio_qtest_dev.h create mode 100644 drivers/net/virtio/virtio_qtest/virtio_qtest_pci.c create mode 100644 drivers/net/virtio/virtio_qtest/virtio_qtest_pci.h -- 2.7.4