From: "Qiu, Michael" <michael.qiu@intel.com>
To: Tetsuya Mukawa <mukawa@igel.co.jp>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "nakajima.yoshihiro@lab.ntt.co.jp"
<nakajima.yoshihiro@lab.ntt.co.jp>,
"zhbzg@huawei.com" <zhbzg@huawei.com>,
"mst@redhat.com" <mst@redhat.com>,
"gaoxiaoqiu@huawei.com" <gaoxiaoqiu@huawei.com>,
"oscar.zhangbo@huawei.com" <oscar.zhangbo@huawei.com>,
"ann.zhuangyanying@huawei.com" <ann.zhuangyanying@huawei.com>,
"zhoujingbin@huawei.com" <zhoujingbin@huawei.com>,
"guohongzhen@huawei.com" <guohongzhen@huawei.com>
Subject: Re: [dpdk-dev] [RFC PATCH 0/2] Virtio-net PMD Extension to work on host.
Date: Mon, 28 Dec 2015 05:15:58 +0000 [thread overview]
Message-ID: <533710CFB86FA344BFBF2D6802E6028622F04DCF@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <1447930650-26023-1-git-send-email-mukawa@igel.co.jp>
Hi, Tetsuya
I have a question about your solution, as I know you plan to run qemu
and dpdk both in container right?
If so, I think it's a bit tricky, DPDK is a lib, and qemu is a App,
seems it is not suitable to let a lib depends on Apps.
Also, till now I don't see any usecase to run qemu inside container.
Thanks,
Michael
On 11/19/2015 6:58 PM, Tetsuya Mukawa wrote:
> THIS IS A PoC IMPLEMENATION.
>
> [Abstraction]
>
> Normally, virtio-net PMD only works on VM, because there is no virtio-net device on host.
> This RFC patch extends virtio-net PMD to be able to work on host as virtual PMD.
> But we didn't implement virtio-net device as a part of virtio-net PMD.
> To prepare virtio-net device for the PMD, start QEMU process with special QTest mode, then connect it from virtio-net PMD through unix domain socket.
>
> The PMD can connect to anywhere QEMU virtio-net device can.
> For example, the PMD can connects to vhost-net kernel module and vhost-user backend application.
> Similar to virtio-net PMD on QEMU, application memory that uses virtio-net PMD will be shared between vhost backend application.
> But vhost backend application memory will not be shared.
>
> Main target of this PMD is container like docker, rkt, lxc and etc.
> We can isolate related processes(virtio-net PMD process, QEMU and vhost-user backend process) by container.
> But, to communicate through unix domain socket, shared directory will be needed.
>
>
> [How to use]
>
> So far, we need QEMU patch to connect to vhost-user backend.
> Please check known issue in later section.
> Because of this, I will describe example of using vhost-net kernel module.
>
> - Compile
> Set "CONFIG_RTE_VIRTIO_VDEV=y" in config/common_linux.
> Then compile it.
>
> - Start QEMU like below.
> $ sudo qemu-system-x86_64 -qtest unix:/tmp/qtest0,server -machine accel=qtest \
> -display none -qtest-log /dev/null \
> -netdev type=tap,script=/etc/qemu-ifup,id=net0,vhost=on \
> -device virtio-net-pci,netdev=net0 \
> -chardev socket,id=chr1,path=/tmp/ivshmem0,server \
> -device ivshmem,size=1G,chardev=chr1,vectors=1
>
> - Start DPDK application like below
> $ sudo ./testpmd -c f -n 1 -m 1024 --shm \
> --vdev="eth_cvio0,qtest=/tmp/qtest0,ivshmem=/tmp/ivshmem0" -- \
> --disable-hw-vlan --txqflags=0xf00 -i
>
> - Check created tap device.
>
> (*1) Please Specify same memory size in QEMU and DPDK command line.
>
>
> [Detailed Description]
>
> - virtio-net device implementation
> The PMD uses QEMU virtio-net device. To do that, QEMU QTest functionality is used.
> QTest is a test framework of QEMU devices. It allows us to implement a device driver outside of QEMU.
> With QTest, we can implement DPDK application and virtio-net PMD as standalone process on host.
> When QEMU is invoked as QTest mode, any guest code will not run.
> To know more about QTest, see below.
> http://wiki.qemu.org/Features/QTest
>
> - probing devices
> QTest provides a unix domain socket. Through this socket, driver process can access to I/O port and memory of QEMU virtual machine.
> The PMD will send I/O port accesses to probe pci devices.
> If we can find virtio-net and ivshmem device, initialize the devices.
> Also, I/O port accesses of virtio-net PMD will be sent through socket, and virtio-net PMD can initialize vitio-net device on QEMU correctly.
>
> - ivshmem device to share memory
> To share memory that virtio-net PMD process uses, ivshmem device will be used.
> Because ivshmem device can only handle one file descriptor, shared memory should be consist of one file.
> To allocate such a memory, EAL has new option called "--shm".
> If the option is specified, EAL will open a file and allocate memory from hugepages.
> While initializing ivshmem device, we can set BAR(Base Address Register).
> It represents which memory QEMU vcpu can access to this shared memory.
> We will specify host physical address of shared memory as this address.
> It is very useful because we don't need to apply patch to QEMU to calculate address offset.
> (For example, if virtio-net PMD process will allocate memory from shared memory, then specify the physical address of it to virtio-net register, QEMU virtio-net device can understand it without calculating address offset.)
>
> - Known limitation
> So far, the PMD doesn't handle interrupts from QEMU devices.
> Because of this, VIRTIO_NET_F_STATUS functionality is dropped.
> But without it, we can use all virtio-net functions.
>
> - Known issues
> So far, to use vhost-user, we need to apply vhost-user patch to QEMU and DPDK vhost library.
> This is because, QEMU will not send memory information and file descriptor of ivshmem device to vhost-user backend.
> (Anyway, vhost-net kernel module can receive the information. So vhost-user behavior will not be correct. I will submit the patch to QEMU soon)
> Also, we may have an issue in DPDK vhost library to handle kickfd and callfd. The patch for it is needed.
> (Let me check it more)
> If someone wants to check vhost-user behavior, I will describe it more in later email.
>
>
> [Addition]
>
> We can apply same manner to handle any kind of QEMU devices from DPDK application.
> So far, I don't have any ideas except for virtio-net device. But someone would have.
>
>
> Tetsuya Mukawa (2):
> EAL: Add new EAL "--shm" option.
> virtio: Extend virtio-net PMD to support container environment
>
> config/common_linuxapp | 5 +
> drivers/net/virtio/Makefile | 4 +
> drivers/net/virtio/qtest.c | 590 +++++++++++++++++++++++++++++
> drivers/net/virtio/virtio_ethdev.c | 214 ++++++++++-
> drivers/net/virtio/virtio_ethdev.h | 16 +
> drivers/net/virtio/virtio_pci.h | 25 ++
> lib/librte_eal/common/eal_common_options.c | 5 +
> lib/librte_eal/common/eal_internal_cfg.h | 1 +
> lib/librte_eal/common/eal_options.h | 2 +
> lib/librte_eal/common/include/rte_memory.h | 5 +
> lib/librte_eal/linuxapp/eal/eal_memory.c | 71 ++++
> 11 files changed, 917 insertions(+), 21 deletions(-)
> create mode 100644 drivers/net/virtio/qtest.c
>
next prev parent reply other threads:[~2015-12-28 5:16 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-19 10:57 Tetsuya Mukawa
2015-11-19 10:57 ` [dpdk-dev] [RFC PATCH 1/2] EAL: Add new EAL "--shm" option Tetsuya Mukawa
2015-12-16 8:37 ` [dpdk-dev] [PATCH v1 0/2] Virtio-net PMD Extension to work on host Tetsuya Mukawa
2015-12-16 8:37 ` [dpdk-dev] [PATCH v1 1/2] EAL: Add new EAL "--contig-mem" option Tetsuya Mukawa
2015-12-16 8:37 ` [dpdk-dev] [PATCH v1 2/2] virtio: Extend virtio-net PMD to support container environment Tetsuya Mukawa
2015-12-28 11:57 ` Pavel Fedin
2016-01-06 3:57 ` Tetsuya Mukawa
2016-01-06 5:56 ` Tan, Jianfeng
2016-01-06 7:27 ` Tetsuya Mukawa
2015-12-24 14:05 ` [dpdk-dev] [PATCH v1 0/2] Virtio-net PMD Extension to work on host Tan, Jianfeng
2015-12-28 11:06 ` Tetsuya Mukawa
2016-01-06 3:57 ` Tetsuya Mukawa
2016-01-06 5:42 ` Tan, Jianfeng
2016-01-06 7:35 ` Tetsuya Mukawa
2016-01-11 5:31 ` Tan, Jianfeng
2015-11-19 10:57 ` [dpdk-dev] [RFC PATCH 2/2] virtio: Extend virtio-net PMD to support container environment Tetsuya Mukawa
2015-11-19 18:16 ` [dpdk-dev] [RFC PATCH 0/2] Virtio-net PMD Extension to work on host Rich Lane
2015-11-20 2:00 ` Xie, Huawei
2015-11-20 2:35 ` Tetsuya Mukawa
2015-11-20 2:53 ` Tetsuya Mukawa
2015-12-28 5:15 ` Qiu, Michael [this message]
2015-12-28 11:06 ` Tetsuya Mukawa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=533710CFB86FA344BFBF2D6802E6028622F04DCF@SHSMSX101.ccr.corp.intel.com \
--to=michael.qiu@intel.com \
--cc=ann.zhuangyanying@huawei.com \
--cc=dev@dpdk.org \
--cc=gaoxiaoqiu@huawei.com \
--cc=guohongzhen@huawei.com \
--cc=mst@redhat.com \
--cc=mukawa@igel.co.jp \
--cc=nakajima.yoshihiro@lab.ntt.co.jp \
--cc=oscar.zhangbo@huawei.com \
--cc=zhbzg@huawei.com \
--cc=zhoujingbin@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).