DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Qiu, Michael" <michael.qiu@intel.com>
To: Tetsuya Mukawa <mukawa@igel.co.jp>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "nakajima.yoshihiro@lab.ntt.co.jp"
	<nakajima.yoshihiro@lab.ntt.co.jp>,
	"zhbzg@huawei.com" <zhbzg@huawei.com>,
	"mst@redhat.com" <mst@redhat.com>,
	"gaoxiaoqiu@huawei.com" <gaoxiaoqiu@huawei.com>,
	"oscar.zhangbo@huawei.com" <oscar.zhangbo@huawei.com>,
	"ann.zhuangyanying@huawei.com" <ann.zhuangyanying@huawei.com>,
	"zhoujingbin@huawei.com" <zhoujingbin@huawei.com>,
	"guohongzhen@huawei.com" <guohongzhen@huawei.com>
Subject: Re: [dpdk-dev] [RFC PATCH 0/2] Virtio-net PMD Extension to work on host.
Date: Mon, 28 Dec 2015 05:15:58 +0000	[thread overview]
Message-ID: <533710CFB86FA344BFBF2D6802E6028622F04DCF@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <1447930650-26023-1-git-send-email-mukawa@igel.co.jp>

Hi, Tetsuya

I have a question about your solution, as I know you plan to run qemu
and dpdk both in container right?

If so, I think it's a bit tricky, DPDK is a lib, and qemu is a App,
seems it is not suitable to let a lib depends on Apps.

Also, till now I don't see any usecase to run qemu inside container.

Thanks,
Michael

On 11/19/2015 6:58 PM, Tetsuya Mukawa wrote:
> THIS IS A PoC IMPLEMENATION.
>
> [Abstraction]
>
> Normally, virtio-net PMD only works on VM, because there is no virtio-net device on host.
> This RFC patch extends virtio-net PMD to be able to work on host as virtual PMD.
> But we didn't implement virtio-net device as a part of virtio-net PMD.
> To prepare virtio-net device for the PMD, start QEMU process with special QTest mode, then connect it from virtio-net PMD through unix domain socket.
>
> The PMD can connect to anywhere QEMU virtio-net device can.
> For example, the PMD can connects to vhost-net kernel module and vhost-user backend application.
> Similar to virtio-net PMD on QEMU, application memory that uses virtio-net PMD will be shared between vhost backend application.
> But vhost backend application memory will not be shared.
>
> Main target of this PMD is container like docker, rkt, lxc and etc.
> We can isolate related processes(virtio-net PMD process, QEMU and vhost-user backend process) by container.
> But, to communicate through unix domain socket, shared directory will be needed.
>
>
> [How to use]
>
> So far, we need QEMU patch to connect to vhost-user backend.
> Please check known issue in later section.
> Because of this, I will describe example of using vhost-net kernel module.
>
>  - Compile
>  Set "CONFIG_RTE_VIRTIO_VDEV=y" in config/common_linux.
>  Then compile it.
>
>  - Start QEMU like below.
>  $ sudo qemu-system-x86_64 -qtest unix:/tmp/qtest0,server -machine accel=qtest \
>                            -display none -qtest-log /dev/null \
>                            -netdev type=tap,script=/etc/qemu-ifup,id=net0,vhost=on \
>                            -device virtio-net-pci,netdev=net0 \
>                            -chardev socket,id=chr1,path=/tmp/ivshmem0,server \
>                            -device ivshmem,size=1G,chardev=chr1,vectors=1
>
>  - Start DPDK application like below
>  $ sudo ./testpmd -c f -n 1 -m 1024 --shm \
>                   --vdev="eth_cvio0,qtest=/tmp/qtest0,ivshmem=/tmp/ivshmem0" -- \
>                   --disable-hw-vlan --txqflags=0xf00 -i
>
>  - Check created tap device.
>
> (*1) Please Specify same memory size in QEMU and DPDK command line.
>
>
> [Detailed Description]
>
>  - virtio-net device implementation
> The PMD uses QEMU virtio-net device. To do that, QEMU QTest functionality is used.
> QTest is a test framework of QEMU devices. It allows us to implement a device driver outside of QEMU.
> With QTest, we can implement DPDK application and virtio-net PMD as standalone process on host.
> When QEMU is invoked as QTest mode, any guest code will not run.
> To know more about QTest, see below.
> http://wiki.qemu.org/Features/QTest
>
>  - probing devices
> QTest provides a unix domain socket. Through this socket, driver process can access to I/O port and memory of QEMU virtual machine.
> The PMD will send I/O port accesses to probe pci devices.
> If we can find virtio-net and ivshmem device, initialize the devices.
> Also, I/O port accesses of virtio-net PMD will be sent through socket, and virtio-net PMD can initialize vitio-net device on QEMU correctly.
>
>  - ivshmem device to share memory
> To share memory that virtio-net PMD process uses, ivshmem device will be used.
> Because ivshmem device can only handle one file descriptor, shared memory should be consist of one file.
> To allocate such a memory, EAL has new option called "--shm".
> If the option is specified, EAL will open a file and allocate memory from hugepages.
> While initializing ivshmem device, we can set BAR(Base Address Register).
> It represents which memory QEMU vcpu can access to this shared memory.
> We will specify host physical address of shared memory as this address.
> It is very useful because we don't need to apply patch to QEMU to calculate address offset.
> (For example, if virtio-net PMD process will allocate memory from shared memory, then specify the physical address of it to virtio-net register, QEMU virtio-net device can understand it without calculating address offset.)
>
>  - Known limitation
> So far, the PMD doesn't handle interrupts from QEMU devices.
> Because of this, VIRTIO_NET_F_STATUS functionality is dropped.
> But without it, we can use all virtio-net functions.
>
>  - Known issues
> So far, to use vhost-user, we need to apply vhost-user patch to QEMU and DPDK vhost library.
> This is because, QEMU will not send memory information and file descriptor of ivshmem device to vhost-user backend.
> (Anyway, vhost-net kernel module can receive the information. So vhost-user behavior will not be correct. I will submit the patch to QEMU soon)
> Also, we may have an issue in DPDK vhost library to handle kickfd and callfd. The patch for it is needed.
> (Let me check it more)
> If someone wants to check vhost-user behavior, I will describe it more in later email.
>
>
> [Addition]
>
> We can apply same manner to handle any kind of QEMU devices from DPDK application.
> So far, I don't have any ideas except for virtio-net device. But someone would have.
>
>
> Tetsuya Mukawa (2):
>   EAL: Add new EAL "--shm" option.
>   virtio: Extend virtio-net PMD to support container environment
>
>  config/common_linuxapp                     |   5 +
>  drivers/net/virtio/Makefile                |   4 +
>  drivers/net/virtio/qtest.c                 | 590 +++++++++++++++++++++++++++++
>  drivers/net/virtio/virtio_ethdev.c         | 214 ++++++++++-
>  drivers/net/virtio/virtio_ethdev.h         |  16 +
>  drivers/net/virtio/virtio_pci.h            |  25 ++
>  lib/librte_eal/common/eal_common_options.c |   5 +
>  lib/librte_eal/common/eal_internal_cfg.h   |   1 +
>  lib/librte_eal/common/eal_options.h        |   2 +
>  lib/librte_eal/common/include/rte_memory.h |   5 +
>  lib/librte_eal/linuxapp/eal/eal_memory.c   |  71 ++++
>  11 files changed, 917 insertions(+), 21 deletions(-)
>  create mode 100644 drivers/net/virtio/qtest.c
>


  parent reply	other threads:[~2015-12-28  5:16 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-19 10:57 Tetsuya Mukawa
2015-11-19 10:57 ` [dpdk-dev] [RFC PATCH 1/2] EAL: Add new EAL "--shm" option Tetsuya Mukawa
2015-12-16  8:37   ` [dpdk-dev] [PATCH v1 0/2] Virtio-net PMD Extension to work on host Tetsuya Mukawa
2015-12-16  8:37     ` [dpdk-dev] [PATCH v1 1/2] EAL: Add new EAL "--contig-mem" option Tetsuya Mukawa
2015-12-16  8:37     ` [dpdk-dev] [PATCH v1 2/2] virtio: Extend virtio-net PMD to support container environment Tetsuya Mukawa
2015-12-28 11:57       ` Pavel Fedin
2016-01-06  3:57         ` Tetsuya Mukawa
2016-01-06  5:56           ` Tan, Jianfeng
2016-01-06  7:27             ` Tetsuya Mukawa
2015-12-24 14:05     ` [dpdk-dev] [PATCH v1 0/2] Virtio-net PMD Extension to work on host Tan, Jianfeng
2015-12-28 11:06       ` Tetsuya Mukawa
2016-01-06  3:57         ` Tetsuya Mukawa
2016-01-06  5:42           ` Tan, Jianfeng
2016-01-06  7:35             ` Tetsuya Mukawa
2016-01-11  5:31               ` Tan, Jianfeng
2015-11-19 10:57 ` [dpdk-dev] [RFC PATCH 2/2] virtio: Extend virtio-net PMD to support container environment Tetsuya Mukawa
2015-11-19 18:16 ` [dpdk-dev] [RFC PATCH 0/2] Virtio-net PMD Extension to work on host Rich Lane
2015-11-20  2:00   ` Xie, Huawei
2015-11-20  2:35     ` Tetsuya Mukawa
2015-11-20  2:53       ` Tetsuya Mukawa
2015-12-28  5:15 ` Qiu, Michael [this message]
2015-12-28 11:06   ` Tetsuya Mukawa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=533710CFB86FA344BFBF2D6802E6028622F04DCF@SHSMSX101.ccr.corp.intel.com \
    --to=michael.qiu@intel.com \
    --cc=ann.zhuangyanying@huawei.com \
    --cc=dev@dpdk.org \
    --cc=gaoxiaoqiu@huawei.com \
    --cc=guohongzhen@huawei.com \
    --cc=mst@redhat.com \
    --cc=mukawa@igel.co.jp \
    --cc=nakajima.yoshihiro@lab.ntt.co.jp \
    --cc=oscar.zhangbo@huawei.com \
    --cc=zhbzg@huawei.com \
    --cc=zhoujingbin@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).