From: Tetsuya Mukawa <mukawa@igel.co.jp>
To: "Xie, Huawei" <huawei.xie@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"yuanhan.liu@linux.intel.com" <yuanhan.liu@linux.intel.com>,
"Tan, Jianfeng" <jianfeng.tan@intel.com>
Subject: Re: [dpdk-dev] [RFC PATCH 5/5] virtio: Extend virtio-net PMD to support container environment
Date: Tue, 26 Jan 2016 11:58:32 +0900 [thread overview]
Message-ID: <56A6E0D8.9010005@igel.co.jp> (raw)
In-Reply-To: <C37D651A908B024F974696C65296B57B4C5ACAC5@SHSMSX101.ccr.corp.intel.com>
On 2016/01/25 19:15, Xie, Huawei wrote:
> On 1/22/2016 6:38 PM, Tetsuya Mukawa wrote:
>> On 2016/01/22 17:14, Xie, Huawei wrote:
>>> On 1/21/2016 7:09 PM, Tetsuya Mukawa wrote:
>>>> virtio: Extend virtio-net PMD to support container environment
>>>>
>>>> The patch adds a new virtio-net PMD configuration that allows the PMD to
>>>> work on host as if the PMD is in VM.
>>>> Here is new configuration for virtio-net PMD.
>>>> - CONFIG_RTE_LIBRTE_VIRTIO_HOST_MODE
>>>> To use this mode, EAL needs physically contiguous memory. To allocate
>>>> such memory, add "--shm" option to application command line.
>>>>
>>>> To prepare virtio-net device on host, the users need to invoke QEMU
>>>> process in special qtest mode. This mode is mainly used for testing QEMU
>>>> devices from outer process. In this mode, no guest runs.
>>>> Here is QEMU command line.
>>>>
>>>> $ qemu-system-x86_64 \
>>>> -machine pc-i440fx-1.4,accel=qtest \
>>>> -display none -qtest-log /dev/null \
>>>> -qtest unix:/tmp/socket,server \
>>>> -netdev type=tap,script=/etc/qemu-ifup,id=net0,queues=1\
>>>> -device virtio-net-pci,netdev=net0,mq=on \
>>>> -chardev socket,id=chr1,path=/tmp/ivshmem,server \
>>>> -device ivshmem,size=1G,chardev=chr1,vectors=1
>>>>
>>>> * QEMU process is needed per port.
>>> Does qtest supports hot plug virtio-net pci device, so that we could run
>>> one QEMU process in host, which provisions the virtio-net virtual
>>> devices for the container?
>> Theoretically, we can use hot plug in some cases.
>> But I guess we have 3 concerns here.
>>
>> 1. Security.
>> If we share QEMU process between multiple DPDK applications, this QEMU
>> process will have all fds of the applications on different containers.
>> In some cases, it will be security concern.
>> So, I guess we need to support current 1:1 configuration at least.
>>
>> 2. shared memory.
>> Currently, QEMU and DPDK application will map shared memory using same
>> virtual address.
>> So if multiple DPDK application connects to one QEMU process, each DPDK
>> application should have different address for shared memory. I guess
>> this will be a big limitation.
>>
>> 3. PCI bridge.
>> So far, QEMU has one PCI bridge, so we can connect almost 10 PCI devices
>> to QEMU.
>> (I forget correct number, but it's almost 10, because some slots are
>> reserved by QEMU)
>> A DPDK application needs both virtio-net and ivshmem device, so I guess
>> almost 5 DPDK applications can connect to one QEMU process, so far.
>> To add more PCI bridges solves this.
>> But we need to add a lot of implementation to support cascaded PCI
>> bridges and PCI devices.
>> (Also we need to solve above "2nd" concern.)
>>
>> Anyway, if we use virtio-net PMD and vhost-user PMD, QEMU process will
>> not do anything after initialization.
>> (QEMU will try to read a qtest socket, then be stopped because there is
>> no message after initialization)
>> So I guess we can ignore overhead of these QEMU processes.
>> If someone cannot ignore it, I guess this is the one of cases that it's
>> nice to use your light weight container implementation.
> Thanks for the explanation, and also in your opinion where is the best
> place to run the QEMU instance? If we run QEMU instances in host, for
> vhost-kernel support, we could get rid of the root privilege issue.
Do you mean below?
If we deploy QEMU instance on host, we can start a container without the
root privilege.
(But on host, still QEMU instance needs the privilege to access to
vhost-kernel)
If so, I agree to deploy QEMU instance on host or other privileged
container will be nice.
In the case of vhost-user, to deploy on host or non-privileged container
will be good.
>
> Another issue is do you plan to support multiple virtio devices in
> container? Currently i find the code assuming only one virtio-net device
> in QEMU, right?
Yes, so far, 1 port needs 1 QEMU instance.
So if you need multiple virtio devices, you need to invoke multiple QEMU
instances.
Do you want to deploy 1 QEMU instance for each DPDK application, even if
the application has multiple virtio-net ports?
So far, I am not sure whether we need it, because this type of DPDK
application will need only one port in most cases.
But if you need this, yes, I can implement using QEMU PCI hotplug feature.
(But probably we can only attach almost 10 ports. This will be limitation.)
>
> Btw, i have read most of your qtest code. No obvious issues found so far
> but quite a couple of nits. You must have spent a lot of time on this.
> It is great work!
I appreciate your reviewing!
BTW, my container implementation needed a QEMU patch in the case of
vhost-user.
But the patch has been merged in upstream QEMU, so we don't have this
limitation any more.
Thanks,
Tetsuya
next prev parent reply other threads:[~2016-01-26 2:58 UTC|newest]
Thread overview: 120+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-18 9:13 [dpdk-dev] [PATCH 0/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-18 9:13 ` [dpdk-dev] [PATCH 1/3] virtio: Change the parameter order of io_write8/16/32() Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 0/5] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-21 11:10 ` Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 1/5] virtio: Change the parameter order of io_write8/16/32() Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 2/5] virtio: move rte_eal_pci_unmap_device() to virtio_pci.c Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 3/5] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-22 7:26 ` Xie, Huawei
2016-01-22 7:35 ` Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 4/5] EAL: Add new EAL "--shm" option Tetsuya Mukawa
2016-01-22 1:43 ` Tan, Jianfeng
2016-01-22 2:07 ` Tan, Jianfeng
2016-01-22 3:23 ` Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 5/5] virtio: Extend virtio-net PMD to support container environment Tetsuya Mukawa
2016-01-22 8:14 ` Xie, Huawei
2016-01-22 10:37 ` Tetsuya Mukawa
2016-01-25 10:15 ` Xie, Huawei
2016-01-26 2:58 ` Tetsuya Mukawa [this message]
2016-01-27 9:39 ` Xie, Huawei
2016-01-28 2:33 ` Tetsuya Mukawa
2016-01-25 10:17 ` Xie, Huawei
2016-01-26 2:58 ` Tetsuya Mukawa
2016-01-25 10:29 ` Xie, Huawei
2016-01-26 2:58 ` Tetsuya Mukawa
2016-01-27 10:03 ` Xie, Huawei
2016-01-28 2:44 ` Tetsuya Mukawa
2016-01-29 8:56 ` Xie, Huawei
2016-01-27 15:58 ` Xie, Huawei
2016-01-28 2:47 ` Tetsuya Mukawa
2016-01-28 9:48 ` Xie, Huawei
2016-01-28 9:53 ` Tetsuya Mukawa
2016-01-27 16:45 ` Xie, Huawei
2016-01-28 2:47 ` Tetsuya Mukawa
2016-01-28 6:15 ` Xie, Huawei
2016-01-28 6:29 ` Tetsuya Mukawa
2016-01-29 8:57 ` Yuanhan Liu
2016-01-29 9:13 ` Yuanhan Liu
2016-02-01 1:49 ` Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 0/5] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 1/5] virtio: Retrieve driver name from eth_dev Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 2/5] EAL: Add new EAL "--qtest-virtio" option Tetsuya Mukawa
2016-02-15 7:52 ` Tan, Jianfeng
2016-02-16 1:32 ` Tetsuya Mukawa
2016-02-16 5:53 ` David Marchand
2016-02-16 11:36 ` Tan, Jianfeng
2016-02-17 3:36 ` Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 0/6] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 1/6] virtio: Retrieve driver name from eth_dev Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 2/6] vhost: Add a function to check virtio device type Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 3/6] EAL: Add new EAL "--range-virtaddr" option Tetsuya Mukawa
2016-03-04 2:20 ` Tan, Jianfeng
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 00/12] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 01/12] virtio: Retrieve driver name from eth_dev Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 02/12] vhost: Add a function to check virtio device type Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 03/12] EAL: Add a new "--range-virtaddr" option Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 04/12] EAL: Add a new "--align-memsize" option Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 05/12] virtio, qtest: Add QTest utility basic functions Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 06/12] virtio, qtest: Add pci device initialization function to qtest utils Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 07/12] virtio, qtest: Add functionality to share memory between QTest guest Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 08/12] virtio, qtest: Add functionality to handle interrupt Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 09/12] virtio, qtest: Add misc functions to handle pci information Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 10/12] virtio: Add QTest support to vtpci abstraction Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 11/12] virtio: Add QTest support for virtio-net PMD Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 0/6] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-06-02 7:31 ` Yuanhan Liu
2016-06-02 9:30 ` Tetsuya Mukawa
2016-06-03 4:17 ` Yuanhan Liu
2016-06-03 13:51 ` Thomas Monjalon
2016-06-06 5:10 ` Tetsuya Mukawa
2016-06-06 7:21 ` Yuanhan Liu
2016-06-06 8:33 ` Tetsuya Mukawa
2016-06-06 8:49 ` Yuanhan Liu
2016-06-06 9:30 ` Tetsuya Mukawa
2016-06-06 9:58 ` Yuanhan Liu
2016-06-06 10:50 ` Tan, Jianfeng
2016-06-07 7:12 ` Tetsuya Mukawa
2016-06-07 7:33 ` Yuanhan Liu
2016-06-06 8:03 ` Tan, Jianfeng
2016-06-06 9:28 ` Tetsuya Mukawa
2016-06-06 10:35 ` Tan, Jianfeng
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 1/6] virtio, qtest: Add QTest utility basic functions Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 2/6] virtio, qtest: Add pci device initialization function to qtest utils Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 3/6] virtio, qtest: Add functionality to share memory between QTest guest Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 4/6] virtio, qtest: Add misc functions to handle pci information Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 5/6] virtio: Add QTest support to vtpci abstraction Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 6/6] virtio: Add QTest support for virtio-net PMD Tetsuya Mukawa
2016-06-02 3:30 ` [dpdk-dev] [PATCH v1 0/2] Supplement patches for virtio-qtest to support LSC interrupt Tetsuya Mukawa
2016-06-02 3:30 ` [dpdk-dev] [PATCH v1 1/2] virtio: Handle interrupt things under vtpci abstraction Tetsuya Mukawa
2016-06-02 3:30 ` [dpdk-dev] [PATCH v1 2/2] virtio, qtest: Add functionality to handle interrupt Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 12/12] docs: add release note for qtest virtio container support Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 4/6] EAL: Add a new "--align-memsize" option Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 5/6] virtio: Add support for qtest virtio-net PMD Tetsuya Mukawa
2016-03-04 2:18 ` Tan, Jianfeng
2016-03-04 5:05 ` Tetsuya Mukawa
2016-03-04 6:10 ` Tan, Jianfeng
2016-03-04 9:53 ` Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 6/6] docs: add release note for qtest virtio container support Tetsuya Mukawa
2016-02-22 15:40 ` Mcnamara, John
2016-02-23 10:28 ` Mcnamara, John
2016-02-24 1:20 ` Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 3/5] vhost: Add a function to check virtio device type Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 4/5] virtio: Add support for qtest virtio-net PMD Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 5/5] docs: add release note for qtest virtio container support Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 0/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 1/3] virtio: Change the parameter order of io_write8/16/32() Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 2/3] virtio: move rte_eal_pci_unmap_device() to virtio_pci.c Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 3/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-29 9:17 ` Yuanhan Liu
2016-02-01 1:50 ` Tetsuya Mukawa
2016-02-01 13:15 ` Yuanhan Liu
2016-02-02 2:19 ` Tetsuya Mukawa
2016-02-02 2:45 ` Yuanhan Liu
2016-02-02 3:55 ` Tetsuya Mukawa
2016-01-18 9:13 ` [dpdk-dev] [PATCH 2/3] virtio: move rte_eal_pci_unmap_device() to virtio_pci.c Tetsuya Mukawa
2016-01-18 9:13 ` [dpdk-dev] [PATCH 3/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-18 13:46 ` Yuanhan Liu
2016-01-19 1:22 ` Tetsuya Mukawa
2016-01-19 2:41 ` Xie, Huawei
2016-01-18 13:13 ` [dpdk-dev] [PATCH 0/3] " Tan, Jianfeng
2016-01-19 1:22 ` Tetsuya Mukawa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56A6E0D8.9010005@igel.co.jp \
--to=mukawa@igel.co.jp \
--cc=dev@dpdk.org \
--cc=huawei.xie@intel.com \
--cc=jianfeng.tan@intel.com \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).