From: "Xie, Huawei" <huawei.xie@intel.com>
To: Tetsuya Mukawa <mukawa@igel.co.jp>, "dev@dpdk.org" <dev@dpdk.org>,
"yuanhan.liu@linux.intel.com" <yuanhan.liu@linux.intel.com>,
"Tan, Jianfeng" <jianfeng.tan@intel.com>
Subject: Re: [dpdk-dev] [RFC PATCH 5/5] virtio: Extend virtio-net PMD to support container environment
Date: Wed, 27 Jan 2016 09:39:04 +0000 [thread overview]
Message-ID: <C37D651A908B024F974696C65296B57B4C5B144A@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <56A6E0D8.9010005@igel.co.jp>
On 1/26/2016 10:58 AM, Tetsuya Mukawa wrote:
> On 2016/01/25 19:15, Xie, Huawei wrote:
>> On 1/22/2016 6:38 PM, Tetsuya Mukawa wrote:
>>> On 2016/01/22 17:14, Xie, Huawei wrote:
>>>> On 1/21/2016 7:09 PM, Tetsuya Mukawa wrote:
>>>>> virtio: Extend virtio-net PMD to support container environment
>>>>>
>>>>> The patch adds a new virtio-net PMD configuration that allows the PMD to
>>>>> work on host as if the PMD is in VM.
>>>>> Here is new configuration for virtio-net PMD.
>>>>> - CONFIG_RTE_LIBRTE_VIRTIO_HOST_MODE
>>>>> To use this mode, EAL needs physically contiguous memory. To allocate
>>>>> such memory, add "--shm" option to application command line.
>>>>>
>>>>> To prepare virtio-net device on host, the users need to invoke QEMU
>>>>> process in special qtest mode. This mode is mainly used for testing QEMU
>>>>> devices from outer process. In this mode, no guest runs.
>>>>> Here is QEMU command line.
>>>>>
>>>>> $ qemu-system-x86_64 \
>>>>> -machine pc-i440fx-1.4,accel=qtest \
>>>>> -display none -qtest-log /dev/null \
>>>>> -qtest unix:/tmp/socket,server \
>>>>> -netdev type=tap,script=/etc/qemu-ifup,id=net0,queues=1\
>>>>> -device virtio-net-pci,netdev=net0,mq=on \
>>>>> -chardev socket,id=chr1,path=/tmp/ivshmem,server \
>>>>> -device ivshmem,size=1G,chardev=chr1,vectors=1
>>>>>
>>>>> * QEMU process is needed per port.
>>>> Does qtest supports hot plug virtio-net pci device, so that we could run
>>>> one QEMU process in host, which provisions the virtio-net virtual
>>>> devices for the container?
>>> Theoretically, we can use hot plug in some cases.
>>> But I guess we have 3 concerns here.
>>>
>>> 1. Security.
>>> If we share QEMU process between multiple DPDK applications, this QEMU
>>> process will have all fds of the applications on different containers.
>>> In some cases, it will be security concern.
>>> So, I guess we need to support current 1:1 configuration at least.
>>>
>>> 2. shared memory.
>>> Currently, QEMU and DPDK application will map shared memory using same
>>> virtual address.
>>> So if multiple DPDK application connects to one QEMU process, each DPDK
>>> application should have different address for shared memory. I guess
>>> this will be a big limitation.
>>>
>>> 3. PCI bridge.
>>> So far, QEMU has one PCI bridge, so we can connect almost 10 PCI devices
>>> to QEMU.
>>> (I forget correct number, but it's almost 10, because some slots are
>>> reserved by QEMU)
>>> A DPDK application needs both virtio-net and ivshmem device, so I guess
>>> almost 5 DPDK applications can connect to one QEMU process, so far.
>>> To add more PCI bridges solves this.
>>> But we need to add a lot of implementation to support cascaded PCI
>>> bridges and PCI devices.
>>> (Also we need to solve above "2nd" concern.)
>>>
>>> Anyway, if we use virtio-net PMD and vhost-user PMD, QEMU process will
>>> not do anything after initialization.
>>> (QEMU will try to read a qtest socket, then be stopped because there is
>>> no message after initialization)
>>> So I guess we can ignore overhead of these QEMU processes.
>>> If someone cannot ignore it, I guess this is the one of cases that it's
>>> nice to use your light weight container implementation.
>> Thanks for the explanation, and also in your opinion where is the best
>> place to run the QEMU instance? If we run QEMU instances in host, for
>> vhost-kernel support, we could get rid of the root privilege issue.
> Do you mean below?
> If we deploy QEMU instance on host, we can start a container without the
> root privilege.
> (But on host, still QEMU instance needs the privilege to access to
> vhost-kernel)
There is no issue running QEMU instance with root privilege on host, but
i think it is not acceptable granting the container root privilege.
>
> If so, I agree to deploy QEMU instance on host or other privileged
> container will be nice.
> In the case of vhost-user, to deploy on host or non-privileged container
> will be good.
>
>> Another issue is do you plan to support multiple virtio devices in
>> container? Currently i find the code assuming only one virtio-net device
>> in QEMU, right?
> Yes, so far, 1 port needs 1 QEMU instance.
> So if you need multiple virtio devices, you need to invoke multiple QEMU
> instances.
>
> Do you want to deploy 1 QEMU instance for each DPDK application, even if
> the application has multiple virtio-net ports?
>
> So far, I am not sure whether we need it, because this type of DPDK
> application will need only one port in most cases.
> But if you need this, yes, I can implement using QEMU PCI hotplug feature.
> (But probably we can only attach almost 10 ports. This will be limitation.)
I am OK with supporting one virtio device for the first version.
>
>> Btw, i have read most of your qtest code. No obvious issues found so far
>> but quite a couple of nits. You must have spent a lot of time on this.
>> It is great work!
> I appreciate your reviewing!
>
> BTW, my container implementation needed a QEMU patch in the case of
> vhost-user.
> But the patch has been merged in upstream QEMU, so we don't have this
> limitation any more.
Great, better put the QEMU dependency information in the commit message
>
> Thanks,
> Tetsuya
>
next prev parent reply other threads:[~2016-01-27 9:39 UTC|newest]
Thread overview: 120+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-18 9:13 [dpdk-dev] [PATCH 0/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-18 9:13 ` [dpdk-dev] [PATCH 1/3] virtio: Change the parameter order of io_write8/16/32() Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 0/5] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-21 11:10 ` Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 1/5] virtio: Change the parameter order of io_write8/16/32() Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 2/5] virtio: move rte_eal_pci_unmap_device() to virtio_pci.c Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 3/5] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-22 7:26 ` Xie, Huawei
2016-01-22 7:35 ` Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 4/5] EAL: Add new EAL "--shm" option Tetsuya Mukawa
2016-01-22 1:43 ` Tan, Jianfeng
2016-01-22 2:07 ` Tan, Jianfeng
2016-01-22 3:23 ` Tetsuya Mukawa
2016-01-21 11:07 ` [dpdk-dev] [RFC PATCH 5/5] virtio: Extend virtio-net PMD to support container environment Tetsuya Mukawa
2016-01-22 8:14 ` Xie, Huawei
2016-01-22 10:37 ` Tetsuya Mukawa
2016-01-25 10:15 ` Xie, Huawei
2016-01-26 2:58 ` Tetsuya Mukawa
2016-01-27 9:39 ` Xie, Huawei [this message]
2016-01-28 2:33 ` Tetsuya Mukawa
2016-01-25 10:17 ` Xie, Huawei
2016-01-26 2:58 ` Tetsuya Mukawa
2016-01-25 10:29 ` Xie, Huawei
2016-01-26 2:58 ` Tetsuya Mukawa
2016-01-27 10:03 ` Xie, Huawei
2016-01-28 2:44 ` Tetsuya Mukawa
2016-01-29 8:56 ` Xie, Huawei
2016-01-27 15:58 ` Xie, Huawei
2016-01-28 2:47 ` Tetsuya Mukawa
2016-01-28 9:48 ` Xie, Huawei
2016-01-28 9:53 ` Tetsuya Mukawa
2016-01-27 16:45 ` Xie, Huawei
2016-01-28 2:47 ` Tetsuya Mukawa
2016-01-28 6:15 ` Xie, Huawei
2016-01-28 6:29 ` Tetsuya Mukawa
2016-01-29 8:57 ` Yuanhan Liu
2016-01-29 9:13 ` Yuanhan Liu
2016-02-01 1:49 ` Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 0/5] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 1/5] virtio: Retrieve driver name from eth_dev Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 2/5] EAL: Add new EAL "--qtest-virtio" option Tetsuya Mukawa
2016-02-15 7:52 ` Tan, Jianfeng
2016-02-16 1:32 ` Tetsuya Mukawa
2016-02-16 5:53 ` David Marchand
2016-02-16 11:36 ` Tan, Jianfeng
2016-02-17 3:36 ` Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 0/6] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 1/6] virtio: Retrieve driver name from eth_dev Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 2/6] vhost: Add a function to check virtio device type Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 3/6] EAL: Add new EAL "--range-virtaddr" option Tetsuya Mukawa
2016-03-04 2:20 ` Tan, Jianfeng
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 00/12] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 01/12] virtio: Retrieve driver name from eth_dev Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 02/12] vhost: Add a function to check virtio device type Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 03/12] EAL: Add a new "--range-virtaddr" option Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 04/12] EAL: Add a new "--align-memsize" option Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 05/12] virtio, qtest: Add QTest utility basic functions Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 06/12] virtio, qtest: Add pci device initialization function to qtest utils Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 07/12] virtio, qtest: Add functionality to share memory between QTest guest Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 08/12] virtio, qtest: Add functionality to handle interrupt Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 09/12] virtio, qtest: Add misc functions to handle pci information Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 10/12] virtio: Add QTest support to vtpci abstraction Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 11/12] virtio: Add QTest support for virtio-net PMD Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 0/6] Virtio-net PMD: QEMU QTest extension for container Tetsuya Mukawa
2016-06-02 7:31 ` Yuanhan Liu
2016-06-02 9:30 ` Tetsuya Mukawa
2016-06-03 4:17 ` Yuanhan Liu
2016-06-03 13:51 ` Thomas Monjalon
2016-06-06 5:10 ` Tetsuya Mukawa
2016-06-06 7:21 ` Yuanhan Liu
2016-06-06 8:33 ` Tetsuya Mukawa
2016-06-06 8:49 ` Yuanhan Liu
2016-06-06 9:30 ` Tetsuya Mukawa
2016-06-06 9:58 ` Yuanhan Liu
2016-06-06 10:50 ` Tan, Jianfeng
2016-06-07 7:12 ` Tetsuya Mukawa
2016-06-07 7:33 ` Yuanhan Liu
2016-06-06 8:03 ` Tan, Jianfeng
2016-06-06 9:28 ` Tetsuya Mukawa
2016-06-06 10:35 ` Tan, Jianfeng
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 1/6] virtio, qtest: Add QTest utility basic functions Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 2/6] virtio, qtest: Add pci device initialization function to qtest utils Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 3/6] virtio, qtest: Add functionality to share memory between QTest guest Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 4/6] virtio, qtest: Add misc functions to handle pci information Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 5/6] virtio: Add QTest support to vtpci abstraction Tetsuya Mukawa
2016-06-02 3:29 ` [dpdk-dev] [PATCH v5 6/6] virtio: Add QTest support for virtio-net PMD Tetsuya Mukawa
2016-06-02 3:30 ` [dpdk-dev] [PATCH v1 0/2] Supplement patches for virtio-qtest to support LSC interrupt Tetsuya Mukawa
2016-06-02 3:30 ` [dpdk-dev] [PATCH v1 1/2] virtio: Handle interrupt things under vtpci abstraction Tetsuya Mukawa
2016-06-02 3:30 ` [dpdk-dev] [PATCH v1 2/2] virtio, qtest: Add functionality to handle interrupt Tetsuya Mukawa
2016-03-09 8:33 ` [dpdk-dev] [PATCH v4 12/12] docs: add release note for qtest virtio container support Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 4/6] EAL: Add a new "--align-memsize" option Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 5/6] virtio: Add support for qtest virtio-net PMD Tetsuya Mukawa
2016-03-04 2:18 ` Tan, Jianfeng
2016-03-04 5:05 ` Tetsuya Mukawa
2016-03-04 6:10 ` Tan, Jianfeng
2016-03-04 9:53 ` Tetsuya Mukawa
2016-02-22 8:17 ` [dpdk-dev] [PATCH v3 6/6] docs: add release note for qtest virtio container support Tetsuya Mukawa
2016-02-22 15:40 ` Mcnamara, John
2016-02-23 10:28 ` Mcnamara, John
2016-02-24 1:20 ` Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 3/5] vhost: Add a function to check virtio device type Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 4/5] virtio: Add support for qtest virtio-net PMD Tetsuya Mukawa
2016-02-10 3:40 ` [dpdk-dev] [PATCH v2 5/5] docs: add release note for qtest virtio container support Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 0/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 1/3] virtio: Change the parameter order of io_write8/16/32() Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 2/3] virtio: move rte_eal_pci_unmap_device() to virtio_pci.c Tetsuya Mukawa
2016-01-28 9:33 ` [dpdk-dev] [PATCH v2 3/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-29 9:17 ` Yuanhan Liu
2016-02-01 1:50 ` Tetsuya Mukawa
2016-02-01 13:15 ` Yuanhan Liu
2016-02-02 2:19 ` Tetsuya Mukawa
2016-02-02 2:45 ` Yuanhan Liu
2016-02-02 3:55 ` Tetsuya Mukawa
2016-01-18 9:13 ` [dpdk-dev] [PATCH 2/3] virtio: move rte_eal_pci_unmap_device() to virtio_pci.c Tetsuya Mukawa
2016-01-18 9:13 ` [dpdk-dev] [PATCH 3/3] virtio: Add a new layer to abstract pci access method Tetsuya Mukawa
2016-01-18 13:46 ` Yuanhan Liu
2016-01-19 1:22 ` Tetsuya Mukawa
2016-01-19 2:41 ` Xie, Huawei
2016-01-18 13:13 ` [dpdk-dev] [PATCH 0/3] " Tan, Jianfeng
2016-01-19 1:22 ` Tetsuya Mukawa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=C37D651A908B024F974696C65296B57B4C5B144A@SHSMSX101.ccr.corp.intel.com \
--to=huawei.xie@intel.com \
--cc=dev@dpdk.org \
--cc=jianfeng.tan@intel.com \
--cc=mukawa@igel.co.jp \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).