From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) by dpdk.org (Postfix) with ESMTP id 9430C8E71 for ; Tue, 26 Jan 2016 03:58:36 +0100 (CET) Received: by mail-pa0-f41.google.com with SMTP id uo6so92542414pac.1 for ; Mon, 25 Jan 2016 18:58:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=0YUDxL+LsITc/kpxrTMcWBF+3QE7mhIzQLnbD/2CHoo=; b=bROAoexi2uQY3HjlXftBomahpz6s2fst3pifwiAS54j4qsxlf3cLAj4WGCHoCE8nk+ hXqXt4D1VRGP3v34LlaW9Uvh5PlsLHv2+P/p6BthVZn3ITeWF83ke3AJsFVJaRci9wMr 1NIJ2apNz5v6CcO24KA+ZEdS56ZWOYxyRGx5+xfNM45uXLjqSJ9LXMOG+llO52T3U+WL c1zI94MYNIOciwcj1Nv2uAHQw0hKr2ygtmaY85l7GLgA7gbndGq0VnNf2143ppqhseRL lnMetUBuvWYDDQo4l54gS9UG2mw+pHKaexxfAviRK5n/SxDV/AFurBTgz+7XGm3+dArq CzRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=0YUDxL+LsITc/kpxrTMcWBF+3QE7mhIzQLnbD/2CHoo=; b=I8s4NGH4oD63w7oDgljUckJ2O4z4KrIdHpNOLXVYVcqCIlIGAtO4RN9DmppK34GXuO HzxC7VzZM67brW6ninVub3AF9z7K5eSLGzdZ0qfCuevc3wiSYFnWsV95KPSenyGw6wEW cYqUsuSFa3R6Pp2pWdHIsA4AI7CRgH0G5F8g+K8nUq9NOqMQU0m+1AHr1JfwMSJ1pvpG xToxoc03FF/Dfr7zB3/trfMml3gi0R5HAP4Y6VjVjd9yF4PC334AIhcj2TibuXe20d9U 5LAwx+ofu9meyQfh3yOMVQawLg4fKpPHESKWC1yCz7gZ/tSqBh7t52vp3K/dQ6T2ldXn n5Ww== X-Gm-Message-State: AG10YORezC9wCCkgjkxcgRikDvcK27DBeXzmtpZPoskjg01zLaILkMsPSHbxWi/RC4xuOQ== X-Received: by 10.66.236.103 with SMTP id ut7mr30407496pac.4.1453777115890; Mon, 25 Jan 2016 18:58:35 -0800 (PST) Received: from [10.16.129.101] (napt.igel.co.jp. [219.106.231.132]) by smtp.googlemail.com with ESMTPSA id c87sm31701522pfj.41.2016.01.25.18.58.33 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 25 Jan 2016 18:58:35 -0800 (PST) To: "Xie, Huawei" , "dev@dpdk.org" , "yuanhan.liu@linux.intel.com" , "Tan, Jianfeng" References: <1453108389-21006-2-git-send-email-mukawa@igel.co.jp> <1453374478-30996-6-git-send-email-mukawa@igel.co.jp> <56A2065A.9020207@igel.co.jp> From: Tetsuya Mukawa X-Enigmail-Draft-Status: N1110 Message-ID: <56A6E0D8.9010005@igel.co.jp> Date: Tue, 26 Jan 2016 11:58:32 +0900 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC PATCH 5/5] virtio: Extend virtio-net PMD to support container environment X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jan 2016 02:58:37 -0000 On 2016/01/25 19:15, Xie, Huawei wrote: > On 1/22/2016 6:38 PM, Tetsuya Mukawa wrote: >> On 2016/01/22 17:14, Xie, Huawei wrote: >>> On 1/21/2016 7:09 PM, Tetsuya Mukawa wrote: >>>> virtio: Extend virtio-net PMD to support container environment >>>> >>>> The patch adds a new virtio-net PMD configuration that allows the PMD to >>>> work on host as if the PMD is in VM. >>>> Here is new configuration for virtio-net PMD. >>>> - CONFIG_RTE_LIBRTE_VIRTIO_HOST_MODE >>>> To use this mode, EAL needs physically contiguous memory. To allocate >>>> such memory, add "--shm" option to application command line. >>>> >>>> To prepare virtio-net device on host, the users need to invoke QEMU >>>> process in special qtest mode. This mode is mainly used for testing QEMU >>>> devices from outer process. In this mode, no guest runs. >>>> Here is QEMU command line. >>>> >>>> $ qemu-system-x86_64 \ >>>> -machine pc-i440fx-1.4,accel=qtest \ >>>> -display none -qtest-log /dev/null \ >>>> -qtest unix:/tmp/socket,server \ >>>> -netdev type=tap,script=/etc/qemu-ifup,id=net0,queues=1\ >>>> -device virtio-net-pci,netdev=net0,mq=on \ >>>> -chardev socket,id=chr1,path=/tmp/ivshmem,server \ >>>> -device ivshmem,size=1G,chardev=chr1,vectors=1 >>>> >>>> * QEMU process is needed per port. >>> Does qtest supports hot plug virtio-net pci device, so that we could run >>> one QEMU process in host, which provisions the virtio-net virtual >>> devices for the container? >> Theoretically, we can use hot plug in some cases. >> But I guess we have 3 concerns here. >> >> 1. Security. >> If we share QEMU process between multiple DPDK applications, this QEMU >> process will have all fds of the applications on different containers. >> In some cases, it will be security concern. >> So, I guess we need to support current 1:1 configuration at least. >> >> 2. shared memory. >> Currently, QEMU and DPDK application will map shared memory using same >> virtual address. >> So if multiple DPDK application connects to one QEMU process, each DPDK >> application should have different address for shared memory. I guess >> this will be a big limitation. >> >> 3. PCI bridge. >> So far, QEMU has one PCI bridge, so we can connect almost 10 PCI devices >> to QEMU. >> (I forget correct number, but it's almost 10, because some slots are >> reserved by QEMU) >> A DPDK application needs both virtio-net and ivshmem device, so I guess >> almost 5 DPDK applications can connect to one QEMU process, so far. >> To add more PCI bridges solves this. >> But we need to add a lot of implementation to support cascaded PCI >> bridges and PCI devices. >> (Also we need to solve above "2nd" concern.) >> >> Anyway, if we use virtio-net PMD and vhost-user PMD, QEMU process will >> not do anything after initialization. >> (QEMU will try to read a qtest socket, then be stopped because there is >> no message after initialization) >> So I guess we can ignore overhead of these QEMU processes. >> If someone cannot ignore it, I guess this is the one of cases that it's >> nice to use your light weight container implementation. > Thanks for the explanation, and also in your opinion where is the best > place to run the QEMU instance? If we run QEMU instances in host, for > vhost-kernel support, we could get rid of the root privilege issue. Do you mean below? If we deploy QEMU instance on host, we can start a container without the root privilege. (But on host, still QEMU instance needs the privilege to access to vhost-kernel) If so, I agree to deploy QEMU instance on host or other privileged container will be nice. In the case of vhost-user, to deploy on host or non-privileged container will be good. > > Another issue is do you plan to support multiple virtio devices in > container? Currently i find the code assuming only one virtio-net device > in QEMU, right? Yes, so far, 1 port needs 1 QEMU instance. So if you need multiple virtio devices, you need to invoke multiple QEMU instances. Do you want to deploy 1 QEMU instance for each DPDK application, even if the application has multiple virtio-net ports? So far, I am not sure whether we need it, because this type of DPDK application will need only one port in most cases. But if you need this, yes, I can implement using QEMU PCI hotplug feature. (But probably we can only attach almost 10 ports. This will be limitation.) > > Btw, i have read most of your qtest code. No obvious issues found so far > but quite a couple of nits. You must have spent a lot of time on this. > It is great work! I appreciate your reviewing! BTW, my container implementation needed a QEMU patch in the case of vhost-user. But the patch has been merged in upstream QEMU, so we don't have this limitation any more. Thanks, Tetsuya