From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <avi.cohen@huawei.com>
Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187])
 by dpdk.org (Postfix) with ESMTP id 9DA797CB3
 for <users@dpdk.org>; Mon, 10 Jul 2017 08:49:53 +0200 (CEST)
Received: from 172.30.72.54 (EHLO NKGEML413-HUB.china.huawei.com)
 ([172.30.72.54])
 by dggrg01-dlp.huawei.com (MOS 4.4.6-GA FastPath queued)
 with ESMTP id ARU35649; Mon, 10 Jul 2017 14:49:50 +0800 (CST)
Received: from FRAEML702-CAH.china.huawei.com (10.206.14.33) by
 NKGEML413-HUB.china.huawei.com (10.98.56.74) with Microsoft SMTP Server (TLS)
 id 14.3.235.1; Mon, 10 Jul 2017 14:49:05 +0800
Received: from FRAEML521-MBX.china.huawei.com ([169.254.1.122]) by
 fraeml702-cah.china.huawei.com ([10.206.14.33]) with mapi id 14.03.0301.000;
 Mon, 10 Jul 2017 08:49:00 +0200
From: "Avi Cohen (A)" <avi.cohen@huawei.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>, "dpdk-ovs@lists.01.org"
 <dpdk-ovs@lists.01.org>, "users@dpdk.org" <users@dpdk.org>
Thread-Topic: VIRTIO for containers
Thread-Index: AQHS71DYC699Y73ftUG6MPPfZcA7c6I503QggAfGd4CAChbRIIAAqC8AgABYSpA=
Date: Mon, 10 Jul 2017 06:49:00 +0000
Message-ID: <B84047ECBD981D4B93EAE5A6245AA361013C987A@FRAEML521-MBX.china.huawei.com>
References: <B84047ECBD981D4B93EAE5A6245AA361013C5A45@FRAEML521-MBS.china.huawei.com>
 <ED26CBA2FAD1BF48A8719AEF02201E3651217C51@SHSMSX103.ccr.corp.intel.com>
 <B84047ECBD981D4B93EAE5A6245AA361013C7D2B@FRAEML521-MBX.china.huawei.com>
 <ED26CBA2FAD1BF48A8719AEF02201E36512186DE@SHSMSX103.ccr.corp.intel.com>
 <B84047ECBD981D4B93EAE5A6245AA361013C8E03@FRAEML521-MBX.china.huawei.com>
 <e93928ec-8e44-dda5-c07a-9118b5ff7f14@intel.com>
 <B84047ECBD981D4B93EAE5A6245AA361013C9047@FRAEML521-MBX.china.huawei.com>
 <ED26CBA2FAD1BF48A8719AEF02201E365122249B@SHSMSX103.ccr.corp.intel.com>
 <B84047ECBD981D4B93EAE5A6245AA361013C9830@FRAEML521-MBX.china.huawei.com>
 <ED26CBA2FAD1BF48A8719AEF02201E365122E47D@SHSMSX103.ccr.corp.intel.com>
In-Reply-To: <ED26CBA2FAD1BF48A8719AEF02201E365122E47D@SHSMSX103.ccr.corp.intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.200.202.183]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-CFilter-Loop: Reflected
X-Mirapoint-Virus-RAPID-Raw: score=unknown(0),
 refid=str=0001.0A020205.59632390.000F, ss=1, re=0.000, recu=0.000, reip=0.000,
 cl=1, cld=1, fgs=0, ip=169.254.1.122,
 so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32
X-Mirapoint-Loop-Id: 93d260bafaaa425b015814967d13f80b
Subject: Re: [dpdk-users] VIRTIO for containers
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/users>,
 <mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/users>,
 <mailto:users-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 10 Jul 2017 06:49:56 -0000



> -----Original Message-----
> From: Tan, Jianfeng [mailto:jianfeng.tan@intel.com]
> Sent: Monday, 10 July, 2017 6:28 AM
> To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org
> Subject: RE: VIRTIO for containers
> > [Avi Cohen (A)]
> > Thanks you Jianfeng
> > I've tested it and the performance looks very good compared to native o=
vs.
> > I have 1 more question:
> > You wrote " there's a vhost kthread to push the data out to user space
> > " - Is that mean a copy from userspace to kernel (and viceversa) or
> > there is a zero-copy mmap  like in AF_PACKET which handles TX/RX rings =
in
> userspace ?
> > Best Regards
> > avi
>=20
> So far it needs data copy at least from kernel to user path; there's an
> experimental feature, named experimental_zcopytx, to avoid data copy, but
> not very useful due to the implementation limitation.
[Avi Cohen (A)]=20
Thanks Jianfeng
The penalty here that this vhost-kthread consumes very much CPU  on a high =
throughput scenario  - more than 80% CPU on ~ 10Gbps throughput
And this is in addition to the 100% CPU of the PMD threads
Also when PMD threads can be shared between multiple containers - the vhost=
-kthread is per container.
Best Regards
avi
>=20
> Packet mmap (similar to AF_PACKET) is exactly the direction we were
> discussing for the further optimization. Plus, an optimized vhost thread
> model (current thread model is: one thread for one rx-tx queue pair) is a=
lso
> considered.
>=20
> Thanks,
> Jianfeng
>=20