From: 王志克 <wangzhike@jd.com>
To: "Tan, Jianfeng (jianfeng.tan@intel.com)" <jianfeng.tan@intel.com>,
"avi.cohen@huawei.com" <avi.cohen@huawei.com>,
"dpdk-ovs@lists.01.org" <dpdk-ovs@lists.01.org>,
"users@dpdk.org" <users@dpdk.org>
Subject: Re: [dpdk-users] VIRTIO for containers
Date: Fri, 20 Oct 2017 09:24:55 +0000 [thread overview]
Message-ID: <6DAF063A35010343823807B082E5681F41D22305@mbx05.360buyAD.local> (raw)
I read this thread, and try to do the same way (legacy containers connect to ovs+dpdk). However, I meet following error when creating ovs port.
ovs-vsctl add-port br0 virtiouser0 -- set Interface virtiouser0 type=dpdk options:dpdk-devargs=net_virtio_user0,path=/dev/vhost-net
ovs-vsctl: Error detected while setting up 'virtiouser0': Error attaching device 'net_virtio_user0,path=/dev/vhost-net' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
Debug shows that it calls virtio_user_dev_init()->vhost_user_setup(), and failed in connect() with target /dev/vhost-net. The errno is ECONNREFUSED.
Below command indeed shows no one is listening.
lsof | grep vhost-net
In kernel OVS, I guess qemu-kvm would listne to /dev/vhost-net. But for ovs_dpdk and container, what extra work need be done? Appreciate any help.
Br,
Wang Zhike
> -----Original Message-----
> From: Avi Cohen (A) [mailto:avi.cohen at huawei.com<http://dpdk.org/ml/listinfo/users>]
> Sent: Sunday, July 9, 2017 11:32 PM
> To: Tan, Jianfeng; dpdk-ovs at lists.01.org<http://dpdk.org/ml/listinfo/users>; users at dpdk.org<http://dpdk.org/ml/listinfo/users>
> Subject: RE: VIRTIO for containers
>
> Thanks you Jianfeng - pls see inline marked [Avi Cohen (A)]
>
> > -----Original Message-----
> > From: Tan, Jianfeng [mailto:jianfeng.tan at intel.com<http://dpdk.org/ml/listinfo/users>]
> > Sent: Monday, 03 July, 2017 10:22 AM
> > To: Avi Cohen (A); dpdk-ovs at lists.01.org<http://dpdk.org/ml/listinfo/users>; users at dpdk.org<http://dpdk.org/ml/listinfo/users>
> > Subject: RE: VIRTIO for containers
> >
> >
> >
> > > -----Original Message-----
> > > From: Avi Cohen (A) [mailto:avi.cohen at huawei.com<http://dpdk.org/ml/listinfo/users>]
> > > Sent: Wednesday, June 28, 2017 2:45 PM
> > > To: Tan, Jianfeng; dpdk-ovs at lists.01.org<http://dpdk.org/ml/listinfo/users>; users at dpdk.org<http://dpdk.org/ml/listinfo/users>
> > > Subject: RE: VIRTIO for containers
> > >
> > > Thank you Jianfeng
> > >
> > > > -----Original Message-----
> > > > From: Tan, Jianfeng [mailto:jianfeng.tan at intel.com<http://dpdk.org/ml/listinfo/users>]
> > > > Sent: Tuesday, 27 June, 2017 5:22 PM
> > > > To: Avi Cohen (A); dpdk-ovs at lists.01.org<http://dpdk.org/ml/listinfo/users>; users at dpdk.org<http://dpdk.org/ml/listinfo/users>
> > > > Subject: Re: VIRTIO for containers
> > > >
> > > >
> > > >
> > > > On 6/26/2017 8:06 PM, Avi Cohen (A) wrote:
> > > > > Thank You Jianfeng
> > > > >
> > > > >> We have developed virtio-user + vhost-kernel as the backend. In
> > > > >> that scenario, you can add the tap interface into a container
> > > > >> network
> > > > namespace.
> > > > >> And there's a vhost kthread to push the data out to user space.
> > > > >>
> > > > >> And I cannot guarantee the performance as it has diametric model
> > > > >> in VM
> > > > >> (virtio) - OVS-DPDK (vhost).
> > > > >>
> > > > > [Avi Cohen (A)]
> > > > > Can you refer to a document how to run this setup?
> > > >
> > > > Please refer to
> > > >
> http://dpdk.org/doc/guides/howto/virtio_user_as_exceptional_path.htm
> > > > l
> > > >
> > > [Avi Cohen (A)]
> > > My setup includes a container and ovs-dpdk , i still not sure about:
> > > - How to set the virtio backend port in the ovs-dpdk ?
> >
> > For OVS-DPDK, you need a version above 2.7.0. Below command is used to
> > create a virtio-user port with vhost-kernel backend:
> > # ovs-vsctl add-port br0 virtiouser0 -- set Interface virtiouser0 type=dpdk
> > options:dpdk-devargs=virtio_user0,path=/dev/vhost-net
> >
> > > - How to set the container with the virtio frontend ?
> >
> > No, containers will not hold the virtio frontend in this case. Above ovs-vsctl
> > command will generate a virtio-user port in OVS, and a tap interface in
> > kernel, you can assign the tap interface into a net namespace of some
> > container so that its networking flow will go through OVS-DPDK then to
> > outside.
> [Avi Cohen (A)]
> Thanks you Jianfeng
> I've tested it and the performance looks very good compared to native ovs.
> I have 1 more question:
> You wrote " there's a vhost kthread to push the data out to user space " -
> Is that mean a copy from userspace to kernel (and viceversa) or there is a
> zero-copy mmap like in AF_PACKET which handles TX/RX rings in userspace ?
> Best Regards
> avi
So far it needs data copy at least from kernel to user path; there's an experimental feature, named experimental_zcopytx, to avoid data copy, but not very useful due to the implementation limitation.
Packet mmap (similar to AF_PACKET) is exactly the direction we were discussing for the further optimization. Plus, an optimized vhost thread model (current thread model is: one thread for one rx-tx queue pair) is also considered.
Thanks,
Jianfeng
> >
> > Thanks,
> > Jianfeng
> >
> > > Best Regards
> > > avi
> > >
> > > > Thanks,
> > > > Jianfeng
> > > >
> > > > > Best Regards
> > > > > avi
> > > > >>> I've tested the performance of a container connected to OVS-
> DPDK
> > > > >>> via vdev-af_packet and processed by virtual PMD, and its
> > > > >>> performance is good [uses mmap'ed to userspace - zero copy
> > > > >>> RX/TX ring buffer] but not good as the performance of a VM
> > > > >>> connected to OVS-DPDK (@host) via vhost-user virtio.
> > > > >>> Best Regards
> > > > >>> avi
> > > > >>>
> > > > >>>> -----Original Message-----
> > > > >>>> From: Tan, Jianfeng [mailto:jianfeng.tan at intel.com<http://dpdk.org/ml/listinfo/users>]
> > > > >>>> Sent: Monday, 26 June, 2017 6:15 AM
> > > > >>>> To: Avi Cohen (A); dpdk-ovs at lists.01.org<http://dpdk.org/ml/listinfo/users>; users at dpdk.org<http://dpdk.org/ml/listinfo/users>
> > > > >>>> Subject: RE: VIRTIO for containers
> > > > >>>>
> > > > >>>> Hi Avi,
> > > > >>>>
> > > > >>>>> -----Original Message-----
> > > > >>>>> From: users [mailto:users-bounces at dpdk.org<http://dpdk.org/ml/listinfo/users>] On Behalf Of Avi
> > > Cohen
> > > > >>>>> (A)
> > > > >>>>> Sent: Sunday, June 25, 2017 11:13 PM
> > > > >>>>> To: dpdk-ovs at lists.01.org<http://dpdk.org/ml/listinfo/users>; users at dpdk.org<http://dpdk.org/ml/listinfo/users>
> > > > >>>>> Subject: [dpdk-users] VIRTIO for containers
> > > > >>>>>
> > > > >>>>> Hello,
> > > > >>>>> Does anyone know the status of this project
> > > > >>>>> http://dpdk.org/ml/archives/dev/2015-November/027732.html
> -
> > > > >>>>> Implementing a virtio device for containers ?
> > > > >>>> It has been upstreamed since v16.07. Here is a howto doc:
> > > > >>>>
> > > > >>>
> > > >
> > http://dpdk.org/doc/guides/howto/virtio_user_for_container_networking.
> > > > >>> h
> > > > >>>> tml
> > > > >>>>
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Jianfeng
> > > > >>>>
> > > > >>>>> Best Regards
> > > > >>>>> avi
________________________________
* Previous message: [dpdk-users] VIRTIO for containers <http://dpdk.org/ml/archives/users/2017-July/002162.html>
* Next message: [dpdk-users] VIRTIO for containers <http://dpdk.org/ml/archives/users/2017-July/002165.html>
* Messages sorted by: [ date ]<http://dpdk.org/ml/archives/users/2017-July/date.html#2164> [ thread ]<http://dpdk.org/ml/archives/users/2017-July/thread.html#2164> [ subject ]<http://dpdk.org/ml/archives/users/2017-July/subject.html#2164> [ author ]<http://dpdk.org/ml/archives/users/2017-July/author.html#2164>
________________________________
More information about the users mailing list<http://dpdk.org/ml/listinfo/users>
next reply other threads:[~2017-10-20 9:25 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-20 9:24 王志克 [this message]
2017-10-20 16:54 ` Tan, Jianfeng
2017-10-24 3:15 ` 王志克
2017-10-24 9:45 ` 王志克
2017-10-25 7:34 ` Tan, Jianfeng
2017-10-25 9:58 ` 王志克
2017-10-26 8:53 ` Tan, Jianfeng
2017-10-26 12:53 ` 王志克
2017-10-27 1:58 ` Tan, Jianfeng
2017-10-31 4:25 ` 王志克
2017-11-01 2:58 ` Tan, Jianfeng
-- strict thread matches above, loose matches on Subject: below --
2017-06-25 15:13 Avi Cohen (A)
2017-06-26 3:14 ` Tan, Jianfeng
2017-06-26 6:16 ` Avi Cohen (A)
2017-06-26 11:58 ` Tan, Jianfeng
2017-06-26 12:06 ` Avi Cohen (A)
2017-06-27 14:22 ` Tan, Jianfeng
2017-06-28 6:45 ` Avi Cohen (A)
2017-07-03 7:21 ` Tan, Jianfeng
2017-07-09 15:32 ` Avi Cohen (A)
2017-07-10 3:28 ` Tan, Jianfeng
2017-07-10 6:49 ` Avi Cohen (A)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6DAF063A35010343823807B082E5681F41D22305@mbx05.360buyAD.local \
--to=wangzhike@jd.com \
--cc=avi.cohen@huawei.com \
--cc=dpdk-ovs@lists.01.org \
--cc=jianfeng.tan@intel.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).