From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: "Xia, Chenbo" <chenbo.xia@intel.com>,
Thomas Monjalon <thomas@monjalon.net>,
David Marchand <david.marchand@redhat.com>
Cc: dev <dev@dpdk.org>,
Stephen Hemminger <stephen@networkplumber.org>,
"Liang, Cunming" <cunming.liang@intel.com>,
"Lu, Xiuchun" <xiuchun.lu@intel.com>,
"Li, Miao" <miao.li@intel.com>,
"Wu, Jingjing" <jingjing.wu@intel.com>
Subject: Re: [dpdk-dev] [PATCH 0/8] Introduce emudev library and iavf emudev driver
Date: Mon, 21 Dec 2020 13:01:43 +0100 [thread overview]
Message-ID: <a42a455f-db82-ebce-22e3-414f3f9d48da@redhat.com> (raw)
In-Reply-To: <22c05dc3-bee9-4662-3f8d-a4dcd7635b42@redhat.com>
On 12/21/20 10:52 AM, Maxime Coquelin wrote:
> Hi Chenbo,
>
> On 12/19/20 7:11 AM, Xia, Chenbo wrote:
>> Hi David,
>>
>>> -----Original Message-----
>>> From: David Marchand <david.marchand@redhat.com>
>>> Sent: Friday, December 18, 2020 5:54 PM
>>> To: Xia, Chenbo <chenbo.xia@intel.com>
>>> Cc: dev <dev@dpdk.org>; Thomas Monjalon <thomas@monjalon.net>; Stephen
>>> Hemminger <stephen@networkplumber.org>; Liang, Cunming
>>> <cunming.liang@intel.com>; Lu, Xiuchun <xiuchun.lu@intel.com>; Li, Miao
>>> <miao.li@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
>>> Subject: Re: [PATCH 0/8] Introduce emudev library and iavf emudev driver
>>>
>>> On Fri, Dec 18, 2020 at 9:02 AM Chenbo Xia <chenbo.xia@intel.com> wrote:
>>>>
>>>> This series introduces a new device abstraction called emudev for
>>> emulated
>>>> devices. A new library (librte_emudev) is implemented. The first emudev
>>>> driver is also introduced, which emulates Intel Adaptive Virtual
>>> Function
>>>> (iavf) as a software network device.
>>>>
>>>> This series has a dependency on librte_vfio_user patch series:
>>>> http://patchwork.dpdk.org/cover/85389/
>>>>
>>>> Background & Motivation
>>>> -----------------------
>>>> The disaggregated/multi-process QEMU is using VFIO-over-socket/vfio-user
>>>> as the main transport mechanism to disaggregate IO services from QEMU.
>>>> Therefore, librte_vfio_user is introduced in DPDK to accommodate
>>>> emulated devices for high performance I/O. Although vfio-user library
>>>> provides possibility of emulating devices in DPDK, DPDK does not have
>>>> a device abstraction for emulated devices. A good device abstraction
>>> will
>>>> be useful for applications or high performance data path driver. With
>>>> this consideration, emudev library is designed and implemented. It also
>>>> make it possbile to keep modular design on emulated devices by
>>> implementing
>>>> data path related logic in a standalone driver (e.g., an ethdev driver)
>>>> and keeps the unrelated logic in the emudev driver.
>>>
>>> Since you mention performance, how does it compare to vhost-user/virtio?
>>
>> I think it depends on the device specification (i.e., how complex its data path
>> handling is). A first try on iavf spec shows better performance than virtio
>> in our simple tests.
>
> That's interesting! How big is the performance difference? And how do
> we explain it?
>
> If there are improvements that could be done in the Virtio
> specification, it would be great to know and work on their
> implementations. It worries me a bit that every one is coming with
> his new device emulation every release, making things like live-
> migration difficult to achieve in the future.
I did a quick review of the IAVF emudev driver to understand what other
factors than ring layout could explain a performance gain.
My understanding is that part of the performance gain may come from
following things that are supported/implemented in Vhost-user backend
and not in IAVF driver:
1. Memory hotplug. It seems the datapath is not safe against memory
hotplug in the VM, which causes the memory tables to be updated
asynchronously from the datapath. In order to support it in Vhost-user
library, we had to introduce locks to ensure the datapath isn't
accessing the shared memory while it is being remapped.
2. Live-migration. This feature does not seem supported in the driver,
as I could not find dirty pages tracking mechanism. On Vhost-user side,
supporting implies adding more branch conditions in the hot path, even
when it is not used.
3. No IOMMU support. Same here, this is supported in Vhost-user library,
and adding its support in IAVF driver would introduce some more branches
in the hot path even when not used.
4. Out of bound memory accesses checks. While
rte_iavf_emu_get_dma_vaddr() provides a way to ensure the full requested
length is mapped, the data path does not use it. It does not even ensure
the translated address is non-NULL. It makes it trivial for a malicious
guest to make the hypervisor's vSwitch to crash by simply passing random
values as buffer's address and length. Fixing it is trivial, but it will
add several more checks and loops (if a buffer is spanned across two
pages) in the hot path.
Other than that, there is for sure a performance gain due to all the
features Virtio-net supports that we have to check and handle in the
hotpath, like indirect descriptors or mergeable buffers for example.
Best regards,
Maxime
> Regards,
> Maxime
>
>> Thanks!
>> Chenbo
>>
>>>
>>>
>>> --
>>> David Marchand
>>
next prev parent reply other threads:[~2020-12-21 12:01 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-18 7:47 Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 1/8] lib: introduce emudev library Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 2/8] doc: add emudev library guide Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 3/8] emu: introduce emulated iavf driver Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 4/8] emu/iavf: add vfio-user device register and unregister Chenbo Xia
2021-01-07 7:18 ` Xing, Beilei
2021-01-07 8:41 ` Xia, Chenbo
2020-12-18 7:47 ` [dpdk-dev] [PATCH 5/8] emu/iavf: add resource management and internal logic of iavf Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 6/8] emu/iavf: add emudev operations to fit in emudev framework Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 7/8] test/emudev: introduce functional test Chenbo Xia
2020-12-18 7:47 ` [dpdk-dev] [PATCH 8/8] doc: update release notes for iavf emudev driver Chenbo Xia
2020-12-18 9:53 ` [dpdk-dev] [PATCH 0/8] Introduce emudev library and " David Marchand
2020-12-19 6:11 ` Xia, Chenbo
2020-12-21 9:52 ` Maxime Coquelin
2020-12-21 12:01 ` Maxime Coquelin [this message]
2020-12-22 3:09 ` Xia, Chenbo
2020-12-22 8:48 ` Maxime Coquelin
2020-12-23 5:28 ` Xia, Chenbo
2020-12-19 6:27 ` [dpdk-dev] [PATCH v2 " Chenbo Xia
2020-12-19 6:27 ` [dpdk-dev] [PATCH v2 1/8] lib: introduce emudev library Chenbo Xia
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 2/8] doc: add emudev library guide Chenbo Xia
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 3/8] emu: introduce emulated iavf driver Chenbo Xia
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 4/8] emu/iavf: add vfio-user device register and unregister Chenbo Xia
2021-01-04 6:45 ` Wu, Jingjing
2021-01-05 1:26 ` Xia, Chenbo
2021-01-05 13:41 ` Wu, Jingjing
2021-01-06 7:41 ` Xia, Chenbo
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 5/8] emu/iavf: add resource management and internal logic of iavf Chenbo Xia
2020-12-29 6:05 ` Wu, Jingjing
2020-12-30 1:59 ` Xia, Chenbo
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 6/8] emu/iavf: add emudev operations to fit in emudev framework Chenbo Xia
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 7/8] test/emudev: introduce functional test Chenbo Xia
2020-12-19 6:28 ` [dpdk-dev] [PATCH v2 8/8] doc: update release notes for iavf emudev driver Chenbo Xia
2021-01-13 16:52 ` [dpdk-dev] [PATCH v2 0/8] Introduce emudev library and " Thomas Monjalon
2021-01-14 1:35 ` Xia, Chenbo
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 " Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 1/8] lib: introduce emudev library Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 2/8] doc: add emudev library guide Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 3/8] emu: introduce emulated iavf driver Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 4/8] emu/iavf: add vfio-user device register and unregister Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 5/8] emu/iavf: add resource management and internal logic of iavf Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 6/8] emu/iavf: add emudev operations to fit in emudev framework Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 7/8] test/emudev: introduce functional test Chenbo Xia
2021-01-14 6:25 ` [dpdk-dev] [PATCH v3 8/8] doc: update release notes for iavf emudev driver Chenbo Xia
2024-02-12 22:49 ` [dpdk-dev] [PATCH v3 0/8] Introduce emudev library and " Stephen Hemminger
2023-06-14 19:47 ` [dpdk-dev] [PATCH " Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a42a455f-db82-ebce-22e3-414f3f9d48da@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=chenbo.xia@intel.com \
--cc=cunming.liang@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=jingjing.wu@intel.com \
--cc=miao.li@intel.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
--cc=xiuchun.lu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).