Re: [dpdk-dev] [PATCH v6 0/2] introduce asynchronous data path for vhost

[Maxime Coquelin <maxime.coquelin@redhat.com>]

Hi Patrick,

Thanks for the series.
I think we miss a chapter in the Vhost lib documentation to explain what
this new API is about.

Do you think you can write something by -rc3?

Thanks in advance,
Maxime

On 7/7/20 7:07 AM, patrick.fu@intel.com wrote:
> From: Patrick Fu <patrick.fu@intel.com>
> 
> Performing large memory copies usually takes up a major part of CPU
> cycles and becomes the hot spot in vhost-user enqueue operation. To
> offload expensive memory operations from the CPU, this patch set
> proposes to leverage DMA engines, e.g., I/OAT, a DMA engine in the
> Intel's processor, to accelerate large copies.
> 
> Large copies are offloaded from the CPU to the DMA in an asynchronous
> manner. The CPU just submits copy jobs to the DMA but without waiting
> for its copy completion. Thus, there is no CPU intervention during
> data transfer; we can save precious CPU cycles and improve the overall
> throughput for vhost-user based applications, like OVS. During packet
> transmission, it offloads large copies to the DMA and performs small
> copies by the CPU, due to startup overheads associated with the DMA.
> 
> This patch set construct a general framework that applications can
> leverage to attach DMA channels with vhost-user transmit queues. Four
> new RTE APIs are introduced to vhost library for applications to
> register and use the asynchronous data path. In addition, two new DMA
> operation callbacks are defined, by which vhost-user asynchronous data
> path can interact with DMA hardware. Currently only enqueue operation
> for split queue is implemented, but the framework is flexible to extend
> support for packed queue.
> 
> v2:
> update meson file for new header file
> update rte_vhost_version.map to include new APIs
> rename async APIs/structures to be prefixed with "rte_vhost"
> rename some variables/structures for readibility
> correct minor typo in comments/license statements
> refine memory allocation logic for vq internal buffer
> add error message printing in some failure cases
> check inflight async packets in unregistration API call
> mark new APIs as experimental
> 
> v3:
> use atomic_xxx() functions in updating ring index
> fix a bug in async enqueue failure handling
> 
> v4:
> part of the fix intended in v3 patch was missed, this patch
> adds all thoes fixes
> 
> v5:
> minor changes on some function/variable names
> reset CPU batch copy packet count when async enqueue error
> occurs
> disable virtio log feature in async copy mode
> minor optimization on async shadow index flush
> 
> v6:
> add some background introduction in the commit message
> 
> Patrick Fu (2):
>   vhost: introduce async enqueue registration API
>   vhost: introduce async enqueue for split ring
> 
>  lib/librte_vhost/Makefile              |   2 +-
>  lib/librte_vhost/meson.build           |   2 +-
>  lib/librte_vhost/rte_vhost.h           |   1 +
>  lib/librte_vhost/rte_vhost_async.h     | 176 ++++++++
>  lib/librte_vhost/rte_vhost_version.map |   4 +
>  lib/librte_vhost/socket.c              |  27 ++
>  lib/librte_vhost/vhost.c               | 127 +++++-
>  lib/librte_vhost/vhost.h               |  30 +-
>  lib/librte_vhost/vhost_user.c          |  23 +-
>  lib/librte_vhost/virtio_net.c          | 551 ++++++++++++++++++++++++-
>  10 files changed, 934 insertions(+), 9 deletions(-)
>  create mode 100644 lib/librte_vhost/rte_vhost_async.h
>