From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F3600A0352; Thu, 30 Dec 2021 14:51:00 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 97112410F1; Thu, 30 Dec 2021 14:51:00 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 578BA40F35 for ; Thu, 30 Dec 2021 14:50:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1640872259; x=1672408259; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8eozcNeYdh84XMiaisqdLXKxUITwgGRGuJNq56vTPcI=; b=PpcTODxbJQ81bZTQnwLPsLy7BWeeiyMM6LRquWEgq1IAo4a+MQUVySId wwPd58lJA3+x6UqMKm8cgL1Gx5Yww7Rx95bzhpiIZqsDRiRTTlo676VV8 N2D70CGLHYYaiW7lnyk6QeUAeJeOTCYgQtIvDr3NLd8pUxDXYxeR9JdY5 vDiF2VVAXYwSGaX2D9Sid6zkcgpkEXxKEN0Gh5sBQeOHDkOtMqalwBW5U XoZAQVRGjipnXBatvtbpdzOzaDu4YzEU0Od00HNHFpPb9n8wmH1zo1+7O i7vEu3N2FwmWi8ae2gjDpqqZZnRq2k02iglBUE9NCFQUbNyhqx6SBXosK A==; X-IronPort-AV: E=McAfee;i="6200,9189,10212"; a="241881256" X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="241881256" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 05:50:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="524397235" Received: from npgdpdkvirtiojiayuhu117.sh.intel.com ([10.67.119.202]) by orsmga008.jf.intel.com with ESMTP; 30 Dec 2021 05:50:55 -0800 From: Jiayu Hu To: dev@dpdk.org Cc: maxime.coquelin@redhat.com, i.maximets@ovn.org, chenbo.xia@intel.com, bruce.richardson@intel.com, harry.van.haaren@intel.com, sunil.pai.g@intel.com, john.mcnamara@intel.com, xuan.ding@intel.com, cheng1.jiang@intel.com, liangma@liangbit.com, Jiayu Hu Subject: [PATCH v1 0/1] integrate dmadev in vhost Date: Thu, 30 Dec 2021 16:55:04 -0500 Message-Id: <20211230215505.329674-1-jiayu.hu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211122105437.3534231-1-jiayu.hu@intel.com> References: <20211122105437.3534231-1-jiayu.hu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA abstraction layer and simplify application logics, this patch integrates dmadev in vhost. To enable the flexibility of using DMA devices in different function modules, not limited in vhost, vhost doesn't manage DMA devices. Applications, like OVS, need to manage and configure DMA devices and tell vhost what DMA device to use in every dataplane function call. In addition, vhost supports M:N mapping between vrings and DMA virtual channels. Specifically, one vring can use multiple different DMA channels and one DMA channel can be shared by multiple vrings at the same time. The reason of enabling one vring to use multiple DMA channels is that it's possible that more than one dataplane threads enqueue packets to the same vring with their own DMA virtual channels. Besides, the number of DMA devices is limited. For the purpose of scaling, it's necessary to support sharing DMA channels among vrings. As only enqueue path is enabled DMA acceleration, the new dataplane functions are like: 1). rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count, dma_id, dma_vchan): Get descriptors and submit copies to DMA virtual channel for the packets that need to be send to VM. 2). rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count, dma_id, dma_vchan): Check completed DMA copies from the given DMA virtual channel and write back corresponding descriptors to vring. OVS needs to call rte_vhost_poll_enqueue_completed to clean in-flight copies on previous call and it can be called inside rxq_recv function, so that it doesn't require big change in OVS datapath. For example: netdev_dpdk_vhost_rxq_recv() { ... qid = rxq->queue_id * VIRTIO_QNUM + VIRTIO_RXQ; rte_vhost_poll_enqueue_completed(vid, qid, ...); } Change log ========== rfc -> v1: - remove useless code - support dynamic DMA vchannel ring size (rte_vhost_async_dma_configure) - fix several bugs - fix typo and coding style issues - replace "while" with "for" - update programmer guide - support share dma among vhost in vhost example - remove "--dma-type" in vhost example Jiayu Hu (1): vhost: integrate dmadev in asynchronous datapath doc/guides/prog_guide/vhost_lib.rst | 70 ++++----- examples/vhost/Makefile | 2 +- examples/vhost/ioat.c | 218 -------------------------- examples/vhost/ioat.h | 63 -------- examples/vhost/main.c | 230 +++++++++++++++++++++++----- examples/vhost/main.h | 11 ++ examples/vhost/meson.build | 6 +- lib/vhost/meson.build | 3 +- lib/vhost/rte_vhost_async.h | 121 +++++---------- lib/vhost/version.map | 3 + lib/vhost/vhost.c | 130 +++++++++++----- lib/vhost/vhost.h | 53 ++++++- lib/vhost/virtio_net.c | 206 +++++++++++++++++++------ 13 files changed, 587 insertions(+), 529 deletions(-) delete mode 100644 examples/vhost/ioat.c delete mode 100644 examples/vhost/ioat.h -- 2.25.1