From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B81D1A0613 for ; Thu, 26 Sep 2019 09:44:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 410A61BEEF; Thu, 26 Sep 2019 09:44:04 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id D70E31BEE6 for ; Thu, 26 Sep 2019 09:44:01 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Sep 2019 00:44:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,551,1559545200"; d="scan'208";a="390505726" Received: from npg_dpdk_virtio_jiayuhu_07.sh.intel.com ([10.67.119.35]) by fmsmga006.fm.intel.com with ESMTP; 26 Sep 2019 00:44:00 -0700 From: Jiayu Hu To: dev@dpdk.org Cc: tiwei.bie@intel.com, maxime.coquelin@redhat.com, zhihong.wang@intel.com, Jiayu Hu Date: Thu, 26 Sep 2019 10:26:11 -0400 Message-Id: <1569507973-247570-1-git-send-email-jiayu.hu@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [dpdk-dev] [RFC 0/2] Add a PMD for I/OAT accelerated vhost-user X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In vhost-user enqueue and dequeue operations, where data movement is heavily involved, performing large memory copies usually takes up a major part of CPU cycles and becomes the hot spot. To offload expensive memory operations from the CPU, this patch set proposes to leverage I/OAT, a DMA engine in the Intel's processor, to accelerate large copies for vhost-user. We implement a new PMD for the I/OAT accelerated vhost-user, called vhost-ioat. This PMD leverages librte_vhost to handle vhost messages, but implements own vring's enqueue and dequeue operations. It offloads large memory copies to the I/OAT in a synchronous mode; that is, the CPU just submits copy jobs to the I/OAT but without waiting for its completion. Thus, there is no CPU intervention during data transfer; we can save precious CPU cycles and improve vhost performance. The PMD provides basic functionality of packet reception and transmission. During packet reception and transmission, it offloads large copies to the I/OAT and performs small copies by the CPU, due to startup overheads associated with the I/OAT. However, the PMD just supports I/OAT acceleration in the PMD's transmit data path (i.e. vring's enqueue operation); it still uses the CPU to perform all copies in the PMD's receive data path (i.e. vring's dequeue operation) currently. Note that the PMD just supports split ring. Users can explicitly assign an I/OAT device to a TX queue by the parameter 'ioats'. But currently, one I/OAT device can only be used by one queue and a queue can use one I/OAT device at a time. In addition, the PMD supports multiqueue and both client and server modes. Users can specify the queue number and client/server mode by 'queues' and 'client' parameters. Jiayu Hu (2): vhost: populate guest memory for DMA-accelerated vhost-user net/vhost_ioat: add vhost I/OAT driver config/common_base | 2 + config/common_linux | 1 + drivers/Makefile | 2 +- drivers/net/Makefile | 1 + drivers/net/vhost_ioat/Makefile | 31 + drivers/net/vhost_ioat/eth_vhost.c | 1439 +++++++++++++++++++++++++++++++++++ drivers/net/vhost_ioat/eth_vhost.h | 255 +++++++ drivers/net/vhost_ioat/internal.h | 225 ++++++ drivers/net/vhost_ioat/virtio_net.c | 1243 ++++++++++++++++++++++++++++++ lib/librte_vhost/rte_vhost.h | 1 + lib/librte_vhost/socket.c | 10 + lib/librte_vhost/vhost.h | 2 + lib/librte_vhost/vhost_user.c | 3 +- mk/rte.app.mk | 1 + 14 files changed, 3214 insertions(+), 2 deletions(-) create mode 100644 drivers/net/vhost_ioat/Makefile create mode 100644 drivers/net/vhost_ioat/eth_vhost.c create mode 100644 drivers/net/vhost_ioat/eth_vhost.h create mode 100644 drivers/net/vhost_ioat/internal.h create mode 100644 drivers/net/vhost_ioat/virtio_net.c -- 2.7.4