[dpdk-dev] [RFC 0/2] Add a PMD for I/OAT accelerated vhost-user

[Jiayu Hu <jiayu.hu@intel.com>]

In vhost-user enqueue and dequeue operations, where data movement is
heavily involved, performing large memory copies usually takes up a
major part of CPU cycles and becomes the hot spot. To offload expensive
memory operations from the CPU, this patch set proposes to leverage I/OAT,
a DMA engine in the Intel's processor, to accelerate large copies for
vhost-user.

We implement a new PMD for the I/OAT accelerated vhost-user, called
vhost-ioat. This PMD leverages librte_vhost to handle vhost messages,
but implements own vring's enqueue and dequeue operations. It offloads
large memory copies to the I/OAT in a synchronous mode; that is, the CPU
just submits copy jobs to the I/OAT but without waiting for its
completion. Thus, there is no CPU intervention during data transfer;
we can save precious CPU cycles and improve vhost performance.

The PMD provides basic functionality of packet reception and
transmission. During packet reception and transmission, it offloads
large copies to the I/OAT and performs small copies by the CPU, due to
startup overheads associated with the I/OAT. However, the PMD just
supports I/OAT acceleration in the PMD's transmit data path (i.e. vring's
enqueue operation); it still uses the CPU to perform all copies in the
PMD's receive data path (i.e. vring's dequeue operation) currently. Note
that the PMD just supports split ring.

Users can explicitly assign an I/OAT device to a TX queue by the
parameter 'ioats'. But currently, one I/OAT device can only be used by
one queue and a queue can use one I/OAT device at a time. In addition,
the PMD supports multiqueue and both client and server modes. Users can
specify the queue number and client/server mode by 'queues' and 'client'
parameters.

Jiayu Hu (2):
  vhost: populate guest memory for DMA-accelerated vhost-user
  net/vhost_ioat: add vhost I/OAT driver

 config/common_base                  |    2 +
 config/common_linux                 |    1 +
 drivers/Makefile                    |    2 +-
 drivers/net/Makefile                |    1 +
 drivers/net/vhost_ioat/Makefile     |   31 +
 drivers/net/vhost_ioat/eth_vhost.c  | 1439 +++++++++++++++++++++++++++++++++++
 drivers/net/vhost_ioat/eth_vhost.h  |  255 +++++++
 drivers/net/vhost_ioat/internal.h   |  225 ++++++
 drivers/net/vhost_ioat/virtio_net.c | 1243 ++++++++++++++++++++++++++++++
 lib/librte_vhost/rte_vhost.h        |    1 +
 lib/librte_vhost/socket.c           |   10 +
 lib/librte_vhost/vhost.h            |    2 +
 lib/librte_vhost/vhost_user.c       |    3 +-
 mk/rte.app.mk                       |    1 +
 14 files changed, 3214 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/vhost_ioat/Makefile
 create mode 100644 drivers/net/vhost_ioat/eth_vhost.c
 create mode 100644 drivers/net/vhost_ioat/eth_vhost.h
 create mode 100644 drivers/net/vhost_ioat/internal.h
 create mode 100644 drivers/net/vhost_ioat/virtio_net.c

-- 
2.7.4