From: Xiao Wang <xiao.w.wang@intel.com>
To: dev@dpdk.org
Cc: jianfeng.tan@intel.com, tiwei.bie@intel.com,
maxime.coquelin@redhat.com, yliu@fridaylinux.org,
cunming.liang@intel.com, dan.daly@intel.com,
zhihong.wang@intel.com, Xiao Wang <xiao.w.wang@intel.com>
Subject: [dpdk-dev] [PATCH 0/3] add vDPA sample driver
Date: Sun, 4 Feb 2018 22:55:39 +0800 [thread overview]
Message-ID: <20180204145542.38345-1-xiao.w.wang@intel.com> (raw)
This patch set has dependency on the vhost lib patch:
http://dpdk.org/dev/patchwork/patch/34872/
This patch set shows a reference sample of making vDPA device driver
, this driver uses a QEMU-emulated virtio-net PCI device as vDPA device,
and make it serve as a backend for a virtio-net pci device in nested VM.
The key driver ops implemented are:
* vdpa_virtio_eng_init
Mapping virtio pci device with VFIO into userspace, and read device
capability and intialize internal data.
* vdpa_virtio_eng_uninit
Release the mapped device.
* vdpa_virtio_info_query
Device capability reporting, e.g. queue number, features.
* vdpa_virtio_dev_config
With the guest virtio information provideed by vhost lib, this
function configures device and IOMMU to set up vhost datapath,
which includes: Rx/Tx vring, VFIO interrupt, kick relay.
* vdpa_virtio_dev_close
Unset the stuff that are configured previously by dev_conf.
This driver requires the virtio device supports VIRTIO_F_IOMMU_PLATFORM
, because the buffer address written in desc is IOVA.
Because vDPA driver needs to set up MSI-X vector to interrupt the guest,
only vfio-pci is supported currently.
Below are setup steps for your reference:
1. Make sure your kernnel vhost module and QEMU support vIOMMU.
- OS: CentOS 7.4
- QEMU: 2.10.1
- Guest OS: CentOS 7.2
- Nested VM OS: CentOS 7.2
2. enable VT-x feature for vCPU in VM.
modprobe kvm_intel nested=1
3. Start a VM with a virtio-net-pci device.
./qemu-2.10.1/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cpu host \
<snip>
-machine q35 \
-device intel-iommu \
-netdev tap,id=mytap,ifname=vdpa,vhostforce=on \
-device virtio-net-pci,netdev=mytap,mac=00:aa:bb:cc:dd:ee,\
disable-modern=off,disable-legacy=on,iommu_platform=on \
4. Bind VFIO-pci to virtio_net_pci device
a) login to VM;
b) modprobe vfio-pci
c) rmmod vfio_iommu_type1
d) modprobe vfio_iommu_type1 allow_unsafe_interrupts=1
e) ./usertools/dpdk-devbind.py -b vfio-pci 00:03.0
5. Start vdpa sample
./examples/vdpa/build/vdpa -c 0x2 -n 4 --socket-mem 1024 --no-pci \
--vdev "net_vdpa_virtio_pci0,bdf=0000:00:03.0" -- --bdf 0000:00:03.0 \
--iface /tmp/vhost-user- --devcnt 1 --queue 1
6. Start nested VM
./qemu-2.10.1/x86_64-softmmu/qemu-system-x86_64 -cpu host -enable-kvm \
<snip>
-mem-prealloc \
-chardev socket,id=char0,path=/tmp/vhost-user-0 \
-netdev type=vhost-user,id=vdpa,chardev=char0,vhostforce \
-device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \
7. Login the nested VM, and verify the virtio in nested VM can communicate
with tap device on host.
Xiao Wang (3):
bus/pci: expose API for vDPA
net/vdpa_virtio_pci: introduce vdpa sample driver
examples/vdpa: add a new sample for vdpa
config/common_base | 6 +
config/common_linuxapp | 1 +
drivers/bus/pci/Makefile | 1 +
drivers/bus/pci/linux/pci.c | 10 +-
drivers/bus/pci/linux/pci_init.h | 22 +-
drivers/bus/pci/linux/pci_vfio.c | 5 +-
drivers/bus/pci/rte_bus_pci_version.map | 13 +
drivers/net/Makefile | 1 +
drivers/net/vdpa_virtio_pci/Makefile | 31 +
.../net/vdpa_virtio_pci/rte_eth_vdpa_virtio_pci.c | 1527 ++++++++++++++++++++
.../rte_vdpa_virtio_pci_version.map | 4 +
examples/vdpa/Makefile | 32 +
examples/vdpa/main.c | 387 +++++
mk/rte.app.mk | 1 +
14 files changed, 2030 insertions(+), 11 deletions(-)
create mode 100644 drivers/net/vdpa_virtio_pci/Makefile
create mode 100644 drivers/net/vdpa_virtio_pci/rte_eth_vdpa_virtio_pci.c
create mode 100644 drivers/net/vdpa_virtio_pci/rte_vdpa_virtio_pci_version.map
create mode 100644 examples/vdpa/Makefile
create mode 100644 examples/vdpa/main.c
--
2.15.1
next reply other threads:[~2018-02-04 14:56 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-04 14:55 Xiao Wang [this message]
2018-02-04 14:55 ` [dpdk-dev] [PATCH 1/3] bus/pci: expose API for vDPA Xiao Wang
2018-02-04 14:55 ` [dpdk-dev] [PATCH 2/3] net/vdpa_virtio_pci: introduce vdpa sample driver Xiao Wang
2018-02-06 14:24 ` Maxime Coquelin
2018-02-08 2:23 ` Wang, Xiao W
2018-02-08 9:08 ` Maxime Coquelin
2018-02-12 15:36 ` Wang, Xiao W
2018-02-04 14:55 ` [dpdk-dev] [PATCH 3/3] examples/vdpa: add a new sample for vdpa Xiao Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180204145542.38345-1-xiao.w.wang@intel.com \
--to=xiao.w.wang@intel.com \
--cc=cunming.liang@intel.com \
--cc=dan.daly@intel.com \
--cc=dev@dpdk.org \
--cc=jianfeng.tan@intel.com \
--cc=maxime.coquelin@redhat.com \
--cc=tiwei.bie@intel.com \
--cc=yliu@fridaylinux.org \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).