From: Jianfeng Tan <jianfeng.tan@intel.com>
To: dev@dpdk.org
Cc: yuanhan.liu@linux.intel.com, stephen@networkplumber.org,
lei.a.yao@intel.com, Jianfeng Tan <jianfeng.tan@intel.com>
Subject: [dpdk-dev] [PATCH v3 00/10] rxq interrupt mode for virtio PMD
Date: Mon, 16 Jan 2017 14:46:52 +0000 [thread overview]
Message-ID: <1484578022-92705-1-git-send-email-jianfeng.tan@intel.com> (raw)
In-Reply-To: <1480810702-114815-1-git-send-email-jianfeng.tan@intel.com>
v3:
- Update documents:
* doc/guides/nics/features/virtio.ini
* doc/guides/nics/features/virtio_vec.ini
* doc/guides/nics/virtio.rst
- Use hw->max_queue_pairs instead of dev->data->nb_rx_queues to
allocate intr_vec array.
- Fix v2 not working on legacy virtio devices by moving msix enabling
before queue/irq binding.
- Reword cover letter to give an overview of this series.
- Remove wrapper to call vtpci->set_config_irq and vtpci->set_queue_irq.
- Rebase on the new code, and fix a bug after changes by the commit
bb30369dc10("eal: allow passing const interrupt handle"). Basically,
it changes the way to get max interrupts. And we need to re-register
callback to update intr_handle->max_intr.
- In l3fwd-power ptype fix, use rte_eth_dev_get_supported_ptypes() to
query if PMD provides needed ptypes.
v2:
- Add PCI queue/irq config ops.
- Move rxq interrupt settings before sending DRIVER OK.
Historically, virtio PMD can only be binded to igb_uio or
uio_pci_generic, and not for vfio-pci (iommu group cannot be created as
vIOMMU is not enabled in QEMU yet). Besides, quote from
http://dpdk.org/doc/guides-16.11/rel_notes/release_2_1.html:
"Per queue RX interrupt events are only allowed in VFIO
which supports multiple MSI-X vectors."
Linux starts to support VFIO NO-IOMMU mode since 4.8.0. It cannot put
devices into groups for separation as normal VFIO does. So it does not
require QEMU to support vIOMMU. But it does inherit other benefits from
VFIO framework, like better interrupts supports (than UIO). It gives a
good chance to enable rxq interrupt for virtio PMD.
To implement it,
a. Firstly, we need to enable msix. This should be done before DRIVER_OK
setting and also before queue/irq binding in step b.
b. Bind queue/irq through portio (legacy devices) or mmio (modern devices).
So far, we hard-code 1:1 queue/irq mapping (each rx queue has one
exclusive interrupt), like this:
vec 0 -> config irq
vec 1 -> rxq0
vec 2 -> rxq1
...
which means, the "vectors" option of QEMU should be configured with
a value >= N+1 (N is the number of the queue pairs).
c. To enable/disable interrupt notification, flags on virtqueues are used
to control devices either sending interrupt or not.
d. Encap above behaviors into callbacks in ether_dev_ops, like
rx_queue_intr_enable/rx_queue_intr_disable/rx_descriptor_done etc.
How to test:
Step 1, prepare a VM image with kernel version >= 4.8.0, and make sure
the kernel is compiled with CONFIG_VFIO_NOIOMMU=y.
Step 2, on the host, start a testpmd with a vhost port:
$ testpmd -c 0x7 -m 1024 --vdev 'eth_vhost0,iface=/tmp/sock0,queues=2' \
--no-pci -- -i --rxq=2 --txq=2 --nb-cores=2
Step 3, boot the VM:
$ qemu ... -chardev socket,id=chr1,path=/tmp/sock0 \
-netdev vhost-user,id=net1,chardev=chr1,vhostforce,queues=2 \
-device virtio-net-pci,netdev=net1,mq=on,vectors=5 ...
Step 4, insert kernel modules
$ modprobe vfio enable_unsafe_noiommu_mode=1
$ modprobe vfio-pci
Step 5, start l3fwd-power in VM:
$ l3fwd-power -c 0x3 -n 4 -- -p 1 -P --config="(0,0,1),(0,1,1)" \
--no-numa --parse-ptype
Step 6, send packets from testpmd on the host:
$ start tx_first
Then l3fwd-power outputs:
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 0
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 1
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Jianfeng Tan (10):
net/virtio: fix rewriting LSC flag
net/virtio: clean up wrapper of set_config_irq
net/virtio: add Rx descriptor check
net/virtio: add PCI ops for queue/irq binding
net/virtio: add Rx queue intr enable/disable functions
net/virtio: setup rxq interrupts
net/virtio: unbind intr/eventfd when stop device
net/virtio: unmapping queue/irq when close device
examples/l3fwd-power: add parse-ptype option
examples/l3fwd-power: fix not stop and close device
drivers/net/virtio/virtio_ethdev.c | 161 +++++++++++++++++++++++++++++++++++--
drivers/net/virtio/virtio_ethdev.h | 3 +
drivers/net/virtio/virtio_pci.c | 31 +++++--
drivers/net/virtio/virtio_pci.h | 5 +-
drivers/net/virtio/virtio_rxtx.c | 9 +++
drivers/net/virtio/virtqueue.c | 11 ---
drivers/net/virtio/virtqueue.h | 16 +++-
examples/l3fwd-power/main.c | 120 ++++++++++++++++++++++++++-
8 files changed, 325 insertions(+), 31 deletions(-)
--
2.7.4
next prev parent reply other threads:[~2017-01-16 14:46 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-04 0:18 [dpdk-dev] [PATCH 0/5] Interrupt " Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 1/5] net/virtio: add Rx descriptor check Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 2/5] net/virtio: setup Rx fastpath interrupts Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 3/5] net/virtio: remove Rx queue interrupts when stopping Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 4/5] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 5/5] examples/l3fwd: add parse-ptype option Jianfeng Tan
2016-12-08 6:59 ` Yuanhan Liu
2016-12-08 7:40 ` Tan, Jianfeng
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 0/9] rxq interrupt mode for virtio PMD Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 1/9] net/virtio: fix rewriting LSC flag Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 2/9] net/virtio: add Rx descriptor check Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 3/9] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2016-12-30 5:46 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 4/9] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2016-12-30 5:59 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 5/9] net/virtio: setup rxq interrupts Jianfeng Tan
2016-12-30 6:27 ` Yuanhan Liu
2017-01-04 6:56 ` Tan, Jianfeng
2017-01-04 7:22 ` Yuanhan Liu
2017-01-04 7:30 ` Tan, Jianfeng
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 6/9] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 7/9] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2016-12-30 6:30 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 8/9] examples/l3fwd: add parse-ptype option Jianfeng Tan
2016-12-30 6:39 ` Yuanhan Liu
2016-12-30 7:30 ` Tan, Jianfeng
2017-01-03 6:59 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 9/9] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2016-12-30 6:44 ` Yuanhan Liu
2016-12-30 6:56 ` Tan, Jianfeng
2016-12-29 7:42 ` [dpdk-dev] [PATCH v2 0/9] rxq interrupt mode for virtio PMD Tan, Jianfeng
2016-12-30 5:41 ` Yuanhan Liu
2016-12-30 6:57 ` Yuanhan Liu
2017-01-16 14:46 ` Jianfeng Tan [this message]
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 01/10] net/virtio: fix rewriting LSC flag Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 02/10] net/virtio: clean up wrapper of set_config_irq Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 03/10] net/virtio: add Rx descriptor check Jianfeng Tan
2017-01-16 19:16 ` Stephen Hemminger
2017-01-17 4:09 ` Yuanhan Liu
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 04/10] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 05/10] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2017-01-17 2:30 ` Jason Wang
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 07/10] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 08/10] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 09/10] examples/l3fwd-power: add parse-ptype option Jianfeng Tan
2017-01-17 5:16 ` Yuanhan Liu
2017-01-17 5:23 ` Tan, Jianfeng
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 10/10] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2017-01-17 2:14 ` [dpdk-dev] [PATCH v3 00/10] rxq interrupt mode for virtio PMD Yao, Lei A
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 " Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 01/10] net/virtio: fix rewriting LSC flag Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 02/10] net/virtio: clean up wrapper of set_config_irq Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 03/10] net/virtio: add Rx descriptor check Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 04/10] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 05/10] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 07/10] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 08/10] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 09/10] examples/l3fwd-power: add parse-ptype option Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 10/10] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2017-01-17 8:28 ` [dpdk-dev] [PATCH v4 00/10] rxq interrupt mode for virtio PMD Yuanhan Liu
2017-01-17 8:00 ` [dpdk-dev] [PATCH v5 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1484578022-92705-1-git-send-email-jianfeng.tan@intel.com \
--to=jianfeng.tan@intel.com \
--cc=dev@dpdk.org \
--cc=lei.a.yao@intel.com \
--cc=stephen@networkplumber.org \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).