From: Jianfeng Tan <jianfeng.tan@intel.com>
To: dev@dpdk.org
Cc: yuanhan.liu@linux.intel.com, stephen@networkplumber.org,
lei.a.yao@intel.com, Jianfeng Tan <jianfeng.tan@intel.com>
Subject: [dpdk-dev] [PATCH v4 06/10] net/virtio: setup rxq interrupts
Date: Tue, 17 Jan 2017 07:10:26 +0000 [thread overview]
Message-ID: <1484637030-106261-7-git-send-email-jianfeng.tan@intel.com> (raw)
In-Reply-To: <1484637030-106261-1-git-send-email-jianfeng.tan@intel.com>
This patch mainly allocates structure to store queue/irq mapping,
and configure queue/irq mapping down through PCI ops. It also creates
eventfds for each Rx queue and tell the kernel about the eventfd/intr
binding.
Note: So far, we hard-code 1:1 queue/irq mapping (each rx queue has
one exclusive interrupt), like this:
vec 0 -> config irq
vec 1 -> rxq0
vec 2 -> rxq1
...
which means, the "vectors" option of QEMU should be configured with
a value >= N+1 (N is the number of the queue pairs).
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Tested-by: Lei Yao <lei.a.yao@intel.com>
---
doc/guides/nics/features/virtio.ini | 1 +
doc/guides/nics/features/virtio_vec.ini | 1 +
doc/guides/nics/virtio.rst | 58 +++++++++++++++++++++++
drivers/net/virtio/virtio_ethdev.c | 84 +++++++++++++++++++++++++++++++++
4 files changed, 144 insertions(+)
diff --git a/doc/guides/nics/features/virtio.ini b/doc/guides/nics/features/virtio.ini
index f5de291..7f038c5 100644
--- a/doc/guides/nics/features/virtio.ini
+++ b/doc/guides/nics/features/virtio.ini
@@ -5,6 +5,7 @@
;
[Features]
Link status = Y
+Rx interrupt = Y
Queue start/stop = Y
Scattered Rx = Y
Promiscuous mode = Y
diff --git a/doc/guides/nics/features/virtio_vec.ini b/doc/guides/nics/features/virtio_vec.ini
index 6dc7cf0..ec93f5c 100644
--- a/doc/guides/nics/features/virtio_vec.ini
+++ b/doc/guides/nics/features/virtio_vec.ini
@@ -5,6 +5,7 @@
;
[Features]
Link status = Y
+Rx interrupt = Y
Queue start/stop = Y
Promiscuous mode = Y
Allmulticast mode = Y
diff --git a/doc/guides/nics/virtio.rst b/doc/guides/nics/virtio.rst
index c90e517..42600f3 100644
--- a/doc/guides/nics/virtio.rst
+++ b/doc/guides/nics/virtio.rst
@@ -87,6 +87,8 @@ In this release, the virtio PMD driver provides the basic functionality of packe
* Virtio supports Link State interrupt.
+* Virtio supports Rx interrupt (so far, only support 1:1 mapping for queue/interrupt).
+
* Virtio supports software vlan stripping and inserting.
* Virtio supports using port IO to get PCI resource when uio/igb_uio module is not available.
@@ -274,3 +276,59 @@ Example of using the vector version of the virtio poll mode driver in
``testpmd``::
testpmd -c 0x7 -n 4 -- -i --txqflags=0xF01 --rxq=1 --txq=1 --nb-cores=1
+
+
+Interrupt mode
+--------------
+
+There are three kinds of interrupts from a virtio device over PCI bus: config
+interrupt, Rx interrupts, and Tx interrupts. Config interrupt is used for
+notification of device configuration changes, especially link status (lsc).
+Interrupt mode is translated into Rx interrupts in the context of DPDK.
+
+Prerequisites for Rx interrupts
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To support Rx interrupts,
+#. Check if guest kernel supports VFIO-NOIOMMU:
+
+ Linux started to support VFIO-NOIOMMU since 4.8.0. Make sure the guest
+ kernel is compiled with:
+
+ .. code-block:: console
+
+ CONFIG_VFIO_NOIOMMU=y
+
+#. Properly set msix vectors when starting VM:
+
+ Enable multi-queue when starting VM, and specify msix vectors in qemu
+ cmdline. (N+1) is the minimum, and (2N+2) is mostly recommended.
+
+ .. code-block:: console
+
+ $(QEMU) ... -device virtio-net-pci,mq=on,vectors=2N+2 ...
+
+#. In VM, insert vfio module in NOIOMMU mode:
+
+ .. code-block:: console
+
+ modprobe vfio enable_unsafe_noiommu_mode=1
+ modprobe vfio-pci
+
+#. In VM, bind the virtio device with vfio-pci:
+
+ .. code-block:: console
+
+ python tools/dpdk-devbind.py -b vfio-pci 00:03.0
+
+Example
+~~~~~~~
+
+Here we use l3fwd-power as an example to show how to get started.
+
+ Example:
+
+ .. code-block:: console
+
+ $ l3fwd-power -c 0x3 -- -p 1 -P --config="(0,0,1)" \
+ --no-numa --parse-ptype
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 4224d17..e870125 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -1210,6 +1210,82 @@ rx_func_get(struct rte_eth_dev *eth_dev)
eth_dev->rx_pkt_burst = &virtio_recv_pkts;
}
+/* Only support 1:1 queue/interrupt mapping so far.
+ * TODO: support n:1 queue/interrupt mapping when there are limited number of
+ * interrupt vectors (<N+1).
+ */
+static int
+virtio_queues_bind_intr(struct rte_eth_dev *dev)
+{
+ uint32_t i;
+ struct virtio_hw *hw = dev->data->dev_private;
+
+ PMD_INIT_LOG(INFO, "queue/interrupt binding\n");
+ for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ dev->intr_handle->intr_vec[i] = i + 1;
+ if (VTPCI_OPS(hw)->set_queue_irq(hw, hw->vqs[i * 2], i + 1) ==
+ VIRTIO_MSI_NO_VECTOR) {
+ PMD_DRV_LOG(ERR, "failed to set queue vector");
+ return -EBUSY;
+ }
+ }
+
+ return 0;
+}
+
+static int
+virtio_configure_intr(struct rte_eth_dev *dev)
+{
+ struct virtio_hw *hw = dev->data->dev_private;
+
+ if (!rte_intr_cap_multiple(dev->intr_handle)) {
+ PMD_INIT_LOG(ERR, "Multiple intr vector not supported");
+ return -ENOTSUP;
+ }
+
+ if (rte_intr_efd_enable(dev->intr_handle, dev->data->nb_rx_queues)) {
+ PMD_INIT_LOG(ERR, "Fail to create eventfd");
+ return -1;
+ }
+
+ if (!dev->intr_handle->intr_vec) {
+ dev->intr_handle->intr_vec =
+ rte_zmalloc("intr_vec",
+ hw->max_queue_pairs * sizeof(int), 0);
+ if (!dev->intr_handle->intr_vec) {
+ PMD_INIT_LOG(ERR, "Failed to allocate %u rxq vectors",
+ hw->max_queue_pairs);
+ return -ENOMEM;
+ }
+ }
+
+ /* Re-register callback to update max_intr */
+ rte_intr_callback_unregister(dev->intr_handle,
+ virtio_interrupt_handler,
+ dev);
+ rte_intr_callback_register(dev->intr_handle,
+ virtio_interrupt_handler,
+ dev);
+
+ /* DO NOT try to remove this! This function will enable msix, or QEMU
+ * will encounter SIGSEGV when DRIVER_OK is sent.
+ * And for legacy devices, this should be done before queue/vec binding
+ * to change the config size from 20 to 24, or VIRTIO_MSI_QUEUE_VECTOR
+ * (22) will be ignored.
+ */
+ if (rte_intr_enable(dev->intr_handle) < 0) {
+ PMD_DRV_LOG(ERR, "interrupt enable failed");
+ return -1;
+ }
+
+ if (virtio_queues_bind_intr(dev) < 0) {
+ PMD_INIT_LOG(ERR, "Failed to bind queue/interrupt");
+ return -1;
+ }
+
+ return 0;
+}
+
/* reset device and renegotiate features if needed */
static int
virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
@@ -1306,6 +1382,14 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features)
ret = virtio_alloc_queues(eth_dev);
if (ret < 0)
return ret;
+
+ if (eth_dev->data->dev_conf.intr_conf.rxq) {
+ if (virtio_configure_intr(eth_dev) < 0) {
+ PMD_INIT_LOG(ERR, "failed to configure interrupt");
+ return -1;
+ }
+ }
+
vtpci_reinit_complete(hw);
if (pci_dev)
--
2.7.4
next prev parent reply other threads:[~2017-01-17 7:09 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-04 0:18 [dpdk-dev] [PATCH 0/5] Interrupt mode for virtio PMD Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 1/5] net/virtio: add Rx descriptor check Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 2/5] net/virtio: setup Rx fastpath interrupts Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 3/5] net/virtio: remove Rx queue interrupts when stopping Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 4/5] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 5/5] examples/l3fwd: add parse-ptype option Jianfeng Tan
2016-12-08 6:59 ` Yuanhan Liu
2016-12-08 7:40 ` Tan, Jianfeng
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 0/9] rxq interrupt mode for virtio PMD Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 1/9] net/virtio: fix rewriting LSC flag Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 2/9] net/virtio: add Rx descriptor check Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 3/9] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2016-12-30 5:46 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 4/9] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2016-12-30 5:59 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 5/9] net/virtio: setup rxq interrupts Jianfeng Tan
2016-12-30 6:27 ` Yuanhan Liu
2017-01-04 6:56 ` Tan, Jianfeng
2017-01-04 7:22 ` Yuanhan Liu
2017-01-04 7:30 ` Tan, Jianfeng
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 6/9] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 7/9] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2016-12-30 6:30 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 8/9] examples/l3fwd: add parse-ptype option Jianfeng Tan
2016-12-30 6:39 ` Yuanhan Liu
2016-12-30 7:30 ` Tan, Jianfeng
2017-01-03 6:59 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 9/9] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2016-12-30 6:44 ` Yuanhan Liu
2016-12-30 6:56 ` Tan, Jianfeng
2016-12-29 7:42 ` [dpdk-dev] [PATCH v2 0/9] rxq interrupt mode for virtio PMD Tan, Jianfeng
2016-12-30 5:41 ` Yuanhan Liu
2016-12-30 6:57 ` Yuanhan Liu
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 00/10] " Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 01/10] net/virtio: fix rewriting LSC flag Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 02/10] net/virtio: clean up wrapper of set_config_irq Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 03/10] net/virtio: add Rx descriptor check Jianfeng Tan
2017-01-16 19:16 ` Stephen Hemminger
2017-01-17 4:09 ` Yuanhan Liu
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 04/10] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 05/10] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2017-01-17 2:30 ` Jason Wang
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 07/10] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 08/10] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 09/10] examples/l3fwd-power: add parse-ptype option Jianfeng Tan
2017-01-17 5:16 ` Yuanhan Liu
2017-01-17 5:23 ` Tan, Jianfeng
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 10/10] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2017-01-17 2:14 ` [dpdk-dev] [PATCH v3 00/10] rxq interrupt mode for virtio PMD Yao, Lei A
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 " Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 01/10] net/virtio: fix rewriting LSC flag Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 02/10] net/virtio: clean up wrapper of set_config_irq Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 03/10] net/virtio: add Rx descriptor check Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 04/10] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 05/10] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2017-01-17 7:10 ` Jianfeng Tan [this message]
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 07/10] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 08/10] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 09/10] examples/l3fwd-power: add parse-ptype option Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 10/10] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2017-01-17 8:28 ` [dpdk-dev] [PATCH v4 00/10] rxq interrupt mode for virtio PMD Yuanhan Liu
2017-01-17 8:00 ` [dpdk-dev] [PATCH v5 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1484637030-106261-7-git-send-email-jianfeng.tan@intel.com \
--to=jianfeng.tan@intel.com \
--cc=dev@dpdk.org \
--cc=lei.a.yao@intel.com \
--cc=stephen@networkplumber.org \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).