From: "Yao, Lei A" <lei.a.yao@intel.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "yuanhan.liu@linux.intel.com" <yuanhan.liu@linux.intel.com>,
"stephen@networkplumber.org" <stephen@networkplumber.org>
Subject: Re: [dpdk-dev] [PATCH v3 00/10] rxq interrupt mode for virtio PMD
Date: Tue, 17 Jan 2017 02:14:51 +0000 [thread overview]
Message-ID: <2DBBFF226F7CF64BAFCA79B681719D953A14EC5C@shsmsx102.ccr.corp.intel.com> (raw)
In-Reply-To: <1484578022-92705-1-git-send-email-jianfeng.tan@intel.com>
Tested-by: Lei Yao <lei.a.yao@intel.com>
Apply patch to dpdk_next_virtio branch.
Qemu version: 2.5.0
Kernel version in VM: 4.8.1
Following TCs are tested and passed:
Test Case1: Basic Virtio Interrupt test
Test Case2: Interrupted received in VM with different Virtio version(0.95 and 1.0)
Test Case3: Interrupted by packet data on all queue
Test Case4: Interrupted by packet data on unique queue
Test Case5: Stop packet transmit, related cores will be back to sleep
> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Monday, January 16, 2017 10:47 PM
> To: dev@dpdk.org
> Cc: yuanhan.liu@linux.intel.com; stephen@networkplumber.org; Yao, Lei A
> <lei.a.yao@intel.com>; Tan, Jianfeng <jianfeng.tan@intel.com>
> Subject: [PATCH v3 00/10] rxq interrupt mode for virtio PMD
>
> v3:
> - Update documents:
> * doc/guides/nics/features/virtio.ini
> * doc/guides/nics/features/virtio_vec.ini
> * doc/guides/nics/virtio.rst
> - Use hw->max_queue_pairs instead of dev->data->nb_rx_queues to
> allocate intr_vec array.
> - Fix v2 not working on legacy virtio devices by moving msix enabling
> before queue/irq binding.
> - Reword cover letter to give an overview of this series.
> - Remove wrapper to call vtpci->set_config_irq and vtpci->set_queue_irq.
> - Rebase on the new code, and fix a bug after changes by the commit
> bb30369dc10("eal: allow passing const interrupt handle"). Basically,
> it changes the way to get max interrupts. And we need to re-register
> callback to update intr_handle->max_intr.
> - In l3fwd-power ptype fix, use rte_eth_dev_get_supported_ptypes() to
> query if PMD provides needed ptypes.
>
> v2:
> - Add PCI queue/irq config ops.
> - Move rxq interrupt settings before sending DRIVER OK.
>
> Historically, virtio PMD can only be binded to igb_uio or
> uio_pci_generic, and not for vfio-pci (iommu group cannot be created as
> vIOMMU is not enabled in QEMU yet). Besides, quote from
> http://dpdk.org/doc/guides-16.11/rel_notes/release_2_1.html:
> "Per queue RX interrupt events are only allowed in VFIO
> which supports multiple MSI-X vectors."
>
> Linux starts to support VFIO NO-IOMMU mode since 4.8.0. It cannot put
> devices into groups for separation as normal VFIO does. So it does not
> require QEMU to support vIOMMU. But it does inherit other benefits from
> VFIO framework, like better interrupts supports (than UIO). It gives a
> good chance to enable rxq interrupt for virtio PMD.
>
> To implement it,
> a. Firstly, we need to enable msix. This should be done before DRIVER_OK
> setting and also before queue/irq binding in step b.
> b. Bind queue/irq through portio (legacy devices) or mmio (modern devices).
> So far, we hard-code 1:1 queue/irq mapping (each rx queue has one
> exclusive interrupt), like this:
> vec 0 -> config irq
> vec 1 -> rxq0
> vec 2 -> rxq1
> ...
>
> which means, the "vectors" option of QEMU should be configured with
> a value >= N+1 (N is the number of the queue pairs).
> c. To enable/disable interrupt notification, flags on virtqueues are used
> to control devices either sending interrupt or not.
> d. Encap above behaviors into callbacks in ether_dev_ops, like
> rx_queue_intr_enable/rx_queue_intr_disable/rx_descriptor_done etc.
>
>
> How to test:
>
> Step 1, prepare a VM image with kernel version >= 4.8.0, and make sure
> the kernel is compiled with CONFIG_VFIO_NOIOMMU=y.
>
> Step 2, on the host, start a testpmd with a vhost port:
> $ testpmd -c 0x7 -m 1024 --vdev 'eth_vhost0,iface=/tmp/sock0,queues=2' \
> --no-pci -- -i --rxq=2 --txq=2 --nb-cores=2
>
> Step 3, boot the VM:
> $ qemu ... -chardev socket,id=chr1,path=/tmp/sock0 \
> -netdev vhost-user,id=net1,chardev=chr1,vhostforce,queues=2 \
> -device virtio-net-pci,netdev=net1,mq=on,vectors=5 ...
>
> Step 4, insert kernel modules
> $ modprobe vfio enable_unsafe_noiommu_mode=1
> $ modprobe vfio-pci
>
> Step 5, start l3fwd-power in VM:
> $ l3fwd-power -c 0x3 -n 4 -- -p 1 -P --config="(0,0,1),(0,1,1)" \
> --no-numa --parse-ptype
>
> Step 6, send packets from testpmd on the host:
> $ start tx_first
>
> Then l3fwd-power outputs:
> L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 0
> L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 1
>
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>
> Jianfeng Tan (10):
> net/virtio: fix rewriting LSC flag
> net/virtio: clean up wrapper of set_config_irq
> net/virtio: add Rx descriptor check
> net/virtio: add PCI ops for queue/irq binding
> net/virtio: add Rx queue intr enable/disable functions
> net/virtio: setup rxq interrupts
> net/virtio: unbind intr/eventfd when stop device
> net/virtio: unmapping queue/irq when close device
> examples/l3fwd-power: add parse-ptype option
> examples/l3fwd-power: fix not stop and close device
>
> drivers/net/virtio/virtio_ethdev.c | 161
> +++++++++++++++++++++++++++++++++++--
> drivers/net/virtio/virtio_ethdev.h | 3 +
> drivers/net/virtio/virtio_pci.c | 31 +++++--
> drivers/net/virtio/virtio_pci.h | 5 +-
> drivers/net/virtio/virtio_rxtx.c | 9 +++
> drivers/net/virtio/virtqueue.c | 11 ---
> drivers/net/virtio/virtqueue.h | 16 +++-
> examples/l3fwd-power/main.c | 120 ++++++++++++++++++++++++++-
> 8 files changed, 325 insertions(+), 31 deletions(-)
>
> --
> 2.7.4
next prev parent reply other threads:[~2017-01-17 2:14 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-04 0:18 [dpdk-dev] [PATCH 0/5] Interrupt " Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 1/5] net/virtio: add Rx descriptor check Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 2/5] net/virtio: setup Rx fastpath interrupts Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 3/5] net/virtio: remove Rx queue interrupts when stopping Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 4/5] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2016-12-04 0:18 ` [dpdk-dev] [PATCH 5/5] examples/l3fwd: add parse-ptype option Jianfeng Tan
2016-12-08 6:59 ` Yuanhan Liu
2016-12-08 7:40 ` Tan, Jianfeng
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 0/9] rxq interrupt mode for virtio PMD Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 1/9] net/virtio: fix rewriting LSC flag Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 2/9] net/virtio: add Rx descriptor check Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 3/9] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2016-12-30 5:46 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 4/9] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2016-12-30 5:59 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 5/9] net/virtio: setup rxq interrupts Jianfeng Tan
2016-12-30 6:27 ` Yuanhan Liu
2017-01-04 6:56 ` Tan, Jianfeng
2017-01-04 7:22 ` Yuanhan Liu
2017-01-04 7:30 ` Tan, Jianfeng
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 6/9] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 7/9] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2016-12-30 6:30 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 8/9] examples/l3fwd: add parse-ptype option Jianfeng Tan
2016-12-30 6:39 ` Yuanhan Liu
2016-12-30 7:30 ` Tan, Jianfeng
2017-01-03 6:59 ` Yuanhan Liu
2016-12-29 7:30 ` [dpdk-dev] [PATCH v2 9/9] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2016-12-30 6:44 ` Yuanhan Liu
2016-12-30 6:56 ` Tan, Jianfeng
2016-12-29 7:42 ` [dpdk-dev] [PATCH v2 0/9] rxq interrupt mode for virtio PMD Tan, Jianfeng
2016-12-30 5:41 ` Yuanhan Liu
2016-12-30 6:57 ` Yuanhan Liu
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 00/10] " Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 01/10] net/virtio: fix rewriting LSC flag Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 02/10] net/virtio: clean up wrapper of set_config_irq Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 03/10] net/virtio: add Rx descriptor check Jianfeng Tan
2017-01-16 19:16 ` Stephen Hemminger
2017-01-17 4:09 ` Yuanhan Liu
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 04/10] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 05/10] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2017-01-17 2:30 ` Jason Wang
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
2017-01-16 14:46 ` [dpdk-dev] [PATCH v3 07/10] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 08/10] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 09/10] examples/l3fwd-power: add parse-ptype option Jianfeng Tan
2017-01-17 5:16 ` Yuanhan Liu
2017-01-17 5:23 ` Tan, Jianfeng
2017-01-16 14:47 ` [dpdk-dev] [PATCH v3 10/10] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2017-01-17 2:14 ` Yao, Lei A [this message]
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 00/10] rxq interrupt mode for virtio PMD Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 01/10] net/virtio: fix rewriting LSC flag Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 02/10] net/virtio: clean up wrapper of set_config_irq Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 03/10] net/virtio: add Rx descriptor check Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 04/10] net/virtio: add PCI ops for queue/irq binding Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 05/10] net/virtio: add Rx queue intr enable/disable functions Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 07/10] net/virtio: unbind intr/eventfd when stop device Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 08/10] net/virtio: unmapping queue/irq when close device Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 09/10] examples/l3fwd-power: add parse-ptype option Jianfeng Tan
2017-01-17 7:10 ` [dpdk-dev] [PATCH v4 10/10] examples/l3fwd-power: fix not stop and close device Jianfeng Tan
2017-01-17 8:28 ` [dpdk-dev] [PATCH v4 00/10] rxq interrupt mode for virtio PMD Yuanhan Liu
2017-01-17 8:00 ` [dpdk-dev] [PATCH v5 06/10] net/virtio: setup rxq interrupts Jianfeng Tan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2DBBFF226F7CF64BAFCA79B681719D953A14EC5C@shsmsx102.ccr.corp.intel.com \
--to=lei.a.yao@intel.com \
--cc=dev@dpdk.org \
--cc=jianfeng.tan@intel.com \
--cc=stephen@networkplumber.org \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).