From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id AAF983238 for ; Tue, 17 Jan 2017 03:14:55 +0100 (CET) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga104.fm.intel.com with ESMTP; 16 Jan 2017 18:14:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,242,1477983600"; d="scan'208";a="214119771" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga004.fm.intel.com with ESMTP; 16 Jan 2017 18:14:54 -0800 Received: from fmsmsx122.amr.corp.intel.com (10.18.125.37) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 16 Jan 2017 18:14:54 -0800 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx122.amr.corp.intel.com (10.18.125.37) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 16 Jan 2017 18:14:54 -0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.88]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.177]) with mapi id 14.03.0248.002; Tue, 17 Jan 2017 10:14:52 +0800 From: "Yao, Lei A" To: "Tan, Jianfeng" , "dev@dpdk.org" CC: "yuanhan.liu@linux.intel.com" , "stephen@networkplumber.org" Thread-Topic: [PATCH v3 00/10] rxq interrupt mode for virtio PMD Thread-Index: AQHScAdR1Gt16r/q/k2rQ0gAIEdfe6E77A/Q Date: Tue, 17 Jan 2017 02:14:51 +0000 Message-ID: <2DBBFF226F7CF64BAFCA79B681719D953A14EC5C@shsmsx102.ccr.corp.intel.com> References: <1480810702-114815-1-git-send-email-jianfeng.tan@intel.com> <1484578022-92705-1-git-send-email-jianfeng.tan@intel.com> In-Reply-To: <1484578022-92705-1-git-send-email-jianfeng.tan@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3 00/10] rxq interrupt mode for virtio PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jan 2017 02:14:56 -0000 Tested-by: Lei Yao Apply patch to dpdk_next_virtio branch. Qemu version: 2.5.0 Kernel version in VM: 4.8.1=20 Following TCs are tested and passed: Test Case1: Basic Virtio Interrupt test Test Case2: Interrupted received in VM with different Virtio version(0.95 a= nd 1.0) Test Case3: Interrupted by packet data on all queue Test Case4: Interrupted by packet data on unique queue Test Case5: Stop packet transmit, related cores will be back to sleep > -----Original Message----- > From: Tan, Jianfeng > Sent: Monday, January 16, 2017 10:47 PM > To: dev@dpdk.org > Cc: yuanhan.liu@linux.intel.com; stephen@networkplumber.org; Yao, Lei A > ; Tan, Jianfeng > Subject: [PATCH v3 00/10] rxq interrupt mode for virtio PMD >=20 > v3: > - Update documents: > * doc/guides/nics/features/virtio.ini > * doc/guides/nics/features/virtio_vec.ini > * doc/guides/nics/virtio.rst > - Use hw->max_queue_pairs instead of dev->data->nb_rx_queues to > allocate intr_vec array. > - Fix v2 not working on legacy virtio devices by moving msix enabling > before queue/irq binding. > - Reword cover letter to give an overview of this series. > - Remove wrapper to call vtpci->set_config_irq and vtpci->set_queue_irq= . > - Rebase on the new code, and fix a bug after changes by the commit > bb30369dc10("eal: allow passing const interrupt handle"). Basically, > it changes the way to get max interrupts. And we need to re-register > callback to update intr_handle->max_intr. > - In l3fwd-power ptype fix, use rte_eth_dev_get_supported_ptypes() to > query if PMD provides needed ptypes. >=20 > v2: > - Add PCI queue/irq config ops. > - Move rxq interrupt settings before sending DRIVER OK. >=20 > Historically, virtio PMD can only be binded to igb_uio or > uio_pci_generic, and not for vfio-pci (iommu group cannot be created as > vIOMMU is not enabled in QEMU yet). Besides, quote from > http://dpdk.org/doc/guides-16.11/rel_notes/release_2_1.html: > "Per queue RX interrupt events are only allowed in VFIO > which supports multiple MSI-X vectors." >=20 > Linux starts to support VFIO NO-IOMMU mode since 4.8.0. It cannot put > devices into groups for separation as normal VFIO does. So it does not > require QEMU to support vIOMMU. But it does inherit other benefits from > VFIO framework, like better interrupts supports (than UIO). It gives a > good chance to enable rxq interrupt for virtio PMD. >=20 > To implement it, > a. Firstly, we need to enable msix. This should be done before DRIVER_OK > setting and also before queue/irq binding in step b. > b. Bind queue/irq through portio (legacy devices) or mmio (modern devices= ). > So far, we hard-code 1:1 queue/irq mapping (each rx queue has one > exclusive interrupt), like this: > vec 0 -> config irq > vec 1 -> rxq0 > vec 2 -> rxq1 > ... >=20 > which means, the "vectors" option of QEMU should be configured with > a value >=3D N+1 (N is the number of the queue pairs). > c. To enable/disable interrupt notification, flags on virtqueues are used > to control devices either sending interrupt or not. > d. Encap above behaviors into callbacks in ether_dev_ops, like > rx_queue_intr_enable/rx_queue_intr_disable/rx_descriptor_done etc. >=20 >=20 > How to test: >=20 > Step 1, prepare a VM image with kernel version >=3D 4.8.0, and make sure > the kernel is compiled with CONFIG_VFIO_NOIOMMU=3Dy. >=20 > Step 2, on the host, start a testpmd with a vhost port: > $ testpmd -c 0x7 -m 1024 --vdev 'eth_vhost0,iface=3D/tmp/sock0,queues= =3D2' \ > --no-pci -- -i --rxq=3D2 --txq=3D2 --nb-cores=3D2 >=20 > Step 3, boot the VM: > $ qemu ... -chardev socket,id=3Dchr1,path=3D/tmp/sock0 \ > -netdev vhost-user,id=3Dnet1,chardev=3Dchr1,vhostforce,queues=3D2 \ > -device virtio-net-pci,netdev=3Dnet1,mq=3Don,vectors=3D5 ... >=20 > Step 4, insert kernel modules > $ modprobe vfio enable_unsafe_noiommu_mode=3D1 > $ modprobe vfio-pci >=20 > Step 5, start l3fwd-power in VM: > $ l3fwd-power -c 0x3 -n 4 -- -p 1 -P --config=3D"(0,0,1),(0,1,1)" \ > --no-numa --parse-ptype >=20 > Step 6, send packets from testpmd on the host: > $ start tx_first >=20 > Then l3fwd-power outputs: > L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 0 > L3FWD_POWER: lcore 1 is waked up from rx interrupt on port 0 queue 1 >=20 > Signed-off-by: Jianfeng Tan >=20 > Jianfeng Tan (10): > net/virtio: fix rewriting LSC flag > net/virtio: clean up wrapper of set_config_irq > net/virtio: add Rx descriptor check > net/virtio: add PCI ops for queue/irq binding > net/virtio: add Rx queue intr enable/disable functions > net/virtio: setup rxq interrupts > net/virtio: unbind intr/eventfd when stop device > net/virtio: unmapping queue/irq when close device > examples/l3fwd-power: add parse-ptype option > examples/l3fwd-power: fix not stop and close device >=20 > drivers/net/virtio/virtio_ethdev.c | 161 > +++++++++++++++++++++++++++++++++++-- > drivers/net/virtio/virtio_ethdev.h | 3 + > drivers/net/virtio/virtio_pci.c | 31 +++++-- > drivers/net/virtio/virtio_pci.h | 5 +- > drivers/net/virtio/virtio_rxtx.c | 9 +++ > drivers/net/virtio/virtqueue.c | 11 --- > drivers/net/virtio/virtqueue.h | 16 +++- > examples/l3fwd-power/main.c | 120 ++++++++++++++++++++++++++- > 8 files changed, 325 insertions(+), 31 deletions(-) >=20 > -- > 2.7.4