From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id E5FCD2B9C for ; Fri, 30 Dec 2016 07:26:03 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP; 29 Dec 2016 22:26:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,428,1477983600"; d="scan'208";a="1077552921" Received: from yliu-dev.sh.intel.com (HELO yliu-dev) ([10.239.67.162]) by orsmga001.jf.intel.com with ESMTP; 29 Dec 2016 22:26:00 -0800 Date: Fri, 30 Dec 2016 14:27:47 +0800 From: Yuanhan Liu To: Jianfeng Tan Cc: dev@dpdk.org, stephen@networkplumber.org Message-ID: <20161230062747.GI21789@yliu-dev.sh.intel.com> References: <1482996643-113253-1-git-send-email-jianfeng.tan@intel.com> <1482996643-113253-6-git-send-email-jianfeng.tan@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1482996643-113253-6-git-send-email-jianfeng.tan@intel.com> User-Agent: Mutt/1.5.23 (2014-03-12) Subject: Re: [dpdk-dev] [PATCH v2 5/9] net/virtio: setup rxq interrupts X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2016 06:26:04 -0000 On Thu, Dec 29, 2016 at 07:30:39AM +0000, Jianfeng Tan wrote: > This patch mainly allocates structure to store queue/irq mapping, > and configure queue/irq mapping down through PCI ops. It also creates > eventfds for each Rx queue and tell the kernel about the eventfd/intr > binding. > > Mostly importantly, different from previous NICs (usually implements > these logic in dev_start()), virtio's interrupt settings should be > configured down to QEMU before sending DRIVER_OK notification. Isn't it obvious we have to have all driver stuff (including interrupt settings) configured properly before setting DRIVER_OK? :) That said, it's meanless to state the fact that virtio acts differently than other nics here on dev_start/stop. > Note: We only support 1:1 queue/irq mapping so far, which means, each > rx queue has one exclusive interrupt (corresponding to irqfd in the > qemu/kvm) to get notified when packets are available on that queue. That means you have to setup the "vectors=N" option has to set correctly in QEMU, otherwise it won't work? If so, you also have to doc it somewhere. > Signed-off-by: Jianfeng Tan > --- > drivers/net/virtio/virtio_ethdev.c | 89 ++++++++++++++++++++++++++++++++++++++ > 1 file changed, 89 insertions(+) > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > index 3f8b90c..082346b 100644 > --- a/drivers/net/virtio/virtio_ethdev.c > +++ b/drivers/net/virtio/virtio_ethdev.c > @@ -1206,6 +1206,76 @@ rx_func_get(struct rte_eth_dev *eth_dev) > eth_dev->rx_pkt_burst = &virtio_recv_pkts; > } > > +/* Only support 1:1 queue/interrupt mapping so far. > + * TODO: under below cases, lsc and rxq interrupt share one interrupt. > + * a) binded to uio, igb_uio, vfio (type1); > + * b) device only has one vec, see _vectors_ option in -device virtio-net-pci. > + * TODO: support n:1 queue/interrupt mapping. Both TODOs are actually the same: supporting n:1 mapping. That said, you don't have to write 2 TODOs here. Please, don't be mean by adding some whitespace lines. > + */ > +static int > +virtio_queues_bind_intr(struct rte_eth_dev *dev) > +{ > + uint32_t i; > + struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; > + struct virtio_hw *hw = dev->data->dev_private; > + > + PMD_INIT_LOG(INFO, "queue/interrupt binding\n"); > + for (i = 0; i < dev->data->nb_rx_queues; ++i) { > + intr_handle->intr_vec[i] = i + 1; > + if (vtpci_irq_queue(hw->vqs[i * VTNET_CQ], i + 1) == It's logically wrong to use VTNET_CQ to get the Rx queue index. You could either use 2 simply, or define a macro for that. > + VIRTIO_MSI_NO_VECTOR) { > + PMD_DRV_LOG(ERR, "failed to set queue vector"); > + return -EBUSY; > + } > + } > + > + return 0; > +} > + > +static int > +virtio_configure_intr(struct rte_eth_dev *dev) > +{ > + uint32_t intr_vector; > + struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; > + > + /* check if rxq interrupt is enabled */ Unnecessary comment, the function name and the error log explains it well. > + if (!rte_intr_cap_multiple(intr_handle)) { > + PMD_INIT_LOG(ERR, "Multiple intr vector not supported"); > + return -ENOTSUP; > + } > + > + intr_vector = dev->data->nb_rx_queues; > + if (rte_intr_efd_enable(intr_handle, intr_vector)) { > + PMD_INIT_LOG(ERR, "Fail to create eventfd"); > + return -1; > + } > + > + if (!intr_handle->intr_vec) { > + intr_handle->intr_vec = > + rte_zmalloc("intr_vec", intr_vector * sizeof(int), 0); > + if (!intr_handle->intr_vec) { > + PMD_INIT_LOG(ERR, "Failed to allocate %d rxq vectors", > + intr_vector); > + return -ENOMEM; > + } > + } > + > + if (virtio_queues_bind_intr(dev) < 0) { > + PMD_INIT_LOG(ERR, "Failed to bind queue/interrupt"); > + return -1; You have to free intr_handle->intr_vec, otherwise, memory leak occurs. > + } > + > + /* DO NOT try remove this! This function will enable msix, or QEMU > + * will encounter SIGSEGV. > + */ Looks like a QEMU bug to me. I mean, even though the driver is badly configured, it should not crash QEMU. > + if (rte_intr_enable(intr_handle) < 0) { > + PMD_DRV_LOG(ERR, "interrupt enable failed"); > + return -1; > + } > + > + return 0; > +} > + > /* reset device and renegotiate features if needed */ > static int > virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features) > @@ -1299,6 +1369,17 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features) > ret = virtio_alloc_queues(eth_dev); > if (ret < 0) > return ret; > + > + /* Make sure rxq interrupt is configured before sending DRIVER_OK, > + * so that QEMU can properly set those irq into kvm. > + */ As said, I don't think such comment is needed: for sure, we have to setup everything properly (about the device) before setting the DRIVER_OK flag. > + if (eth_dev->data->dev_conf.intr_conf.rxq) { > + if (virtio_configure_intr(eth_dev) < 0) { > + PMD_INIT_LOG(ERR, "failed to configure interrupt"); > + return -1; > + } > + } > + > vtpci_reinit_complete(hw); > > if (pci_dev) > @@ -1503,7 +1584,15 @@ virtio_dev_start(struct rte_eth_dev *dev) > PMD_DRV_LOG(ERR, "link status not supported by host"); > return -ENOTSUP; > } > + } > > + /* Enable uio/vfio intr/eventfd mapping: althrough we already did that > + * in device configure, but it could be unmapped when device is > + * stopped. Well, I didn't see you do that; I did see you do that in next patch though. That said, the next patch should be merged here, into one patch. --yliu > + */ > + if (dev->data->dev_conf.intr_conf.lsc || > + dev->data->dev_conf.intr_conf.rxq) { > + rte_intr_disable(&dev->pci_dev->intr_handle); > if (rte_intr_enable(&dev->pci_dev->intr_handle) < 0) { > PMD_DRV_LOG(ERR, "interrupt enable failed"); > return -EIO; > -- > 2.7.4