From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id C0563378E for ; Wed, 1 Jun 2016 03:54:03 +0200 (CEST) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP; 31 May 2016 18:53:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,398,1459839600"; d="scan'208";a="711943938" Received: from shvmail01.sh.intel.com ([10.239.29.42]) by FMSMGA003.fm.intel.com with ESMTP; 31 May 2016 18:53:51 -0700 Received: from shecgisg004.sh.intel.com (shecgisg004.sh.intel.com [10.239.29.89]) by shvmail01.sh.intel.com with ESMTP id u511rnpw011231; Wed, 1 Jun 2016 09:53:49 +0800 Received: from shecgisg004.sh.intel.com (localhost [127.0.0.1]) by shecgisg004.sh.intel.com (8.13.6/8.13.6/SuSE Linux 0.8) with ESMTP id u511rk5j028330; Wed, 1 Jun 2016 09:53:48 +0800 Received: (from wenzhuol@localhost) by shecgisg004.sh.intel.com (8.13.6/8.13.6/Submit) id u511rkHW028326; Wed, 1 Jun 2016 09:53:46 +0800 From: Wenzhuo Lu To: dev@dpdk.org Cc: jingjing.wu@intel.com, Wenzhuo Lu Date: Wed, 1 Jun 2016 09:53:09 +0800 Message-Id: <1464745989-28280-3-git-send-email-wenzhuo.lu@intel.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1464745989-28280-1-git-send-email-wenzhuo.lu@intel.com> References: <1464069991-31051-1-git-send-email-wenzhuo.lu@intel.com> <1464745989-28280-1-git-send-email-wenzhuo.lu@intel.com> Subject: [dpdk-dev] [PATCH v2 2/2] igb: VF supports mailbox interruption for PF link up/down X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jun 2016 01:54:04 -0000 In this scenario, kernel PF + DPDK VF, when PF finds the link state changes, up -> down or down -> up, it will send a message to VF by mailbox. This link state change may be triggered by PHY disconnection/reconnection, configuration like *ifconfig down/up* or interface parameter, like MTU, change. This patch enables the support of the mailbox interruption, so VF can receive the message of link up/down. After VF receives this message, VF port need to be reset to recover. So the handler of this message registers a reset callback to let APP reset the VF port. Signed-off-by: Wenzhuo Lu --- doc/guides/rel_notes/release_16_07.rst | 2 +- drivers/net/e1000/igb_ethdev.c | 159 +++++++++++++++++++++++++++++++++ 2 files changed, 160 insertions(+), 1 deletion(-) diff --git a/doc/guides/rel_notes/release_16_07.rst b/doc/guides/rel_notes/release_16_07.rst index 990bd46..a761e3c 100644 --- a/doc/guides/rel_notes/release_16_07.rst +++ b/doc/guides/rel_notes/release_16_07.rst @@ -47,7 +47,7 @@ New Features * Dropped specific Xen Dom0 code. * Dropped specific anonymous mempool code in testpmd. -* **Added mailbox interruption support for ixgbe VF.** +* **Added mailbox interruption support for ixgbe/igb VF.** When the link becomes down or up, PF will use mailbox message to notice VF. To handle this link up/down event, add the mailbox interruption diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c index f0921ee..b0e5e6a 100644 --- a/drivers/net/e1000/igb_ethdev.c +++ b/drivers/net/e1000/igb_ethdev.c @@ -86,6 +86,12 @@ #define E1000_INCVALUE_82576 (16 << IGB_82576_TSYNC_SHIFT) #define E1000_TSAUXC_DISABLE_SYSTIME 0x80000000 +#define E1000_VTIVAR_MISC 0x01740 +#define E1000_VTIVAR_MISC_MASK 0xFF +#define E1000_VTIVAR_VALID 0x80 +#define E1000_VTIVAR_MISC_MAILBOX 0 +#define E1000_VTIVAR_MISC_INTR_MASK 0x3 + static int eth_igb_configure(struct rte_eth_dev *dev); static int eth_igb_start(struct rte_eth_dev *dev); static void eth_igb_stop(struct rte_eth_dev *dev); @@ -259,6 +265,9 @@ static void eth_igb_assign_msix_vector(struct e1000_hw *hw, int8_t direction, static void eth_igb_write_ivar(struct e1000_hw *hw, uint8_t msix_vector, uint8_t index, uint8_t offset); static void eth_igb_configure_msix_intr(struct rte_eth_dev *dev); +static void eth_igbvf_interrupt_handler(struct rte_intr_handle *handle, + void *param); +static void igbvf_mbx_process(struct rte_eth_dev *dev); /* * Define VF Stats MACRO for Non "cleared on read" register @@ -554,6 +563,41 @@ igb_intr_disable(struct e1000_hw *hw) E1000_WRITE_FLUSH(hw); } +static inline void +igbvf_intr_enable(struct rte_eth_dev *dev) +{ + struct e1000_hw *hw = + E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private); + + /* only for mailbox */ + E1000_WRITE_REG(hw, E1000_EIAM, 1 << E1000_VTIVAR_MISC_MAILBOX); + E1000_WRITE_REG(hw, E1000_EIAC, 1 << E1000_VTIVAR_MISC_MAILBOX); + E1000_WRITE_REG(hw, E1000_EIMS, 1 << E1000_VTIVAR_MISC_MAILBOX); + E1000_WRITE_FLUSH(hw); +} + +/* only for mailbox now. If RX/TX needed, should extend this function. */ +static void +igbvf_set_ivar_map(struct e1000_hw *hw, uint8_t msix_vector) +{ + uint32_t tmp = 0; + + /* mailbox */ + tmp |= (msix_vector & E1000_VTIVAR_MISC_INTR_MASK); + tmp |= E1000_VTIVAR_VALID; + E1000_WRITE_REG(hw, E1000_VTIVAR_MISC, tmp); +} + +static void +eth_igbvf_configure_msix_intr(struct rte_eth_dev *dev) +{ + struct e1000_hw *hw = + E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private); + + /* Configure VF other cause ivar */ + igbvf_set_ivar_map(hw, E1000_VTIVAR_MISC_MAILBOX); +} + static inline int32_t igb_pf_reset_hw(struct e1000_hw *hw) { @@ -942,6 +986,10 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev) eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id, "igb_mac_82576_vf"); + rte_intr_callback_register(&pci_dev->intr_handle, + eth_igbvf_interrupt_handler, + (void *)eth_dev); + return 0; } @@ -950,6 +998,7 @@ eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev) { struct e1000_adapter *adapter = E1000_DEV_PRIVATE(eth_dev->data->dev_private); + struct rte_pci_device *pci_dev = eth_dev->pci_dev; PMD_INIT_FUNC_TRACE(); @@ -966,6 +1015,12 @@ eth_igbvf_dev_uninit(struct rte_eth_dev *eth_dev) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; + /* disable uio intr before callback unregister */ + rte_intr_disable(&pci_dev->intr_handle); + rte_intr_callback_unregister(&pci_dev->intr_handle, + eth_igbvf_interrupt_handler, + (void *)eth_dev); + return 0; } @@ -2564,6 +2619,69 @@ eth_igb_interrupt_handler(__rte_unused struct rte_intr_handle *handle, } static int +eth_igbvf_interrupt_get_status(struct rte_eth_dev *dev) +{ + uint32_t eicr; + struct e1000_hw *hw = + E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private); + struct e1000_interrupt *intr = + E1000_DEV_PRIVATE_TO_INTR(dev->data->dev_private); + + igbvf_intr_disable(hw); + + /* read-on-clear nic registers here */ + eicr = E1000_READ_REG(hw, E1000_EICR); + intr->flags = 0; + + if (eicr == E1000_VTIVAR_MISC_MAILBOX) + intr->flags |= E1000_FLAG_MAILBOX; + + return 0; +} + +void igbvf_mbx_process(struct rte_eth_dev *dev) +{ + struct e1000_hw *hw = + E1000_DEV_PRIVATE_TO_HW(dev->data->dev_private); + struct e1000_mbx_info *mbx = &hw->mbx; + u32 in_msg = 0; + + if (mbx->ops.read(hw, &in_msg, 1, 0)) + return; + + /* PF reset VF event */ + if (in_msg == E1000_PF_CONTROL_MSG) + _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET); +} + +static int +eth_igbvf_interrupt_action(struct rte_eth_dev *dev) +{ + struct e1000_interrupt *intr = + E1000_DEV_PRIVATE_TO_INTR(dev->data->dev_private); + + if (intr->flags & E1000_FLAG_MAILBOX) { + igbvf_mbx_process(dev); + intr->flags &= ~E1000_FLAG_MAILBOX; + } + + igbvf_intr_enable(dev); + rte_intr_enable(&dev->pci_dev->intr_handle); + + return 0; +} + +static void +eth_igbvf_interrupt_handler(__rte_unused struct rte_intr_handle *handle, + void *param) +{ + struct rte_eth_dev *dev = (struct rte_eth_dev *)param; + + eth_igbvf_interrupt_get_status(dev); + eth_igbvf_interrupt_action(dev); +} + +static int eth_igb_led_on(struct rte_eth_dev *dev) { struct e1000_hw *hw; @@ -2834,6 +2952,8 @@ igbvf_dev_start(struct rte_eth_dev *dev) struct e1000_adapter *adapter = E1000_DEV_PRIVATE(dev->data->dev_private); int ret; + struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; + uint32_t intr_vector = 0; PMD_INIT_FUNC_TRACE(); @@ -2853,12 +2973,41 @@ igbvf_dev_start(struct rte_eth_dev *dev) return ret; } + /* check and configure queue intr-vector mapping */ + if (dev->data->dev_conf.intr_conf.rxq != 0) { + intr_vector = dev->data->nb_rx_queues; + ret = rte_intr_efd_enable(intr_handle, intr_vector); + if (ret) + return ret; + } + + if (rte_intr_dp_is_en(intr_handle) && !intr_handle->intr_vec) { + intr_handle->intr_vec = + rte_zmalloc("intr_vec", + dev->data->nb_rx_queues * sizeof(int), 0); + if (!intr_handle->intr_vec) { + PMD_INIT_LOG(ERR, "Failed to allocate %d rx_queues" + " intr_vec\n", dev->data->nb_rx_queues); + return -ENOMEM; + } + } + + eth_igbvf_configure_msix_intr(dev); + + /* enable uio/vfio intr/eventfd mapping */ + rte_intr_enable(intr_handle); + + /* resume enabled intr since hw reset */ + igbvf_intr_enable(dev); + return 0; } static void igbvf_dev_stop(struct rte_eth_dev *dev) { + struct rte_intr_handle *intr_handle = &dev->pci_dev->intr_handle; + PMD_INIT_FUNC_TRACE(); igbvf_stop_adapter(dev); @@ -2870,6 +3019,16 @@ igbvf_dev_stop(struct rte_eth_dev *dev) igbvf_set_vfta_all(dev,0); igb_dev_clear_queues(dev); + + /* disable intr eventfd mapping */ + rte_intr_disable(intr_handle); + + /* Clean datapath event and queue/vec mapping */ + rte_intr_efd_disable(intr_handle); + if (intr_handle->intr_vec) { + rte_free(intr_handle->intr_vec); + intr_handle->intr_vec = NULL; + } } static void -- 1.9.3