From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 5E2498E65 for ; Tue, 7 Jun 2016 08:13:14 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP; 06 Jun 2016 23:13:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,432,1459839600"; d="scan'208";a="982306293" Received: from shvmail01.sh.intel.com ([10.239.29.42]) by fmsmga001.fm.intel.com with ESMTP; 06 Jun 2016 23:13:14 -0700 Received: from shecgisg004.sh.intel.com (shecgisg004.sh.intel.com [10.239.29.89]) by shvmail01.sh.intel.com with ESMTP id u576DAH2030022; Tue, 7 Jun 2016 14:13:10 +0800 Received: from shecgisg004.sh.intel.com (localhost [127.0.0.1]) by shecgisg004.sh.intel.com (8.13.6/8.13.6/SuSE Linux 0.8) with ESMTP id u576D6jk005607; Tue, 7 Jun 2016 14:13:08 +0800 Received: (from zhetao@localhost) by shecgisg004.sh.intel.com (8.13.6/8.13.6/Submit) id u576D6A9005603; Tue, 7 Jun 2016 14:13:06 +0800 From: Zhe Tao To: dev@dpdk.org Cc: wenzhuo.lu@intel.com, zhe.tao@intel.com, konstantin.ananyev@intel.com, bruce.richardson@intel.com, jing.d.chen@intel.com, cunming.liang@intel.com, jingjing.wu@intel.com, helin.zhang@intel.com Date: Tue, 7 Jun 2016 14:12:48 +0800 Message-Id: <1465279968-5515-9-git-send-email-zhe.tao@intel.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1465279968-5515-1-git-send-email-zhe.tao@intel.com> References: <1465279968-5515-1-git-send-email-zhe.tao@intel.com> Subject: [dpdk-dev] [PATCH v3 8/8] i40e: implement device reset on VF X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jun 2016 06:13:15 -0000 Implement the device reset function. 1, Add the fake RX/TX functions. 2, The reset function tries to stop RX/TX by replacing the RX/TX functions with the fake ones and getting the locks to make sure the regular RX/TX finished. 3, After the RX/TX stopped, reset the VF port, and then release the locks. Signed-off-by: Zhe Tao --- app/test-pmd/config.c | 3 + doc/guides/rel_notes/release_16_07.rst | 5 ++ drivers/net/i40e/i40e_ethdev.h | 7 +- drivers/net/i40e/i40e_ethdev_vf.c | 152 ++++++++++++++++++++++++++++++++- drivers/net/i40e/i40e_rxtx.c | 10 +++ drivers/net/i40e/i40e_rxtx.h | 4 + 6 files changed, 175 insertions(+), 6 deletions(-) diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index 1c552e4..a120261 100644 --- a/app/test-pmd/config.c +++ b/app/test-pmd/config.c @@ -378,6 +378,7 @@ tx_queue_infos_display(portid_t port_id, uint16_t queue_id) printf("\nNumber of TXDs: %hu", qinfo.nb_desc); printf("\n"); } +void i40evf_emulate_vf_reset(uint8_t port_id); void port_infos_display(portid_t port_id) @@ -398,6 +399,8 @@ port_infos_display(portid_t port_id) printf("]\n"); return; } + printf("emulate vf reset\n"); + i40evf_emulate_vf_reset(port_id); port = &ports[port_id]; rte_eth_link_get_nowait(port_id, &link); printf("\n%s Infos for port %-2d %s\n", diff --git a/doc/guides/rel_notes/release_16_07.rst b/doc/guides/rel_notes/release_16_07.rst index a4c0cc3..f43b867 100644 --- a/doc/guides/rel_notes/release_16_07.rst +++ b/doc/guides/rel_notes/release_16_07.rst @@ -62,6 +62,11 @@ New Features callback in the message handler to notice the APP. APP need call the device reset API to reset the VF port. +* **Added VF reset support for i40e VF driver.** + + Added a new implementaion to allow i40e VF driver to + reset the functionality and state of itself. + Resolved Issues --------------- diff --git a/drivers/net/i40e/i40e_ethdev.h b/drivers/net/i40e/i40e_ethdev.h index 672d920..dcd6e0f 100644 --- a/drivers/net/i40e/i40e_ethdev.h +++ b/drivers/net/i40e/i40e_ethdev.h @@ -541,9 +541,8 @@ struct i40e_adapter { struct rte_timecounter rx_tstamp_tc; struct rte_timecounter tx_tstamp_tc; - /* For VF reset backup */ - eth_rx_burst_t rx_backup; - eth_tx_burst_t tx_backup; + /* For VF reset */ + uint8_t reset_number; }; int i40e_dev_switch_queues(struct i40e_pf *pf, bool on); @@ -597,6 +596,8 @@ void i40e_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, void i40e_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_txq_info *qinfo); +void i40evf_emulate_vf_reset(uint8_t port_id); + /* I40E_DEV_PRIVATE_TO */ #define I40E_DEV_PRIVATE_TO_PF(adapter) \ (&((struct i40e_adapter *)adapter)->pf) diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c index 46d8a7c..d873d2d 100644 --- a/drivers/net/i40e/i40e_ethdev_vf.c +++ b/drivers/net/i40e/i40e_ethdev_vf.c @@ -157,6 +157,12 @@ i40evf_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id); static void i40evf_handle_pf_event(__rte_unused struct rte_eth_dev *dev, uint8_t *msg, uint16_t msglen); +static int i40evf_dev_uninit(struct rte_eth_dev *eth_dev); +static int i40evf_dev_init(struct rte_eth_dev *eth_dev); +static void i40evf_dev_close(struct rte_eth_dev *dev); +static int i40evf_dev_start(struct rte_eth_dev *dev); +static int i40evf_dev_configure(struct rte_eth_dev *dev); +static int i40evf_handle_vf_reset(struct rte_eth_dev *dev); /* Default hash key buffer for RSS */ static uint32_t rss_key_default[I40E_VFQF_HKEY_MAX_INDEX + 1]; @@ -223,6 +229,7 @@ static const struct eth_dev_ops i40evf_eth_dev_ops = { .reta_query = i40evf_dev_rss_reta_query, .rss_hash_update = i40evf_dev_rss_hash_update, .rss_hash_conf_get = i40evf_dev_rss_hash_conf_get, + .dev_reset = i40evf_handle_vf_reset }; /* @@ -1309,6 +1316,140 @@ i40evf_uninit_vf(struct rte_eth_dev *dev) } static void +i40e_vf_queue_reset(struct rte_eth_dev *dev) +{ + uint16_t i; + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + struct i40e_rx_queue *rxq = dev->data->rx_queues[i]; + + if (rxq->q_set) { + i40e_dev_rx_queue_setup(dev, + rxq->queue_id, + rxq->nb_rx_desc, + rxq->socket_id, + &rxq->rxconf, + rxq->mp); + } + + rxq = dev->data->rx_queues[i]; + rte_spinlock_trylock(&rxq->rx_lock); + } + for (i = 0; i < dev->data->nb_tx_queues; i++) { + struct i40e_tx_queue *txq = dev->data->tx_queues[i]; + + if (txq->q_set) { + i40e_dev_tx_queue_setup(dev, + txq->queue_id, + txq->nb_tx_desc, + txq->socket_id, + &txq->txconf); + } + + txq = dev->data->tx_queues[i]; + rte_spinlock_trylock(&txq->tx_lock); + } +} + +static void +i40e_vf_reset_dev(struct rte_eth_dev *dev) +{ + struct i40e_adapter *adapter = + I40E_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); + + i40evf_dev_close(dev); + PMD_DRV_LOG(DEBUG, "i40evf dev close complete"); + i40evf_dev_uninit(dev); + PMD_DRV_LOG(DEBUG, "i40evf dev detached"); + memset(dev->data->dev_private, 0, + (uint64_t)&adapter->reset_number - (uint64_t)adapter); + + i40evf_dev_configure(dev); + i40evf_dev_init(dev); + PMD_DRV_LOG(DEBUG, "i40evf dev attached"); + i40e_vf_queue_reset(dev); + PMD_DRV_LOG(DEBUG, "i40evf queue reset"); + i40evf_dev_start(dev); + PMD_DRV_LOG(DEBUG, "i40evf dev restart"); +} + +static uint16_t +i40evf_recv_pkts_detach(void __rte_unused *rx_queue, + struct rte_mbuf __rte_unused **rx_pkts, + uint16_t __rte_unused nb_pkts) +{ + return 0; +} + +static uint16_t +i40evf_xmit_pkts_detach(void __rte_unused *tx_queue, + struct rte_mbuf __rte_unused **tx_pkts, + uint16_t __rte_unused nb_pkts) +{ + return 0; +} + +static int +i40evf_handle_vf_reset(struct rte_eth_dev *dev) +{ + struct i40e_adapter *adapter = + I40E_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); + uint16_t i = 0; + struct i40e_rx_queue *rxq; + struct i40e_tx_queue *txq; + + if (!dev->data->dev_started) + return 0; + + adapter->reset_number = 1; + + /** + * Stop RX/TX by fake functions and locks. + * Fake functions are used to make RX/TX lock easier. + */ + dev->rx_pkt_burst = i40evf_recv_pkts_detach; + dev->tx_pkt_burst = i40evf_xmit_pkts_detach; + + if (dev->data->rx_queues) + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + rte_spinlock_lock(&rxq->rx_lock); + } + + if (dev->data->tx_queues) + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + rte_spinlock_lock(&txq->tx_lock); + } + + i40e_vf_reset_dev(dev); + + adapter->reset_number = 0; + + if (dev->data->rx_queues) + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxq = dev->data->rx_queues[i]; + rte_spinlock_unlock(&rxq->rx_lock); + } + + if (dev->data->tx_queues) + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txq = dev->data->tx_queues[i]; + rte_spinlock_unlock(&txq->tx_lock); + } + + return 0; +} + +void +i40evf_emulate_vf_reset(uint8_t port_id) +{ + struct rte_eth_dev *dev = &rte_eth_devices[port_id]; + + i40evf_handle_vf_reset(dev); +} + +static void i40evf_handle_pf_event(__rte_unused struct rte_eth_dev *dev, uint8_t *msg, __rte_unused uint16_t msglen) @@ -1446,13 +1587,18 @@ i40evf_dev_init(struct rte_eth_dev *eth_dev) struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(\ eth_dev->data->dev_private); struct rte_pci_device *pci_dev = eth_dev->pci_dev; + struct i40e_adapter *adapter = + I40E_DEV_PRIVATE_TO_ADAPTER(eth_dev->data->dev_private); PMD_INIT_FUNC_TRACE(); - /* assign ops func pointer */ eth_dev->dev_ops = &i40evf_eth_dev_ops; - eth_dev->rx_pkt_burst = RX_LOCK_FUNCTION(eth_dev, i40e_recv_pkts); - eth_dev->tx_pkt_burst = TX_LOCK_FUNCTION(eth_dev, i40e_xmit_pkts); + if (adapter->reset_number) { + eth_dev->rx_pkt_burst = + RX_LOCK_FUNCTION(eth_dev, i40e_recv_pkts); + eth_dev->tx_pkt_burst = + TX_LOCK_FUNCTION(eth_dev, i40e_xmit_pkts); + } /* * For secondary processes, we don't initialise any further as primary diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c index 0a6dcfb..f6caf4e 100644 --- a/drivers/net/i40e/i40e_rxtx.c +++ b/drivers/net/i40e/i40e_rxtx.c @@ -2147,6 +2147,7 @@ i40e_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t len, i; uint16_t base, bsf, tc_mapping; int use_def_burst_func = 1; + struct rte_eth_rxconf conf = *rx_conf; if (hw->mac.type == I40E_MAC_VF || hw->mac.type == I40E_MAC_X722_VF) { struct i40e_vf *vf = @@ -2185,6 +2186,8 @@ i40e_dev_rx_queue_setup(struct rte_eth_dev *dev, return -ENOMEM; } rxq->mp = mp; + rxq->socket_id = socket_id; + rxq->rxconf = conf; rxq->nb_rx_desc = nb_desc; rxq->rx_free_thresh = rx_conf->rx_free_thresh; rxq->queue_id = queue_idx; @@ -2364,6 +2367,7 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev, uint32_t ring_size; uint16_t tx_rs_thresh, tx_free_thresh; uint16_t i, base, bsf, tc_mapping; + struct rte_eth_txconf conf = *tx_conf; if (hw->mac.type == I40E_MAC_VF || hw->mac.type == I40E_MAC_X722_VF) { struct i40e_vf *vf = @@ -2487,6 +2491,8 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev, } txq->nb_tx_desc = nb_desc; + txq->socket_id = socket_id; + txq->txconf = conf; txq->tx_rs_thresh = tx_rs_thresh; txq->tx_free_thresh = tx_free_thresh; txq->pthresh = tx_conf->tx_thresh.pthresh; @@ -2951,8 +2957,12 @@ void i40e_dev_free_queues(struct rte_eth_dev *dev) { uint16_t i; + struct i40e_adapter *adapter = + I40E_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); PMD_INIT_FUNC_TRACE(); + if (adapter->reset_number) + return; for (i = 0; i < dev->data->nb_rx_queues; i++) { i40e_dev_rx_queue_release(dev->data->rx_queues[i]); diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h index a1c13b8..7ee33dc 100644 --- a/drivers/net/i40e/i40e_rxtx.h +++ b/drivers/net/i40e/i40e_rxtx.h @@ -141,6 +141,8 @@ struct i40e_rx_queue { uint16_t rx_using_sse; /**