* [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements @ 2020-05-06 19:27 Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 1/4] net/netvsc: support per-queue info requests Stephen Hemminger ` (5 more replies) 0 siblings, 6 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:27 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger This patch set is based on top of my previous netvsc PMD bug fixes. It provides additional standard diagnostic hooks for queue info and improves the debug output. Stephen Hemminger (4): net/netvsc: support per-queue info requests net/netvsc: implement rx/tx descriptor status functions net/netvsc: change tx/rx error handling bus/vmbus: improve debug output drivers/bus/vmbus/vmbus_channel.c | 42 +++++++++------ drivers/net/netvsc/hn_ethdev.c | 6 +++ drivers/net/netvsc/hn_rxtx.c | 88 +++++++++++++++++++++++++++---- drivers/net/netvsc/hn_var.h | 8 +++ 4 files changed, 119 insertions(+), 25 deletions(-) -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH 1/4] net/netvsc: support per-queue info requests 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger @ 2020-05-06 19:27 ` Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger ` (4 subsequent siblings) 5 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:27 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger There is not a lot of info here from this driver but worth supporting the standard values for number of descriptors and offloads flags. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 2 ++ drivers/net/netvsc/hn_rxtx.c | 20 ++++++++++++++++++++ drivers/net/netvsc/hn_var.h | 2 ++ 3 files changed, 24 insertions(+) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 55b8a63804a9..b890fa3befa8 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -860,6 +860,8 @@ static const struct eth_dev_ops hn_eth_dev_ops = { .dev_stop = hn_dev_stop, .dev_close = hn_dev_close, .dev_infos_get = hn_dev_info_get, + .txq_info_get = hn_dev_tx_queue_info, + .rxq_info_get = hn_dev_rx_queue_info, .dev_supported_ptypes_get = hn_vf_supported_ptypes, .promiscuous_enable = hn_dev_promiscuous_enable, .promiscuous_disable = hn_dev_promiscuous_disable, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 31fae5597598..668f48fcbf15 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -319,6 +319,15 @@ hn_dev_tx_queue_setup(struct rte_eth_dev *dev, return err; } +void +hn_dev_tx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_txq_info *qinfo) +{ + struct hn_tx_queue *txq = dev->data->tx_queues[queue_id]; + + qinfo->nb_desc = txq->txdesc_pool->size; + qinfo->conf.offloads = dev->data->dev_conf.txmode.offloads; +} static struct hn_txdesc *hn_txd_get(struct hn_tx_queue *txq) { @@ -859,6 +868,17 @@ struct hn_rx_queue *hn_rx_queue_alloc(struct hn_data *hv, return rxq; } +void +hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_rxq_info *qinfo) +{ + struct hn_rx_queue *rxq = dev->data->rx_queues[queue_id]; + + qinfo->mp = rxq->mb_pool; + qinfo->nb_desc = rxq->rx_ring->size; + qinfo->conf.offloads = dev->data->dev_conf.rxmode.offloads; +} + int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index d1d38b459601..93168d2e0102 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -178,6 +178,8 @@ int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +void hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_rxq_info *qinfo); void hn_dev_rx_queue_release(void *arg); void hn_dev_free_queues(struct rte_eth_dev *dev); -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH 2/4] net/netvsc: implement rx/tx descriptor status functions 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 1/4] net/netvsc: support per-queue info requests Stephen Hemminger @ 2020-05-06 19:27 ` Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger ` (3 subsequent siblings) 5 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:27 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger These functions are useful for applications and debugging. The netvsc PMD also transparently handles the rx/tx descriptor functions for underlying VF device. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 3 +++ drivers/net/netvsc/hn_rxtx.c | 39 ++++++++++++++++++++++++++++++++++ drivers/net/netvsc/hn_var.h | 5 +++++ 3 files changed, 47 insertions(+) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index b890fa3befa8..675a49e66a57 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -875,8 +875,11 @@ static const struct eth_dev_ops hn_eth_dev_ops = { .tx_queue_setup = hn_dev_tx_queue_setup, .tx_queue_release = hn_dev_tx_queue_release, .tx_done_cleanup = hn_dev_tx_done_cleanup, + .tx_descriptor_status = hn_dev_tx_descriptor_status, .rx_queue_setup = hn_dev_rx_queue_setup, .rx_queue_release = hn_dev_rx_queue_release, + .rx_queue_count = hn_dev_rx_queue_count, + .rx_descriptor_status = hn_dev_rx_queue_status, .link_update = hn_dev_link_update, .stats_get = hn_dev_stats_get, .stats_reset = hn_dev_stats_reset, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 668f48fcbf15..537fd0125687 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -369,6 +369,20 @@ hn_dev_tx_queue_release(void *arg) rte_free(txq); } +int hn_dev_tx_descriptor_status(void *arg, uint16_t offset) +{ + const struct hn_tx_queue *txq = arg; + uint16_t used; + + hn_process_events(txq->hv, txq->queue_id, 0); + + used = rte_mempool_in_use_count(txq->txdesc_pool); + if (offset < used) + return RTE_ETH_TX_DESC_FULL; + else + return RTE_ETH_TX_DESC_DONE; +} + static void hn_nvs_send_completed(struct rte_eth_dev *dev, uint16_t queue_id, unsigned long xactid, const struct hn_nvs_rndis_ack *ack) @@ -966,6 +980,31 @@ hn_dev_rx_queue_release(void *arg) hn_rx_queue_free(rxq, true); } +/* + * Get the number of used descriptor in a rx queue + * For this device that means how many packets are pending in the ring. + */ +uint32_t +hn_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct hn_rx_queue *rxq = dev->data->rx_queues[queue_id]; + + return rte_ring_count(rxq->rx_ring); +} + +int hn_dev_rx_queue_status(void *arg, uint16_t offset) +{ + const struct hn_rx_queue *rxq = arg; + + hn_process_events(rxq->hv, rxq->queue_id, 0); + if (offset >= rxq->rx_ring->capacity) + return -EINVAL; + else if (offset < rte_ring_count(rxq->rx_ring)) + return RTE_ETH_RX_DESC_DONE; + else + return RTE_ETH_RX_DESC_AVAIL; +} + int hn_dev_tx_done_cleanup(void *arg, uint32_t free_cnt) { diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index 93168d2e0102..ded0d4c60a04 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -169,6 +169,7 @@ void hn_dev_tx_queue_release(void *arg); void hn_dev_tx_queue_info(struct rte_eth_dev *dev, uint16_t queue_idx, struct rte_eth_txq_info *qinfo); int hn_dev_tx_done_cleanup(void *arg, uint32_t free_cnt); +int hn_dev_tx_descriptor_status(void *arg, uint16_t offset); struct hn_rx_queue *hn_rx_queue_alloc(struct hn_data *hv, uint16_t queue_id, @@ -181,6 +182,8 @@ int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, void hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_info *qinfo); void hn_dev_rx_queue_release(void *arg); +uint32_t hn_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id); +int hn_dev_rx_queue_status(void *rxq, uint16_t offset); void hn_dev_free_queues(struct rte_eth_dev *dev); /* Check if VF is attached */ @@ -231,6 +234,8 @@ int hn_vf_tx_queue_setup(struct rte_eth_dev *dev, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); void hn_vf_tx_queue_release(struct hn_data *hv, uint16_t queue_id); +int hn_vf_tx_queue_status(struct hn_data *hv, uint16_t queue_id, uint16_t offset); + int hn_vf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH 3/4] net/netvsc: change tx/rx error handling 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 1/4] net/netvsc: support per-queue info requests Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger @ 2020-05-06 19:27 ` Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 4/4] bus/vmbus: improve debug output Stephen Hemminger ` (2 subsequent siblings) 5 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:27 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger The PMD_TX_LOG and PMD_RX_LOG can hide errors since the debug log is typically disabled. Change the code to use PMD_DRV_LOG for errors. Under load, the ring buffer to the host can fill. Add some statistics to estimate the impact and see other errors. --- drivers/net/netvsc/hn_ethdev.c | 1 + drivers/net/netvsc/hn_rxtx.c | 29 ++++++++++++++++++++--------- drivers/net/netvsc/hn_var.h | 1 + 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 675a49e66a57..0f4827c54be4 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -58,6 +58,7 @@ static const struct hn_xstats_name_off hn_stat_strings[] = { { "good_bytes", offsetof(struct hn_stats, bytes) }, { "errors", offsetof(struct hn_stats, errors) }, { "ring full", offsetof(struct hn_stats, ring_full) }, + { "channel full", offsetof(struct hn_stats, channel_full) }, { "multicast_packets", offsetof(struct hn_stats, multicast) }, { "broadcast_packets", offsetof(struct hn_stats, broadcast) }, { "undersize_packets", offsetof(struct hn_stats, size_bins[0]) }, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 537fd0125687..fa53c7becc12 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -403,8 +403,8 @@ hn_nvs_send_completed(struct rte_eth_dev *dev, uint16_t queue_id, txq->stats.bytes += txd->data_size; txq->stats.packets += txd->packets; } else { - PMD_TX_LOG(NOTICE, "port %u:%u complete tx %u failed status %u", - txq->port_id, txq->queue_id, txd->chim_index, ack->status); + PMD_DRV_LOG(NOTICE, "port %u:%u complete tx %u failed status %u", + txq->port_id, txq->queue_id, txd->chim_index, ack->status); ++txq->stats.errors; } @@ -429,8 +429,7 @@ hn_nvs_handle_comp(struct rte_eth_dev *dev, uint16_t queue_id, break; default: - PMD_TX_LOG(NOTICE, - "unexpected send completion type %u", + PMD_DRV_LOG(NOTICE, "unexpected send completion type %u", hdr->type); } } @@ -648,6 +647,7 @@ static void hn_rxpkt(struct hn_rx_queue *rxq, struct hn_rx_bufinfo *rxb, if (unlikely(rte_ring_sp_enqueue(rxq->rx_ring, m) != 0)) { ++rxq->stats.ring_full; + PMD_RX_LOG(DEBUG, "rx ring full"); rte_pktmbuf_free(m); } } @@ -1156,10 +1156,16 @@ static int hn_flush_txagg(struct hn_tx_queue *txq, bool *need_sig) if (likely(ret == 0)) hn_reset_txagg(txq); - else - PMD_TX_LOG(NOTICE, "port %u:%u send failed: %d", - txq->port_id, txq->queue_id, ret); + else if (ret == -EAGAIN) { + PMD_TX_LOG(DEBUG, "port %u:%u channel full", + txq->port_id, txq->queue_id); + ++txq->stats.channel_full; + } else { + ++txq->stats.errors; + PMD_DRV_LOG(NOTICE, "port %u:%u send failed: %d", + txq->port_id, txq->queue_id, ret); + } return ret; } @@ -1505,8 +1511,13 @@ hn_xmit_pkts(void *ptxq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) ret = hn_xmit_sg(txq, txd, m, &need_sig); if (unlikely(ret != 0)) { - PMD_TX_LOG(NOTICE, "sg send failed: %d", ret); - ++txq->stats.errors; + if (ret == -EAGAIN) { + PMD_TX_LOG(DEBUG, "sg channel full"); + ++txq->stats.channel_full; + } else { + PMD_DRV_LOG(NOTICE, "sg send failed: %d", ret); + ++txq->stats.errors; + } hn_txd_put(txq, txd); goto fail; } diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index ded0d4c60a04..20363389a0d6 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -40,6 +40,7 @@ struct hn_stats { uint64_t bytes; uint64_t errors; uint64_t ring_full; + uint64_t channel_full; uint64_t multicast; uint64_t broadcast; /* Size bins in array as RFC 2819, undersized [0], 64 [1], etc */ -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH 4/4] bus/vmbus: improve debug output 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger ` (2 preceding siblings ...) 2020-05-06 19:27 ` [dpdk-dev] [PATCH 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger @ 2020-05-06 19:27 ` Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger 5 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:27 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger Make debug output more compact. Don't print pending size or interrupt mask for the transmit side (not applicable). Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/bus/vmbus/vmbus_channel.c | 42 +++++++++++++++++++------------ 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/drivers/bus/vmbus/vmbus_channel.c b/drivers/bus/vmbus/vmbus_channel.c index ff2985c25758..5df7deadd3f1 100644 --- a/drivers/bus/vmbus/vmbus_channel.c +++ b/drivers/bus/vmbus/vmbus_channel.c @@ -421,26 +421,36 @@ void rte_vmbus_chan_close(struct vmbus_channel *chan) static void vmbus_dump_ring(FILE *f, const char *id, const struct vmbus_br *br) { const struct vmbus_bufring *vbr = br->vbr; - struct vmbus_chanpkt_hdr pkt; - fprintf(f, "%s windex=%u rindex=%u mask=%u pending=%u feature=%#x\n", - id, vbr->windex, vbr->rindex, vbr->imask, - vbr->pending_send, vbr->feature_bits.value); - fprintf(f, " size=%u avail write=%u read=%u\n", - br->dsize, vmbus_br_availwrite(br, vbr->windex), + fprintf(f, "%s index=%u/%u avail=%u/%u\n", + id, vbr->windex, vbr->rindex, + vmbus_br_availwrite(br, vbr->windex), vmbus_br_availread(br)); - - if (vmbus_rxbr_peek(br, &pkt, sizeof(pkt)) == 0) - fprintf(f, " pkt type %#x len %u flags %#x xactid %#"PRIx64"\n", - pkt.type, - pkt.tlen << VMBUS_CHANPKT_SIZE_SHIFT, - pkt.flags, pkt.xactid); } void rte_vmbus_chan_dump(FILE *f, const struct vmbus_channel *chan) { - fprintf(f, "channel[%u] relid=%u monitor=%u\n", - chan->subchannel_id, chan->relid, chan->monitor_id); - vmbus_dump_ring(f, "rxbr", &chan->rxbr); - vmbus_dump_ring(f, "txbr", &chan->txbr); + const struct vmbus_br *rbr = &chan->rxbr; + const struct vmbus_bufring *vbr = rbr->vbr; + struct vmbus_chanpkt_hdr pkt; + + fprintf(f, "chan %u: relid=%u mask=%u", + chan->subchannel_id, chan->relid, vbr->imask); + + if (vbr->feature_bits.feat_pending_send_sz) + fprintf(f, " pending_sz=%u\n", + vbr->pending_send); + else + fprintf(f, "\n"); + + vmbus_dump_ring(f, " tx", &chan->txbr); + vmbus_dump_ring(f, " rx", rbr); + + /* show first packet if any from host */ + if (vmbus_rxbr_peek(rbr, &pkt, sizeof(pkt))) + return; + + fprintf(f, "[pkt type %#x len %u flags %#x xactid %#"PRIx64"]\n", + pkt.type, pkt.tlen << VMBUS_CHANPKT_SIZE_SHIFT, + pkt.flags, pkt.xactid); } -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger ` (3 preceding siblings ...) 2020-05-06 19:27 ` [dpdk-dev] [PATCH 4/4] bus/vmbus: improve debug output Stephen Hemminger @ 2020-05-06 19:52 ` Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 1/4] net/netvsc: support per-queue info requests Stephen Hemminger ` (3 more replies) 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger 5 siblings, 4 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger This patch set is based on top of my previous netvsc PMD bug fixes. It provides additional standard diagnostic hooks for queue info and improves the debug output. v2 - add missing Signed-off-by Stephen Hemminger (4): net/netvsc: support per-queue info requests net/netvsc: implement rx/tx descriptor status functions net/netvsc: change tx/rx error handling bus/vmbus: improve debug output drivers/bus/vmbus/vmbus_channel.c | 42 +++++++++------ drivers/net/netvsc/hn_ethdev.c | 6 +++ drivers/net/netvsc/hn_rxtx.c | 88 +++++++++++++++++++++++++++---- drivers/net/netvsc/hn_var.h | 8 +++ 4 files changed, 119 insertions(+), 25 deletions(-) -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v2 1/4] net/netvsc: support per-queue info requests 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger @ 2020-05-06 19:52 ` Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger ` (2 subsequent siblings) 3 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger There is not a lot of info here from this driver. But worth supporting these additional info queries. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 2 ++ drivers/net/netvsc/hn_rxtx.c | 20 ++++++++++++++++++++ drivers/net/netvsc/hn_var.h | 2 ++ 3 files changed, 24 insertions(+) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 55b8a63804a9..b890fa3befa8 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -860,6 +860,8 @@ static const struct eth_dev_ops hn_eth_dev_ops = { .dev_stop = hn_dev_stop, .dev_close = hn_dev_close, .dev_infos_get = hn_dev_info_get, + .txq_info_get = hn_dev_tx_queue_info, + .rxq_info_get = hn_dev_rx_queue_info, .dev_supported_ptypes_get = hn_vf_supported_ptypes, .promiscuous_enable = hn_dev_promiscuous_enable, .promiscuous_disable = hn_dev_promiscuous_disable, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 31fae5597598..668f48fcbf15 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -319,6 +319,15 @@ hn_dev_tx_queue_setup(struct rte_eth_dev *dev, return err; } +void +hn_dev_tx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_txq_info *qinfo) +{ + struct hn_tx_queue *txq = dev->data->tx_queues[queue_id]; + + qinfo->nb_desc = txq->txdesc_pool->size; + qinfo->conf.offloads = dev->data->dev_conf.txmode.offloads; +} static struct hn_txdesc *hn_txd_get(struct hn_tx_queue *txq) { @@ -859,6 +868,17 @@ struct hn_rx_queue *hn_rx_queue_alloc(struct hn_data *hv, return rxq; } +void +hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_rxq_info *qinfo) +{ + struct hn_rx_queue *rxq = dev->data->rx_queues[queue_id]; + + qinfo->mp = rxq->mb_pool; + qinfo->nb_desc = rxq->rx_ring->size; + qinfo->conf.offloads = dev->data->dev_conf.rxmode.offloads; +} + int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index d1d38b459601..93168d2e0102 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -178,6 +178,8 @@ int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +void hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_rxq_info *qinfo); void hn_dev_rx_queue_release(void *arg); void hn_dev_free_queues(struct rte_eth_dev *dev); -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v2 2/4] net/netvsc: implement rx/tx descriptor status functions 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 1/4] net/netvsc: support per-queue info requests Stephen Hemminger @ 2020-05-06 19:52 ` Stephen Hemminger 2020-05-07 15:27 ` Ferruh Yigit 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 4/4] bus/vmbus: improve debug output Stephen Hemminger 3 siblings, 1 reply; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger These functions are useful for applications and debugging. The netvsc PMD also transparently handles the rx/tx descriptor functions for underlying VF device. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 3 +++ drivers/net/netvsc/hn_rxtx.c | 39 ++++++++++++++++++++++++++++++++++ drivers/net/netvsc/hn_var.h | 5 +++++ 3 files changed, 47 insertions(+) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index b890fa3befa8..675a49e66a57 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -875,8 +875,11 @@ static const struct eth_dev_ops hn_eth_dev_ops = { .tx_queue_setup = hn_dev_tx_queue_setup, .tx_queue_release = hn_dev_tx_queue_release, .tx_done_cleanup = hn_dev_tx_done_cleanup, + .tx_descriptor_status = hn_dev_tx_descriptor_status, .rx_queue_setup = hn_dev_rx_queue_setup, .rx_queue_release = hn_dev_rx_queue_release, + .rx_queue_count = hn_dev_rx_queue_count, + .rx_descriptor_status = hn_dev_rx_queue_status, .link_update = hn_dev_link_update, .stats_get = hn_dev_stats_get, .stats_reset = hn_dev_stats_reset, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 668f48fcbf15..537fd0125687 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -369,6 +369,20 @@ hn_dev_tx_queue_release(void *arg) rte_free(txq); } +int hn_dev_tx_descriptor_status(void *arg, uint16_t offset) +{ + const struct hn_tx_queue *txq = arg; + uint16_t used; + + hn_process_events(txq->hv, txq->queue_id, 0); + + used = rte_mempool_in_use_count(txq->txdesc_pool); + if (offset < used) + return RTE_ETH_TX_DESC_FULL; + else + return RTE_ETH_TX_DESC_DONE; +} + static void hn_nvs_send_completed(struct rte_eth_dev *dev, uint16_t queue_id, unsigned long xactid, const struct hn_nvs_rndis_ack *ack) @@ -966,6 +980,31 @@ hn_dev_rx_queue_release(void *arg) hn_rx_queue_free(rxq, true); } +/* + * Get the number of used descriptor in a rx queue + * For this device that means how many packets are pending in the ring. + */ +uint32_t +hn_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct hn_rx_queue *rxq = dev->data->rx_queues[queue_id]; + + return rte_ring_count(rxq->rx_ring); +} + +int hn_dev_rx_queue_status(void *arg, uint16_t offset) +{ + const struct hn_rx_queue *rxq = arg; + + hn_process_events(rxq->hv, rxq->queue_id, 0); + if (offset >= rxq->rx_ring->capacity) + return -EINVAL; + else if (offset < rte_ring_count(rxq->rx_ring)) + return RTE_ETH_RX_DESC_DONE; + else + return RTE_ETH_RX_DESC_AVAIL; +} + int hn_dev_tx_done_cleanup(void *arg, uint32_t free_cnt) { diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index 93168d2e0102..ded0d4c60a04 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -169,6 +169,7 @@ void hn_dev_tx_queue_release(void *arg); void hn_dev_tx_queue_info(struct rte_eth_dev *dev, uint16_t queue_idx, struct rte_eth_txq_info *qinfo); int hn_dev_tx_done_cleanup(void *arg, uint32_t free_cnt); +int hn_dev_tx_descriptor_status(void *arg, uint16_t offset); struct hn_rx_queue *hn_rx_queue_alloc(struct hn_data *hv, uint16_t queue_id, @@ -181,6 +182,8 @@ int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, void hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_info *qinfo); void hn_dev_rx_queue_release(void *arg); +uint32_t hn_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id); +int hn_dev_rx_queue_status(void *rxq, uint16_t offset); void hn_dev_free_queues(struct rte_eth_dev *dev); /* Check if VF is attached */ @@ -231,6 +234,8 @@ int hn_vf_tx_queue_setup(struct rte_eth_dev *dev, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); void hn_vf_tx_queue_release(struct hn_data *hv, uint16_t queue_id); +int hn_vf_tx_queue_status(struct hn_data *hv, uint16_t queue_id, uint16_t offset); + int hn_vf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [dpdk-dev] [PATCH v2 2/4] net/netvsc: implement rx/tx descriptor status functions 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger @ 2020-05-07 15:27 ` Ferruh Yigit 0 siblings, 0 replies; 23+ messages in thread From: Ferruh Yigit @ 2020-05-07 15:27 UTC (permalink / raw) To: Stephen Hemminger, dev On 5/6/2020 8:52 PM, Stephen Hemminger wrote: > These functions are useful for applications and debugging. > The netvsc PMD also transparently handles the rx/tx descriptor > functions for underlying VF device. > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> <...> > +int hn_dev_rx_queue_status(void *arg, uint16_t offset) > +{ > + const struct hn_rx_queue *rxq = arg; > + > + hn_process_events(rxq->hv, rxq->queue_id, 0); > + if (offset >= rxq->rx_ring->capacity) > + return -EINVAL; > + else if (offset < rte_ring_count(rxq->rx_ring)) Is this logic correct, can we compare 'offset' with used entry count in circular buffer? > + return RTE_ETH_RX_DESC_DONE; > + else > + return RTE_ETH_RX_DESC_AVAIL; > +} Can't there be descriptors with 'RTE_ETH_RX_DESC_UNAVAIL' status? ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v2 3/4] net/netvsc: change tx/rx error handling 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 1/4] net/netvsc: support per-queue info requests Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger @ 2020-05-06 19:52 ` Stephen Hemminger 2020-05-07 15:29 ` Ferruh Yigit 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 4/4] bus/vmbus: improve debug output Stephen Hemminger 3 siblings, 1 reply; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger The PMD_TX_LOG and PMD_RX_LOG can hide errors since the debug log is typically disabled. Change the code to use PMD_DRV_LOG for errors. Under load, the ring buffer to the host can fill. Add some statistics to estimate the impact and see other errors. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 1 + drivers/net/netvsc/hn_rxtx.c | 29 ++++++++++++++++++++--------- drivers/net/netvsc/hn_var.h | 1 + 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 675a49e66a57..0f4827c54be4 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -58,6 +58,7 @@ static const struct hn_xstats_name_off hn_stat_strings[] = { { "good_bytes", offsetof(struct hn_stats, bytes) }, { "errors", offsetof(struct hn_stats, errors) }, { "ring full", offsetof(struct hn_stats, ring_full) }, + { "channel full", offsetof(struct hn_stats, channel_full) }, { "multicast_packets", offsetof(struct hn_stats, multicast) }, { "broadcast_packets", offsetof(struct hn_stats, broadcast) }, { "undersize_packets", offsetof(struct hn_stats, size_bins[0]) }, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 537fd0125687..fa53c7becc12 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -403,8 +403,8 @@ hn_nvs_send_completed(struct rte_eth_dev *dev, uint16_t queue_id, txq->stats.bytes += txd->data_size; txq->stats.packets += txd->packets; } else { - PMD_TX_LOG(NOTICE, "port %u:%u complete tx %u failed status %u", - txq->port_id, txq->queue_id, txd->chim_index, ack->status); + PMD_DRV_LOG(NOTICE, "port %u:%u complete tx %u failed status %u", + txq->port_id, txq->queue_id, txd->chim_index, ack->status); ++txq->stats.errors; } @@ -429,8 +429,7 @@ hn_nvs_handle_comp(struct rte_eth_dev *dev, uint16_t queue_id, break; default: - PMD_TX_LOG(NOTICE, - "unexpected send completion type %u", + PMD_DRV_LOG(NOTICE, "unexpected send completion type %u", hdr->type); } } @@ -648,6 +647,7 @@ static void hn_rxpkt(struct hn_rx_queue *rxq, struct hn_rx_bufinfo *rxb, if (unlikely(rte_ring_sp_enqueue(rxq->rx_ring, m) != 0)) { ++rxq->stats.ring_full; + PMD_RX_LOG(DEBUG, "rx ring full"); rte_pktmbuf_free(m); } } @@ -1156,10 +1156,16 @@ static int hn_flush_txagg(struct hn_tx_queue *txq, bool *need_sig) if (likely(ret == 0)) hn_reset_txagg(txq); - else - PMD_TX_LOG(NOTICE, "port %u:%u send failed: %d", - txq->port_id, txq->queue_id, ret); + else if (ret == -EAGAIN) { + PMD_TX_LOG(DEBUG, "port %u:%u channel full", + txq->port_id, txq->queue_id); + ++txq->stats.channel_full; + } else { + ++txq->stats.errors; + PMD_DRV_LOG(NOTICE, "port %u:%u send failed: %d", + txq->port_id, txq->queue_id, ret); + } return ret; } @@ -1505,8 +1511,13 @@ hn_xmit_pkts(void *ptxq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) ret = hn_xmit_sg(txq, txd, m, &need_sig); if (unlikely(ret != 0)) { - PMD_TX_LOG(NOTICE, "sg send failed: %d", ret); - ++txq->stats.errors; + if (ret == -EAGAIN) { + PMD_TX_LOG(DEBUG, "sg channel full"); + ++txq->stats.channel_full; + } else { + PMD_DRV_LOG(NOTICE, "sg send failed: %d", ret); + ++txq->stats.errors; + } hn_txd_put(txq, txd); goto fail; } diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index ded0d4c60a04..20363389a0d6 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -40,6 +40,7 @@ struct hn_stats { uint64_t bytes; uint64_t errors; uint64_t ring_full; + uint64_t channel_full; uint64_t multicast; uint64_t broadcast; /* Size bins in array as RFC 2819, undersized [0], 64 [1], etc */ -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [dpdk-dev] [PATCH v2 3/4] net/netvsc: change tx/rx error handling 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger @ 2020-05-07 15:29 ` Ferruh Yigit 0 siblings, 0 replies; 23+ messages in thread From: Ferruh Yigit @ 2020-05-07 15:29 UTC (permalink / raw) To: Stephen Hemminger, dev On 5/6/2020 8:52 PM, Stephen Hemminger wrote: > The PMD_TX_LOG and PMD_RX_LOG can hide errors since the > debug log is typically disabled. Change the code to use > PMD_DRV_LOG for errors. This will cost additional cycles in fast path, but I assume you know and accept it. > > Under load, the ring buffer to the host can fill. > Add some statistics to estimate the impact and see other errors. > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> <...> ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v2 4/4] bus/vmbus: improve debug output 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger ` (2 preceding siblings ...) 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger @ 2020-05-06 19:52 ` Stephen Hemminger 3 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-06 19:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger Make debug output more compact. Don't print pending size or interrupt mask for the transmit side (not applicable). Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/bus/vmbus/vmbus_channel.c | 42 +++++++++++++++++++------------ 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/drivers/bus/vmbus/vmbus_channel.c b/drivers/bus/vmbus/vmbus_channel.c index ff2985c25758..5df7deadd3f1 100644 --- a/drivers/bus/vmbus/vmbus_channel.c +++ b/drivers/bus/vmbus/vmbus_channel.c @@ -421,26 +421,36 @@ void rte_vmbus_chan_close(struct vmbus_channel *chan) static void vmbus_dump_ring(FILE *f, const char *id, const struct vmbus_br *br) { const struct vmbus_bufring *vbr = br->vbr; - struct vmbus_chanpkt_hdr pkt; - fprintf(f, "%s windex=%u rindex=%u mask=%u pending=%u feature=%#x\n", - id, vbr->windex, vbr->rindex, vbr->imask, - vbr->pending_send, vbr->feature_bits.value); - fprintf(f, " size=%u avail write=%u read=%u\n", - br->dsize, vmbus_br_availwrite(br, vbr->windex), + fprintf(f, "%s index=%u/%u avail=%u/%u\n", + id, vbr->windex, vbr->rindex, + vmbus_br_availwrite(br, vbr->windex), vmbus_br_availread(br)); - - if (vmbus_rxbr_peek(br, &pkt, sizeof(pkt)) == 0) - fprintf(f, " pkt type %#x len %u flags %#x xactid %#"PRIx64"\n", - pkt.type, - pkt.tlen << VMBUS_CHANPKT_SIZE_SHIFT, - pkt.flags, pkt.xactid); } void rte_vmbus_chan_dump(FILE *f, const struct vmbus_channel *chan) { - fprintf(f, "channel[%u] relid=%u monitor=%u\n", - chan->subchannel_id, chan->relid, chan->monitor_id); - vmbus_dump_ring(f, "rxbr", &chan->rxbr); - vmbus_dump_ring(f, "txbr", &chan->txbr); + const struct vmbus_br *rbr = &chan->rxbr; + const struct vmbus_bufring *vbr = rbr->vbr; + struct vmbus_chanpkt_hdr pkt; + + fprintf(f, "chan %u: relid=%u mask=%u", + chan->subchannel_id, chan->relid, vbr->imask); + + if (vbr->feature_bits.feat_pending_send_sz) + fprintf(f, " pending_sz=%u\n", + vbr->pending_send); + else + fprintf(f, "\n"); + + vmbus_dump_ring(f, " tx", &chan->txbr); + vmbus_dump_ring(f, " rx", rbr); + + /* show first packet if any from host */ + if (vmbus_rxbr_peek(rbr, &pkt, sizeof(pkt))) + return; + + fprintf(f, "[pkt type %#x len %u flags %#x xactid %#"PRIx64"]\n", + pkt.type, pkt.tlen << VMBUS_CHANPKT_SIZE_SHIFT, + pkt.flags, pkt.xactid); } -- 2.20.1 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger ` (4 preceding siblings ...) 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 1/7] net/netvsc: support per-queue info requests Stephen Hemminger ` (7 more replies) 5 siblings, 8 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger This set of patches includes bugfixes for races in VF handling and implements the queue and descriptor status functions in netvsc PMD. This patch set is likely to cause merge warnings for the automated tools because it is based on top of my previous netvsc PMD patches already in net-next branch. v3 - review feedback from v2 fixes for VF notification races Stephen Hemminger (7): net/netvsc: support per-queue info requests net/netvsc: implement rx/tx descriptor status functions net/netvsc: change tx/rx logging net/netvsc: fix warning when VF is removed net/netvsc: don't query VF link state net/netvsc: process link change messages in alarm net/netvsc: do not spin forever waiting for reply drivers/net/netvsc/hn_ethdev.c | 10 +++- drivers/net/netvsc/hn_rndis.c | 51 +++++++++++++--- drivers/net/netvsc/hn_rxtx.c | 106 ++++++++++++++++++++++++++++++--- drivers/net/netvsc/hn_var.h | 11 +++- drivers/net/netvsc/hn_vf.c | 104 ++++++-------------------------- 5 files changed, 173 insertions(+), 109 deletions(-) -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 1/7] net/netvsc: support per-queue info requests 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger ` (6 subsequent siblings) 7 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger There is not a lot of info here from this driver. But worth supporting these additional info queries. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 2 ++ drivers/net/netvsc/hn_rxtx.c | 20 ++++++++++++++++++++ drivers/net/netvsc/hn_var.h | 2 ++ 3 files changed, 24 insertions(+) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 55b8a63804a9..b890fa3befa8 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -860,6 +860,8 @@ static const struct eth_dev_ops hn_eth_dev_ops = { .dev_stop = hn_dev_stop, .dev_close = hn_dev_close, .dev_infos_get = hn_dev_info_get, + .txq_info_get = hn_dev_tx_queue_info, + .rxq_info_get = hn_dev_rx_queue_info, .dev_supported_ptypes_get = hn_vf_supported_ptypes, .promiscuous_enable = hn_dev_promiscuous_enable, .promiscuous_disable = hn_dev_promiscuous_disable, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 31fae5597598..668f48fcbf15 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -319,6 +319,15 @@ hn_dev_tx_queue_setup(struct rte_eth_dev *dev, return err; } +void +hn_dev_tx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_txq_info *qinfo) +{ + struct hn_tx_queue *txq = dev->data->tx_queues[queue_id]; + + qinfo->nb_desc = txq->txdesc_pool->size; + qinfo->conf.offloads = dev->data->dev_conf.txmode.offloads; +} static struct hn_txdesc *hn_txd_get(struct hn_tx_queue *txq) { @@ -859,6 +868,17 @@ struct hn_rx_queue *hn_rx_queue_alloc(struct hn_data *hv, return rxq; } +void +hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_rxq_info *qinfo) +{ + struct hn_rx_queue *rxq = dev->data->rx_queues[queue_id]; + + qinfo->mp = rxq->mb_pool; + qinfo->nb_desc = rxq->rx_ring->size; + qinfo->conf.offloads = dev->data->dev_conf.rxmode.offloads; +} + int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index d1d38b459601..93168d2e0102 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -178,6 +178,8 @@ int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +void hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, + struct rte_eth_rxq_info *qinfo); void hn_dev_rx_queue_release(void *arg); void hn_dev_free_queues(struct rte_eth_dev *dev); -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 1/7] net/netvsc: support per-queue info requests Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-27 18:14 ` Ferruh Yigit 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 3/7] net/netvsc: change tx/rx logging Stephen Hemminger ` (5 subsequent siblings) 7 siblings, 1 reply; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger These functions are useful for applications and debugging. The netvsc PMD also transparently handles the rx/tx descriptor functions for underlying VF device. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 3 ++ drivers/net/netvsc/hn_rxtx.c | 57 ++++++++++++++++++++++++++++++++++ drivers/net/netvsc/hn_var.h | 5 +++ 3 files changed, 65 insertions(+) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index b890fa3befa8..675a49e66a57 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -875,8 +875,11 @@ static const struct eth_dev_ops hn_eth_dev_ops = { .tx_queue_setup = hn_dev_tx_queue_setup, .tx_queue_release = hn_dev_tx_queue_release, .tx_done_cleanup = hn_dev_tx_done_cleanup, + .tx_descriptor_status = hn_dev_tx_descriptor_status, .rx_queue_setup = hn_dev_rx_queue_setup, .rx_queue_release = hn_dev_rx_queue_release, + .rx_queue_count = hn_dev_rx_queue_count, + .rx_descriptor_status = hn_dev_rx_queue_status, .link_update = hn_dev_link_update, .stats_get = hn_dev_stats_get, .stats_reset = hn_dev_stats_reset, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index 668f48fcbf15..b22d8ea7feb0 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -369,6 +369,29 @@ hn_dev_tx_queue_release(void *arg) rte_free(txq); } +/* + * Check the status of a Tx descriptor in the queue. + * + * returns: + * - -EINVAL - offset outside of tx_descriptor pool. + * - RTE_ETH_TX_DESC_FULL - descriptor is not acknowleged by host. + * - RTE_ETH_TX_DESC_DONE - descriptor is available. + */ +int hn_dev_tx_descriptor_status(void *arg, uint16_t offset) +{ + const struct hn_tx_queue *txq = arg; + + hn_process_events(txq->hv, txq->queue_id, 0); + + if (offset >= rte_mempool_avail_count(txq->txdesc_pool)) + return -EINVAL; + + if (offset < rte_mempool_in_use_count(txq->txdesc_pool)) + return RTE_ETH_TX_DESC_FULL; + else + return RTE_ETH_TX_DESC_DONE; +} + static void hn_nvs_send_completed(struct rte_eth_dev *dev, uint16_t queue_id, unsigned long xactid, const struct hn_nvs_rndis_ack *ack) @@ -966,6 +989,40 @@ hn_dev_rx_queue_release(void *arg) hn_rx_queue_free(rxq, true); } +/* + * Get the number of used descriptor in a rx queue + * For this device that means how many packets are pending in the ring. + */ +uint32_t +hn_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id) +{ + struct hn_rx_queue *rxq = dev->data->rx_queues[queue_id]; + + return rte_ring_count(rxq->rx_ring); +} + +/* + * Check the status of a Rx descriptor in the queue + * + * returns: + * - -EINVAL - offset outside of ring + * - RTE_ETH_RX_DESC_AVAIL - no data available yet + * - RTE_ETH_RX_DESC_DONE - data is waiting in stagin ring + */ +int hn_dev_rx_queue_status(void *arg, uint16_t offset) +{ + const struct hn_rx_queue *rxq = arg; + + hn_process_events(rxq->hv, rxq->queue_id, 0); + if (offset >= rxq->rx_ring->capacity) + return -EINVAL; + + if (offset < rte_ring_count(rxq->rx_ring)) + return RTE_ETH_RX_DESC_DONE; + else + return RTE_ETH_RX_DESC_AVAIL; +} + int hn_dev_tx_done_cleanup(void *arg, uint32_t free_cnt) { diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index 93168d2e0102..ded0d4c60a04 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -169,6 +169,7 @@ void hn_dev_tx_queue_release(void *arg); void hn_dev_tx_queue_info(struct rte_eth_dev *dev, uint16_t queue_idx, struct rte_eth_txq_info *qinfo); int hn_dev_tx_done_cleanup(void *arg, uint32_t free_cnt); +int hn_dev_tx_descriptor_status(void *arg, uint16_t offset); struct hn_rx_queue *hn_rx_queue_alloc(struct hn_data *hv, uint16_t queue_id, @@ -181,6 +182,8 @@ int hn_dev_rx_queue_setup(struct rte_eth_dev *dev, void hn_dev_rx_queue_info(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_info *qinfo); void hn_dev_rx_queue_release(void *arg); +uint32_t hn_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id); +int hn_dev_rx_queue_status(void *rxq, uint16_t offset); void hn_dev_free_queues(struct rte_eth_dev *dev); /* Check if VF is attached */ @@ -231,6 +234,8 @@ int hn_vf_tx_queue_setup(struct rte_eth_dev *dev, unsigned int socket_id, const struct rte_eth_txconf *tx_conf); void hn_vf_tx_queue_release(struct hn_data *hv, uint16_t queue_id); +int hn_vf_tx_queue_status(struct hn_data *hv, uint16_t queue_id, uint16_t offset); + int hn_vf_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger @ 2020-05-27 18:14 ` Ferruh Yigit 2020-05-27 21:00 ` Stephen Hemminger 0 siblings, 1 reply; 23+ messages in thread From: Ferruh Yigit @ 2020-05-27 18:14 UTC (permalink / raw) To: Stephen Hemminger, dev On 5/19/2020 5:52 PM, Stephen Hemminger wrote: > These functions are useful for applications and debugging. > The netvsc PMD also transparently handles the rx/tx descriptor > functions for underlying VF device. > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> > --- > drivers/net/netvsc/hn_ethdev.c | 3 ++ > drivers/net/netvsc/hn_rxtx.c | 57 ++++++++++++++++++++++++++++++++++ > drivers/net/netvsc/hn_var.h | 5 +++ > 3 files changed, 65 insertions(+) > > diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c > index b890fa3befa8..675a49e66a57 100644 > --- a/drivers/net/netvsc/hn_ethdev.c > +++ b/drivers/net/netvsc/hn_ethdev.c > @@ -875,8 +875,11 @@ static const struct eth_dev_ops hn_eth_dev_ops = { > .tx_queue_setup = hn_dev_tx_queue_setup, > .tx_queue_release = hn_dev_tx_queue_release, > .tx_done_cleanup = hn_dev_tx_done_cleanup, > + .tx_descriptor_status = hn_dev_tx_descriptor_status, > .rx_queue_setup = hn_dev_rx_queue_setup, > .rx_queue_release = hn_dev_rx_queue_release, > + .rx_queue_count = hn_dev_rx_queue_count, > + .rx_descriptor_status = hn_dev_rx_queue_status, I guess this should announce 'Rx descriptor status' & 'Tx descriptor status' features for NIC, in 'doc/guides/nics/features/netvsc.ini' file. If so please send an incremental patch for it, I can squash it in the next-net. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions 2020-05-27 18:14 ` Ferruh Yigit @ 2020-05-27 21:00 ` Stephen Hemminger 0 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-27 21:00 UTC (permalink / raw) To: Ferruh Yigit; +Cc: dev On Wed, 27 May 2020 19:14:49 +0100 Ferruh Yigit <ferruh.yigit@intel.com> wrote: > On 5/19/2020 5:52 PM, Stephen Hemminger wrote: > > These functions are useful for applications and debugging. > > The netvsc PMD also transparently handles the rx/tx descriptor > > functions for underlying VF device. > > > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> > > --- > > drivers/net/netvsc/hn_ethdev.c | 3 ++ > > drivers/net/netvsc/hn_rxtx.c | 57 ++++++++++++++++++++++++++++++++++ > > drivers/net/netvsc/hn_var.h | 5 +++ > > 3 files changed, 65 insertions(+) > > > > diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c > > index b890fa3befa8..675a49e66a57 100644 > > --- a/drivers/net/netvsc/hn_ethdev.c > > +++ b/drivers/net/netvsc/hn_ethdev.c > > @@ -875,8 +875,11 @@ static const struct eth_dev_ops hn_eth_dev_ops = { > > .tx_queue_setup = hn_dev_tx_queue_setup, > > .tx_queue_release = hn_dev_tx_queue_release, > > .tx_done_cleanup = hn_dev_tx_done_cleanup, > > + .tx_descriptor_status = hn_dev_tx_descriptor_status, > > .rx_queue_setup = hn_dev_rx_queue_setup, > > .rx_queue_release = hn_dev_rx_queue_release, > > + .rx_queue_count = hn_dev_rx_queue_count, > > + .rx_descriptor_status = hn_dev_rx_queue_status, > > I guess this should announce 'Rx descriptor status' & 'Tx descriptor status' > features for NIC, in 'doc/guides/nics/features/netvsc.ini' file. > > If so please send an incremental patch for it, I can squash it in the next-net. Thanks, will do ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 3/7] net/netvsc: change tx/rx logging 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 1/7] net/netvsc: support per-queue info requests Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 4/7] net/netvsc: fix warning when VF is removed Stephen Hemminger ` (4 subsequent siblings) 7 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger The PMD_TX_LOG and PMD_RX_LOG can hide errors since this debug log is typically disabled. Change the code to use PMD_DRV_LOG for errors. Under load, the ring buffer to the host can fill. Add some statistics to estimate the impact and see other errors. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 1 + drivers/net/netvsc/hn_rxtx.c | 29 ++++++++++++++++++++--------- drivers/net/netvsc/hn_var.h | 1 + 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 675a49e66a57..0f4827c54be4 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -58,6 +58,7 @@ static const struct hn_xstats_name_off hn_stat_strings[] = { { "good_bytes", offsetof(struct hn_stats, bytes) }, { "errors", offsetof(struct hn_stats, errors) }, { "ring full", offsetof(struct hn_stats, ring_full) }, + { "channel full", offsetof(struct hn_stats, channel_full) }, { "multicast_packets", offsetof(struct hn_stats, multicast) }, { "broadcast_packets", offsetof(struct hn_stats, broadcast) }, { "undersize_packets", offsetof(struct hn_stats, size_bins[0]) }, diff --git a/drivers/net/netvsc/hn_rxtx.c b/drivers/net/netvsc/hn_rxtx.c index b22d8ea7feb0..853de2daf7e7 100644 --- a/drivers/net/netvsc/hn_rxtx.c +++ b/drivers/net/netvsc/hn_rxtx.c @@ -412,8 +412,8 @@ hn_nvs_send_completed(struct rte_eth_dev *dev, uint16_t queue_id, txq->stats.bytes += txd->data_size; txq->stats.packets += txd->packets; } else { - PMD_TX_LOG(NOTICE, "port %u:%u complete tx %u failed status %u", - txq->port_id, txq->queue_id, txd->chim_index, ack->status); + PMD_DRV_LOG(NOTICE, "port %u:%u complete tx %u failed status %u", + txq->port_id, txq->queue_id, txd->chim_index, ack->status); ++txq->stats.errors; } @@ -438,8 +438,7 @@ hn_nvs_handle_comp(struct rte_eth_dev *dev, uint16_t queue_id, break; default: - PMD_TX_LOG(NOTICE, - "unexpected send completion type %u", + PMD_DRV_LOG(NOTICE, "unexpected send completion type %u", hdr->type); } } @@ -657,6 +656,7 @@ static void hn_rxpkt(struct hn_rx_queue *rxq, struct hn_rx_bufinfo *rxb, if (unlikely(rte_ring_sp_enqueue(rxq->rx_ring, m) != 0)) { ++rxq->stats.ring_full; + PMD_RX_LOG(DEBUG, "rx ring full"); rte_pktmbuf_free(m); } } @@ -1174,10 +1174,16 @@ static int hn_flush_txagg(struct hn_tx_queue *txq, bool *need_sig) if (likely(ret == 0)) hn_reset_txagg(txq); - else - PMD_TX_LOG(NOTICE, "port %u:%u send failed: %d", - txq->port_id, txq->queue_id, ret); + else if (ret == -EAGAIN) { + PMD_TX_LOG(DEBUG, "port %u:%u channel full", + txq->port_id, txq->queue_id); + ++txq->stats.channel_full; + } else { + ++txq->stats.errors; + PMD_DRV_LOG(NOTICE, "port %u:%u send failed: %d", + txq->port_id, txq->queue_id, ret); + } return ret; } @@ -1523,8 +1529,13 @@ hn_xmit_pkts(void *ptxq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) ret = hn_xmit_sg(txq, txd, m, &need_sig); if (unlikely(ret != 0)) { - PMD_TX_LOG(NOTICE, "sg send failed: %d", ret); - ++txq->stats.errors; + if (ret == -EAGAIN) { + PMD_TX_LOG(DEBUG, "sg channel full"); + ++txq->stats.channel_full; + } else { + PMD_DRV_LOG(NOTICE, "sg send failed: %d", ret); + ++txq->stats.errors; + } hn_txd_put(txq, txd); goto fail; } diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index ded0d4c60a04..20363389a0d6 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -40,6 +40,7 @@ struct hn_stats { uint64_t bytes; uint64_t errors; uint64_t ring_full; + uint64_t channel_full; uint64_t multicast; uint64_t broadcast; /* Size bins in array as RFC 2819, undersized [0], 64 [1], etc */ -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 4/7] net/netvsc: fix warning when VF is removed 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger ` (2 preceding siblings ...) 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 3/7] net/netvsc: change tx/rx logging Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 5/7] net/netvsc: don't query VF link state Stephen Hemminger ` (3 subsequent siblings) 7 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger The code to unset owner of VF device was changing port to invalid value before calling unset. Fixes: 4a9efcddaddd ("net/netvsc: fix VF support with secondary process") Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_vf.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c index b7e3ba46bf63..a745c7c36ea3 100644 --- a/drivers/net/netvsc/hn_vf.c +++ b/drivers/net/netvsc/hn_vf.c @@ -120,11 +120,11 @@ static void hn_vf_remove(struct hn_data *hv) /* Stop incoming packets from arriving on VF */ hn_nvs_set_datapath(hv, NVS_DATAPATH_SYNTHETIC); - /* Stop transmission over VF */ - hv->vf_port = HN_INVALID_PORT; - /* Give back ownership */ rte_eth_dev_owner_unset(hv->vf_port, hv->owner.id); + + /* Stop transmission over VF */ + hv->vf_port = HN_INVALID_PORT; } } -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 5/7] net/netvsc: don't query VF link state 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger ` (3 preceding siblings ...) 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 4/7] net/netvsc: fix warning when VF is removed Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 6/7] net/netvsc: process link change messages in alarm Stephen Hemminger ` (2 subsequent siblings) 7 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger When the primary device link state is queried, there is no need to query the VF state as well. The application only sees the state of the synthetic device. Fixes: dc7680e8597c ("net/netvsc: support integrated VF") Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_ethdev.c | 4 +- drivers/net/netvsc/hn_var.h | 3 -- drivers/net/netvsc/hn_vf.c | 98 +++++----------------------------- 3 files changed, 15 insertions(+), 90 deletions(-) diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c index 0f4827c54be4..ac182a548645 100644 --- a/drivers/net/netvsc/hn_ethdev.c +++ b/drivers/net/netvsc/hn_ethdev.c @@ -202,7 +202,7 @@ static int hn_parse_args(const struct rte_eth_dev *dev) */ int hn_dev_link_update(struct rte_eth_dev *dev, - int wait_to_complete) + int wait_to_complete __rte_unused) { struct hn_data *hv = dev->data->dev_private; struct rte_eth_link link, old; @@ -216,8 +216,6 @@ hn_dev_link_update(struct rte_eth_dev *dev, hn_rndis_get_linkspeed(hv); - hn_vf_link_update(dev, wait_to_complete); - link = (struct rte_eth_link) { .link_duplex = ETH_LINK_FULL_DUPLEX, .link_autoneg = ETH_LINK_SPEED_FIXED, diff --git a/drivers/net/netvsc/hn_var.h b/drivers/net/netvsc/hn_var.h index 20363389a0d6..881832d85719 100644 --- a/drivers/net/netvsc/hn_var.h +++ b/drivers/net/netvsc/hn_var.h @@ -141,7 +141,6 @@ struct hn_data { uint16_t rss_ind[128]; struct rte_eth_dev_owner owner; - struct rte_intr_handle vf_intr; struct vmbus_channel *channels[HN_MAX_CHANNELS]; }; @@ -228,8 +227,6 @@ int hn_vf_mc_addr_list(struct rte_eth_dev *dev, struct rte_ether_addr *mc_addr_set, uint32_t nb_mc_addr); -int hn_vf_link_update(struct rte_eth_dev *dev, - int wait_to_complete); int hn_vf_tx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, uint16_t nb_desc, unsigned int socket_id, diff --git a/drivers/net/netvsc/hn_vf.c b/drivers/net/netvsc/hn_vf.c index a745c7c36ea3..f5f15c046207 100644 --- a/drivers/net/netvsc/hn_vf.c +++ b/drivers/net/netvsc/hn_vf.c @@ -97,15 +97,8 @@ int hn_vf_add(struct rte_eth_dev *dev, struct hn_data *hv) } err = hn_vf_attach(hv, port); - if (err == 0) { - dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC; - hv->vf_intr = (struct rte_intr_handle) { - .fd = -1, - .type = RTE_INTR_HANDLE_EXT, - }; - dev->intr_handle = &hv->vf_intr; + if (err == 0) hn_nvs_set_datapath(hv, NVS_DATAPATH_VF); - } return err; } @@ -223,80 +216,6 @@ int hn_vf_info_get(struct hn_data *hv, struct rte_eth_dev_info *info) return ret; } -int hn_vf_link_update(struct rte_eth_dev *dev, - int wait_to_complete) -{ - struct hn_data *hv = dev->data->dev_private; - struct rte_eth_dev *vf_dev; - int ret = 0; - - rte_rwlock_read_lock(&hv->vf_lock); - vf_dev = hn_get_vf_dev(hv); - if (vf_dev && vf_dev->dev_ops->link_update) - ret = (*vf_dev->dev_ops->link_update)(vf_dev, wait_to_complete); - rte_rwlock_read_unlock(&hv->vf_lock); - - return ret; -} - -/* called when VF has link state interrupts enabled */ -static int hn_vf_lsc_event(uint16_t port_id __rte_unused, - enum rte_eth_event_type event, - void *cb_arg, void *out __rte_unused) -{ - struct rte_eth_dev *dev = cb_arg; - - if (event != RTE_ETH_EVENT_INTR_LSC) - return 0; - - /* if link state has changed pass on */ - if (hn_dev_link_update(dev, 0) == 0) - return 0; /* no change */ - - return _rte_eth_dev_callback_process(dev, - RTE_ETH_EVENT_INTR_LSC, - NULL); -} - -static int _hn_vf_configure(struct rte_eth_dev *dev, - uint16_t vf_port, - const struct rte_eth_conf *dev_conf) -{ - struct rte_eth_conf vf_conf = *dev_conf; - struct rte_eth_dev *vf_dev; - int ret; - - vf_dev = &rte_eth_devices[vf_port]; - if (dev_conf->intr_conf.lsc && - (vf_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC)) { - PMD_DRV_LOG(DEBUG, "enabling LSC for VF %u", - vf_port); - vf_conf.intr_conf.lsc = 1; - } else { - PMD_DRV_LOG(DEBUG, "disabling LSC for VF %u", - vf_port); - vf_conf.intr_conf.lsc = 0; - } - - ret = rte_eth_dev_configure(vf_port, - dev->data->nb_rx_queues, - dev->data->nb_tx_queues, - &vf_conf); - if (ret) { - PMD_DRV_LOG(ERR, - "VF configuration failed: %d", ret); - } else if (vf_conf.intr_conf.lsc) { - ret = rte_eth_dev_callback_register(vf_port, - RTE_ETH_DEV_INTR_LSC, - hn_vf_lsc_event, dev); - if (ret) - PMD_DRV_LOG(ERR, - "Failed to register LSC callback for VF %u", - vf_port); - } - return ret; -} - /* * Configure VF if present. * Force VF to have same number of queues as synthetic device @@ -305,11 +224,22 @@ int hn_vf_configure(struct rte_eth_dev *dev, const struct rte_eth_conf *dev_conf) { struct hn_data *hv = dev->data->dev_private; + struct rte_eth_conf vf_conf = *dev_conf; int ret = 0; + /* link state interrupt does not matter here. */ + vf_conf.intr_conf.lsc = 0; + rte_rwlock_read_lock(&hv->vf_lock); - if (hv->vf_port != HN_INVALID_PORT) - ret = _hn_vf_configure(dev, hv->vf_port, dev_conf); + if (hv->vf_port != HN_INVALID_PORT) { + ret = rte_eth_dev_configure(hv->vf_port, + dev->data->nb_rx_queues, + dev->data->nb_tx_queues, + &vf_conf); + if (ret != 0) + PMD_DRV_LOG(ERR, + "VF configuration failed: %d", ret); + } rte_rwlock_read_unlock(&hv->vf_lock); return ret; } -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 6/7] net/netvsc: process link change messages in alarm 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger ` (4 preceding siblings ...) 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 5/7] net/netvsc: don't query VF link state Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 7/7] net/netvsc: do not spin forever waiting for reply Stephen Hemminger 2020-05-27 18:16 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Ferruh Yigit 7 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger The original code would deadlock itself if a link change event happened with link state interrupt enabled. The problem is that the link state changed message would be seen while reading the host to guest ring (under lock) and then the driver would send a query to the host to see the new link state. The response would never be seen (stuck in a while loop) waiting for the response. The solution is to use the link change indication to trigger a DPDK alarm. The alarm will happen in a different thread and in that context it can send request for new link state and also do interrupt callback. This is similar to how the bonding driver is handling the same thing. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_rndis.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/net/netvsc/hn_rndis.c b/drivers/net/netvsc/hn_rndis.c index 7947ca2331e2..6a850ce5f226 100644 --- a/drivers/net/netvsc/hn_rndis.c +++ b/drivers/net/netvsc/hn_rndis.c @@ -17,6 +17,7 @@ #include <rte_memzone.h> #include <rte_malloc.h> #include <rte_atomic.h> +#include <rte_alarm.h> #include <rte_branch_prediction.h> #include <rte_ether.h> #include <rte_common.h> @@ -281,6 +282,15 @@ static int hn_nvs_send_rndis_ctrl(struct vmbus_channel *chan, &nvs_rndis, sizeof(nvs_rndis), 0U, NULL); } +/* + * Alarm callback to process link changed notifications. + * Can not directly since link_status is discovered while reading ring + */ +static void hn_rndis_link_alarm(void *arg) +{ + _rte_eth_dev_callback_process(arg, RTE_ETH_EVENT_INTR_LSC, NULL); +} + void hn_rndis_link_status(struct rte_eth_dev *dev, const void *msg) { const struct rndis_status_msg *indicate = msg; @@ -298,11 +308,8 @@ void hn_rndis_link_status(struct rte_eth_dev *dev, const void *msg) case RNDIS_STATUS_LINK_SPEED_CHANGE: case RNDIS_STATUS_MEDIA_CONNECT: case RNDIS_STATUS_MEDIA_DISCONNECT: - if (dev->data->dev_conf.intr_conf.lsc && - hn_dev_link_update(dev, 0) == 0) - _rte_eth_dev_callback_process(dev, - RTE_ETH_EVENT_INTR_LSC, - NULL); + if (dev->data->dev_conf.intr_conf.lsc) + rte_eal_alarm_set(10, hn_rndis_link_alarm, dev); break; default: PMD_DRV_LOG(NOTICE, "unknown RNDIS indication: %#x", @@ -1101,6 +1108,10 @@ hn_rndis_attach(struct hn_data *hv) void hn_rndis_detach(struct hn_data *hv) { + struct rte_eth_dev *dev = &rte_eth_devices[hv->port_id]; + + rte_eal_alarm_cancel(hn_rndis_link_alarm, dev); + /* Halt the RNDIS. */ hn_rndis_halt(hv); } -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [dpdk-dev] [PATCH v3 7/7] net/netvsc: do not spin forever waiting for reply 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger ` (5 preceding siblings ...) 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 6/7] net/netvsc: process link change messages in alarm Stephen Hemminger @ 2020-05-19 16:52 ` Stephen Hemminger 2020-05-27 18:16 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Ferruh Yigit 7 siblings, 0 replies; 23+ messages in thread From: Stephen Hemminger @ 2020-05-19 16:52 UTC (permalink / raw) To: dev; +Cc: Stephen Hemminger Because of bugs in driver or host a reply to a request might never occur. Better to give an error than spin forever. Fixes: 4e9c73e96e83 ("net/netvsc: add Hyper-V network device") Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> --- drivers/net/netvsc/hn_rndis.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/drivers/net/netvsc/hn_rndis.c b/drivers/net/netvsc/hn_rndis.c index 6a850ce5f226..d186ddeabda9 100644 --- a/drivers/net/netvsc/hn_rndis.c +++ b/drivers/net/netvsc/hn_rndis.c @@ -10,6 +10,7 @@ #include <stdio.h> #include <errno.h> #include <unistd.h> +#include <time.h> #include <rte_ethdev_driver.h> #include <rte_ethdev.h> @@ -34,6 +35,9 @@ #include "hn_rndis.h" #include "ndis.h" +#define RNDIS_TIMEOUT_SEC 5 +#define RNDIS_DELAY_MS 10 + #define HN_RNDIS_XFER_SIZE 0x4000 #define HN_NDIS_TXCSUM_CAP_IP4 \ @@ -354,7 +358,7 @@ void hn_rndis_receive_response(struct hn_data *hv, rte_smp_wmb(); if (rte_atomic32_cmpset(&hv->rndis_pending, hdr->rid, 0) == 0) { - PMD_DRV_LOG(ERR, + PMD_DRV_LOG(NOTICE, "received id %#x pending id %#x", hdr->rid, (uint32_t)hv->rndis_pending); } @@ -377,6 +381,11 @@ static int hn_rndis_exec1(struct hn_data *hv, return -EIO; } + if (rid == 0) { + PMD_DRV_LOG(ERR, "Invalid request id"); + return -EINVAL; + } + if (comp != NULL && rte_atomic32_cmpset(&hv->rndis_pending, 0, rid) == 0) { PMD_DRV_LOG(ERR, @@ -391,9 +400,26 @@ static int hn_rndis_exec1(struct hn_data *hv, } if (comp) { + time_t start = time(NULL); + /* Poll primary channel until response received */ - while (hv->rndis_pending == rid) + while (hv->rndis_pending == rid) { + if (hv->closed) + return -ENETDOWN; + + if (time(NULL) - start > RNDIS_TIMEOUT_SEC) { + PMD_DRV_LOG(ERR, + "RNDIS response timed out"); + + rte_atomic32_cmpset(&hv->rndis_pending, rid, 0); + return -ETIMEDOUT; + } + + if (rte_vmbus_chan_rx_empty(hv->primary->chan)) + rte_delay_ms(RNDIS_DELAY_MS); + hn_process_events(hv, 0, 1); + } memcpy(comp, hv->rndis_resp, comp_len); } -- 2.26.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger ` (6 preceding siblings ...) 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 7/7] net/netvsc: do not spin forever waiting for reply Stephen Hemminger @ 2020-05-27 18:16 ` Ferruh Yigit 7 siblings, 0 replies; 23+ messages in thread From: Ferruh Yigit @ 2020-05-27 18:16 UTC (permalink / raw) To: Stephen Hemminger, dev On 5/19/2020 5:52 PM, Stephen Hemminger wrote: > This set of patches includes bugfixes for races in VF handling > and implements the queue and descriptor status functions in netvsc PMD. > > This patch set is likely to cause merge warnings for the automated > tools because it is based on top of my previous netvsc PMD patches > already in net-next branch. > > v3 - review feedback from v2 > fixes for VF notification races > > Stephen Hemminger (7): > net/netvsc: support per-queue info requests > net/netvsc: implement rx/tx descriptor status functions > net/netvsc: change tx/rx logging > net/netvsc: fix warning when VF is removed > net/netvsc: don't query VF link state > net/netvsc: process link change messages in alarm > net/netvsc: do not spin forever waiting for reply Series applied to dpdk-next-net/master, thanks. (Please check comment on 2/7 for .ini file update) ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2020-05-27 21:01 UTC | newest] Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-05-06 19:27 [dpdk-dev] [PATCH 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 1/4] net/netvsc: support per-queue info requests Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger 2020-05-06 19:27 ` [dpdk-dev] [PATCH 4/4] bus/vmbus: improve debug output Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 0/4] net/netvsc: diagnostic enhancements Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 1/4] net/netvsc: support per-queue info requests Stephen Hemminger 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 2/4] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger 2020-05-07 15:27 ` Ferruh Yigit 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 3/4] net/netvsc: change tx/rx error handling Stephen Hemminger 2020-05-07 15:29 ` Ferruh Yigit 2020-05-06 19:52 ` [dpdk-dev] [PATCH v2 4/4] bus/vmbus: improve debug output Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 1/7] net/netvsc: support per-queue info requests Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 2/7] net/netvsc: implement rx/tx descriptor status functions Stephen Hemminger 2020-05-27 18:14 ` Ferruh Yigit 2020-05-27 21:00 ` Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 3/7] net/netvsc: change tx/rx logging Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 4/7] net/netvsc: fix warning when VF is removed Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 5/7] net/netvsc: don't query VF link state Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 6/7] net/netvsc: process link change messages in alarm Stephen Hemminger 2020-05-19 16:52 ` [dpdk-dev] [PATCH v3 7/7] net/netvsc: do not spin forever waiting for reply Stephen Hemminger 2020-05-27 18:16 ` [dpdk-dev] [PATCH v3 0/7] net/netvsc: VF bug fix and diagnostic support Ferruh Yigit
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).