From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: "Di, ChenxuX" <chenxux.di@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "Yang, Qiming" <qiming.yang@intel.com>,
"Di, ChenxuX" <chenxux.di@intel.com>
Subject: Re: [dpdk-dev] [PATCH v7 3/4] net/ixgbe: cleanup Tx buffers
Date: Thu, 9 Jan 2020 14:01:41 +0000 [thread overview]
Message-ID: <SN6PR11MB2558AF818E543328E5F48C249A390@SN6PR11MB2558.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20200109103822.89011-4-chenxux.di@intel.com>
Hi Chenxu,
Good progress wih _full_version, but still some issues remains I think.
More comments inline.
Konstantin
>
> Signed-off-by: Chenxu Di <chenxux.di@intel.com>
> ---
> drivers/net/ixgbe/ixgbe_ethdev.c | 4 +
> drivers/net/ixgbe/ixgbe_rxtx.c | 156 ++++++++++++++++++++++++++++++-
> drivers/net/ixgbe/ixgbe_rxtx.h | 10 ++
> 3 files changed, 169 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 2c6fd0f13..668c36188 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -601,6 +601,7 @@ static const struct eth_dev_ops ixgbe_eth_dev_ops = {
> .udp_tunnel_port_add = ixgbe_dev_udp_tunnel_port_add,
> .udp_tunnel_port_del = ixgbe_dev_udp_tunnel_port_del,
> .tm_ops_get = ixgbe_tm_ops_get,
> + .tx_done_cleanup = ixgbe_tx_done_cleanup,
> };
>
> /*
> @@ -649,6 +650,7 @@ static const struct eth_dev_ops ixgbevf_eth_dev_ops = {
> .reta_query = ixgbe_dev_rss_reta_query,
> .rss_hash_update = ixgbe_dev_rss_hash_update,
> .rss_hash_conf_get = ixgbe_dev_rss_hash_conf_get,
> + .tx_done_cleanup = ixgbe_tx_done_cleanup,
> };
>
> /* store statistics names and its offset in stats structure */
> @@ -1101,6 +1103,7 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev, void *init_params __rte_unused)
> eth_dev->rx_pkt_burst = &ixgbe_recv_pkts;
> eth_dev->tx_pkt_burst = &ixgbe_xmit_pkts;
> eth_dev->tx_pkt_prepare = &ixgbe_prep_pkts;
> + ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_scalar);
>
> /*
> * For secondary processes, we don't initialise any further as primary
> @@ -1580,6 +1583,7 @@ eth_ixgbevf_dev_init(struct rte_eth_dev *eth_dev)
> eth_dev->dev_ops = &ixgbevf_eth_dev_ops;
> eth_dev->rx_pkt_burst = &ixgbe_recv_pkts;
> eth_dev->tx_pkt_burst = &ixgbe_xmit_pkts;
> + ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_scalar);
>
> /* for secondary processes, we don't initialise any further as primary
> * has already done this work. Only check we don't need a different
> diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
> index fa572d184..122dae425 100644
> --- a/drivers/net/ixgbe/ixgbe_rxtx.c
> +++ b/drivers/net/ixgbe/ixgbe_rxtx.c
> @@ -92,6 +92,8 @@ uint16_t ixgbe_xmit_fixed_burst_vec(void *tx_queue, struct rte_mbuf **tx_pkts,
> uint16_t nb_pkts);
> #endif
>
> +static ixgbe_tx_done_cleanup_t ixgbe_tx_done_cleanup_op;
You can't have just one static variable here.
There could be several ixgbe devices and they could be configured in a different way.
I.E. txpkt_burst() is per device, so tx_done_cleanup() also has to be per device.
Probably the easiest way is to add new entry for tx_done_cleanup into struct ixgbe_txq_ops,
and set it properly in ixgbe_set_tx_function().
> +
> /*********************************************************************
> *
> * TX functions
> @@ -2306,6 +2308,152 @@ ixgbe_tx_queue_release_mbufs(struct ixgbe_tx_queue *txq)
> }
> }
>
> +int
> +ixgbe_tx_done_cleanup_scalar(struct ixgbe_tx_queue *txq, uint32_t free_cnt)
As a nit I would change _scalar to _full or so.
> +{
> + uint32_t pkt_cnt;
> + uint16_t i;
> + uint16_t tx_last;
> + uint16_t tx_id;
> + uint16_t nb_tx_to_clean;
> + uint16_t nb_tx_free_last;
> + struct ixgbe_tx_entry *swr_ring = txq->sw_ring;
> +
> + /* Start free mbuf from the next of tx_tail */
> + tx_last = txq->tx_tail;
> + tx_id = swr_ring[tx_last].next_id;
> +
> + if (txq->nb_tx_free == 0)
> + if (ixgbe_xmit_cleanup(txq))
As a nit it could be just if (ixgbe_set_tx_function && ixgbe_xmit_cleanup(txq))
> + return 0;
> +
> + nb_tx_to_clean = txq->nb_tx_free;
> + nb_tx_free_last = txq->nb_tx_free;
> + if (!free_cnt)
> + free_cnt = txq->nb_tx_desc;
> +
> + /* Loop through swr_ring to count the amount of
> + * freeable mubfs and packets.
> + */
> + for (pkt_cnt = 0; pkt_cnt < free_cnt; ) {
> + for (i = 0; i < nb_tx_to_clean &&
> + pkt_cnt < free_cnt &&
> + tx_id != tx_last; i++) {
> + if (swr_ring[tx_id].mbuf != NULL) {
> + rte_pktmbuf_free_seg(swr_ring[tx_id].mbuf);
> + swr_ring[tx_id].mbuf = NULL;
> +
> + /*
> + * last segment in the packet,
> + * increment packet count
> + */
> + pkt_cnt += (swr_ring[tx_id].last_id == tx_id);
> + }
> +
> + tx_id = swr_ring[tx_id].next_id;
> + }
> +
> + if (tx_id == tx_last || txq->tx_rs_thresh
> + > txq->nb_tx_desc - txq->nb_tx_free)
First condition (tx_id == tx_last) is porbably redundant here.
> + break;
> +
> + if (pkt_cnt < free_cnt) {
> + if (ixgbe_xmit_cleanup(txq))
> + break;
> +
> + nb_tx_to_clean = txq->nb_tx_free - nb_tx_free_last;
> + nb_tx_free_last = txq->nb_tx_free;
> + }
> + }
> +
> + PMD_TX_FREE_LOG(DEBUG,
> + "Free %u Packets successfully "
> + "(port=%d queue=%d)",
> + pkt_cnt, txq->port_id, txq->queue_id);
> +
> + return (int)pkt_cnt;
> +}
> +
> +int
> +ixgbe_tx_done_cleanup_vec(struct ixgbe_tx_queue *txq __rte_unused,
> + uint32_t free_cnt __rte_unused)
> +{
> + return -ENOTSUP;
> +}
> +
> +int
> +ixgbe_tx_done_cleanup_simple(struct ixgbe_tx_queue *txq,
> + uint32_t free_cnt)
> +{
> + uint16_t i;
> + uint16_t tx_first;
> + uint16_t tx_id;
> + uint32_t pkt_cnt;
> + struct ixgbe_tx_entry *swr_ring = txq->sw_ring;
Looks overcomplicated here.
TX simple (and vec) doesn't support mulsti-seg packets,
So one TXD - one mbuf, and one packet.
And ixgbe_tx_free_bufs() always retunrs/frees either 0 or tx_rs_thresh mbufs/packets.
So it probably can be something like that:
ixgbe_tx_done_cleanup_simple(struct ixgbe_tx_queue *txq, uint32_t free_cnt)
{
If (free_cnt == 0)
free_cnt = txq->nb_desc;
cnt = free_cnt - free_cnt % txq->tx_rs_thesh;
for (i = 0; i < cnt; i+= n) {
n = ixgbe_tx_free_bufs(txq);
if (n == 0)
break;
}
return i;
}
> +
> + /* Start free mbuf from tx_first */
> + tx_first = txq->tx_next_dd - (txq->tx_rs_thresh - 1);
> + tx_id = tx_first;
> +
> + /* while free_cnt is 0,
> + * suppose one mbuf per packet,
> + * try to free packets as many as possible
> + */
> + if (free_cnt == 0)
> + free_cnt = txq->nb_tx_desc;
> +
> + /* Loop through swr_ring to count freeable packets */
> + for (pkt_cnt = 0; pkt_cnt < free_cnt; ) {
> + if (txq->nb_tx_desc - txq->nb_tx_free < txq->tx_rs_thresh)
> + break;
> +
> + if (!ixgbe_tx_free_bufs(txq))
> + break;
> +
> + for (i = 0; i != txq->tx_rs_thresh &&
> + tx_id != tx_first; i++) {
> + /* last segment in the packet,
> + * increment packet count
> + */
> + pkt_cnt += (tx_id == swr_ring[tx_id].last_id);
> + tx_id = swr_ring[tx_id].next_id;
> + }
> +
> + if (tx_id == tx_first)
> + break;
> + }
> +
> + PMD_TX_FREE_LOG(DEBUG,
> + "Free %u packets successfully "
> + "(port=%d queue=%d)",
> + pkt_cnt, txq->port_id, txq->queue_id);
> +
> + return (int)pkt_cnt;
> +}
> +
> +int
> +ixgbe_tx_done_cleanup(void *txq, uint32_t free_cnt)
> +{
> + ixgbe_tx_done_cleanup_t func = ixgbe_get_tx_done_cleanup_func();
> +
> + if (!func)
> + return -ENOTSUP;
> +
> + return func(txq, free_cnt);
> +}
> +
> +void
> +ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_t fn)
> +{
> + ixgbe_tx_done_cleanup_op = fn;
> +}
> +
> +ixgbe_tx_done_cleanup_t
> +ixgbe_get_tx_done_cleanup_func(void)
> +{
> + return ixgbe_tx_done_cleanup_op;
> +}
> +
> static void __attribute__((cold))
> ixgbe_tx_free_swring(struct ixgbe_tx_queue *txq)
> {
> @@ -2398,9 +2546,14 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq)
> ixgbe_txq_vec_setup(txq) == 0)) {
> PMD_INIT_LOG(DEBUG, "Vector tx enabled.");
> dev->tx_pkt_burst = ixgbe_xmit_pkts_vec;
> - } else
> + ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_vec);
> + } else {
> #endif
> dev->tx_pkt_burst = ixgbe_xmit_pkts_simple;
> + ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_simple);
> +#ifdef RTE_IXGBE_INC_VECTOR
> + }
> +#endif
> } else {
> PMD_INIT_LOG(DEBUG, "Using full-featured tx code path");
> PMD_INIT_LOG(DEBUG,
> @@ -2412,6 +2565,7 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq)
> (unsigned long)RTE_PMD_IXGBE_TX_MAX_BURST);
> dev->tx_pkt_burst = ixgbe_xmit_pkts;
> dev->tx_pkt_prepare = ixgbe_prep_pkts;
> + ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_scalar);
> }
> }
>
> diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h b/drivers/net/ixgbe/ixgbe_rxtx.h
> index 505d344b9..a52597aa9 100644
> --- a/drivers/net/ixgbe/ixgbe_rxtx.h
> +++ b/drivers/net/ixgbe/ixgbe_rxtx.h
> @@ -253,6 +253,8 @@ struct ixgbe_txq_ops {
> IXGBE_ADVTXD_DCMD_DEXT |\
> IXGBE_ADVTXD_DCMD_EOP)
>
> +typedef int (*ixgbe_tx_done_cleanup_t)(struct ixgbe_tx_queue *txq,
> + uint32_t free_cnt);
>
> /* Takes an ethdev and a queue and sets up the tx function to be used based on
> * the queue parameters. Used in tx_queue_setup by primary process and then
> @@ -285,6 +287,14 @@ int ixgbe_rx_vec_dev_conf_condition_check(struct rte_eth_dev *dev);
> int ixgbe_rxq_vec_setup(struct ixgbe_rx_queue *rxq);
> void ixgbe_rx_queue_release_mbufs_vec(struct ixgbe_rx_queue *rxq);
>
> +void ixgbe_set_tx_done_cleanup_func(ixgbe_tx_done_cleanup_t fn);
> +ixgbe_tx_done_cleanup_t ixgbe_get_tx_done_cleanup_func(void);
> +
> +int ixgbe_tx_done_cleanup(void *txq, uint32_t free_cnt);
> +int ixgbe_tx_done_cleanup_scalar(struct ixgbe_tx_queue *txq, uint32_t free_cnt);
> +int ixgbe_tx_done_cleanup_vec(struct ixgbe_tx_queue *txq, uint32_t free_cnt);
> +int ixgbe_tx_done_cleanup_simple(struct ixgbe_tx_queue *txq, uint32_t free_cnt);
> +
> extern const uint32_t ptype_table[IXGBE_PACKET_TYPE_MAX];
> extern const uint32_t ptype_table_tn[IXGBE_PACKET_TYPE_TN_MAX];
>
> --
> 2.17.1
next prev parent reply other threads:[~2020-01-09 14:01 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-03 5:51 [dpdk-dev] [PATCH 0/4] drivers/net: " Chenxu Di
2019-12-03 5:51 ` [dpdk-dev] [PATCH 1/4] net/fm10k: " Chenxu Di
2019-12-03 5:51 ` [dpdk-dev] [PATCH 2/4] net/i40e: " Chenxu Di
2019-12-03 5:51 ` [dpdk-dev] [PATCH 3/4] net/ice: " Chenxu Di
2019-12-03 5:51 ` [dpdk-dev] [PATCH 4/4] net/ixgbe: " Chenxu Di
2019-12-20 3:02 ` [dpdk-dev] [PATCH v2 0/5] drivers/net: " Chenxu Di
2019-12-20 3:02 ` [dpdk-dev] [PATCH v2 1/5] net/fm10k: " Chenxu Di
2019-12-20 3:02 ` [dpdk-dev] [PATCH v2 2/5] net/i40e: " Chenxu Di
2019-12-20 3:02 ` [dpdk-dev] [PATCH v2 3/5] net/ice: " Chenxu Di
2019-12-20 3:02 ` [dpdk-dev] [PATCH v2 4/5] net/ixgbe: " Chenxu Di
2019-12-20 3:02 ` [dpdk-dev] [PATCH v2 5/5] net/e1000: " Chenxu Di
2019-12-20 3:15 ` [dpdk-dev] [PATCH v3 0/5] drivers/net: " Chenxu Di
2019-12-20 3:15 ` [dpdk-dev] [PATCH v3 1/5] net/fm10k: " Chenxu Di
2019-12-20 3:15 ` [dpdk-dev] [PATCH v3 2/5] net/i40e: " Chenxu Di
2019-12-20 3:15 ` [dpdk-dev] [PATCH v3 3/5] net/ice: " Chenxu Di
2019-12-20 3:15 ` [dpdk-dev] [PATCH v3 4/5] net/ixgbe: " Chenxu Di
2019-12-20 3:15 ` [dpdk-dev] [PATCH v3 5/5] net/e1000: " Chenxu Di
2019-12-24 2:39 ` [dpdk-dev] [PATCH v4 0/5] drivers/net: " Chenxu Di
2019-12-24 2:39 ` [dpdk-dev] [PATCH v4 1/5] net/fm10k: " Chenxu Di
2019-12-24 2:39 ` [dpdk-dev] [PATCH v4 2/5] net/i40e: " Chenxu Di
2019-12-26 8:24 ` Xing, Beilei
2019-12-24 2:39 ` [dpdk-dev] [PATCH v4 3/5] net/ice: " Chenxu Di
2019-12-24 2:39 ` [dpdk-dev] [PATCH v4 4/5] net/ixgbe: " Chenxu Di
2019-12-24 2:39 ` [dpdk-dev] [PATCH v4 5/5] net/e1000: " Chenxu Di
2019-12-30 9:38 ` [dpdk-dev] [PATCH v6 0/4] drivers/net: " Chenxu Di
2019-12-30 9:38 ` [dpdk-dev] [PATCH v6 1/4] net/i40e: " Chenxu Di
2019-12-30 13:01 ` Ananyev, Konstantin
2019-12-30 9:38 ` [dpdk-dev] [PATCH v6 2/4] net/ice: " Chenxu Di
2019-12-30 9:38 ` [dpdk-dev] [PATCH v6 3/4] net/ixgbe: " Chenxu Di
2019-12-30 12:53 ` Ananyev, Konstantin
2020-01-03 9:01 ` Di, ChenxuX
2020-01-05 23:36 ` Ananyev, Konstantin
2020-01-06 9:03 ` Di, ChenxuX
2020-01-06 13:26 ` Ananyev, Konstantin
2020-01-07 10:46 ` Di, ChenxuX
2020-01-07 14:09 ` Ananyev, Konstantin
2020-01-08 10:15 ` Di, ChenxuX
2020-01-08 15:12 ` Ananyev, Konstantin
2019-12-30 9:38 ` [dpdk-dev] [PATCH v6 4/4] net/e1000: " Chenxu Di
2020-01-09 10:38 ` [dpdk-dev] [PATCH v7 0/4] drivers/net: " Chenxu Di
2020-01-09 10:38 ` [dpdk-dev] [PATCH v7 1/4] net/i40e: " Chenxu Di
2020-01-09 10:38 ` [dpdk-dev] [PATCH v7 2/4] net/ice: " Chenxu Di
2020-01-09 10:38 ` [dpdk-dev] [PATCH v7 3/4] net/ixgbe: " Chenxu Di
2020-01-09 14:01 ` Ananyev, Konstantin [this message]
2020-01-10 10:08 ` Di, ChenxuX
2020-01-10 12:46 ` Ananyev, Konstantin
2020-01-09 10:38 ` [dpdk-dev] [PATCH v7 4/4] net/e1000: " Chenxu Di
2020-01-10 9:58 ` [dpdk-dev] [PATCH v8 0/4] drivers/net: " Chenxu Di
2020-01-10 9:58 ` [dpdk-dev] [PATCH v8 1/4] net/i40e: " Chenxu Di
2020-01-10 9:58 ` [dpdk-dev] [PATCH v8 2/4] net/ice: " Chenxu Di
2020-01-10 9:58 ` [dpdk-dev] [PATCH v8 3/4] net/ixgbe: " Chenxu Di
2020-01-10 13:49 ` Ananyev, Konstantin
2020-01-10 9:59 ` [dpdk-dev] [PATCH v8 4/4] net/e1000: " Chenxu Di
2020-01-13 9:57 ` [dpdk-dev] [PATCH v9 0/4] drivers/net: " Chenxu Di
2020-01-13 9:57 ` [dpdk-dev] [PATCH v9 1/4] net/i40e: " Chenxu Di
2020-01-13 11:08 ` Ananyev, Konstantin
2020-01-13 9:57 ` [dpdk-dev] [PATCH v9 2/4] net/ice: " Chenxu Di
2020-01-14 1:55 ` Yang, Qiming
2020-01-14 12:40 ` Ferruh Yigit
2020-01-15 14:34 ` Ferruh Yigit
2020-01-16 1:40 ` Di, ChenxuX
2020-01-16 7:09 ` [dpdk-dev] [PATCH] net/ice: cleanup for vec path check Xiaolong Ye
2020-01-16 10:19 ` Ferruh Yigit
2020-01-17 2:21 ` Yang, Qiming
2020-01-16 8:43 ` [dpdk-dev] [PATCH v9 2/4] net/ice: cleanup Tx buffers Ferruh Yigit
2020-01-13 9:57 ` [dpdk-dev] [PATCH v9 3/4] net/ixgbe: " Chenxu Di
2020-01-13 11:07 ` Ananyev, Konstantin
2020-01-16 8:44 ` Ferruh Yigit
2020-01-16 14:47 ` Ferruh Yigit
2020-01-16 15:23 ` Ferruh Yigit
2020-01-13 9:57 ` [dpdk-dev] [PATCH v9 4/4] net/e1000: " Chenxu Di
2020-01-13 11:08 ` Ananyev, Konstantin
2020-01-14 2:49 ` Ye Xiaolong
2020-01-14 2:22 ` [dpdk-dev] [PATCH 0/4] drivers/net: " Ye Xiaolong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=SN6PR11MB2558AF818E543328E5F48C249A390@SN6PR11MB2558.namprd11.prod.outlook.com \
--to=konstantin.ananyev@intel.com \
--cc=chenxux.di@intel.com \
--cc=dev@dpdk.org \
--cc=qiming.yang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).