DPDK patches and discussions
 help / color / mirror / Atom feed
From: Michal Krawczyk <mk@semihalf.com>
To: dev@dpdk.org
Cc: mw@semihalf.com, mba@semihalf.com, gtzalik@amazon.com,
	evgenys@amazon.com, igorch@amazon.com,
	Michal Krawczyk <mk@semihalf.com>
Subject: [dpdk-dev] [PATCH 23/29] net/ena: limit refill threshold by fixed value
Date: Fri, 27 Mar 2020 11:18:17 +0100	[thread overview]
Message-ID: <20200327101823.12646-24-mk@semihalf.com> (raw)
In-Reply-To: <20200327101823.12646-1-mk@semihalf.com>

Divider used for both Tx and Rx cleanup/refill threshold can cause too
big delay in case of the really big rings - for example if the 8k Rx
ring will be used, the refill won't trigger unless 1024 threshold will
be reached. It will also cause driver to try to allocate that much
descriptors.

Limiting it by fixed value - 256 in that case, would limit maximum
time spent in repopulate function.

Signed-off-by: Michal Krawczyk <mk@semihalf.com>
Reviewed-by: Igor Chauskin <igorch@amazon.com>
Reviewed-by: Guy Tzalik <gtzalik@amazon.com>
---
 drivers/net/ena/ena_ethdev.c | 27 ++++++++++++++-------------
 drivers/net/ena/ena_ethdev.h | 10 ++++++++++
 2 files changed, 24 insertions(+), 13 deletions(-)

diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index 8f8a06d5ba..3e288d56c4 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -34,14 +34,6 @@
 /*reverse version of ENA_IO_RXQ_IDX*/
 #define ENA_IO_RXQ_IDX_REV(q)	((q - 1) / 2)
 
-/* While processing submitted and completed descriptors (rx and tx path
- * respectively) in a loop it is desired to:
- *  - perform batch submissions while populating sumbissmion queue
- *  - avoid blocking transmission of other packets during cleanup phase
- * Hence the utilization ratio of 1/8 of a queue size.
- */
-#define ENA_RING_DESCS_RATIO(ring_size)	(ring_size / 8)
-
 #define __MERGE_64B_H_L(h, l) (((uint64_t)h << 32) | l)
 #define TEST_BIT(val, bit_shift) (val & (1UL << bit_shift))
 
@@ -2130,7 +2122,8 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct ena_ring *rx_ring = (struct ena_ring *)(rx_queue);
 	unsigned int ring_size = rx_ring->ring_size;
 	unsigned int ring_mask = ring_size - 1;
-	unsigned int refill_required;
+	unsigned int free_queue_entries;
+	unsigned int refill_threshold;
 	uint16_t next_to_clean = rx_ring->next_to_clean;
 	uint16_t descs_in_use;
 	struct rte_mbuf *mbuf;
@@ -2199,11 +2192,15 @@ static uint16_t eth_ena_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 	rx_ring->rx_stats.cnt += completed;
 	rx_ring->next_to_clean = next_to_clean;
 
-	refill_required = ena_com_free_q_entries(rx_ring->ena_com_io_sq);
+	free_queue_entries = ena_com_free_q_entries(rx_ring->ena_com_io_sq);
+	refill_threshold =
+		RTE_MIN(ring_size / ENA_REFILL_THRESH_DIVIDER,
+		(unsigned int)ENA_REFILL_THRESH_PACKET);
+
 	/* Burst refill to save doorbells, memory barriers, const interval */
-	if (refill_required > ENA_RING_DESCS_RATIO(ring_size)) {
+	if (free_queue_entries > refill_threshold) {
 		ena_com_update_dev_comp_head(rx_ring->ena_com_io_cq);
-		ena_populate_rx_queue(rx_ring, refill_required);
+		ena_populate_rx_queue(rx_ring, free_queue_entries);
 	}
 
 	return completed;
@@ -2342,6 +2339,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 	uint16_t seg_len;
 	unsigned int ring_size = tx_ring->ring_size;
 	unsigned int ring_mask = ring_size - 1;
+	unsigned int cleanup_budget;
 	struct ena_com_tx_ctx ena_tx_ctx;
 	struct ena_tx_buffer *tx_info;
 	struct ena_com_buf *ebuf;
@@ -2499,9 +2497,12 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 		/* Put back descriptor to the ring for reuse */
 		tx_ring->empty_tx_reqs[next_to_clean & ring_mask] = req_id;
 		next_to_clean++;
+		cleanup_budget =
+			RTE_MIN(ring_size / ENA_REFILL_THRESH_DIVIDER,
+			(unsigned int)ENA_REFILL_THRESH_PACKET);
 
 		/* If too many descs to clean, leave it for another run */
-		if (unlikely(total_tx_descs > ENA_RING_DESCS_RATIO(ring_size)))
+		if (unlikely(total_tx_descs > cleanup_budget))
 			break;
 	}
 	tx_ring->tx_stats.available_desc =
diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h
index 64d93dac02..6634d0134f 100644
--- a/drivers/net/ena/ena_ethdev.h
+++ b/drivers/net/ena/ena_ethdev.h
@@ -30,6 +30,16 @@
 #define ENA_WD_TIMEOUT_SEC	3
 #define ENA_DEVICE_KALIVE_TIMEOUT (ENA_WD_TIMEOUT_SEC * rte_get_timer_hz())
 
+/* While processing submitted and completed descriptors (rx and tx path
+ * respectively) in a loop it is desired to:
+ *  - perform batch submissions while populating sumbissmion queue
+ *  - avoid blocking transmission of other packets during cleanup phase
+ * Hence the utilization ratio of 1/8 of a queue size or max value if the size
+ * of the ring is very big - like 8k Rx rings.
+ */
+#define ENA_REFILL_THRESH_DIVIDER      8
+#define ENA_REFILL_THRESH_PACKET       256
+
 struct ena_adapter;
 
 enum ena_ring_type {
-- 
2.20.1


  parent reply	other threads:[~2020-03-27 10:33 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-27 10:17 [dpdk-dev] [PATCH 00/29] Update ENA driver to v2.1.0 Michal Krawczyk
2020-03-27 10:17 ` [dpdk-dev] [PATCH 01/29] net/ena: check if size of buffer is at least 1400B Michal Krawczyk
2020-03-27 10:55   ` Andrew Rybchenko
2020-03-31  9:47     ` Michał Krawczyk
2020-03-27 14:51   ` Stephen Hemminger
2020-03-31  9:48     ` Michał Krawczyk
2020-03-27 10:17 ` [dpdk-dev] [PATCH 02/29] net/ena/base: make allocation macros thread-safe Michal Krawczyk
2020-03-27 14:54   ` Stephen Hemminger
2020-03-31  9:47     ` Michał Krawczyk
2020-03-27 10:17 ` [dpdk-dev] [PATCH 03/29] net/ena/base: prevent allocation of 0-sized memory Michal Krawczyk
2020-03-27 10:17 ` [dpdk-dev] [PATCH 04/29] net/ena/base: set default hash key Michal Krawczyk
2020-03-27 11:12   ` Andrew Rybchenko
2020-03-31  9:40     ` Michał Krawczyk
2020-03-31  9:51       ` Michał Krawczyk
2020-03-27 10:17 ` [dpdk-dev] [PATCH 05/29] net/ena/base: rework interrupt moderation Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 06/29] net/ena/base: remove extra properties strings Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 07/29] net/ena/base: add accelerated LLQ mode Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 08/29] net/ena/base: fix documentation of the functions Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 09/29] net/ena/base: fix indentation in cq polling Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 10/29] net/ena/base: add error logs when preparing Tx Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 11/29] net/ena/base: use 48-bit memory addresses in ena_com Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 12/29] net/ena/base: fix types for printing timestamps Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 13/29] net/ena/base: fix indentation of multiple defines Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 14/29] net/ena/base: update gen date and commit Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 15/29] net/ena: set IO ring size to the valid value Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 16/29] net/ena: refactor getting IO queues capabilities Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 17/29] net/ena: add support for large LLQ headers Michal Krawczyk
2020-03-27 11:20   ` Andrew Rybchenko
2020-03-31  9:42     ` Michał Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 18/29] net/ena: remove memory barriers before doorbells Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 19/29] net/ena: add Tx drops statistic Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 20/29] net/ena: disable meta caching Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 21/29] net/ena: refactor Rx path Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 22/29] net/ena: rework getting number of available descs Michal Krawczyk
2020-03-27 10:18 ` Michal Krawczyk [this message]
2020-03-27 10:18 ` [dpdk-dev] [PATCH 24/29] net/ena: use macros for ring idx operations Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 25/29] net/ena: refactor Tx path Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 26/29] net/ena: reuse 0 length Rx descriptor Michal Krawczyk
2020-03-27 11:29   ` Andrew Rybchenko
2020-03-31  9:45     ` Michał Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 27/29] doc: add notes on ENA usage on metal instances Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 28/29] net/ena: update copyright date Michal Krawczyk
2020-03-27 10:18 ` [dpdk-dev] [PATCH 29/29] net/ena: update version of the driver to v2.1.0 Michal Krawczyk

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200327101823.12646-24-mk@semihalf.com \
    --to=mk@semihalf.com \
    --cc=dev@dpdk.org \
    --cc=evgenys@amazon.com \
    --cc=gtzalik@amazon.com \
    --cc=igorch@amazon.com \
    --cc=mba@semihalf.com \
    --cc=mw@semihalf.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).