DPDK patches and discussions
 help / color / mirror / Atom feed
From: alangordondewar@gmail.com
To: cristian.dumitrescu@intel.com
Cc: dev@dpdk.org, Alan Dewar <alan.dewar@att.com>
Subject: [dpdk-dev] [PATCH] sched: fix port time rounding error
Date: Thu, 16 Apr 2020 09:48:21 +0100	[thread overview]
Message-ID: <20200416084821.12591-1-alan.dewar@att.com> (raw)

From: Alan Dewar <alan.dewar@att.com>

The QoS scheduler works off port time that is computed from the number
of CPU cycles that have elapsed since the last time the port was
polled.   It divides the number of elapsed cycles to calculate how
many bytes can be sent, however this division can generate rounding
errors, where some fraction of a byte sent may be lost.

Lose enough of these fractional bytes and the QoS scheduler
underperforms.  The problem is worse with low bandwidths.

To compensate for this rounding error this fix doesn't advance the
port's time_cpu_cycles by the number of cycles that have elapsed,
but by multiplying the computed number of bytes that can be sent
(which has been rounded down) by number of cycles per byte.
This will mean that port's time_cpu_cycles will lag behind the CPU
cycles momentarily.  At the next poll, the lag will be taken into
account.

Fixes: de3cfa2c98 ("sched: initial import")

Signed-off-by: Alan Dewar <alan.dewar@att.com>
---
 lib/librte_sched/rte_sched.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c
index c0983ddda..c656dba2d 100644
--- a/lib/librte_sched/rte_sched.c
+++ b/lib/librte_sched/rte_sched.c
@@ -222,6 +222,7 @@ struct rte_sched_port {
 	uint64_t time_cpu_bytes;      /* Current CPU time measured in bytes */
 	uint64_t time;                /* Current NIC TX time measured in bytes */
 	struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */
+	uint64_t cycles_per_byte;
 
 	/* Grinders */
 	struct rte_mbuf **pkts_out;
@@ -852,6 +853,7 @@ rte_sched_port_config(struct rte_sched_port_params *params)
 	cycles_per_byte = (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT)
 		/ params->rate;
 	port->inv_cycles_per_byte = rte_reciprocal_value(cycles_per_byte);
+	port->cycles_per_byte = cycles_per_byte;
 
 	/* Grinders */
 	port->pkts_out = NULL;
@@ -2673,20 +2675,26 @@ static inline void
 rte_sched_port_time_resync(struct rte_sched_port *port)
 {
 	uint64_t cycles = rte_get_tsc_cycles();
-	uint64_t cycles_diff = cycles - port->time_cpu_cycles;
+	uint64_t cycles_diff;
 	uint64_t bytes_diff;
 	uint32_t i;
 
+	if (cycles < port->time_cpu_cycles)
+		goto end;
+
+	cycles_diff = cycles - port->time_cpu_cycles;
 	/* Compute elapsed time in bytes */
 	bytes_diff = rte_reciprocal_divide(cycles_diff << RTE_SCHED_TIME_SHIFT,
 					   port->inv_cycles_per_byte);
 
 	/* Advance port time */
-	port->time_cpu_cycles = cycles;
+	port->time_cpu_cycles +=
+		(bytes_diff * port->cycles_per_byte) >> RTE_SCHED_TIME_SHIFT;
 	port->time_cpu_bytes += bytes_diff;
 	if (port->time < port->time_cpu_bytes)
 		port->time = port->time_cpu_bytes;
 
+end:
 	/* Reset pipe loop detection */
 	for (i = 0; i < port->n_subports_per_port; i++)
 		port->subports[i]->pipe_loop = RTE_SCHED_PIPE_INVALID;
-- 
2.17.1


             reply	other threads:[~2020-04-16  8:48 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-16  8:48 alangordondewar [this message]
2020-04-17 21:19 ` Dumitrescu, Cristian
2020-04-20 11:23   ` Singh, Jasvinder
2020-04-21  8:21     ` Dewar, Alan
2020-06-24 22:50       ` Thomas Monjalon
2020-06-25  8:32         ` Singh, Jasvinder
2020-06-25  8:40           ` Alan Dewar
2020-06-25  9:59 ` [dpdk-dev] [PATCH v2] " alangordondewar
2020-07-05 20:41   ` Thomas Monjalon
2020-07-06 21:20     ` Singh, Jasvinder
2020-07-06 23:01       ` Thomas Monjalon
2020-08-20 14:32   ` Kevin Traynor
2020-08-21 15:28     ` Kinsella, Ray
2020-09-07 10:09       ` Kevin Traynor

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200416084821.12591-1-alan.dewar@att.com \
    --to=alangordondewar@gmail.com \
    --cc=alan.dewar@att.com \
    --cc=cristian.dumitrescu@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).