From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8290DA0588; Thu, 16 Apr 2020 10:48:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 661D11DB71; Thu, 16 Apr 2020 10:48:36 +0200 (CEST) Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by dpdk.org (Postfix) with ESMTP id 2033B1DB4F for ; Thu, 16 Apr 2020 10:48:35 +0200 (CEST) Received: by mail-wm1-f65.google.com with SMTP id a81so3872089wmf.5 for ; Thu, 16 Apr 2020 01:48:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=bKHqSUrTwcKRkgXiT4qqDEEjx29FHCRLXpk00Oe/RYo=; b=KPAxFh/NnRNvZYTM1emYnIrldG+cqtcJhr5JPox68CbnCwemrJINtI15TahS4UBgf5 GzVj3tLLshOqMgN2TMNi2RvGb3S+82VHsxCLetUm2+LtQJniacH4NcIGUOospT8g5EG/ DJb0TLVtS63Q2n6YsGDGEaMQFZ4Zt5YI2p29ohSbE1W4rmniCKtCfEBjw2h90f0KVkNz esF8GRDAYBr2iSGd+VMNnEd+VH068yuBurB969BwZMfjGi64hCy0TclhkUbMT/PJiZ5N HmcAUWbOjvijGQM7QBIqEinZ4VcZbOxugEe58wBYV9k23G8A4Ih+niItUOREWvIai5vJ qF0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=bKHqSUrTwcKRkgXiT4qqDEEjx29FHCRLXpk00Oe/RYo=; b=dvUR7YZJ3zxEIm5tN35CxKzUDoAbXsjQV8ayql0FwtZnQednTsPdAY8+BNysmL4qfe iFBrfP8gZbOwiAqOa2iYb2nUlcVzkKaf/ZBMP0xm67qCpqcjxMDWTuD3YQoAh/msePnx yHc2E0UotLFPL6DbQhzE3xggvEU3fL+fdmzxioQsVtf5ulyUdV8j9Gvf9t5Y4Le0rtSD 9wkmVA2YmbWvokKCwNAVFWnBpscJo2sWdNJDAMP6IsHrlf42+DzsTTYzUnAs1dS23ll/ 6leYLIMP/sxker4kd4davy6Z/sO7MWs9/CGHk5uIsG/EhZVCUFXKgFiOxYwGgURVlEHf 4vwA== X-Gm-Message-State: AGi0PuYudWQUrYolz/NMLSTp0HR/CD2cL7tsrt3vP/z9OluZ6Wh2lZAn Lb49Bw+jFBs5IxKCwqdO+A== X-Google-Smtp-Source: APiQypIC3Kabn7Si7qekivBLYUQICfpkles5cNe5/TGZ1/HjLM4CCzJhKo2qaSZZXfqVZzZYYy6Fjg== X-Received: by 2002:a1c:dfc6:: with SMTP id w189mr3802320wmg.180.1587026914883; Thu, 16 Apr 2020 01:48:34 -0700 (PDT) Received: from adewar-Precision-7520.fritz.box (82-69-103-115.dsl.in-addr.zen.co.uk. [82.69.103.115]) by smtp.gmail.com with ESMTPSA id t20sm2915578wmi.2.2020.04.16.01.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Apr 2020 01:48:34 -0700 (PDT) From: alangordondewar@gmail.com X-Google-Original-From: alan.dewar@att.com To: cristian.dumitrescu@intel.com Cc: dev@dpdk.org, Alan Dewar Date: Thu, 16 Apr 2020 09:48:21 +0100 Message-Id: <20200416084821.12591-1-alan.dewar@att.com> X-Mailer: git-send-email 2.17.1 Subject: [dpdk-dev] [PATCH] sched: fix port time rounding error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Alan Dewar The QoS scheduler works off port time that is computed from the number of CPU cycles that have elapsed since the last time the port was polled. It divides the number of elapsed cycles to calculate how many bytes can be sent, however this division can generate rounding errors, where some fraction of a byte sent may be lost. Lose enough of these fractional bytes and the QoS scheduler underperforms. The problem is worse with low bandwidths. To compensate for this rounding error this fix doesn't advance the port's time_cpu_cycles by the number of cycles that have elapsed, but by multiplying the computed number of bytes that can be sent (which has been rounded down) by number of cycles per byte. This will mean that port's time_cpu_cycles will lag behind the CPU cycles momentarily. At the next poll, the lag will be taken into account. Fixes: de3cfa2c98 ("sched: initial import") Signed-off-by: Alan Dewar --- lib/librte_sched/rte_sched.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c index c0983ddda..c656dba2d 100644 --- a/lib/librte_sched/rte_sched.c +++ b/lib/librte_sched/rte_sched.c @@ -222,6 +222,7 @@ struct rte_sched_port { uint64_t time_cpu_bytes; /* Current CPU time measured in bytes */ uint64_t time; /* Current NIC TX time measured in bytes */ struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */ + uint64_t cycles_per_byte; /* Grinders */ struct rte_mbuf **pkts_out; @@ -852,6 +853,7 @@ rte_sched_port_config(struct rte_sched_port_params *params) cycles_per_byte = (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT) / params->rate; port->inv_cycles_per_byte = rte_reciprocal_value(cycles_per_byte); + port->cycles_per_byte = cycles_per_byte; /* Grinders */ port->pkts_out = NULL; @@ -2673,20 +2675,26 @@ static inline void rte_sched_port_time_resync(struct rte_sched_port *port) { uint64_t cycles = rte_get_tsc_cycles(); - uint64_t cycles_diff = cycles - port->time_cpu_cycles; + uint64_t cycles_diff; uint64_t bytes_diff; uint32_t i; + if (cycles < port->time_cpu_cycles) + goto end; + + cycles_diff = cycles - port->time_cpu_cycles; /* Compute elapsed time in bytes */ bytes_diff = rte_reciprocal_divide(cycles_diff << RTE_SCHED_TIME_SHIFT, port->inv_cycles_per_byte); /* Advance port time */ - port->time_cpu_cycles = cycles; + port->time_cpu_cycles += + (bytes_diff * port->cycles_per_byte) >> RTE_SCHED_TIME_SHIFT; port->time_cpu_bytes += bytes_diff; if (port->time < port->time_cpu_bytes) port->time = port->time_cpu_bytes; +end: /* Reset pipe loop detection */ for (i = 0; i < port->n_subports_per_port; i++) port->subports[i]->pipe_loop = RTE_SCHED_PIPE_INVALID; -- 2.17.1