From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F0AB2A04AF; Thu, 20 Aug 2020 16:32:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 55CE41BEC4; Thu, 20 Aug 2020 16:32:37 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id CF7CE2BE2 for ; Thu, 20 Aug 2020 16:32:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597933955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ln401fbUUl+tWbOB5c1eJy1eMscK2YNG+BQqi2ko5sg=; b=CPkv4FvnDtress6jBcrNRXm+u/T3cjWztfIwa9DBvj57xQhVg0FnyN9eYBGoD0CXdNQqzW 5S8kd17EwPBLUO0jTZPb9fcojc5VgWeV2C/z++uELAKYRD508IUIOwLmqbiCImqOCeG6wX qBCYqrzQRpQcPCfpAam8drx2PAgClgc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-yrPR_rliOo6lqXEt6es2PQ-1; Thu, 20 Aug 2020 10:32:32 -0400 X-MC-Unique: yrPR_rliOo6lqXEt6es2PQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E5B568030C8; Thu, 20 Aug 2020 14:32:30 +0000 (UTC) Received: from [10.33.36.18] (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5EC647C0B3; Thu, 20 Aug 2020 14:32:25 +0000 (UTC) To: alangordondewar@gmail.com, cristian.dumitrescu@intel.com Cc: dev@dpdk.org, Alan Dewar , David Marchand , Ray Kinsella , Luca Boccassi , Jasvinder Singh References: <20200416084821.12591-1-alan.dewar@att.com> <20200625095930.18429-1-alan.dewar@att.com> From: Kevin Traynor Message-ID: <6404e021-336b-eef3-c34a-99aa17c6c3b1@redhat.com> Date: Thu, 20 Aug 2020 15:32:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <20200625095930.18429-1-alan.dewar@att.com> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ktraynor@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2] sched: fix port time rounding error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, On 25/06/2020 10:59, alangordondewar@gmail.com wrote: > From: Alan Dewar > > The QoS scheduler works off port time that is computed from the number > of CPU cycles that have elapsed since the last time the port was > polled. It divides the number of elapsed cycles to calculate how > many bytes can be sent, however this division can generate rounding > errors, where some fraction of a byte sent may be lost. > > Lose enough of these fractional bytes and the QoS scheduler > underperforms. The problem is worse with low bandwidths. > > To compensate for this rounding error this fix doesn't advance the > port's time_cpu_cycles by the number of cycles that have elapsed, > but by multiplying the computed number of bytes that can be sent > (which has been rounded down) by number of cycles per byte. > This will mean that port's time_cpu_cycles will lag behind the CPU > cycles momentarily. At the next poll, the lag will be taken into > account. > > v2: > If the cycles value wraps (100 year+) reset the port's cpu cycle back > to zero. > > Fixes: de3cfa2c98 ("sched: initial import") > > Signed-off-by: Alan Dewar > --- > lib/librte_sched/rte_sched.c | 11 +++++++++-- > 1 file changed, 9 insertions(+), 2 deletions(-) > > diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c > index c0983ddda..7c022cd61 100644 > --- a/lib/librte_sched/rte_sched.c > +++ b/lib/librte_sched/rte_sched.c > @@ -222,6 +222,7 @@ struct rte_sched_port { > uint64_t time_cpu_bytes; /* Current CPU time measured in bytes */ > uint64_t time; /* Current NIC TX time measured in bytes */ > struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */ > + uint64_t cycles_per_byte; > I was backporting this patch to 18.11. The older ABI checker complains about this structure change. "cycles_per_byte has been added at the middle position of this structural type." Isn't this an ABI break? Dropping from 18.11 for time being. > /* Grinders */ > struct rte_mbuf **pkts_out; > @@ -852,6 +853,7 @@ rte_sched_port_config(struct rte_sched_port_params *params) > cycles_per_byte = (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT) > / params->rate; > port->inv_cycles_per_byte = rte_reciprocal_value(cycles_per_byte); > + port->cycles_per_byte = cycles_per_byte; > > /* Grinders */ > port->pkts_out = NULL; > @@ -2673,16 +2675,21 @@ static inline void > rte_sched_port_time_resync(struct rte_sched_port *port) > { > uint64_t cycles = rte_get_tsc_cycles(); > - uint64_t cycles_diff = cycles - port->time_cpu_cycles; > + uint64_t cycles_diff; > uint64_t bytes_diff; > uint32_t i; > > + if (cycles < port->time_cpu_cycles) > + port->time_cpu_cycles = 0; > + > + cycles_diff = cycles - port->time_cpu_cycles; > /* Compute elapsed time in bytes */ > bytes_diff = rte_reciprocal_divide(cycles_diff << RTE_SCHED_TIME_SHIFT, > port->inv_cycles_per_byte); > > /* Advance port time */ > - port->time_cpu_cycles = cycles; > + port->time_cpu_cycles += > + (bytes_diff * port->cycles_per_byte) >> RTE_SCHED_TIME_SHIFT; > port->time_cpu_bytes += bytes_diff; > if (port->time < port->time_cpu_bytes) > port->time = port->time_cpu_bytes; >