From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6BEB0A05A0; Tue, 21 Apr 2020 23:55:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2AE6D1D53E; Tue, 21 Apr 2020 23:55:11 +0200 (CEST) Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) by dpdk.org (Postfix) with ESMTP id 545EA1D8D9 for ; Tue, 21 Apr 2020 10:22:00 +0200 (CEST) Received: from pps.filterd (m0083689.ppops.net [127.0.0.1]) by m0083689.ppops.net-00191d01. (8.16.0.42/8.16.0.42) with SMTP id 03L8LXsV001344; Tue, 21 Apr 2020 04:21:59 -0400 Received: from alpi155.enaf.aldc.att.com (sbcsmtp7.sbc.com [144.160.229.24]) by m0083689.ppops.net-00191d01. with ESMTP id 30hj9jsf1d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 21 Apr 2020 04:21:58 -0400 Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id 03L8LwqS026768; Tue, 21 Apr 2020 04:21:58 -0400 Received: from zlp27128.vci.att.com (zlp27128.vci.att.com [135.66.87.50]) by alpi155.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id 03L8Lp3l026695 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 21 Apr 2020 04:21:51 -0400 Received: from zlp27128.vci.att.com (zlp27128.vci.att.com [127.0.0.1]) by zlp27128.vci.att.com (Service) with ESMTP id 90DEF4000421; Tue, 21 Apr 2020 08:21:51 +0000 (GMT) Received: from gbcdcmbx13.intl.att.com (unknown [135.76.180.49]) by zlp27128.vci.att.com (Service) with ESMTPS id 40E794009E81; Tue, 21 Apr 2020 08:21:51 +0000 (GMT) Received: from gbcdcmbx14.intl.att.com (135.76.180.50) by gbcdcmbx13.intl.att.com (135.76.180.49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1913.5; Tue, 21 Apr 2020 09:21:49 +0100 Received: from gbcdcmbx14.intl.att.com ([fe80::5d33:6e38:633a:32a]) by gbcdcmbx14.intl.att.com ([fe80::5d33:6e38:633a:32a%4]) with mapi id 15.01.1913.005; Tue, 21 Apr 2020 09:21:48 +0100 From: "Dewar, Alan" To: "'Singh, Jasvinder'" , "'Dumitrescu, Cristian'" , "'alangordondewar@gmail.com'" CC: "'dev@dpdk.org'" , "'Alan Dewar'" Thread-Topic: [PATCH] sched: fix port time rounding error Thread-Index: AQHWE8vWvtKtHfZWYU6hIAnHxWiPvKh9w0+AgAQQgQCAAWurcA== Date: Tue, 21 Apr 2020 08:21:48 +0000 Message-ID: <80847da6f3b44c49a3884da76818302e@intl.att.com> References: <20200416084821.12591-1-alan.dewar@att.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.76.180.249] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-21_02:2020-04-20, 2020-04-21 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 mlxscore=0 bulkscore=0 adultscore=0 suspectscore=0 impostorscore=0 mlxlogscore=999 lowpriorityscore=0 malwarescore=0 clxscore=1011 phishscore=0 priorityscore=1501 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004210070 X-Mailman-Approved-At: Tue, 21 Apr 2020 23:55:10 +0200 Subject: Re: [dpdk-dev] [PATCH] sched: fix port time rounding error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Singh, Jasvinder =20 > Sent: Monday, April 20, 2020 12:23 PM > To: Dumitrescu, Cristian ; alangordondewar= @gmail.com > Cc: dev@dpdk.org; Alan Dewar > Subject: RE: [PATCH] sched: fix port time rounding error >=20 >=20 >=20 > > -----Original Message----- > > From: Dumitrescu, Cristian > > Sent: Friday, April 17, 2020 10:19 PM > > To: alangordondewar@gmail.com > > Cc: dev@dpdk.org; Alan Dewar ; Singh, Jasvinder=20 > > > > Subject: RE: [PATCH] sched: fix port time rounding error > >=20 > >=20 > >=20 > > > -----Original Message----- > > > From: alangordondewar@gmail.com > > > Sent: Thursday, April 16, 2020 9:48 AM > > > To: Dumitrescu, Cristian > > > Cc: dev@dpdk.org; Alan Dewar > > > Subject: [PATCH] sched: fix port time rounding error > > > > > > From: Alan Dewar > > > > > > The QoS scheduler works off port time that is computed from the=20 > > > number of CPU cycles that have elapsed since the last time the port w= as > > > polled. It divides the number of elapsed cycles to calculate how > > > many bytes can be sent, however this division can generate rounding=20 > > > errors, where some fraction of a byte sent may be lost. > > > > > > Lose enough of these fractional bytes and the QoS scheduler=20 > > > underperforms. The problem is worse with low bandwidths. > > > > > > To compensate for this rounding error this fix doesn't advance the=20 > > > port's time_cpu_cycles by the number of cycles that have elapsed,=20 > > > but by multiplying the computed number of bytes that can be sent=20 > > > (which has been rounded down) by number of cycles per byte. > > > This will mean that port's time_cpu_cycles will lag behind the CPU=20 > > > cycles momentarily. At the next poll, the lag will be taken into=20 > > > account. > > > > > > Fixes: de3cfa2c98 ("sched: initial import") > > > > > > Signed-off-by: Alan Dewar > > > --- > > > lib/librte_sched/rte_sched.c | 12 ++++++++++-- > > > 1 file changed, 10 insertions(+), 2 deletions(-) > > > > > > diff --git a/lib/librte_sched/rte_sched.c=20 > > > b/lib/librte_sched/rte_sched.c index c0983ddda..c656dba2d 100644 > > > --- a/lib/librte_sched/rte_sched.c > > > +++ b/lib/librte_sched/rte_sched.c > > > @@ -222,6 +222,7 @@ struct rte_sched_port { > > > uint64_t time_cpu_bytes; /* Current CPU time measured in bytes > > > */ > > > uint64_t time; /* Current NIC TX time measured in by= tes */ > > > struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte=20 > > > */ > > > + uint64_t cycles_per_byte; > > > > > > /* Grinders */ > > > struct rte_mbuf **pkts_out; > > > @@ -852,6 +853,7 @@ rte_sched_port_config(struct > > rte_sched_port_params > > > *params) > > > cycles_per_byte =3D (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT) > > > / params->rate; > > > port->inv_cycles_per_byte =3D rte_reciprocal_value(cycles_per_byte)= ; > > > + port->cycles_per_byte =3D cycles_per_byte; > > > > > > /* Grinders */ > > > port->pkts_out =3D NULL; > > > @@ -2673,20 +2675,26 @@ static inline void=20 > > > rte_sched_port_time_resync(struct rte_sched_port *port) { > > > uint64_t cycles =3D rte_get_tsc_cycles(); > > > - uint64_t cycles_diff =3D cycles - port->time_cpu_cycles; > > > + uint64_t cycles_diff; > > > uint64_t bytes_diff; > > > uint32_t i; > > > > > > + if (cycles < port->time_cpu_cycles) > > > + goto end; >=20 > Above check seems redundant as port->time_cpu_cycles will always be less = than the current cycles due to roundoff in previous iteration. >=20 This was to catch the condition where the cycles wraps back to zero (after = 100+ years?? depending on clock speed). =20 Rather than just going to end: the conditional should at least reset port->= time_cpu_cycles back to zero. So there would be a very temporary glitch in accuracy once every 100+ years= .=20 > > > > + cycles_diff =3D cycles - port->time_cpu_cycles; > > > /* Compute elapsed time in bytes */ > > > bytes_diff =3D rte_reciprocal_divide(cycles_diff <<=20 > > > RTE_SCHED_TIME_SHIFT, > > > port->inv_cycles_per_byte); > > > > > > /* Advance port time */ > > > - port->time_cpu_cycles =3D cycles; > > > + port->time_cpu_cycles +=3D > > > + (bytes_diff * port->cycles_per_byte) >> > > > RTE_SCHED_TIME_SHIFT; > > > port->time_cpu_bytes +=3D bytes_diff; > > > if (port->time < port->time_cpu_bytes) > > > port->time =3D port->time_cpu_bytes; > > > > > > +end: > > > /* Reset pipe loop detection */ > > > for (i =3D 0; i < port->n_subports_per_port; i++) > > > port->subports[i]->pipe_loop =3D RTE_SCHED_PIPE_INVALID; > > > -- > > > 2.17.1 > >=20 > > Adding Jasvinder.