From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7C16A0597; Fri, 17 Apr 2020 23:19:16 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 40E4E1EB48; Fri, 17 Apr 2020 23:19:16 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id E59991EB44 for ; Fri, 17 Apr 2020 23:19:13 +0200 (CEST) IronPort-SDR: J2fgtXWwJ8ShE9Hsp6AJc1yILtlb/0pFVcHp1nTfHTvgsyqZ/XzCIBkxaH2P/CczFC7UtJTRYk rLfPaBVRtVtw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Apr 2020 14:19:12 -0700 IronPort-SDR: ealsc4V7b25UJJeIPWpVy2ISsdAOEV3rHw5Z8ZMKNKM0yjE9ieMDkd7lGhI0qh/l+PVW2tW73a KcpQFjHZJJqA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,395,1580803200"; d="scan'208";a="289381101" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga002.fm.intel.com with ESMTP; 17 Apr 2020 14:19:12 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 17 Apr 2020 14:19:11 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 17 Apr 2020 14:19:11 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Fri, 17 Apr 2020 14:19:11 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.101) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 17 Apr 2020 14:19:11 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ejbwcppBzxezg5Uq3H+s2KrhsTA3TYxIyD0zl4l49eN5h2piv8dWya2BUjl410FvAi5uL3godWiOB68a+Je/2XrNtJzFP9GK3kPM2FbL/0L0B/hZA8CPwxNArExvi2kbFscnA7Ttvh7KqZaqkWDil8IBl4Rv6aRCzBQrxATw9XLl/HAKPhQP65Z0kOGqkzRAIE4DZUiCgcT6EU/xCsNEvpsmMu61O2vXjKV1soiCDjnhIGzmZt5HlhJcp516oP4ENZs6TYsgIiuRnbgEN97ZEv8WGCyAjvM9/k7E+xuIad8mlUF4gdmnSRGqUivl/P3+ATN1DsEO2a0aRHw7vWB1Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=whgxrJ0G+vfdoBVu63MEjvJ+mu4d7HayuhwzZueoGnw=; b=Q6Rr1y/4Svx2quWk+eJ/KBUebTycZJQ7VJY4r90xmHd2RCr2GAnZePhi9bK67ImGVzXmw8McOeheh+RAnjItis7WAO1DcdFef6mQg7MHOZdi0n5hfBELqvSm60udgV0o5gj2e5wfN//ahm7p53zmdHAi4+Aa0KN1mi9Crs7CTOtmtRdZQRvGB+Qf/c/tqneJJ5Smz2wffCwrQ+cXD86HuXsJG8BRD7Y5VzX+qtjpxqlV1Wb5pomIoS4Q0wvPGVCxm6TiFVP3bel+feNSnUEwSh4gX6I3PzwoGQpxVIRFJ0BtsrppclIhPTqBwo+kz4nUTr+L7mczlj4DajnBlOKebw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=whgxrJ0G+vfdoBVu63MEjvJ+mu4d7HayuhwzZueoGnw=; b=uLK5s1fzUOy1dDr4/X44c/vQGUhmo+1bEA9d4fmAq3fixMmh//GUT6Ca0llLrN+is3CG4903IaSQ2Ub9faOxgPvAEkpiPlJlfqDI0VyYAe34MPsieiJSEtY/LSx/4U0bJ15SBvczk/xxcJTC+/A8QWPY/e+5fKgZg4DWoodvKcs= Received: from BYAPR11MB2935.namprd11.prod.outlook.com (2603:10b6:a03:82::24) by BYAPR11MB2727.namprd11.prod.outlook.com (2603:10b6:a02:c7::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.26; Fri, 17 Apr 2020 21:19:10 +0000 Received: from BYAPR11MB2935.namprd11.prod.outlook.com ([fe80::786e:a42b:df03:a829]) by BYAPR11MB2935.namprd11.prod.outlook.com ([fe80::786e:a42b:df03:a829%5]) with mapi id 15.20.2921.027; Fri, 17 Apr 2020 21:19:10 +0000 From: "Dumitrescu, Cristian" To: "alangordondewar@gmail.com" CC: "dev@dpdk.org" , Alan Dewar , "Singh, Jasvinder" Thread-Topic: [PATCH] sched: fix port time rounding error Thread-Index: AQHWE8vl/JCAZB/5WkSEx8OfZZQNIqh90+Iw Date: Fri, 17 Apr 2020 21:19:09 +0000 Message-ID: References: <20200416084821.12591-1-alan.dewar@att.com> In-Reply-To: <20200416084821.12591-1-alan.dewar@att.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-reaction: no-action dlp-product: dlpe-windows authentication-results: spf=none (sender IP is ) smtp.mailfrom=cristian.dumitrescu@intel.com; x-originating-ip: [192.198.151.178] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d30cc009-0390-4677-bc87-08d7e314f853 x-ms-traffictypediagnostic: BYAPR11MB2727: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-forefront-prvs: 0376ECF4DD x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB2935.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10019020)(366004)(376002)(136003)(39860400002)(346002)(396003)(2906002)(186003)(8936002)(54906003)(5660300002)(52536014)(107886003)(71200400001)(316002)(55016002)(81156014)(86362001)(8676002)(478600001)(9686003)(7696005)(6506007)(66556008)(66946007)(66446008)(76116006)(64756008)(66476007)(6916009)(53546011)(4326008)(26005)(33656002); DIR:OUT; SFP:1102; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3Rv+cckRq18BVD2AjdQK7IlTTlkIdQQM98T0nQ7hF6Be545W5LQHfMVO/Lqi2RC2bVldFaLJzF+xWkLpBpbSEaw3lsnTkZpW9J+29hbhj6UIVk9gqPVDydPfm/P05AOhlvEE9t2cTp+yrTVj4tnct6MAVHrsAFAVkmqnsdmEXiebGLoeU63gPwSJLIVj3pOmwFLTACWnv6xH2MfMymSW9knxWfDle4k0a2zqIv8yv0JTqSdPFJOOYdTHl8qlBMb31VK3SsfOXkhpT6+F1ClUaXn+t3QtaDZyqIZuF6fC3BhXulTm2JlctsIzEMFQFdt3Mk2EETdwdtiPiw/LCKV04DW3KyfvZnV6YIomEzX4H7EdOge+hbJXFFpxRLVM570TCs/txFAZUERnfwhbqCapjnxsYhMIFMH+D3DFDN2CgmYzGyWpgS3Mu/rDj1lfrwc2 x-ms-exchange-antispam-messagedata: pU9tOSK5s0ZTi9Fr7E7+AS4USlzf8ecF2u4ytQPK/m8hcacXv8zpk6+CrJnddtbchIxwwIjFpPII7uWNSkpR5uSi+ZTY53LnwNDyMjkzm8DY+cJ3vqDtta2Chpvgpveg+bFqEfEwJukQThj3LylXaA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: d30cc009-0390-4677-bc87-08d7e314f853 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Apr 2020 21:19:09.8907 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: d+CXHOn4Myk9xQ3RuqWSGGL8ZQ+5uzm36glj65vBgtX4uasapzKeHf/ZNNTNME2XDKTsQoujxyTHgu/z/uh+6kXzTdGqkZzCin2FX530hqA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2727 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] sched: fix port time rounding error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: alangordondewar@gmail.com > Sent: Thursday, April 16, 2020 9:48 AM > To: Dumitrescu, Cristian > Cc: dev@dpdk.org; Alan Dewar > Subject: [PATCH] sched: fix port time rounding error >=20 > From: Alan Dewar >=20 > The QoS scheduler works off port time that is computed from the number > of CPU cycles that have elapsed since the last time the port was > polled. It divides the number of elapsed cycles to calculate how > many bytes can be sent, however this division can generate rounding > errors, where some fraction of a byte sent may be lost. >=20 > Lose enough of these fractional bytes and the QoS scheduler > underperforms. The problem is worse with low bandwidths. >=20 > To compensate for this rounding error this fix doesn't advance the > port's time_cpu_cycles by the number of cycles that have elapsed, > but by multiplying the computed number of bytes that can be sent > (which has been rounded down) by number of cycles per byte. > This will mean that port's time_cpu_cycles will lag behind the CPU > cycles momentarily. At the next poll, the lag will be taken into > account. >=20 > Fixes: de3cfa2c98 ("sched: initial import") >=20 > Signed-off-by: Alan Dewar > --- > lib/librte_sched/rte_sched.c | 12 ++++++++++-- > 1 file changed, 10 insertions(+), 2 deletions(-) >=20 > diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c > index c0983ddda..c656dba2d 100644 > --- a/lib/librte_sched/rte_sched.c > +++ b/lib/librte_sched/rte_sched.c > @@ -222,6 +222,7 @@ struct rte_sched_port { > uint64_t time_cpu_bytes; /* Current CPU time measured in bytes > */ > uint64_t time; /* Current NIC TX time measured in bytes = */ > struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */ > + uint64_t cycles_per_byte; >=20 > /* Grinders */ > struct rte_mbuf **pkts_out; > @@ -852,6 +853,7 @@ rte_sched_port_config(struct > rte_sched_port_params *params) > cycles_per_byte =3D (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT) > / params->rate; > port->inv_cycles_per_byte =3D rte_reciprocal_value(cycles_per_byte); > + port->cycles_per_byte =3D cycles_per_byte; >=20 > /* Grinders */ > port->pkts_out =3D NULL; > @@ -2673,20 +2675,26 @@ static inline void > rte_sched_port_time_resync(struct rte_sched_port *port) > { > uint64_t cycles =3D rte_get_tsc_cycles(); > - uint64_t cycles_diff =3D cycles - port->time_cpu_cycles; > + uint64_t cycles_diff; > uint64_t bytes_diff; > uint32_t i; >=20 > + if (cycles < port->time_cpu_cycles) > + goto end; > + > + cycles_diff =3D cycles - port->time_cpu_cycles; > /* Compute elapsed time in bytes */ > bytes_diff =3D rte_reciprocal_divide(cycles_diff << > RTE_SCHED_TIME_SHIFT, > port->inv_cycles_per_byte); >=20 > /* Advance port time */ > - port->time_cpu_cycles =3D cycles; > + port->time_cpu_cycles +=3D > + (bytes_diff * port->cycles_per_byte) >> > RTE_SCHED_TIME_SHIFT; > port->time_cpu_bytes +=3D bytes_diff; > if (port->time < port->time_cpu_bytes) > port->time =3D port->time_cpu_bytes; >=20 > +end: > /* Reset pipe loop detection */ > for (i =3D 0; i < port->n_subports_per_port; i++) > port->subports[i]->pipe_loop =3D RTE_SCHED_PIPE_INVALID; > -- > 2.17.1 Adding Jasvinder.