From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 65510A04AF; Fri, 21 Aug 2020 17:28:45 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 58DDF2BEA; Fri, 21 Aug 2020 17:28:44 +0200 (CEST) Received: from dal3relay151.mxroute.com (dal3relay151.mxroute.com [64.40.27.151]) by dpdk.org (Postfix) with ESMTP id CDAC82BE2 for ; Fri, 21 Aug 2020 17:28:42 +0200 (CEST) Received: from filter004.mxroute.com ([149.28.56.236] 149.28.56.236.vultr.com) (Authenticated sender: mN4UYu2MZsgR) by dal3relay151.mxroute.com (ZoneMTA) with ESMTPSA id 17411a2d1250006808.003 for (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256); Fri, 21 Aug 2020 15:28:37 +0000 X-Zone-Loop: 3a72c44641ea7e843e068fd14a8b073a7a5b7f9e4fa8 X-Originating-IP: [149.28.56.236] Received: from echo.mxrouting.net (echo.mxrouting.net [116.202.222.109]) by filter004.mxroute.com (Postfix) with ESMTPS id B72A63E9D1; Fri, 21 Aug 2020 15:28:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ashroe.eu; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=hAuOBnBbdgVhq6Ifi2W+/yKjNrYSRhjw7YOEZ0dra64=; b=ELWTJfw6l8ZykMyPJh4wygI8kq okf1P3bonGIIa+ffq53hUXbt+yAXyBCq1CAWtehyCGkKjIrWvpYDuWdjjmngq7K3899BH0FUlWKt0 xC6uarzi0RNiLIHJcHv0TckQNiOS81c9lrPTDxRP40wK0BLBc9ubPyOoxV2mOFlp8iw/KHVz2x1Bt Wk47OibJA/hjAY3qDjkXOKWL0ATMfwK7ouU14fHSsXNmiweRv4qRWpeFgaqreVsKgZ9BnUpXmfKcD NDZ8VLqKRHU+074n+LKu9XwRacKJSwoq6y2N8rFlf7SU03YQAi2DlOfmLlM9qXomYbH4ph79qIIZY Sp1GiOLg==; To: Kevin Traynor , alangordondewar@gmail.com, cristian.dumitrescu@intel.com Cc: dev@dpdk.org, Alan Dewar , David Marchand , Luca Boccassi , Jasvinder Singh References: <20200416084821.12591-1-alan.dewar@att.com> <20200625095930.18429-1-alan.dewar@att.com> <6404e021-336b-eef3-c34a-99aa17c6c3b1@redhat.com> From: "Kinsella, Ray" Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: <406b64c8-0c88-5d71-2179-f48d8384b8e7@ashroe.eu> Date: Fri, 21 Aug 2020 16:28:33 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <6404e021-336b-eef3-c34a-99aa17c6c3b1@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCH v2] sched: fix port time rounding error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 20/08/2020 15:32, Kevin Traynor wrote: > Hi, > > On 25/06/2020 10:59, alangordondewar@gmail.com wrote: >> From: Alan Dewar >> >> The QoS scheduler works off port time that is computed from the number >> of CPU cycles that have elapsed since the last time the port was >> polled. It divides the number of elapsed cycles to calculate how >> many bytes can be sent, however this division can generate rounding >> errors, where some fraction of a byte sent may be lost. >> >> Lose enough of these fractional bytes and the QoS scheduler >> underperforms. The problem is worse with low bandwidths. >> >> To compensate for this rounding error this fix doesn't advance the >> port's time_cpu_cycles by the number of cycles that have elapsed, >> but by multiplying the computed number of bytes that can be sent >> (which has been rounded down) by number of cycles per byte. >> This will mean that port's time_cpu_cycles will lag behind the CPU >> cycles momentarily. At the next poll, the lag will be taken into >> account. >> >> v2: >> If the cycles value wraps (100 year+) reset the port's cpu cycle back >> to zero. >> >> Fixes: de3cfa2c98 ("sched: initial import") >> >> Signed-off-by: Alan Dewar >> --- >> lib/librte_sched/rte_sched.c | 11 +++++++++-- >> 1 file changed, 9 insertions(+), 2 deletions(-) >> >> diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c >> index c0983ddda..7c022cd61 100644 >> --- a/lib/librte_sched/rte_sched.c >> +++ b/lib/librte_sched/rte_sched.c >> @@ -222,6 +222,7 @@ struct rte_sched_port { >> uint64_t time_cpu_bytes; /* Current CPU time measured in bytes */ >> uint64_t time; /* Current NIC TX time measured in bytes */ >> struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */ >> + uint64_t cycles_per_byte; >> > > I was backporting this patch to 18.11. The older ABI checker complains > about this structure change. > > "cycles_per_byte has been added at the middle position of this > structural type." > > Isn't this an ABI break? Dropping from 18.11 for time being. So it looks like rte_sched_port * is an opaque pointers as it's contents are only known to rte_sched.c and not outside. To everyone else it is an opaque data structure, so structural changes to it would not be an ABI breakage. > >> /* Grinders */ >> struct rte_mbuf **pkts_out; >> @@ -852,6 +853,7 @@ rte_sched_port_config(struct rte_sched_port_params *params) >> cycles_per_byte = (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT) >> / params->rate; >> port->inv_cycles_per_byte = rte_reciprocal_value(cycles_per_byte); >> + port->cycles_per_byte = cycles_per_byte; >> >> /* Grinders */ >> port->pkts_out = NULL; >> @@ -2673,16 +2675,21 @@ static inline void >> rte_sched_port_time_resync(struct rte_sched_port *port) >> { >> uint64_t cycles = rte_get_tsc_cycles(); >> - uint64_t cycles_diff = cycles - port->time_cpu_cycles; >> + uint64_t cycles_diff; >> uint64_t bytes_diff; >> uint32_t i; >> >> + if (cycles < port->time_cpu_cycles) >> + port->time_cpu_cycles = 0; >> + >> + cycles_diff = cycles - port->time_cpu_cycles; >> /* Compute elapsed time in bytes */ >> bytes_diff = rte_reciprocal_divide(cycles_diff << RTE_SCHED_TIME_SHIFT, >> port->inv_cycles_per_byte); >> >> /* Advance port time */ >> - port->time_cpu_cycles = cycles; >> + port->time_cpu_cycles += >> + (bytes_diff * port->cycles_per_byte) >> RTE_SCHED_TIME_SHIFT; >> port->time_cpu_bytes += bytes_diff; >> if (port->time < port->time_cpu_bytes) >> port->time = port->time_cpu_bytes; >> >