From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D89ABA052A; Mon, 25 Jan 2021 18:00:55 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AA75A140FDC; Mon, 25 Jan 2021 18:00:55 +0100 (CET) Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) by mails.dpdk.org (Postfix) with ESMTP id 38F99140FDB for ; Mon, 25 Jan 2021 18:00:54 +0100 (CET) Received: by mail-io1-f44.google.com with SMTP id q1so27914090ion.8 for ; Mon, 25 Jan 2021 09:00:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=EWjF3YneW3njElyE2jkyRlLkRNtiWzzLU4x8pjYsJgA=; b=T8BYPeQAo8SINXk272GGeP6jlhULCaO+mTQnyg8SBVCRuKrvsNUpaTttmGnKL/c9gy uYCoQwDSwI0Cu3bwE6MdjX5eiOX1OoquwKwmJ1PayJE7EWK2lnb4YJ3JaAlIUPrtWXJD vjgsIucK4IyPnWRJnI+LeRp6QDMcYPLkrYGsk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=EWjF3YneW3njElyE2jkyRlLkRNtiWzzLU4x8pjYsJgA=; b=GQFKjb1+Elep8T8ZuJ1VAoOoYq74tBcjevmuvz+toG5Ij2bPWs4kEx/RSEc3iTU9rX gSzMiV1VTiEHk8AygwypLb348e/hGO4RlyFu2j+SENYtYyTsGT/cpPu8iuNrWsotPWC5 TzL9XWPH9DrV4yoDR7pKECKaKKyD0hhF7o1QB6U2qFz5KrKdFqCJ3BdrnCBwdgE2A297 uez+1ziqT4BvjV+zVCNrI3j+m/ZISt6Swg7GDopaHAqiT7KvtObwXRiJsON9/4CHMJmN eSI9Yza7tUt7949PFEcEQVWuvH3no3WbpqoycX4T+UO/6buM3zOhBPOkxxni9PA06ovt Phww== X-Gm-Message-State: AOAM530w3L7WMxQPQQ4kKDnEbzDzTuNU3BQJsqGO8wvAusNGKUjYEDh3 e603FfFm4GSlMwU59FHRNpGipdDFbEqUuiWmJgBWHg== X-Google-Smtp-Source: ABdhPJxJ2CYyC2I6dFPJ/BzLRlC0HnNDsTL1q8lDjVzCEr8mGhSUyzDPvAU6ZQ5A3o/HKOJ88L0hZp5jjwNoIVAyBS4= X-Received: by 2002:a02:3843:: with SMTP id v3mr1467212jae.70.1611594053546; Mon, 25 Jan 2021 09:00:53 -0800 (PST) MIME-Version: 1.0 References: <20201104170007.8026-1-olivier.matz@6wind.com> <20210113132734.1636-1-olivier.matz@6wind.com> <98CBD80474FA8B44BF855DF32C47DC35C61593@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35C61599@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35C61599@smartserver.smartshare.dk> From: Brandon Lo Date: Mon, 25 Jan 2021 12:00:18 -0500 Message-ID: To: =?UTF-8?Q?Morten_Br=C3=B8rup?= Cc: Lincoln Lavoie , Ferruh Yigit , Ali Alnubani , David Marchand , Olivier Matz , "Chen, Zhaoyan" , dev , Andrew Rybchenko , "Ananyev, Konstantin" , ajitkhaparde@gmail.com, dpdk stable , Ajit Khaparde , dpdklab Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [dpdklab] RE: [dpdk-stable] [PATCH v4] mbuf: fix reset on mbuf free X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, This seems like a good task for us to do. I will see what it would take in order to convert the difference into a decimal-formatted percentage. I have put this into bugzilla to keep track of this issue: https://bugs.dpdk.org/show_bug.cgi?id=3D626 Thanks, Brandon On Sat, Jan 23, 2021 at 3:57 AM Morten Br=C3=B8rup wrote: > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Lincoln Lavoie > > Sent: Thursday, January 21, 2021 5:35 PM > > To: Morten Br=C3=B8rup > > > > Hi All, > > > > Trying to follow the specific conversation. It is correct, the lab > > does > > not list the specific throughput values achieved by the hardware, as > > that > > data can be sensitive to the hardware vendors, etc. The purpose of the > > lab > > is to check for degradations caused by patches, so the difference is > > really > > the important factor. The comparison is against a prior run on the > > same > > hardware, via the DPDK main branch, so any delta should be caused by > > the > > specific patch changes (excluding statistical "wiggle"). > > > > Thank you for clarifying, Lincoln. > > This sounds like a perfect solution to the meet the purpose. > > I request that you change the output to show the relative difference (i.e= . percentage above/below baseline), instead of the absolute difference (i.e= . number of packets per second). > > By showing a percentage, anyone reading the test report can understand if= the difference is insignificant, or big enough to require further discussi= on before accepting the patch. Showing the difference in packets per second= requires the reader of the test report to have prior knowledge about the e= xpected performance. > > > If the group would prefer, we could calculate additional references if > > desired (i.e. difference from the last official release, or a monthly > > run > > of the current, etc.). We just need the community to define their > > needs, > > and we can add this to the development queue. > > > > I retract my suggestion about adding additional references. You clearly e= xplained how your baselining works, and I think it fully serves the needs o= f a regression test. > > So please just put this small change in your development queue: Output th= e difference in percent with a few decimals after the comma, instead of the= difference in packets per second. > > For readability, performance drops should be shown as negative values, an= d increases as positive, e.g.: > > Difference =3D (NewPPS - BaselinePPS) / BaselinePPS * 100.00 %. > > > If we want to compare performance against official releases, current or o= lder, it should go elsewhere, not in the continuous testing environment. E.= g. the release notes could include a report showing differences from the la= st few official releases. But that is a task for another day. > > > > Cheers, > > Lincoln > > > > > > On Thu, Jan 21, 2021 at 4:29 AM Morten Br=C3=B8rup > > > > wrote: > > > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit > > > > Sent: Thursday, January 21, 2021 10:19 AM > > > > > > > > On 1/15/2021 6:39 PM, Ali Alnubani wrote: > > > > > Hi, > > > > > Adding Ferruh and Zhaoyan, > > > > > > > > > >> Ali, > > > > >> > > > > >> You reported some performance regression, did you confirm it? > > > > >> If I get no reply by monday, I'll proceed with this patch. > > > > > > > > > > Sure I'll confirm by Monday. > > > > > > > > > > Doesn't the regression also reproduce on the Lab's Intel servers? > > > > > Even though the check iol-intel-Performance isn't failing, I can > > see > > > > that the throughput differences from expected for this patch are > > less > > > > than those of another patch that was tested only 20 minutes > > earlier. > > > > Both patches were applied to the same tree: > > > > > > > > > > https://mails.dpdk.org/archives/test-report/2021- > > January/173927.html > > > > >> | 64 | 512 | 1.571 | > > > > > > > > > > https://mails.dpdk.org/archives/test-report/2021- > > January/173919.html > > > > >> | 64 | 512 | 2.698 | > > > > > > > > > > Assuming that pw86457 doesn't have an effect on this test, it > > looks > > > > to me that this patch caused a regression in Intel hardware as > > well. > > > > > > > > > > Can someone update the baseline's expected values for the Intel > > NICs > > > > and rerun the test on this patch? > > > > > > > > > > > > > Zhaoyan said that the baseline is calculated dynamically, > > > > what I understand is baseline set based on previous days > > performance > > > > result, so > > > > it shouldn't require updating. > > > > > > That sounds smart! > > > > > > Perhaps another reference baseline could be added, for informational > > > purposes only: > > > Deviation from the performance of the last official release. > > > > > > > > > > > But cc'ed the lab for more details. > > > > > > > > > > -- > > *Lincoln Lavoie* > > Senior Engineer, Broadband Technologies > > 21 Madbury Rd., Ste. 100, Durham, NH 03824 > > lylavoie@iol.unh.edu > > https://www.iol.unh.edu > > +1-603-674-2755 (m) > > > --=20 Brandon Lo UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 blo@iol.unh.edu www.iol.unh.edu