From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <3chas3@gmail.com> Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by dpdk.org (Postfix) with ESMTP id 9866D1B19; Sun, 9 Sep 2018 22:58:10 +0200 (CEST) Received: by mail-io1-f66.google.com with SMTP id 75-v6so5409586iou.11; Sun, 09 Sep 2018 13:58:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:sender:from:date:message-id :subject:to:cc; bh=tzv/Ie81AswywXey+qkPtdr3wMRgDxgv5zw63ZYnPlQ=; b=MvNh3Ep05OfI/1vCT9sf0svbGVtiNzb4SXW3iqk9CeHMnwbm7HS8rhoes1B1hdNB1g uomAWE5uY35V041ykbAhZTBn6NCQ/WhCufRtbJ94aGOCxQ5jttaiWYOJhrxvapWzevbH G/P/gQM/RkYiqKYLUVMp4DQVnhXLnXXKnlCoYQDvm1CzaKpmQxJg9pTz/0oybEMkHwJC mzB0O5ggax/48NeV6zt6C9PEVMX9znQ/tply2mLZzv0JN0kKZ3JuA6YWM9MARkMgUZvu f1gmfFgQjCFQrViKERf37K8ZyI9JVm6alsnONjUDVqqEdP5jD3rsnEQ+ExYnUj+8g8Og eRIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:sender:from :date:message-id:subject:to:cc; bh=tzv/Ie81AswywXey+qkPtdr3wMRgDxgv5zw63ZYnPlQ=; b=gQNFIfQBAzLLNhGItx4IUQHxFxj+yJXeCKKXnvBQYHcMucex2G3Rpq5l6DNaiJliVu 3yI/xWuUg/PmR/Ju8UGSNqMATzWk28ab8G0GTAn4CZ4Bb9Wxgs9iCVIuu0qguY2hgsIL cMJgQVW9omtHcQeegVusUEEtjjmiygN8KJMlGj0jQSTZJCXzw4I0bBe7H29+roGLvK7H zBqQ2kX2KvDm/1AubXCkAuZzuQNK1d9ChKv2VLxqFRy5XZUtHyGV8lqbazCFHu5Gu6t0 RKIBZdInGnpc8Hox49CosB8iBwnTVa5SNt82v42GdVk61BhC8leMEQeF+3uXZ20NqLc1 uGdg== X-Gm-Message-State: APzg51BhDx9laoU/8Xn0bQkSLoBD/GWMaAo3bL3jNCJPEpZL1CoER32F WLz4yRZ9vh/kiDpDuVbZVIvxkciKuiROQ9ZFzXA= X-Google-Smtp-Source: ANB0Vdb+3zsfJ//drD3yzJNNtxlzEZPZMkxkuaoExMQa4Erch3+YjOE9aSqr6lwm3kpJbJo9z0ilR2ltkLKytdKNoQM= X-Received: by 2002:a6b:9704:: with SMTP id z4-v6mr14278645iod.110.1536526689874; Sun, 09 Sep 2018 13:58:09 -0700 (PDT) MIME-Version: 1.0 References: <20180816125202.15980-1-bluca@debian.org> <20180816133208.26566-1-bluca@debian.org> <1534933159.5764.107.camel@debian.org> <20180822174316.GA29821@roosta> <1535731291.11823.33.camel@debian.org> In-Reply-To: Sender: chasmosaurus@gmail.com X-Google-Sender-Delegation: chasmosaurus@gmail.com From: Chas Williams <3chas3@gmail.com> Date: Sun, 9 Sep 2018 16:57:58 -0400 X-Google-Sender-Auth: 638N6MU6lTA_8UWjN0lVGpmsI4E Message-ID: To: Matan Azrad Cc: bluca@debian.org, Eric Kinzie , dev@dpdk.org, Declan Doherty , Chas Williams , stable@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [dpdk-stable] [PATCH v4] net/bonding: per-slave intermediate rx ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2018 20:58:10 -0000 On Sun, Sep 2, 2018 at 7:34 AM Matan Azrad wrote: > > Hi Luca\Chas > > From: Luca Boccassi > > On Wed, 2018-08-29 at 15:20 +0000, Matan Azrad wrote: > > > > > > From: Chas Williams > > > > On Tue, Aug 28, 2018 at 5:51 AM Matan Azrad > > > com> wrote: > > > > > > > > > > > > From: Chas Williams > > > > > On Mon, Aug 27, 2018 at 11:30 AM Matan Azrad > > > > @mellanox.com> wrote: > > > > > > > > > > > > > > > Because rings are generally quite efficient. > > > > > > > > > > > > But you are using a ring in addition to regular array > > > > > > management, it must hurt performance of the bonding PMD (means > > > > > > the bonding itself - not the slaves PMDs which are called from > > > > > > the bonding) > > > > > > > > > > It adds latency. > > > > > > > > And by that hurts the application performance because it takes more > > > > CPU time in the bonding PMD. > > > > > > > > No, as I said before it takes _less_ CPU time in the bonding PMD > > > > because we use a more optimal read from the slaves. > > > > > > Each packet pointer should be copied more 2 times because of this > > > patch + some management(the ring overhead) So in the bonding code you > > > lose performance. > > > > > > > > > > > > It increases performance because we spend less CPU time reading > > > > > from the PMDs. > > > > > > > > So, it's hack in the bonding PMD to improve some slaves code > > > > performance but hurt the bonding code performance, Over all the > > > > performance we gain for those slaves improves the application > > > > performance only when working with those slaves. > > > > But may hurt the application performance when working with other > > > > slaves. > > > > > > > > What is your evidence that is hurts bonding performance? Your > > > > argument is purely theoretical. > > > > > > Yes, we cannot test all the scenarios cross the PMDs. > > > > Chas has evidence that this helps, a _lot_, in some very common cases. > > We haven't seen evidence of negative impact anywhere in 2 years. Given > > this, surely it's not unreasonable to ask to substantiate theoretical a= rguments > > with some testing? > > What is the common cases of the bond usage? > Do you really know all the variance of the bond usages spreading all over= the world? We actually have a fairly large number of deployments using this bonding co= de across a couple different adapter types (mostly Intel though and some virtu= al usage). The patch was designed to address starvation of slaves because of the way that vector receives tend on the Intel PMDs. If there isn't enough space to attempt a vector receive (a minimum of 4 buffers), then the rx burst will return a value of 0 -- no buffers read. The rx burst in bonding moves to the next adapter. So this tend to starve any slaves that aren't first in line. The issue doe= sn't really show up in single streams. You need to run multiple streams that multiplex across all the slaves. > > I=E2=80=99m saying that using a hack in the bond code which helps for som= e slaves PMDs\application scenarios (your common cases) but hurting > the bond code performance and latency is not the right thing to do becaus= e it may hurt other scenarios\PMDs using the bond. What do you think about the attached patch? It implements an explicit round-robin for the "first" slave in order to enforce some sort of fairness. Some limited testing has shown that this our application scan scale polling to read the PMDs fast enough. Note, I have only tested the 802.3ad paths. The changes are likely necessary for the other RX burst routines since they should suffer the same issue.