From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87C76A0503 for ; Thu, 19 May 2022 17:05:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1E39940222; Thu, 19 May 2022 17:05:46 +0200 (CEST) Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by mails.dpdk.org (Postfix) with ESMTP id 0AB5440156 for ; Thu, 19 May 2022 17:05:43 +0200 (CEST) Received: by mail-pf1-f178.google.com with SMTP id u15so5371241pfi.3 for ; Thu, 19 May 2022 08:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HrMB9GcAdDI1/ff2JMXBR3Zg20HJDbBpty7+UUFxulA=; b=dxXm9MYS65YjY0Zh855KTiREg94V31w86LM8ieSlt+Osf/6lDleBE8i4lGfRoU5Udo vU8S4Is0LnaGHIFwPzBEmeBYYzkD9VXuFoxvDcCHomQWXsLROLQtPlUuqh7jRuMZbni0 qBE7Y1RQAg5OKpjFav831qPgCw44CzgD2yszP1eCizsBo9tPxQj7t/QH/r8gV/Y0KT5v kPXlK9kIxTHahU+MI5Z0IIqEQhFyy6az6IWrtfH+xWe1lTWeLSCb4NEqKMX/Y+XW5fb2 IRVq03wFfa6wHnawlO7no2+UJCOvjU1zb4fTdqPlu2KXhbyJGzlAQTa7LldlVO//UM80 pDZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HrMB9GcAdDI1/ff2JMXBR3Zg20HJDbBpty7+UUFxulA=; b=OPwSxzReSmHsKYL828o6NPav9X2QIj52REy799fM/vwP+r0P9vpgEFKSR6pcRAeZVF qjGheNPgccnjsfuEcJ0vQ1ReUeis4uKVA4J+dDNnaCNJKGiUzjB7P4C3D00Kn3aHPtjU ktJ4g5UF9teDglaSm8r6TSFrVLWn8kJAey8fZ6vJk2bxYHSbWln29EkzivZszI8cComH +XZl2bs3+ka4GLUVrA3b5oazpIDGUeV9FwHuz3iEquz32NIk4jGnbdB5UCSxwc43HIoe 4lUI67rj3TVdHE65BL3ju2+A0Asdzdg2bDbKh3nPMcNh1osLAfkd673tCpUja8aT/+fs Vs3Q== X-Gm-Message-State: AOAM5331Irjz6tZtG5tgYRBgnE6ec4T4LfBZ9h8QcKdmvD6LvsCXUD8V rUxyM3JLHrG17NW1iOqqdL2mWg== X-Google-Smtp-Source: ABdhPJwscEcrNVlLQe9mzTTllg7JPJz0zdOTe0LAdawqeOj5g5iW2kX6cBOd9GyB2I3oM6hxObdh0w== X-Received: by 2002:a05:6a00:114b:b0:4f7:915:3ec3 with SMTP id b11-20020a056a00114b00b004f709153ec3mr5470336pfm.8.1652972742965; Thu, 19 May 2022 08:05:42 -0700 (PDT) Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199]) by smtp.gmail.com with ESMTPSA id r24-20020a635d18000000b003c1bf4c064fsm3627794pgb.72.2022.05.19.08.05.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 May 2022 08:05:42 -0700 (PDT) Date: Thu, 19 May 2022 08:05:39 -0700 From: Stephen Hemminger To: "Kinsella, Ray" Cc: Antonio Di Bacco , "Sanford, Robert" , "users@dpdk.org" Subject: Re: DPDK performances surprise Message-ID: <20220519080539.3c3a344a@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Thu, 19 May 2022 09:07:28 +0000 "Kinsella, Ray" wrote: > Better is a _very_ subjective. >=20 > pcm-memory does one thing well. > That whole suite is worth playing with though. >=20 > Ray K >=20 > From: Antonio Di Bacco > Sent: Thursday 19 May 2022 10:04 > To: Kinsella, Ray > Cc: Sanford, Robert ; users@dpdk.org > Subject: Re: DPDK performances surprise >=20 > This tool seems awesome!!! Better than VTUNE? >=20 > On Thu, May 19, 2022 at 10:29 AM Kinsella, Ray > wrote: > I=E2=80=99d say that is likely yes. >=20 > FYI - pcm-memory is very handy tool for looking at memory traffic. > https://github.com/opcm/pcm >=20 > Thanks, >=20 > Ray K >=20 > From: Sanford, Robert > > Sent: Wednesday 18 May 2022 17:53 > To: Antonio Di Bacco >; users@dpdk.org > Subject: Re: DPDK performances surprise >=20 > My guess is that most of the packet data has a short life in the L3 cache= (before being overwritten by newer packets), but is never flushed to memor= y. >=20 > From: Antonio Di Bacco > > Date: Wednesday, May 18, 2022 at 12:40 PM > To: "users@dpdk.org" > > Subject: DPDK performances surprise >=20 > I recently read a performance test where l2fwd was able to receive packet= s (8000B) from a 100 Gbps card, swap the L2 addresses and send them back to= the same port to be received by an ethernet analyzer. The throughput achie= ved was close to 100 Gbps on a XEON machine (Intel(R) Xeon(R) Platinum 8176= CPU @ 2.10GHz) . This is the same processor I have and I know that, if I t= ry to write around 8000B to the attached DDR4 (2666MT/s) on an allocated 1G= B hugepage, I get a maximum throughput of around 20GB/s. >=20 > Now, a 100 Gbps can generate a flow of around 12 GB/s, these packets have= to be written to the DDR and then read back to swap L2 addresses and this = leads to a cumulative bandwidth on the DDR that is around 2x12 GB/s and is = more than the 20GB/s of available bandwidth on the DDR4. >=20 > How can this be possible ? If you are comparing forwarding versus writing the whole packet: - for the forwarding case swapping mac address is a single cache line rea= d/write - for software writing the whole packet it will end up walking through ma= ny cache lines and dirtying them. Also for that test are you rewriting same packet or walking much= larger memory area