From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C9D2A0503 for ; Thu, 19 May 2022 11:03:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6C03740150; Thu, 19 May 2022 11:03:16 +0200 (CEST) Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by mails.dpdk.org (Postfix) with ESMTP id 7BC2540140 for ; Thu, 19 May 2022 11:03:14 +0200 (CEST) Received: by mail-ot1-f54.google.com with SMTP id y20-20020a056830071400b00606a2ebd91bso3115699ots.5 for ; Thu, 19 May 2022 02:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MIP/nUGQCDBlN2S+HDrOdHtIuQJHY0w/RoKnEw65K1Y=; b=WRk26nUXGOyoaZpMkV7bZ+tQ2/CfvzOsn/BoH6PViHRPYs6akFMub/pZ52lieHhpiy tJRyonhvwLi4/GVCi/jwUb43eOuqhIHYuiqkdaZtWx51hqRvVFD/oGhm8FImO9r/YvqP Pnv/OII02yrczVi0TBCn1pra3GsqHO4oo3YQ/lVcwqbYu1HxVcoVakAmk0hwSwrjFpJ6 Zx9XyFvEHkfAy9uN7eXJqMr3YEdhsJpdeS2f6zxlWurZzhKuyhbTWIQPkMUqv9J+hTbR g000SdzIrPq72RznogWp/veT3rWDt2wxt96HaIvVE2CJfTPNcrhPDezX04lIXYW5NNWw T6pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MIP/nUGQCDBlN2S+HDrOdHtIuQJHY0w/RoKnEw65K1Y=; b=qhapYqBKGkHh8zoYRxvIDOOVzv48MboXP7WmT8+d1At2qGEbVCfXfNqCPKflFlx7AZ gCdjWLeOPe9rguoeX4+tf+yrocDw5sDf5u0wjJ7oZ/CmYyZP/xFHbUlu3mlQS7MCIMra eRuOh52Du8LeQxqNAT1kw3Gd2MMYEmhN9Tgo+BCUGUHT6eWqiG+Tnb88RRnl+SJCDwp7 RetH4vPMK3wzvrSElEMHx9T8NvpotfaCtBb86cF612wJC3apKfj6cXLU+LCO+Hfdcob1 aadfsPjwOX28ULUAMEyt5wBrIQnw5ylW9vkaLGm8rV+tM7IcvGeaops8TQ+E5WGBGTvO G3EQ== X-Gm-Message-State: AOAM532A3jSPbXYCWqZ0URvEVlNksw/teUsorlKKdfWHZaQ5oD1N3bxA FMHmXIy/5d2Pft1LiyDynCKIn5op/xUtiC7Qslo= X-Google-Smtp-Source: ABdhPJzbRp6GElQyX5ayWg9CSotfhdccQJfWqXr7kKPlY3Ie62jJ3EkpM3RvcwNBfmk1AtIZMWawTfDeI1qHyHou2KY= X-Received: by 2002:a9d:b8a:0:b0:605:d870:e3b7 with SMTP id 10-20020a9d0b8a000000b00605d870e3b7mr1419030oth.61.1652950993769; Thu, 19 May 2022 02:03:13 -0700 (PDT) MIME-Version: 1.0 References: <20220518100404.2cb93c04@hermes.local> In-Reply-To: <20220518100404.2cb93c04@hermes.local> From: Antonio Di Bacco Date: Thu, 19 May 2022 11:03:02 +0200 Message-ID: Subject: Re: DPDK performances surprise To: Stephen Hemminger Cc: "Sanford, Robert" , "users@dpdk.org" Content-Type: multipart/alternative; boundary="000000000000d0974005df59a4b3" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000d0974005df59a4b3 Content-Type: text/plain; charset="UTF-8" The packets are 8000B long. On Wed, May 18, 2022 at 7:04 PM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Wed, 18 May 2022 16:53:04 +0000 > "Sanford, Robert" wrote: > > > My guess is that most of the packet data has a short life in the L3 > cache (before being overwritten by newer packets), but is never flushed to > memory. > > > > From: Antonio Di Bacco > > Date: Wednesday, May 18, 2022 at 12:40 PM > > To: "users@dpdk.org" > > Subject: DPDK performances surprise > > > > I recently read a performance test where l2fwd was able to receive > packets (8000B) from a 100 Gbps card, swap the L2 addresses and send them > back to the same port to be received by an ethernet analyzer. The > throughput achieved was close to 100 Gbps on a XEON machine (Intel(R) > Xeon(R) Platinum 8176 CPU @ 2.10GHz) . This is the same processor I have > and I know that, if I try to write around 8000B to the attached DDR4 > (2666MT/s) on an allocated 1GB hugepage, I get a maximum throughput of > around 20GB/s. > > > > Now, a 100 Gbps can generate a flow of around 12 GB/s, these packets > have to be written to the DDR and then read back to swap L2 addresses and > this leads to a cumulative bandwidth on the DDR that is around 2x12 GB/s > and is more than the 20GB/s of available bandwidth on the DDR4. > > > > How can this be possible ? > > Likely cache effects from DDIO. What is your packet size. > --000000000000d0974005df59a4b3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The packets are 8000B long.

On Wed, May 18, 2022 at 7:04 PM= Stephen Hemminger <stephe= n@networkplumber.org> wrote:
On Wed, 18 May 2022 16:53:04 +0000
"Sanford, Robert" <rsanford@akamai.com> wrote:

> My guess is that most of the packet data has a short life in the L3 ca= che (before being overwritten by newer packets), but is never flushed to me= mory.
>
> From: Antonio Di Bacco <a.dibacco.ks@gmail.com>
> Date: Wednesday, May 18, 2022 at 12:40 PM
> To: "users@dp= dk.org" <us= ers@dpdk.org>
> Subject: DPDK performances surprise
>
> I recently read a performance test where l2fwd was able to receive pac= kets (8000B) from a 100 Gbps card, swap the L2 addresses and send them back= to the same port to be received by an ethernet analyzer. The throughput ac= hieved was close to 100 Gbps on a XEON machine (Intel(R) Xeon(R) Platinum 8= 176 CPU @ 2.10GHz) . This is the same processor I have and I know that, if = I try to write around 8000B to the attached DDR4 (2666MT/s) on an allocated= 1GB hugepage, I get a maximum throughput of around 20GB/s.
>
> Now, a 100 Gbps can generate a flow of around 12 GB/s, these packets h= ave to be written to the DDR and then read back to swap L2 addresses and th= is leads to a cumulative bandwidth on the DDR that is around 2x12 GB/s and = is more than the 20GB/s of available bandwidth on the DDR4.
>
> How can this be possible ?

Likely cache effects from DDIO. What is your packet size.
--000000000000d0974005df59a4b3--