From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5627AA0503 for ; Wed, 18 May 2022 19:04:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 21EE640156; Wed, 18 May 2022 19:04:09 +0200 (CEST) Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by mails.dpdk.org (Postfix) with ESMTP id 7F925400D6 for ; Wed, 18 May 2022 19:04:08 +0200 (CEST) Received: by mail-pj1-f52.google.com with SMTP id ds11so2703984pjb.0 for ; Wed, 18 May 2022 10:04:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CnTjp0pjwCzT2/Z85Sa2sOcFZiJjcqzsRRZqVViHV78=; b=MQdHRM2NUQ1lpUxBmoZbn3xbtXJD1dFFZGt4v3Zlov0WrrmupCAaQohcKKQZ/kFqdt mkourtZGnbl35c7x3dnPpEeSMahjvG7CXaG9o9UMLo4jHMTkhVQAwczR/gIAmASF+bmF VQQnqdIM4Go8FJib0qpxTmes7IJ2PhcfnMZhWv0ZhgFkHu7pviGZArM2UlCanB54RMpR CdzxX6m9O/LyhlQZfSPvHqaiaRicILdq13o2CvALbdX7FzfThRSb+WiOJvWp1ZWyzH4q EJ0lSdDCmVVdVaouuPfhkIeZnWicsYOxwheunF0UfqFA3UKjDt1bBIX6UWckA+/4AnlA 6zLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CnTjp0pjwCzT2/Z85Sa2sOcFZiJjcqzsRRZqVViHV78=; b=XpOnh73E890mCKWzWAXkGPH0Fsg+OF2ORL2jV+gCFkARPYEwlN4xVbYu6XTL+pnWxl NnVd3E2fOh68xmTzqdqglQALaWvorbtEOIGCmjuehq23KK1COnA6SGLA6yj366RBkQ9q QthyH7TJLLdz13zi6MbVafBaUuJFSTUaxhYUhCi53E/DnhqiTuVqzeqm8daLs/njgfMM LFqsl/aMcUc/dlF3Q25gleWX2tGfC8dd8mblyMiQkfhPUFqD6Xkn1DG1Z3afYDLrU3S/ rjxPYZpKSDCUc2/3uTvDOS4lMw18M+Y/6ZEdm89XFENSFyOTpc8Yc7CJeRpl59hkw7LV ZR1Q== X-Gm-Message-State: AOAM531Hj+wnKNNFsbR6NK2U/IVKA6lmru7qBULz+piLSt8glwmiShNI 4lup08WDVxnLNLg6LTckAWq2eQ== X-Google-Smtp-Source: ABdhPJx6CrsGHF0FsE/j05RjQcl+rRQe8WSU6NAyq8MLWiRgch/Rfi7cc+YqLmqMN7kZULp435n6PQ== X-Received: by 2002:a17:90b:505:b0:1de:ffef:6167 with SMTP id r5-20020a17090b050500b001deffef6167mr980496pjz.172.1652893447374; Wed, 18 May 2022 10:04:07 -0700 (PDT) Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199]) by smtp.gmail.com with ESMTPSA id s15-20020a63524f000000b003c14af5062dsm1745391pgl.69.2022.05.18.10.04.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 May 2022 10:04:06 -0700 (PDT) Date: Wed, 18 May 2022 10:04:04 -0700 From: Stephen Hemminger To: "Sanford, Robert" Cc: Antonio Di Bacco , "users@dpdk.org" Subject: Re: DPDK performances surprise Message-ID: <20220518100404.2cb93c04@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Wed, 18 May 2022 16:53:04 +0000 "Sanford, Robert" wrote: > My guess is that most of the packet data has a short life in the L3 cache (before being overwritten by newer packets), but is never flushed to memory. > > From: Antonio Di Bacco > Date: Wednesday, May 18, 2022 at 12:40 PM > To: "users@dpdk.org" > Subject: DPDK performances surprise > > I recently read a performance test where l2fwd was able to receive packets (8000B) from a 100 Gbps card, swap the L2 addresses and send them back to the same port to be received by an ethernet analyzer. The throughput achieved was close to 100 Gbps on a XEON machine (Intel(R) Xeon(R) Platinum 8176 CPU @ 2.10GHz) . This is the same processor I have and I know that, if I try to write around 8000B to the attached DDR4 (2666MT/s) on an allocated 1GB hugepage, I get a maximum throughput of around 20GB/s. > > Now, a 100 Gbps can generate a flow of around 12 GB/s, these packets have to be written to the DDR and then read back to swap L2 addresses and this leads to a cumulative bandwidth on the DDR that is around 2x12 GB/s and is more than the 20GB/s of available bandwidth on the DDR4. > > How can this be possible ? Likely cache effects from DDIO. What is your packet size.