From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3830945521 for ; Fri, 28 Jun 2024 11:57:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AC7C040E96; Fri, 28 Jun 2024 11:57:48 +0200 (CEST) Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by mails.dpdk.org (Postfix) with ESMTP id 1811F4029E for ; Fri, 28 Jun 2024 11:57:47 +0200 (CEST) Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-70df213542bso240885a12.3 for ; Fri, 28 Jun 2024 02:57:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=domainhart-com.20230601.gappssmtp.com; s=20230601; t=1719568666; x=1720173466; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=PZI0aDcEcB/pYJM9fKOCg4K5WqM65dxa5u7Xx6ZT/ZI=; b=l3+HpXgZONCURV/w7hduOLWIt1EQ2nSGyir5rIR+Y00DQQZGosBhVJMN2eE1WKavn7 /IlziLBFZB3oSAhWWRnUOHk5CcOfy4D33K79CN+ydAr3bp6LNGltzH4QpfBUkshnW0mP OinWdWGE7ZEviB5JyqlNuxuASAz+s0s11s4/L8rplR5Dsbm9zeleKQvlAV2TLWN7GMy+ /jaK3y19Nf45lyiDuzsrd+dT1dblshR47iWhwpOvTtCLonNSYDvI3gsaJvVf+y+Bgkwf cKJfKDgBwcUpFyobQHiv+YB5VdsKOEBQYOWuSS7PDhlMNBBF8fsnvEp8bkZZ98gaB+9Y KXMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719568666; x=1720173466; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PZI0aDcEcB/pYJM9fKOCg4K5WqM65dxa5u7Xx6ZT/ZI=; b=to1OG+/PqaoVXyQazURGUUXTSuw5vOiHEZpJUkqAQuEtEwEdTQabcR+j39tEpH16yB 3cGo0GHS4Gt6TPq+Tn5C42Z+cGsSLvBpy51b+KUlHDQz20E4PtA8FN5Q+X38MmcL+5D5 R5jOvzs9QYHY/q0Xbc6BE27oXplx9hydJwYoFmxUtjy8niCdUwTjQpfsrvU3Pf/cleKV ShEVmAbA4yA2/5Vt5uJulKQjBKwBFmRqytLfvaj6B4M4UrJeQjQh6DtJY7Kekn7r+5DF DdZkcWbn8BPhiPxHJZDtQJk1SETXo2asaRenMTsCBYXXhl92PY84CSrbTe3sNvAOLF+R x1dw== X-Gm-Message-State: AOJu0YwNRbH7RpZeYV+Go1fA8y3wrfYBL3qg8YacQ3KDW+jq6cEvXhj5 NKEAZts1NExTK8hYjNsSzzlAJyKNFrqcev8b/IhBQxkbLjtjm8HP789WLc3X65usyuaJQc7ydvJ jlXQozVHdtE3ceGhcR4LVRrqipY44WD7d2aADYg== X-Google-Smtp-Source: AGHT+IFsrBaDIbuQJgchtZFVPjxDhCLIi9ZzRMiOiDH0ETeBEB0bFgmM96SBedx0GL7j58pFaTEzVhLdY2nDJPgny0Y= X-Received: by 2002:a05:6a20:8190:b0:1be:bff2:b1bb with SMTP id adf61e73a8af0-1bebff2b7d2mr6494859637.44.1719568666154; Fri, 28 Jun 2024 02:57:46 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Tony Hart Date: Fri, 28 Jun 2024 05:57:34 -0400 Message-ID: Subject: Re: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin To: Bing Zhao Cc: "users@dpdk.org" Content-Type: multipart/alternative; boundary="00000000000083750e061bf047c9" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000083750e061bf047c9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable FYI if I add a 'eth' only line (e.g. to match ARP packets) the performance remains the same (which is good). flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end actions count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 priority 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 priority 2 pattern eth / end actions count / rss queues 6 7 8 9 end / end Any thoughts why the single 'pattern eth / end' performance is worse? On Wed, Jun 26, 2024 at 9:24=E2=80=AFAM Tony Hart wrote: > Hi Bing, > Thanks for the quick reply. The results are... > > With a single hairpin queue I get approx the same rate for both > patterns, ~54Gbps. I assume this is less than the RSS rates due to > fewer queues? > flow create 0 ingress group 1 pattern eth / end actions count / queue > index 6 / end > flow create 0 ingress group 1 pattern eth / ipv4 / end actions count / > queue index 6 / end > > With the split ipv6/ipv4 I'm getting ~124Gbps > > flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end > actions count / rss queues 6 7 8 9 end / end > flow create 0 ingress group 1 priority 1 pattern eth / ipv4 / end > actions count / rss queues 6 7 8 9 end / end > > testpmd> flow list 0 > ID Group Prio Attr Rule > 0 0 0 i-- =3D> JUMP > 1 1 1 i-- ETH IPV6 =3D> COUNT RSS > 2 1 1 i-- ETH IPV4 =3D> COUNT RSS > > On Wed, Jun 26, 2024 at 8:10=E2=80=AFAM Bing Zhao wrot= e: > > > > Hi Tony, > > > > Could you also try to test with: > > 1. QUEUE action instead of RSS and check 1 queue performance. > > 2. when trying to test IPv4 only case, try the following 3 commands wit= h > this order - > > flow create 0 ingress group 0 pattern end actions jump group 1 = / > end > > flow create 0 ingress group 1 pattern priority 1 eth / ipv6 / > end actions count / rss queues 6 7 8 9 end / end > > flow create 0 ingress group 1 pattern priority 1 eth / ipv4 / > end actions count / rss queues 6 7 8 9 end / end > > > > BR. Bing > > > > > -----Original Message----- > > > From: Tony Hart > > > Sent: Wednesday, June 26, 2024 7:39 PM > > > To: users@dpdk.org > > > Subject: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in > hairpin > > > > > > External email: Use caution opening links or attachments > > > > > > > > > I'm using a CX7 and testing hairpin queues. The test traffic is > entirely > > > IPv4+UDP with distributed SIP,DIP pairs and received packets are > u-turned via > > > hairpin in the CX7 (single 400G interface). > > > > > > I see different performance when I use a pattern of 'eth' versus > 'eth/ipv4' in > > > the hairpin flow entry. From testing it seems that specifying just > 'eth' is > > > sufficient to invoke RSS and 'eth/ipv4' > > > should be equivalent since the traffic is all ipv4, but I'm getting > ~104Gbps for > > > the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. > > > > > > Any thoughts on why there is such a performance difference here? > > > > > > thanks > > > tony > > > > > > This is the 'eth' pattern testpmd commands flow create 0 ingress grou= p > 0 > > > pattern end actions jump group 1 / end flow create 0 ingress group 1 > pattern > > > eth / end actions count / rss queues 6 7 8 9 end / end > > > > > > The testpmd commands for 'eth/ipv4' > > > flow create 0 ingress group 0 pattern end actions jump group 1 / end > flow > > > create 0 ingress group 1 pattern eth / ipv4 / end actions count / rss > queues 6 7 > > > 8 9 end / end > > > > > > > > > This is the testpmd command line... > > > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=3D1 -- -i --nb-cores 6 --rxq= 6 > --txq 6 > > > --port-topology loop --forward-mode=3Drxonly --hairpinq 4 --hairpin-m= ode > > > 0x10 > > > > > > Versions > > > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 > > > > > > DPDK: v24.03 > > > > -- > tony > --=20 tony --00000000000083750e061bf047c9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
FYI if I add a 'eth' only line (e.g. to match ARP = packets) the performance remains the same (which is good).=C2=A0

flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end<= span class=3D"gmail-im" style=3D"color:rgb(80,0,80)">
actions count / rs= s queues 6 7 8 9 end / end
flow create 0 ingress group 1 priority= 1 pattern eth / ipv4 / end
actions count / rss queues 6 7 8 9 end / end
flow create 0 ingress group 1 priority 2 pattern eth / end<= /span>
actions count / rss queues 6 7 8 9 end /= end

Any t= houghts why the single 'pattern eth / end' performance is worse?


On Wed, Jun 26, 2024 at 9= :24=E2=80=AFAM Tony Hart <to= ny.hart@domainhart.com> wrote:
Hi Bing,
Thanks for the quick reply.=C2=A0 The results are...

With a single hairpin queue I get approx the same rate for both
patterns, ~54Gbps.=C2=A0 I assume this is less than the RSS rates due to fewer queues?
flow create 0 ingress group 1 pattern eth / end actions count / queue
index 6 / end
flow create 0 ingress group 1 pattern eth / ipv4 / end actions count /
queue index 6 / end

With the split ipv6/ipv4 I'm getting ~124Gbps

flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end
actions count / rss queues 6 7 8 9 end / end
flow create 0 ingress group 1 priority 1 pattern eth / ipv4 / end
actions count / rss queues 6 7 8 9 end / end

testpmd> flow list 0
ID Group Prio Attr Rule
0 0 0 i-- =3D> JUMP
1 1 1 i-- ETH IPV6 =3D> COUNT RSS
2 1 1 i-- ETH IPV4 =3D> COUNT RSS

On Wed, Jun 26, 2024 at 8:10=E2=80=AFAM Bing Zhao <bingz@nvidia.com> wrote:
>
> Hi Tony,
>
> Could you also try to test with:
> 1. QUEUE action instead of RSS and check 1 queue performance.
> 2. when trying to test IPv4 only case, try the following 3 commands wi= th this order -
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0flow create 0 ingress group 0 pattern= end actions jump group 1 / end
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0flow create 0 ingress group 1 pattern= priority 1 eth / ipv6 / end actions count / rss queues 6 7 8 9 end / end >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0flow create 0 ingress group 1 pattern= priority 1 eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end >
> BR. Bing
>
> > -----Original Message-----
> > From: Tony Hart <tony.hart@domainhart.com>
> > Sent: Wednesday, June 26, 2024 7:39 PM
> > To: users@dpd= k.org
> > Subject: Performance of CX7 with 'eth' pattern versus = 9;eth/ipv4' in hairpin
> >
> > External email: Use caution opening links or attachments
> >
> >
> > I'm using a CX7 and testing hairpin queues.=C2=A0 The test tr= affic is entirely
> > IPv4+UDP with distributed SIP,DIP pairs and received packets are = u-turned via
> > hairpin in the CX7 (single 400G interface).
> >
> > I see different performance when I use a pattern of 'eth'= versus 'eth/ipv4' in
> > the hairpin flow entry.=C2=A0 From testing it seems that specifyi= ng just 'eth' is
> > sufficient to invoke RSS and 'eth/ipv4'
> > should be equivalent since the traffic is all ipv4, but I'm g= etting ~104Gbps for
> > the 'eth' pattern and=C2=A0 ~124Gbps for 'eth/ipv4= 9; pattern.
> >
> > Any thoughts on why there is such a performance difference here?<= br> > >
> > thanks
> > tony
> >
> > This is the 'eth' pattern testpmd commands flow create 0 = ingress group 0
> > pattern end actions jump group 1 / end flow create 0 ingress grou= p 1 pattern
> > eth / end actions count / rss queues 6 7 8 9 end / end
> >
> > The testpmd commands for 'eth/ipv4'
> > flow create 0 ingress group 0 pattern end actions jump group 1 / = end flow
> > create 0 ingress group 1 pattern eth / ipv4 / end actions count /= rss queues 6 7
> > 8 9 end / end
> >
> >
> > This is the testpmd command line...
> > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=3D1 -- -i --nb-cores 6 -= -rxq 6 --txq 6
> > --port-topology loop --forward-mode=3Drxonly --hairpinq 4 --hairp= in-mode
> > 0x10
> >
> > Versions
> > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64
> > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64
> > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64
> > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64
> >
> > DPDK: v24.03



--
tony


--
to= ny
--00000000000083750e061bf047c9--