From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CBC0A45521 for ; Fri, 28 Jun 2024 11:54:03 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9B63D40E96; Fri, 28 Jun 2024 11:54:03 +0200 (CEST) Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by mails.dpdk.org (Postfix) with ESMTP id A0EF54029E for ; Fri, 28 Jun 2024 11:54:02 +0200 (CEST) Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-71910dfb8c0so239027a12.3 for ; Fri, 28 Jun 2024 02:54:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=domainhart-com.20230601.gappssmtp.com; s=20230601; t=1719568441; x=1720173241; darn=dpdk.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=YLLq3oMNmfnmHRgAHCpFHbSseowgBT/WTtoYPA32DAU=; b=vOf3365kpLnMBuBF5iEelCjnHZUtQJpDItFPyC2fED6ereLY3XT54tb4lAnBhiEJaO 331wN+CNsJzn8RDmd8581ZQ9bkKPt/IVrrRm2lzwKayDYMmIduJPY5kQowaPqC7fohnM syvDSE1OyYVXKbgooTeEXLOhEPduWlQqb6f+dkoSMxWS/ANH95e+436eNzEZkWG8roRY P5erweyX6MXpDcGkvogwdN2+CvLCR1FyYYbh6jxkAHHo2OBk2gGFH4Jg5FfGND5LtPBY cBnveELIWaW7seNPek4PcxEYvCX01uz0gm6XgHlxcXik+XQkGsRep0rqjNDNRdGSsYy4 F3XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719568441; x=1720173241; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=YLLq3oMNmfnmHRgAHCpFHbSseowgBT/WTtoYPA32DAU=; b=bV2YXxTc8z4BtZK9fxAVXsSu6lhCMBK7pDVeErhQDCrGn2duLPhh9ShJz/wtw6LQog Wfsf3+rMJ6HV0OjojY3kgEgveDAfAatZoL0dKwmXHbsTfYEVuoeHvQ6ci6o/SU8mzLm3 gtY6xWmZA2vuTg+UgT9uU8/ofT+SO+TtH4+CgCBcDgwsxl+CM43txt5GS4JNcKGd2hv4 HO6NFXszcBhZylklSIVgmO/v3iVD/FBHfQ9vWLEq1l8eFRelha/uTEPDfa9+dC/ovpXd hEo+fdZywzKZK+phGnwaugC2r28w/pDGKFro5PH4JvVckztppY3nNQ7W2530g++bgkhu whig== X-Gm-Message-State: AOJu0YxEKUFCvzuEbPWVoMQybfbJ7/j8a9q3Go8qRdNM5WKPU3bnbPda fq3WlyFpbo0WWAopmcHjEKZsy2Fjffe4c5mvbwFYden3CiT/kYMUQQeduWFmLo97EBDwsCom+i0 opNeHGoMbLgAMVQEBxQvpprdCaskmmV2MpINqzwXEyJ4BRLf3MbU= X-Google-Smtp-Source: AGHT+IFr8nMdlQpNUDoj8EZNSjncwQjV/9oXD76ChgPvsZFUzDRWM0cL2uLGHV8vxM8+llo5Fdh4kNYP3uC9W3c/p84= X-Received: by 2002:a17:90a:4e4e:b0:2c8:f3b4:a3df with SMTP id 98e67ed59e1d1-2c8f3b4a487mr4471196a91.42.1719568441298; Fri, 28 Jun 2024 02:54:01 -0700 (PDT) MIME-Version: 1.0 From: Tony Hart Date: Fri, 28 Jun 2024 05:53:49 -0400 Message-ID: Subject: Performance of CX7 with 3 flow groups versus 2 To: users@dpdk.org Content-Type: multipart/alternative; boundary="0000000000001c6c88061bf03a6c" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000001c6c88061bf03a6c Content-Type: text/plain; charset="UTF-8" I'm seeing an unexpected performance drop on the CX7 when using 3 groups (with a policer) versus 3 groups (without policer) versus (2 groups without policer). The performance of each configuration is 72 Gbps, 104 Gbps and 124 Gbps respectively. So the first configuration drops to almost half the performance of the third even though all 3 are just hairpinning packets (the policier is used to color the packet only, no fate actions are taken as a result). This is on a 400G link and using SWS mode. I know there was a similar issue reported on this mailing list recently related to SWS versus HWS performance, but this issue seems different. Any thoughts welcome. thanks tony These are the testpmd commands used to recreate the issue... *Common commands:*add port meter profile trtcm_rfc4115 0 1 1000 150000000 1000 1000 1 add port meter policy 0 1 g_actions end y_actions end r_actions drop / end create port meter 0 1 1 1 yes 0xffff 0 g 0 *3 groups with policer* flow create 0 ingress group 0 pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern end actions meter mtr_id 1 / jump group 2 / end flow create 0 ingress group 2 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end *3 groups without policer* flow create 0 ingress group 0 pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern end actions jump group 2 / end flow create 0 ingress group 2 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end *2 groups without policer* flow create 0 ingress group 0 pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end thanks, tony *testpmd command line* /dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq 6 --txq 6 --port-topology loop --forward-mode=rxonly --hairpinq 4 --hairpin-mode 0x10 *Versions* mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 DPDK: v24.03 --0000000000001c6c88061bf03a6c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I'm seeing an unexpected performance=C2=A0drop on the = CX7 when using 3 groups (with a policer) versus 3 groups (without=C2=A0poli= cer) versus (2 groups without policer).=C2=A0 The performance of each confi= guration is=C2=A0 72 Gbps, 104 Gbps=C2=A0and 124 Gbps=C2=A0respectively.=C2= =A0 =C2=A0So the first configuration drops to almost half the performance o= f the third even though all 3 are just hairpinning packets (the policier is= used to color the packet only, no fate actions are taken as a result).
=
This is on a 400G link and using SWS mode.=C2=A0 I know there was a sim= ilar issue reported on this mailing list recently related to SWS versus HWS= performance, but this issue seems different.

Any=C2=A0thoughts welc= ome.

thanks
tony

Thes= e are the testpmd commands used to recreate the issue...

Common c= ommands:
add port meter profile trtcm_rfc4115 0 1 1000 150000000 100= 0 1000 1

add port meter policy 0 1 g_actions end y_actions end r_act= ions drop =C2=A0/ end

create port meter 0 1 1 1 yes 0xffff 0 g 0
=

3 groups with policer

flow create 0 ingress group 0 p= attern end actions jump group 1 / end

flow create 0 ingress group 1 = pattern end actions meter mtr_id 1 / jump group 2 / end

flow create = 0 ingress group 2 pattern eth / ipv4 / end actions count / rss queues 6 7 8= 9 end / end


3 groups without policer

flow create = 0 ingress group 0 pattern end actions jump group 1 / end

flow create= 0 ingress group 1 pattern end actions jump group 2 / end

flow creat= e 0 ingress group 2 pattern eth / ipv4 / end actions count / rss queues 6 7= 8 9 end / end


2 groups without policer

flow creat= e 0 ingress group 0 pattern end actions jump group 1 / end

flow crea= te 0 ingress group 1 pattern eth / ipv4 / end actions count / rss queues 6 = 7 8 9 end / end

thanks,
tony

testpmd comman= d line
/dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=3D1 -- -i --= nb-cores 6 --rxq 6 --txq 6 --port-topology loop --forward-mode=3Drxonly --h= airpinq 4 --hairpin-mode 0x10


<= u>Versions
mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhe= l9u4.x86_64
kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64=
mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64
ofed-s= cripts-24.04-OFED.24.04.0.6.6.x86_64

DPDK: v24.03

--0000000000001c6c88061bf03a6c--