From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D1B40A056B for ; Thu, 12 Mar 2020 00:40:06 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1430B1BF7F; Thu, 12 Mar 2020 00:40:06 +0100 (CET) Received: from smtp4.epfl.ch (smtp4.epfl.ch [128.178.224.219]) by dpdk.org (Postfix) with ESMTP id BF3C32BE6 for ; Thu, 12 Mar 2020 00:40:04 +0100 (CET) Received: (qmail 57369 invoked by uid 107); 11 Mar 2020 23:40:04 -0000 Received: from ax-snat-224-170.epfl.ch (HELO ewa04.intranet.epfl.ch) (192.168.224.170) (TLS, AES256-GCM-SHA384 cipher) by mail.epfl.ch (AngelmatoPhylax SMTP proxy) with ESMTPS; Thu, 12 Mar 2020 00:40:04 +0100 X-EPFL-Auth: FN/ZhM8wJGzl9k8Hc9iUYadyFD99hGd9KIOKpFw+9H8kNa7sGoA= Received: from ewa09.intranet.epfl.ch (128.178.224.180) by ewa04.intranet.epfl.ch (128.178.224.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Thu, 12 Mar 2020 00:40:03 +0100 Received: from ewa09.intranet.epfl.ch ([fe80::282f:1598:ce61:a9ca]) by ewa09.intranet.epfl.ch ([fe80::282f:1598:ce61:a9ca%3]) with mapi id 15.01.1913.007; Thu, 12 Mar 2020 00:40:03 +0100 From: Yan Lei To: "users@dpdk.org" Thread-Topic: [ConnectX-5 MCX515A-CCAT / MCX516A-CCAT] Can only generate 53Gb/s with 64B packets Thread-Index: AQHV9/4Ra+Ts/cQoqEegSuWXUVpR/g== Date: Wed, 11 Mar 2020 23:40:03 +0000 Message-ID: <7f693d4cbadf4516b204c4cbbc39894f@epfl.ch> Accept-Language: en-US, fr-CH Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [128.179.253.11] MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] [ConnectX-5 MCX515A-CCAT / MCX516A-CCAT] Can only generate 53Gb/s with 64B packets X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, I am currently struggling in getting more than 53Gb/s with 64B packets = on both of the MCX515A-CCAT and MCX516A-CCAT adapter when running a DPDK ap= p that generates and transmits packets. WIth 256B packets I can get 98Gb/s. Has anyone saw the same performance on these NICs? I checked the perf. repo= rt on https://core.dpdk.org/perf-reports/ but there are no numbers of these= NICs. Is this inherent limitation of these NICs (only reach 100Gb/s with larger p= ackets)? If not, which firmware/driver/DPDK/system configurations could I t= une to get 100Gb/s with 64B packets? My setup is as following: - CPU: E5-26= 97 v3 (14 cores, SMT disabled, CPU frequency fixed @ 2.6 GHz) - NIC: Mellan= ox MCX515A-CCAT / MCX516A-CCAT (Using only one port for TX, installed on PC= Ie Gen3 x16) - DPDK: 19.05 - RDMA-CORE: v28.0 - Kernel: 5.3.0 - OS: Ubuntu = 18.04 - Firmware: 16.26.1040 I measured the TX rate with DPDK's testpmd: $ = ./testpmd -l 3-13 -n 4 -w 02:00.0 -- -i --port-topology=3Dchained --nb-port= s=3D1 --rxq=3D10 --txq=3D10 --nb-cores=3D10 --burst=3D128 --rxd=3D512 --txd= =3D512 --mbcache=3D512 --forward-mode=3Dtxonly So 10 cores generating and t= ransmits 64B packets on 10 NIC queues. Your feedbacks will be much apprecia= ted. Thanks, Lei