From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CF1DFA0559 for ; Mon, 16 Mar 2020 08:45:02 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AED091C037; Mon, 16 Mar 2020 08:45:02 +0100 (CET) Received: from sonic302-20.consmr.mail.gq1.yahoo.com (sonic302-20.consmr.mail.gq1.yahoo.com [98.137.68.146]) by dpdk.org (Postfix) with ESMTP id D2FCE1C037 for ; Mon, 16 Mar 2020 08:45:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1584344701; bh=cyEvmKyQvaG64oAWqmRh0UT9xP86kpbYu5kXUX36ctM=; h=Date:From:To:In-Reply-To:References:Subject:From:Subject; b=YeAj058oCboFhBn8/sjJ+ywzEtaWbB/GscyJeDm6MvZc5NxRtQimfbpBaRlRGkIJjTSXQP+amdy2c61MeTBIKAlI0EOqJ/Wf71aNZVfV352E8lftDF/XHpQ7aTpxMh25QeZSA7NXEW1NQifVzFk/j62//578JGl9RxZZoc04sh9JEqPbHk2A99Akf6/RmVLcZreKO0cRxYMxjo+ImghOKApfcKEm3kU3SS7bAq+pUVGbYSRgh4XU5vfcfDmwUtJAvvbQUPKPMCq4oeIJeaYswvZdUOq/Hcpivi0pjq+tbSYJqZPfvm1iz/x8NlEur19++lAoqWy3ahkBOqWRtmTx1Q== X-YMail-OSG: KYbPxBoVM1lrBxgLuaij71g8JMpzeupjaEaTFjvqE3hVGlq4fq1RpQ_PevocjDc adnuWqJON.Ryy_KvI.u5OD4c_yQwxEjbcdE98NvIlrt4VolHLegnSas6YFwZt3NQ31SDuNllnJQR PMsBondnPuCLj4DfuiIVWs3IgMlYcF31H9wpwDHWwAbOIs2mLJX01eRoyNhpm4EGg_0H8SUh4.AD gwO5H8IsJ4.ttXgmpl6rTWE0SKcptENZvgWu4Lok1J_aFfzA5Jo3wLDp09xiqB_Gici6rlfeZ84i XuHYtRYRSD.l2d5QT.c.VIGrFDbCkclV3l59bwombYLWadxFUm6OMp9b6SmybmS_cPochp5AoM7X qCYdesFrKgXJKkx1AjcnqZ12OBlqS6_7FvLb0fTDJyiQSljP7BNNOZVsznigw3F9UTrOwozfJr4G 0eA9RKhjGZxGx1BWWghEQgAla7dKqUMNiw2wl85bwb7RARiuspAhRrHaUW_hDAWAZ8HyF81oauRw La.UuqIpT4nLYSxUlWe5u3YDOFI5aQdI354R5.lIzrg.UBt_MrpBLmTVZFea2.kiIihAlZ2vRftV oipUu4rNo2loZVJzombxUO1khGl2Tfs4qUKf6A5xIc5WyH0eFghp5YQ_9qgV9ZOivimkhDTb5Rk0 8LcGDV7Mn7tCc4A0NJwJzmX2ZrwIUdL5kSFszVTB_uOIPzgs7BJpT.004ll2nwfJNLuNMBqhWtNY OFtZ_a91gOKlu1i1peKCtMQRNZ9QeRHEph4MqmRy1YYF7HHPcUSPTRlK0PviNJPvD_PvT3G8v7Y8 nOSXxhUInpqG7Fai_FWjpRpPTg7IAhdgntzdylx10Wwx22H6WToKQbNxp9QGTZoAd.SksHJz5Zk_ 9mOWtLmJeN3TuXBT6LLkRUHjMlq1fG1651hJbWHis1uSRvVf3Vt4oRwOs6MU_tsPT93mwb042bi6 fKTuGylHViThrBwWVq3ijvgEscwh7z6OqcEOoGmG5bQI8.cwMX9Tk_o_46.CdbCeqD3rEeZKqVql WQFrSj7DGKLmRhNCdD4UAFIUmPVdhyPwtQG1RcY1MoVa8AbOPCenzhdtUlgkKuWe5fwmil0bOLd4 paQ1lzzDkIeRqLq7uD_ufD0oHKB7yAgtF87M3aZIbbjL2wfd0p.HfpEBUH_mEbCcJsRcNBRdEi4f vmdVb8wvSOYzmW48YfwI9oM7K4dploVjORZx.blZYbzCAWits3lsf2Mb.iQhq0EPRQHP1fEqx.zB _Wj1AERcrOxEEQAviGj5B41MeYBrgmvlGchFaHkhbmWlN4N7x5FEwO35eLdzYyZnT76hEG8w7n6K dxrsaaMm94yMBKrQ9kgPvGbrv0EW0kZ0uJQL4XZRC4LoSuNCfAWAiTycnckbeccT5A0K3z5hy6LS Kioj3dPFeCymX5sYdAw-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.gq1.yahoo.com with HTTP; Mon, 16 Mar 2020 07:45:01 +0000 Date: Mon, 16 Mar 2020 07:44:53 +0000 (UTC) From: sachin gupta To: Yan Lei , "users@dpdk.org" Message-ID: <1002819811.5318253.1584344693085@mail.yahoo.com> In-Reply-To: <22238258b98f4758b5dfec541f0664e8@epfl.ch> References: <7f693d4cbadf4516b204c4cbbc39894f@epfl.ch> <1069532531.4293352.1583995531785@mail.yahoo.com> <22238258b98f4758b5dfec541f0664e8@epfl.ch> MIME-Version: 1.0 X-Mailer: WebService/1.1.15342 YahooMailIosMobile Yahoo%20Mail/48646 CFNetwork/1121.2.2 Darwin/19.3.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] [ConnectX-5 MCX515A-CCAT / MCX516A-CCAT] Can only generate 53Gb/s with 64B packets X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Cool YanThanks for letting me know as as well. Can you also let me know the= link capacity=C2=A0 Sachin=C2=A0 Sent from Yahoo Mail for iPhone On Sunday, March 15, 2020, 12:07 AM, Yan Lei wrote: #yiv6938110405 #yiv6938110405 -- P {margin-top:0;margin-bottom:0;}#yiv69381= 10405 Hi Sachin, Thanks a lot=C2=A0for the answer.=C2=A0The issue is resolved, was able to g= et 98Gb/s with 64B packets after set pci maxReadRequest to 1024 and turn of= f NIC flow control. These optimization settings are actually posted in the = mlx5 PMD guide, my bad to have ignored them... Cheers,LeiFrom: sachin gupta Sent: Thursday, March 12, 2020 7:45:31 AM To: users@dpdk.org; Yan Lei Subject: Re: [dpdk-users] [ConnectX-5 MCX515A-CCAT / MCX516A-CCAT] Can only= generate 53Gb/s with 64B packets=C2=A0Hi Lei, The smaller the Packet size, the more the number of Packets per second. I b= elieve this is the inherent problem in all systems, even the ones which hav= e proprietary hardware. In general applications which uses such small packets are rare and you will= see a mix of traffic in the system. RegardsSachin On Thursday, March 12, 2020, 5:10:14 AM GMT+5:30, Yan Lei w= rote: Hi, I am currently struggling in getting more than 53Gb/s with 64B packets = on both of the MCX515A-CCAT and MCX516A-CCAT adapter when running a DPDK ap= p that generates and transmits packets. WIth 256B packets I can get 98Gb/s. Has anyone saw the same performance on these NICs? I checked the perf. repo= rt onhttps://core.dpdk.org/perf-reports/ but there are no numbers of these = NICs. Is this inherent limitation of these NICs (only reach 100Gb/s with larger p= ackets)? If not, which firmware/driver/DPDK/system configurations could I t= une to get 100Gb/s with 64B packets? My setup is as following: - CPU: E5-26= 97 v3 (14 cores, SMT disabled, CPU frequency fixed @ 2.6 GHz) - NIC: Mellan= ox MCX515A-CCAT / MCX516A-CCAT (Using only one port for TX, installed on PC= Ie Gen3 x16) - DPDK: 19.05 - RDMA-CORE: v28.0 - Kernel: 5.3.0 - OS: Ubuntu = 18.04 - Firmware: 16.26.1040 I measured the TX rate with DPDK's testpmd: $ = ./testpmd -l 3-13 -n 4 -w 02:00.0 -- -i --port-topology=3Dchained --nb-port= s=3D1 --rxq=3D10 --txq=3D10 --nb-cores=3D10 --burst=3D128 --rxd=3D512 --txd= =3D512 --mbcache=3D512 --forward-mode=3Dtxonly So 10 cores generating and t= ransmits 64B packets on 10 NIC queues. Your feedbacks will be much apprecia= ted. Thanks, Lei