From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8E284A0569 for ; Thu, 12 Mar 2020 07:46:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D8B481BFA5; Thu, 12 Mar 2020 07:46:10 +0100 (CET) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) by dpdk.org (Postfix) with ESMTP id 2F0622BE6 for ; Thu, 12 Mar 2020 07:46:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1583995568; bh=axn/X4bFuW7gsSVAXLzilgSMy9bsg8T2Qg3N2xX2Z80=; h=Date:From:To:In-Reply-To:References:Subject:From:Subject; b=cLtFPcGKT7541ZhQ5r54Q+AD+4omYTRAouppzdtxeXbMGomyClWWBs7HdoXNLRqOHWvB0xcZP6peBKWOWn9EmnakuduxC77SmcIVexAbk6oQ2FtiHaXvZar0xRY52DNkMb9sJVDCV2wbs/9Q08kkoWtB7FBoPB23HBsFEaO33oIbNEYxX9WytAFS6UYPu7d0nAvBDiCpNRt1aqG+TNt3VA8nXw86fppn5zELpMQtRyAdGzY4nODzRWI3H4KsYwMzKd2Be8zejLSXkY0ExLafC2c1HP66ZzyQE2TPhvUNnMXdhWk822YnWoc+Crtcnykv+u7AXYCmi/ifQaP1kj4cRw== X-YMail-OSG: aGgPM2MVM1mPbi7bQxEeCzZK1w2kn5ErON7USmFrlChc211XpUtxG68vaEhdHVS qn1RNdfWwPq5vz29NrG9aIKbOvr0s1ujXSl8LV2grP_wGEKiEXbeLameL_W6_jjDBSCavR0lF0gw yj1T_6FbMkGrkviT59AqCZweKFRuIcm_YrL8Y9M.6LhKmp3pQNVnUj2SD2DzEpogEhbAgSR6eCcB l4UeJnMV.V1pZyxp3lm1WMZiZdMB_Hat.VmjvPg9Bo0yDi3Rj.1PfLPC3j_3SCNl8y9UWXmOoYGW NNgTxwq61DKkZ.noBtxfsGbQ_wDr.AwCyVDgbwEPKjVBACvQwxCzfc9LT4stsx0TkSuiOYnuj_BB nFLbNa6KV3uFpeJE.tuHoqMH1ev.0kuBqjG7ACAXOkgC1KFjTGPZpY9S155qxSyXOyoPC9hiMeTA KZWOiyFd_jIQZCDovgi4ziPTCdB.vdWWNR6A2tXsZHdlNfxZkyMwLTH2HWi4i6jQb4kwwGWZ4w18 hCw7.2WX76ugcXMmnzrvAhN3YfvwDRMMVBYt3yVUlGHQpsiSgoRleRoWYJ.iwzYFlJG_jRKiXnwX NfSCJ_RxX9jxPqPtBoyYPq5vDwXkO5pQV4NB6cZc4GpP5EpLNGgn0bPHVdRTBYO_o6J0HMkLRvJI 7li0QMm1II9xtUWOO_zWhdH1Aq1o.nQ16fOlPeqhUKQgJpX5IPMXQHVk6cXb9JxPMCmsBcs9xTlC 0zCKg8IJanTGM9J6zLYNUNAoxQcdyk9KJXzAO0tdhAo0Shp2IiGYIknfNFoPtju_0EJlEff15bgn TDHxw_p7JY8OJ2mOmji.NVP98assphpFldphOn8A29cMMOPQ0LUx55hU9_f_c46KTAUN9ZW7fasA m.OQHw4wa44f9sZICvY_ZFI_WeyMW.b3N6qFuhjAwn.L9tKdUDNFaBZX2zXlTjRQffud.BdwBrqh ptuOHmOk6.K5GZp4Ym0hWJGuo1GOZq0p8F9JDkFaKtDIcmk2pCVGlQJ1VHoTEyLIP6MkYeohN2kj 8KfhPUX76JN0hpgnwjaIbtU.FuF9UuHg0cPsWfmhMeA9KKoRMXhnaWoqLE2Y0uijMw4STI02PQvC SmKPexd2ZL.iIC9AhiFFiiSSs.ni59fQZ0r.vPd_CrefURJZdZrNJS0gA2LGjYhh7HpBWHen13T7 n15Xz.ikPzBhDZ3It23EiqtEVMs71aXeRxiwc7nk.6009WDkdZFdv9xXmciJ2kexjcz3gNxhuUB. IsKky0B7IywUhWL72sIBL4oCL3BzajZK7.qTpTGvZt8Q18M7Y4Hiho6800qbZdDYqsPNrDqGf9ve 4D3mL1TozEBPyH6KwXg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Thu, 12 Mar 2020 06:46:08 +0000 Date: Thu, 12 Mar 2020 06:45:31 +0000 (UTC) From: sachin gupta To: "users@dpdk.org" , Yan Lei Message-ID: <1069532531.4293352.1583995531785@mail.yahoo.com> In-Reply-To: <7f693d4cbadf4516b204c4cbbc39894f@epfl.ch> References: <7f693d4cbadf4516b204c4cbbc39894f@epfl.ch> MIME-Version: 1.0 X-Mailer: WebService/1.1.15342 YMailNorrin Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] [ConnectX-5 MCX515A-CCAT / MCX516A-CCAT] Can only generate 53Gb/s with 64B packets X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi Lei, The smaller the Packet size, the more the number of Packets per second. I believe this is the inherent problem in all systems, even the ones which have proprietary hardware. In general applications which uses such small packets are rare and you will see a mix of traffic in the system. RegardsSachin On Thursday, March 12, 2020, 5:10:14 AM GMT+5:30, Yan Lei wrote: Hi, I am currently struggling in getting more than 53Gb/s with 64B packets on both of the MCX515A-CCAT and MCX516A-CCAT adapter when running a DPDK app that generates and transmits packets. WIth 256B packets I can get 98Gb/s. Has anyone saw the same performance on these NICs? I checked the perf. report on https://core.dpdk.org/perf-reports/ but there are no numbers of these NICs. Is this inherent limitation of these NICs (only reach 100Gb/s with larger packets)? If not, which firmware/driver/DPDK/system configurations could I tune to get 100Gb/s with 64B packets? My setup is as following: - CPU: E5-2697 v3 (14 cores, SMT disabled, CPU frequency fixed @ 2.6 GHz) - NIC: Mellanox MCX515A-CCAT / MCX516A-CCAT (Using only one port for TX, installed on PCIe Gen3 x16) - DPDK: 19.05 - RDMA-CORE: v28.0 - Kernel: 5.3.0 - OS: Ubuntu 18.04 - Firmware: 16.26.1040 I measured the TX rate with DPDK's testpmd: $ ./testpmd -l 3-13 -n 4 -w 02:00.0 -- -i --port-topology=chained --nb-ports=1 --rxq=10 --txq=10 --nb-cores=10 --burst=128 --rxd=512 --txd=512 --mbcache=512 --forward-mode=txonly So 10 cores generating and transmits 64B packets on 10 NIC queues. Your feedbacks will be much appreciated. Thanks, Lei