From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 45A2D4569E for ; Wed, 24 Jul 2024 03:19:40 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 134D640E12; Wed, 24 Jul 2024 03:19:40 +0200 (CEST) Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by mails.dpdk.org (Postfix) with ESMTP id 515C7402C0 for ; Wed, 24 Jul 2024 03:19:38 +0200 (CEST) Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-79f16cad2a7so372341785a.0 for ; Tue, 23 Jul 2024 18:19:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721783977; x=1722388777; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=X9oEnToTZz10VWp28TtE57vI7jP0OBhwMd9EI4u/kqs=; b=RWssEk4KZaJyMYHDii873m8zgJa3FU4TELpP0qo645cYP30CNQ+dvUgJqm7Cl9Yv73 Ef7+59acnBRtHdkdiMumuiI4mPlx6PjwPqWFB8AsZQfrc9eZ8sb965SGpQAQMRwe3Ini nF3ySDQYOtPtrUbxI/wErT0nZ6GujjfYz4cWfcabg5fmb6z571fx2RGMTjBw5VlNpy4o 1NvoWu8RsFba1hnreHDWJeUJcMEvdzcvIZ8mjUWTBx5W92ERvbSiQ6et7Z/1iKLDhAXu JUSw8GasVBCuTH9CCcuGCtGIkgUkCsMfJfNKWqQAH6Jimifhj5Ny62i+alPtWDfJCYY6 5W+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721783977; x=1722388777; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X9oEnToTZz10VWp28TtE57vI7jP0OBhwMd9EI4u/kqs=; b=uenCUIRuryRTorXi/5lexVsg93zDZbHx2VBLlX4TWd7BY+eAUGSczgpfu3gukk4BuM GR9cGwswSYrTA7SSnIHTLwQnyUntIk7mRk5hpRCUZHubQ0MOMX2BPNax3AUjz1oCAXZ/ m7NBDUfO8gE9+huRMMw//3Mg7vFzKSnJFXlG76S2YtpECdruWcUnfu5TRFPSzE0+6AVR G/OJ0j2Aa82qaGnGEYxT+v/283/4c3CVErhgjaA39j3bH8gka1AZSxfxXZKU16vMN/DM w6BONesWRITEgyykuOa110bcUNTWy9Ca2gheKH8K8S1hilMISRadSoZEptW9QbDOUNKM 7IWQ== X-Gm-Message-State: AOJu0YwKbBV6/nP7JCJVIkmHeoS7tygvxKLRut+1jn8yGP9DIQIhlDoW eeSBJ1OOesOvb7UuVQLcx2DJbnuiWqOp7bDc/fCy872hLi2B6XOqe+YhuRFFuM/wCOHOT6b4OOV vVceHWgOlTxuzcbgBgyfzTbh2nDPgmxd1 X-Google-Smtp-Source: AGHT+IHEB1EFLnDyv/WOnmu6X6G4mmqC10LZsO7IeKotMloMk+ayEoJh9T4ik89+yY6OmNdrg1sWsI6DahLRsj56I1M= X-Received: by 2002:a05:620a:372a:b0:79f:1873:5463 with SMTP id af79cd13be357-7a1cbcd2b2amr102367185a.6.1721783977469; Tue, 23 Jul 2024 18:19:37 -0700 (PDT) MIME-Version: 1.0 References: <5485686.Sb9uPGUboI@thomas> In-Reply-To: <5485686.Sb9uPGUboI@thomas> From: Yasuhiro Ohara Date: Wed, 24 Jul 2024 10:19:26 +0900 Message-ID: Subject: Re: ConnectX-7 400GbE wire-rate? To: Thomas Monjalon Cc: users@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi Thomas, Thank you for getting back to this. >From what we have seen, it seemed to be rather a Mellanox firmware issue than the pktgen or DPDK issue. When we were using ConnectX-7 with fw ver 28.41.1000 on a Core i9 machine, it limited its tx-bandwidth to 100Gbps, though the physical max should be 400Gbps. If we lower the fw version to 28.39.2048 or 28.39.3004, it could send 256Gbps, on the same core i9 machine. Our bandwidth demonstration event completed successfully, so we are fine with the small knowledge. I think my colleague can help if someone wants to debug further about this issue. Thank you anyway. Regards, Yasu 2024=E5=B9=B47=E6=9C=8823=E6=97=A5(=E7=81=AB) 2:37 Thomas Monjalon : > > Hello, > > I see there is no answer. > > Did you try with testpmd? > I don't know whether it can be a limitation of dpdk-pktgen? > > > > 29/05/2024 03:32, Yasuhiro Ohara: > > Hi. My colleague is trying to generate 400Gbps traffic using ConnectX-7= , > > but with no luck. > > > > Has anyone succeeded in generating 400Gbps (by larger than 1500B is ok)= , > > or are there any known issues? > > > > Using dpdk-pktgen, the link is successfully recognized as 400GbE > > but when we start the traffic, it seems to be capped at 100Gbps. > > > > Some info follows. > > > > [ 1.490256] mlx5_core 0000:01:00.0: firmware version: 28.41.1000 > > [ 1.492853] mlx5_core 0000:01:00.0: 504.112 Gb/s available PCIe > > bandwidth (32.0 GT/s PCIe x16 link) > > [ 1.805419] mlx5_core 0000:01:00.0: Rate limit: 127 rates are > > supported, range: 0Mbps to 195312Mbps > > [ 1.808477] mlx5_core 0000:01:00.0: E-Switch: Total vports 18, per > > vport: max uc(128) max mc(2048) > > [ 1.827270] mlx5_core 0000:01:00.0: Port module event: module 0, > > Cable unplugged > > [ 1.830317] mlx5_core 0000:01:00.0: mlx5_pcie_event:298:(pid 9): > > PCIe slot advertised sufficient power (75W). > > [ 1.830610] mlx5_core 0000:01:00.0: MLX5E: StrdRq(1) RqSz(8) > > StrdSz(2048) RxCqeCmprss(0) > > [ 1.929813] mlx5_core 0000:01:00.0: Supported tc offload range - > > chains: 4294967294, prios: 4294967295 > > > > Device type: ConnectX7 > > Name: MCX75310AAS-NEA_Ax > > Description: NVIDIA ConnectX-7 HHHL Adapter card; 400GbE / NDR IB > > (default mode); Single-port OSFP; PCIe 5.0 x16; Crypto Disabled; > > Secure Boot Enabled; > > Device: /dev/mst/mt4129_pciconf0 > > > > Enabled Link Speed (Ext.) : 0x00010000 (400G_4X) > > Supported Cable Speed (Ext.) : 0x00010800 (400G_4X,100G_1X) > > > > pktgen result: > > > > Pkts/s Rx : 0 > > TX : 8059264 > > > > MBits/s Rx/Tx : 0/97742 > > > > > > Any info is appreciated. Thanks. > > > > regards, > > Yasu > > > > > > >