From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f48.google.com (mail-it0-f48.google.com [209.85.214.48]) by dpdk.org (Postfix) with ESMTP id E5CCE961C for ; Tue, 14 Jun 2016 20:12:05 +0200 (CEST) Received: by mail-it0-f48.google.com with SMTP id a5so84495746ita.1 for ; Tue, 14 Jun 2016 11:12:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=riotgames.com; s=riotgames; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=SogmO5bJ/8CGw9LhPV2U5WjEBa+dKO5vPUWadIkvVdw=; b=cRnkvw8Vz9FHyoyQ9iSKuDv7Flj/fT+q2o26KDaWo8ZHVNTW0KSsXRt7CdTsD9u15Q pWLgND42jqGudT02g1GXn0JMdCeBTrBJ/Yc1ravIIwSOKBbChDJd9fAVcDNDEVQ4U4X0 wlJc8LZe+0rDoqEjtH1R1G2ca7KvexG0ETHBs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=SogmO5bJ/8CGw9LhPV2U5WjEBa+dKO5vPUWadIkvVdw=; b=F0J5Neg6hwk0W/WATDzeuu+aeV21B1wd1ojwR6UN2plA34QI1vrJACh0dHEgv0OS+n MyQclEsACNGU4rTNu9xXekE8nrctugn55faJSAYljEd93i+f6Xv+srAtZsbduYod9k9C eN4BzNLdPnmD+kEJR5Frg8lPTIGLN0j9ZYkjl4i9GkSFGOYBYkOPDxxpB0JhHqR5DdzM TDbBMfYSmIRMVI77nVkk30HIjf9s5hE4K5n1+jUn4MJNWqYUVKx+jVDuXtUuw8r0bQla m9N4QF5wMim5kz8fMJitnU525yRmwAnNY8MLEpNZnpTqy5/QyfTdlKF7BAmlou2JIo3J 4b7g== X-Gm-Message-State: ALyK8tKz/R5dAf02HJTgnES1HwnsemFlHKWmjBoRcdKW7bTeVSr2KX+Qnfs4YHdwqpO+go+kkGdT8u8uAUGhYzke X-Received: by 10.36.209.5 with SMTP id w5mr30418631itg.47.1465927924732; Tue, 14 Jun 2016 11:12:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.52.205 with HTTP; Tue, 14 Jun 2016 11:11:45 -0700 (PDT) In-Reply-To: References: From: Javier Blazquez Date: Tue, 14 Jun 2016 11:11:45 -0700 Message-ID: To: Olga Shern Cc: "users@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] RSS not working on Mellanox ConnectX-3 NIC X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jun 2016 18:12:06 -0000 Thanks Olga, you're right, I looked at the per-queue counters more closely and RSS is indeed distributing packets to all queues. I can also confirm that with ConnectX-4 RX performance scales with multiple queues. On Tue, Jun 14, 2016 at 9:15 AM, Olga Shern wrote: > Hi Javier, > > RSS on ConnectX-3 cards is working, but doesn't improve the Maximum rate > of the NIC, it helps for real application to spread the traffic among > different cores. > Therefore with benchmark application you will see degradation with RSS, > but with real application the performance should be better with RSS than > without. > > ConnectX-4 doesn't have this limitation and we suggest using it instead o= f > ConnectX-3 > > Let me know if you have any more questions. > > Best Regards, > Olga > > -----Original Message----- > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Javier Blazquez > Sent: Thursday, June 09, 2016 7:10 AM > To: users@dpdk.org > Subject: [dpdk-users] RSS not working on Mellanox ConnectX-3 NIC > > Hi, > > I have a pair of Mellanox MCX354A-FCBT NICs and I'm having trouble scalin= g > up RX performance. It appears that RSS is not working and RX speed is > limited by a single queue. > > According to the documentation[1] RSS is supported on the mlx4 driver, an= d > debugging the eth dev initialization code I can see the driver setting up > RSS apparently with success. I can generate 34Mpps over the TX port using= 8 > queues, but I can only ever receive at 20Mpps on the RX end, no matter ho= w > many queues I use. > > The generated packets have randomized source/destination IP addresses and > source/destination UDP ports, so they should hash to different RX queues. > > The NICs are connected directly to each other with a DAC cable. They are > on different NUMA nodes and I'm placing TX/RX lcores on the appropriate > socket for each NIC. It doesn't matter which NIC I use as the sender, the > results are exactly the same. I have tried both pktgen and my own code bu= t > didn't see any difference. > > The server is a 2x 12-core Intel E5-2680 v3 2.5GHz. The Mellanox NICs are > flashed with the latest firmware and I'm using MLNX_OFED 3.3. I'm using t= he > MLNX_DPDK 2.2 distribution, but I also tried the standard DPDK v16.04 and > the result was the same (with lower TX and RX performance overall). > > Here's the output of ibstat: > > CA 'mlx4_0' > CA type: MT4099 > Number of ports: 2 > Firmware version: 2.36.5000 > Hardware version: 1 > Node GUID: 0x0002c90300310c30 > System image GUID: 0x0002c90300310c33 > Port 1: > State: Active > Physical state: LinkUp > Rate: 56 > Base lid: 0 > LMC: 0 > SM lid: 0 > Capability mask: 0x0c010000 > Port GUID: 0x0202c9fffe310c30 > Link layer: Ethernet > Port 2: > State: Active > Physical state: LinkUp > Rate: 56 > Base lid: 0 > LMC: 0 > SM lid: 0 > Capability mask: 0x0c010000 > Port GUID: 0x0202c9fffe310c31 > Link layer: Ethernet > CA 'mlx4_1' > CA type: MT4099 > Number of ports: 2 > Firmware version: 2.36.5000 > Hardware version: 1 > Node GUID: 0x0002c90300318200 > System image GUID: 0x0002c90300318203 > Port 1: > State: Active > Physical state: LinkUp > Rate: 56 > Base lid: 0 > LMC: 0 > SM lid: 0 > Capability mask: 0x0c010000 > Port GUID: 0x0202c9fffe318200 > Link layer: Ethernet > Port 2: > State: Active > Physical state: LinkUp > Rate: 56 > Base lid: 0 > LMC: 0 > SM lid: 0 > Capability mask: 0x0c010000 > Port GUID: 0x0202c9fffe318201 > Link layer: Ethernet > > Below are the pktgen results. Note that the first NIC is 0000:03:00.0 and > is assigned ports 0-1, and the second NIC is 0000:a1:00.0 and is assigned > ports 2-3. I'm testing TX on port 0 and RX on port 2, which are connected > directly. Random packets are generated by using the pktgen script found a= t > [2]. > > $ app/pktgen -c ffffff -n 4 -w 0000:03:00.0 -w 0000:a1:00.0 > --socket-mem=3D1024,1024 -- -N -T -P -m "[0-7].0,[12-19].2 > > Copyright (c) <2010-2016>, Intel Corporation. All rights reserved. Powere= d > by Intel=C2=AE DPDK > EAL: Detected lcore 0 as core 0 on socket 0 > EAL: Detected lcore 1 as core 1 on socket 0 > EAL: Detected lcore 2 as core 2 on socket 0 > EAL: Detected lcore 3 as core 3 on socket 0 > EAL: Detected lcore 4 as core 4 on socket 0 > EAL: Detected lcore 5 as core 5 on socket 0 > EAL: Detected lcore 6 as core 8 on socket 0 > EAL: Detected lcore 7 as core 9 on socket 0 > EAL: Detected lcore 8 as core 10 on socket 0 > EAL: Detected lcore 9 as core 11 on socket 0 > EAL: Detected lcore 10 as core 12 on socket 0 > EAL: Detected lcore 11 as core 13 on socket 0 > EAL: Detected lcore 12 as core 0 on socket 1 > EAL: Detected lcore 13 as core 1 on socket 1 > EAL: Detected lcore 14 as core 2 on socket 1 > EAL: Detected lcore 15 as core 3 on socket 1 > EAL: Detected lcore 16 as core 4 on socket 1 > EAL: Detected lcore 17 as core 5 on socket 1 > EAL: Detected lcore 18 as core 8 on socket 1 > EAL: Detected lcore 19 as core 9 on socket 1 > EAL: Detected lcore 20 as core 10 on socket 1 > EAL: Detected lcore 21 as core 11 on socket 1 > EAL: Detected lcore 22 as core 12 on socket 1 > EAL: Detected lcore 23 as core 13 on socket 1 > EAL: Detected lcore 24 as core 0 on socket 0 > EAL: Detected lcore 25 as core 1 on socket 0 > EAL: Detected lcore 26 as core 2 on socket 0 > EAL: Detected lcore 27 as core 3 on socket 0 > EAL: Detected lcore 28 as core 4 on socket 0 > EAL: Detected lcore 29 as core 5 on socket 0 > EAL: Detected lcore 30 as core 8 on socket 0 > EAL: Detected lcore 31 as core 9 on socket 0 > EAL: Detected lcore 32 as core 10 on socket 0 > EAL: Detected lcore 33 as core 11 on socket 0 > EAL: Detected lcore 34 as core 12 on socket 0 > EAL: Detected lcore 35 as core 13 on socket 0 > EAL: Detected lcore 36 as core 0 on socket 1 > EAL: Detected lcore 37 as core 1 on socket 1 > EAL: Detected lcore 38 as core 2 on socket 1 > EAL: Detected lcore 39 as core 3 on socket 1 > EAL: Detected lcore 40 as core 4 on socket 1 > EAL: Detected lcore 41 as core 5 on socket 1 > EAL: Detected lcore 42 as core 8 on socket 1 > EAL: Detected lcore 43 as core 9 on socket 1 > EAL: Detected lcore 44 as core 10 on socket 1 > EAL: Detected lcore 45 as core 11 on socket 1 > EAL: Detected lcore 46 as core 12 on socket 1 > EAL: Detected lcore 47 as core 13 on socket 1 > EAL: Support maximum 128 logical core(s) by configuration. > EAL: Detected 48 lcore(s) > EAL: Setting up physically contiguous memory... > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Virtual area found at 0x7f38c0000000 (size =3D 0x80000000) > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Virtual area found at 0x7f3800000000 (size =3D 0x80000000) > EAL: Requesting 1 pages of size 1024MB from socket 0 > EAL: Requesting 1 pages of size 1024MB from socket 1 > EAL: TSC frequency is ~2494222 KHz > EAL: Master lcore 0 is ready (tid=3Deca398c0;cpuset=3D[0]) > EAL: lcore 6 is ready (tid=3De7833700;cpuset=3D[6]) > EAL: lcore 7 is ready (tid=3De7032700;cpuset=3D[7]) > EAL: lcore 8 is ready (tid=3De6831700;cpuset=3D[8]) > EAL: lcore 4 is ready (tid=3De8835700;cpuset=3D[4]) > EAL: lcore 1 is ready (tid=3Dea038700;cpuset=3D[1]) > EAL: lcore 9 is ready (tid=3De6030700;cpuset=3D[9]) > EAL: lcore 3 is ready (tid=3De9036700;cpuset=3D[3]) > EAL: lcore 2 is ready (tid=3De9837700;cpuset=3D[2]) > EAL: lcore 13 is ready (tid=3De402c700;cpuset=3D[13]) > EAL: lcore 10 is ready (tid=3De582f700;cpuset=3D[10]) > EAL: lcore 12 is ready (tid=3De482d700;cpuset=3D[12]) > EAL: lcore 11 is ready (tid=3De502e700;cpuset=3D[11]) > EAL: lcore 5 is ready (tid=3De8034700;cpuset=3D[5]) > EAL: lcore 20 is ready (tid=3De0825700;cpuset=3D[20]) > EAL: lcore 19 is ready (tid=3De1026700;cpuset=3D[19]) > EAL: lcore 18 is ready (tid=3De1827700;cpuset=3D[18]) > EAL: lcore 21 is ready (tid=3Dbbfff700;cpuset=3D[21]) > EAL: lcore 22 is ready (tid=3Dbb7fe700;cpuset=3D[22]) > EAL: lcore 14 is ready (tid=3De382b700;cpuset=3D[14]) > EAL: lcore 17 is ready (tid=3De2028700;cpuset=3D[17]) > EAL: lcore 23 is ready (tid=3Dbaffd700;cpuset=3D[23]) > EAL: lcore 15 is ready (tid=3De302a700;cpuset=3D[15]) > EAL: lcore 16 is ready (tid=3De2829700;cpuset=3D[16]) > EAL: PCI device 0000:03:00.0 on NUMA socket 0 > EAL: probe driver: 15b3:1003 librte_pmd_mlx4 > PMD: librte_pmd_mlx4: PCI information matches, using device "mlx4_0" (VF: > false) > PMD: librte_pmd_mlx4: 2 port(s) detected > PMD: librte_pmd_mlx4: port 1 MAC address is 00:02:c9:31:0c:30 > PMD: librte_pmd_mlx4: port 2 MAC address is 00:02:c9:31:0c:31 > EAL: PCI device 0000:a1:00.0 on NUMA socket 1 > EAL: probe driver: 15b3:1003 librte_pmd_mlx4 > PMD: librte_pmd_mlx4: PCI information matches, using device "mlx4_1" (VF: > false) > PMD: librte_pmd_mlx4: 2 port(s) detected > PMD: librte_pmd_mlx4: port 1 MAC address is 00:02:c9:31:82:00 > PMD: librte_pmd_mlx4: port 2 MAC address is 00:02:c9:31:82:01 > [0-7].0 lcores: RX( 0 1 2 3 4 5 6 7 )TX( 0 1 2 3 4 5 6 7 ) ports= : > RX( 0 )TX( 0 ) > [12-19].2 lcores: RX( 12 13 14 15 16 17 18 19 )TX( 12 13 14 15 16 = 17 > 18 19 ) ports: RX( 2 )TX( 2 ) > > Copyright (c) <2010-2016>, Intel Corporation. All rights reserved. > Pktgen created by: Keith Wiles -- >>> Powered by Intel=C2=AE DPDK <<< > > Lua 5.3.2 Copyright (C) 1994-2015 Lua.org, PUC-Rio > >>> Packet Burst 32, RX Desc 512, TX Desc 512, mbufs/port 4096, mbuf > >>> cache > 512 > > =3D=3D=3D port to lcore mapping table (# lcores 24) =3D=3D=3D > lcore: 0 1 2 3 4 5 6 7 8 9 = 10 > 11 12 13 14 15 16 17 18 19 20 21 22 > 23 > port 0: D: T 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 0: 0 0: 0 0:= 0 > 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 = 0: > 0 =3D 8: 8 > port 2: D: T 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0: 0 0:= 0 > 0: 0 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 0: 0 0: 0 0: 0 = 0: > 0 =3D 8: 8 > Total : 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 0: 0 0: 0 0:= 0 > 0: 0 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 1: 1 0: 0 0: 0 0: 0 = 0: > 0 > Display and Timer on lcore 0, rx:tx counts per port/lcore > > Configuring 4 ports, MBUF Size 1920, MBUF Cache Size 512 > Lcore: > 0, RX-TX > RX( 1): ( 0: 0) > TX( 1): ( 0: 0) > 1, RX-TX > RX( 1): ( 0: 1) > TX( 1): ( 0: 1) > 2, RX-TX > RX( 1): ( 0: 2) > TX( 1): ( 0: 2) > 3, RX-TX > RX( 1): ( 0: 3) > TX( 1): ( 0: 3) > 4, RX-TX > RX( 1): ( 0: 4) > TX( 1): ( 0: 4) > 5, RX-TX > RX( 1): ( 0: 5) > TX( 1): ( 0: 5) > 6, RX-TX > RX( 1): ( 0: 6) > TX( 1): ( 0: 6) > 7, RX-TX > RX( 1): ( 0: 7) > TX( 1): ( 0: 7) > 12, RX-TX > RX( 1): ( 2: 0) > TX( 1): ( 2: 0) > 13, RX-TX > RX( 1): ( 2: 1) > TX( 1): ( 2: 1) > 14, RX-TX > RX( 1): ( 2: 2) > TX( 1): ( 2: 2) > 15, RX-TX > RX( 1): ( 2: 3) > TX( 1): ( 2: 3) > 16, RX-TX > RX( 1): ( 2: 4) > TX( 1): ( 2: 4) > 17, RX-TX > RX( 1): ( 2: 5) > TX( 1): ( 2: 5) > 18, RX-TX > RX( 1): ( 2: 6) > TX( 1): ( 2: 6) > 19, RX-TX > RX( 1): ( 2: 7) > TX( 1): ( 2: 7) > > Port : > 0, nb_lcores 8, private 0x8f09f0, lcores: 0 1 2 3 4 5 6 7 > 2, nb_lcores 8, private 0x8f5270, lcores: 12 13 14 15 16 17 18 19 > > > > ** Dev Info (librte_pmd_mlx4:17) ** > max_vfs : 0 min_rx_bufsize : 32 max_rx_pktlen : 65536 > max_rx_queues :65408 max_tx_queues:65408 > max_mac_addrs : 127 max_hash_mac_addrs: 0 max_vmdq_pools: 0 > rx_offload_capa: 0 tx_offload_capa : 0 reta_size : 0 > flow_type_rss_offloads:0000000000000000 > vmdq_queue_base: 0 vmdq_queue_num : 0 vmdq_pool_base: 0 > ** RX Conf ** > pthreash : 0 hthresh : 0 wthresh : 0 > Free Thresh : 0 Drop Enable : 0 Deferred Start : 0 > ** TX Conf ** > pthreash : 0 hthresh : 0 wthresh : 0 > Free Thresh : 0 RS Thresh : 0 Deferred Start : 0 TXQ > Flags:00000000 > > PMD: librte_pmd_mlx4: 0x94b7e0: TX queues number update: 0 -> 8 > PMD: librte_pmd_mlx4: 0x94b7e0: RX queues number update: 0 -> 8 Initializ= e > Port 0 -- TxQ 8, RxQ 8, Src MAC 00:02:c9:31:0c:30 > Create: Default RX 0:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 0:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > > Create: Default TX 0:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:0 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:1 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:2 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:3 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:4 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:5 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:6 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 0:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 0:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 0:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 0:7 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > > Port memory used =3D 324936 KB > > ** Dev Info (librte_pmd_mlx4:19) ** > max_vfs : 0 min_rx_bufsize : 32 max_rx_pktlen : 65536 > max_rx_queues :65408 max_tx_queues:65408 > max_mac_addrs : 127 max_hash_mac_addrs: 0 max_vmdq_pools: 0 > rx_offload_capa: 0 tx_offload_capa : 0 reta_size : 0 > flow_type_rss_offloads:0000000000000000 > vmdq_queue_base: 0 vmdq_queue_num : 0 vmdq_pool_base: 0 > ** RX Conf ** > pthreash : 0 hthresh : 0 wthresh : 0 > Free Thresh : 0 Drop Enable : 0 Deferred Start : 0 > ** TX Conf ** > pthreash : 0 hthresh : 0 wthresh : 0 > Free Thresh : 0 RS Thresh : 0 Deferred Start : 0 TXQ > Flags:00000000 > > PMD: librte_pmd_mlx4: 0x953870: TX queues number update: 0 -> 8 > PMD: librte_pmd_mlx4: 0x953870: RX queues number update: 0 -> 8 Initializ= e > Port 2 -- TxQ 8, RxQ 8, Src MAC 00:02:c9:31:82:00 > Create: Default RX 2:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Default RX 2:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > > Create: Default TX 2:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:0 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:0 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:1 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:1 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:2 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:2 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:3 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:3 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:4 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:4 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:5 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:5 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:6 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:6 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > Create: Default TX 2:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Range TX 2:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Sequence TX 2:7 - Memory used (MBUFs 4096 x (size 1920 + Hdr > 128)) + 1581248 =3D 9737 KB headroom 128 2176 > Create: Special TX 2:7 - Memory used (MBUFs 64 x (size 1920 + Hdr > 128)) + 1581248 =3D 1673 KB headroom 128 2176 > > > Port memory used =3D 324936 KB > > Total memory used =3D 649871 KB Port 0: Link Up - speed 56000 Mbps - > full-duplex Port 2: Link Up - speed 56000 Mbps= - > full-duplex > > > =3D=3D=3D Display processing on lcore 0 > WARNING: Nothing to do on lcore 8: exiting > WARNING: Nothing to do on lcore 9: exiting > WARNING: Nothing to do on lcore 10: exiting > WARNING: Nothing to do on lcore 11: exiting > WARNING: Nothing to do on lcore 20: exiting > WARNING: Nothing to do on lcore 21: exiting > WARNING: Nothing to do on lcore 22: exiting > WARNING: Nothing to do on lcore 23: exiting =3D=3D=3D RX/TX processing lc= ore 1 > rxcnt 1 txcnt 1 port/qid, 0/1 =3D=3D=3D RX/TX processing lcore 2 rxcnt 1= txcnt 1 > port/qid, 0/2 =3D=3D=3D RX/TX processing lcore 3 rxcnt 1 txcnt 1 port/qi= d, 0/3 > =3D=3D=3D RX/TX processing lcore 4 rxcnt 1 txcnt 1 port/qid, 0/4 =3D=3D= =3D RX/TX > processing lcore 5 rxcnt 1 txcnt 1 port/qid, 0/5 =3D=3D=3D RX/TX process= ing > lcore 6 rxcnt 1 txcnt 1 port/qid, 0/6 =3D=3D=3D RX/TX processing lcore = 7 rxcnt > 1 txcnt 1 port/qid, 0/7 =3D=3D=3D RX/TX processing lcore 12 rxcnt 1 txcnt= 1 > port/qid, 2/0 =3D=3D=3D RX/TX processing lcore 13 rxcnt 1 txcnt 1 port/qi= d, 2/1 > =3D=3D=3D RX/TX processing lcore 14 rxcnt 1 txcnt 1 port/qid, 2/2 =3D=3D= =3D RX/TX > processing lcore 15 rxcnt 1 txcnt 1 port/qid, 2/3 =3D=3D=3D RX/TX process= ing > lcore 16 rxcnt 1 txcnt 1 port/qid, 2/4 =3D=3D=3D RX/TX processing lcore 1= 7 rxcnt > 1 txcnt 1 port/qid, 2/5 =3D=3D=3D RX/TX processing lcore 18 rxcnt 1 txcnt= 1 > port/qid, 2/6 =3D=3D=3D RX/TX processing lcore 19 rxcnt 1 txcnt 1 port/qi= d, 2/7 > > > Pktgen > load random.txt > > geometry 132x44 > mac_from_arp disable > set 0 count 0 > set 0 size 64 > set 0 rate 100 > set 0 burst 32 > set 0 sport 1234 > set 0 dport 5678 > set 0 prime 1 > type ipv4 0 > range.proto 0 udp > proto udp 0 > set ip dst 0 10.1.72.17 > set ip src 0 10.1.72.154/24 > set mac 0 00:23:e9:64:c0:03 > vlanid 0 1 > pattern 0 abc > latency 0 disable > mpls 0 disable > mpls_entry 0 0 > qinq 0 disable > qinqids 0 0 0 > gre 0 disable > gre_eth 0 disable > gre_key 0 0 > icmp.echo 0 disable > pcap 0 disable > range 0 enable > process 0 disable > capture 0 disable > rxtap 0 disable > txtap 0 disable > vlan 0 disable > src.mac start 0 00:50:56:86:10:76 > src.mac min 0 00:00:00:00:00:00 > src.mac max 0 00:00:00:00:00:00 > src.mac inc 0 00:00:00:00:00:00 > dst.mac start 0 00:23:e9:64:c0:03 > dst.mac min 0 00:00:00:00:00:00 > dst.mac max 0 00:00:00:00:00:00 > dst.mac inc 0 00:00:00:00:00:00 > src.ip start 0 10.1.72.154 > src.ip min 0 10.1.72.154 > src.ip max 0 10.1.72.254 > src.ip inc 0 0.0.0.1 > dst.ip start 0 10.1.72.17 > dst.ip min 0 10.1.72.17 > dst.ip max 0 10.1.72.17 > dst.ip inc 0 0 > src.port start 0 1025 > src.port min 0 1025 > src.port max 0 65512 > src.port inc 0 1 > dst.port start 0 0 > dst.port min 0 0 > dst.port max 0 254 > dst.port inc 0 1 > vlan.id start 0 1 > vlan.id min 0 1 > vlan.id max 0 4095 > vlan.id inc 0 0 > pkt.size start 0 64 > pkt.size min 0 64 > pkt.size max 0 1518 > pkt.size inc 0 0 > set 0 seqCnt 0 > > Pktgen > start 0 > > Flags:Port : P-----R--------:0 > P--------------:2 > Link State : > ---- > Pkts/s Max/Rx : 0/0 > 19839945/19839945 > Max/Tx : 34199936/34135552 > 34199936/34135552 > MBits/s Rx/Tx : 0/21846 > 12697/21846 > Broadcast : 0 > 0 > Multicast : 0 > 0 > 64 Bytes : 0 > 78156990 > 65-127 : 0 > 0 > 128-255 : 0 > 0 > 256-511 : 0 > 0 > 512-1023 : 0 > 0 > 1024-1518 : 0 > 0 > Runts/Jumbos : 0/0 > 0/0 > Errors Rx/Tx : 0/0 > 0/0 > Total Rx Pkts : 0 > 368764259 > Tx Pkts : 669245499 > 0 > Rx MBs : 0 > 236009 > Tx MBs : 428317 > 0 > ARP/ICMP Pkts : 0/0 > 0/0 > : > Pattern Type : abcd... > abcd... > Tx Count/% Rate : Forever / 100% Forever / > 100% > PktSize/Tx Burst: 64 / 32 64 / > 32 > Src/Dest Port : 1234 / 5678 1234 / > 5678 > Pkt Type:VLAN ID: IPv4 / UDP:0001 IPv4 / > TCP:0001 > Dst IP Address : 10.1.72.17 > 192.168.3.1 > Src IP Address : 10.1.72.154/24 > 192.168.2.1/24 > Dst MAC Address : 00:23:e9:64:c0:03 > 00:00:00:00:00:00 > Src MAC Address 00:02:c9:31:0c:30 > 00:02:c9:31:82:00 > > [1] http://dpdk.org/doc/guides/nics/mlx4.html > [2] http://dpdk.org/ml/archives/users/2016-April/000489.html >