[dpdk-dev] DPDK-pktgen not giving expected throughput

[dpdk-dev] DPDK-pktgen not giving expected throughput

[Shailja Pandey]

Hi,

I tried this command line and blacklisted 04:00.0 and 04:00.1-

/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 //-b 04:00.0 -b 04:00.1 //-- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”/

But I am still not able to see the ports using comma separated cores. I 
have also tried the following configuration but not getting 100 Gbps for 
packet size of 64 bytes and 1500 bytes:

  make; ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffffffff -n 4 
-b 04:00.0 -b 04:00.1 -- -T -P -m "[2-6:7-11].0,[12-16:17-21].1"

Any help would be highly appreciated.

>/On Jan 13, 2018, at 12:51 PM, Shailja Pandey <csz168117 at iitd.ac.in 
<https://dpdk.org/ml/listinfo/dev>> wrote: />//>/Hi, />//>/Output of command ‘lspci | grep Ether’ is- />//>/04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 
10-Gigabit X540-AT2 (rev 01) />/04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 
10-Gigabit X540-AT2 (rev 01) />/05:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 
for 40GbE QSFP+ (rev 02) />/05:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 
for 40GbE QSFP+ (rev 02) />//>/I tried running pktgen with this command />/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m 
“[2,4,6:8,10,12].0, [14,16,18:20,22,24].1” /
Need to blacklist 04:00.0 and 04:00.1 using -b 04:00.0 -b 04:00.1 just before the  '--'

>//>/but I am unable to see the ports while using comma separated cores like 
2,4,6. However when I tried with multiple CPUs, the result is still the 
same. />//>/After allocating multiple cores, I also tried the range commands to 
distribute the packets on different cores(RSS). />//>//>/ > />/On Jan 11, 2018, at 12:45 AM, Shailja Pandey <csz168117 at iitd.ac.in />/ > wrote: />//>/ > />//>//>/ > />/I have also tried with multiple cpus using the command line- />//>/ > />//>//>/ > />/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m 
"[0:4].0,[5:8].1,[9-12].2,[13-16].3” />//>//>/What is port 0 and 1, I guess that port 2 and 3 are the 2x40 card. I 
guess I need the ‘lspci | grep Ether’ output too. />//>/Lets assume port 2 and 3 are the 2 ports of 40G and the NIC is on PCI 
bus0 attached to NUMA node 0. />//>/May need to blacklist the first two port 0 and 1 to remove them from 
being used. />//>/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m 
“[2,4,6:8,10,12].0, [14,16,18:20,22,24].1” />//>/Now you have 6 cores per port 3 on TX and 3 on RX on each port. You 
maybe be able to get away with 4 per port with the speed of the machine 
you have. />//>/Next we have to use the range command to make sure the packets are 
transmitted with a varying set of 5 tuples. />//>/Do a help command an look at the range commands, there is an example in 
the test directory. />//>/To see the range information use ‘page range’ and then you can modify 
each port. />//>/After you are happy with the configuration use the ‘save <filename>’ 
command to save the configuration. Then you can use ‘load <filename>’ or 
add it to the command line with the -f <filename> option. />//>//>//>//>/ > />//>//>/ > />/On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote: />//>/ >> />/The command line for Pktgen is- />//>/ >> />//>//>/ >> />/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m 
"[2].0,[4].1,[8].2,[10].3" />//>/ >> />//>//>/ >> />/Configuration of the machine is- />//>/ >> />//>//>/ >> />/Architecture: x86_64 />//>/ >> />/CPU op-mode(s): 32-bit, 64-bit />//>/ >> />/Byte Order: Little Endian />//>/ >> />/CPU(s): 32 />//>/ >> />/On-line CPU(s) list: 0-31 />//>/ >> />/Thread(s) per core: 2 />//>/ >> />/Core(s) per socket: 8 />//>/ >> />/Socket(s): 2 />//>/ >> />/NUMA node(s): 2 />//>/ >> />/Vendor ID: GenuineIntel />//>/ >> />/CPU family: 6 />//>/ >> />/Model: 63 />//>/ >> />/Model name: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz />//>/ >> />/Stepping: 2 />//>/ >> />/CPU MHz: 1204.226 />//>/ >> />/CPU max MHz: 3400.0000 />//>/ >> />/CPU min MHz: 1200.0000 />//>/ >> />/BogoMIPS: 5195.08 />//>/ >> />/Virtualization: VT-x />//>/ >> />/L1d cache: 32K />//>/ >> />/L1i cache: 32K />//>/ >> />/L2 cache: 256K />//>/ >> />/L3 cache: 20480K />//>/ >> />/NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 />//>/ >> />/NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 />//>/ >> />//>//>/ >> />/Linux XeonE5 4.4.0-93-generic />//>/ >> />//>//>/ >> />/Thanks! />//>/ >> />//>//>/ >> />//>//>/ >> />//>//>/ >> />//>//>/ >> />//>//>/ >> />/ > />//>/ >> />/On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in />//>/ >> />/ > wrote: />//>/ >> />//>//>/ >> />/ > />//>/ >> />//>//>/ >> />//>//>/ >> />/ > />//>/ >> />/Hi, />//>/ >> />//>//>/ >> />/ > />//>/ >> />//>//>/ >> />//>//>/ >> />/ > />//>/ >> />/We are performing experiments on Dell Poweredge R430 server, which is 
based on Haswell architecture based xeon-2640 v3 processor. We have 
attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet 
generation per port. But pktgen is able to generate only 21 Mpps per 
port and we are not sure about the problem. />//>/ >> />//>//>/ >> />/ > />//>/ >> />//>//>/ >> />//>//>/ >> />/ > />//>/ >> />/We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there 
any way to generate packets from pktgen at 118 Mpps ? />//>/ >> />//>//>/ >> />//>//>/ >> />/What is the command line for Pktgen? />//>/ >> />//>//>/ >> />/Using a single CPU you should be able to generate about 21Mpps, which 
tells me the configuration is not correct. />//>/ >> />//>//>/ >> />/Pktgen needs N number of TX and RX cores to generate more traffic, plus 
the packets need to be spread across the cores using RSS. This means 
pktgen needs to be setup to generate packets with the 5 tuple difference 
to get all of the cores to receive the packets. />//>/ >> />//>//>/ >> />/I need to know the command line and the configuration of the machine to 
help. />//>/ >> />/ > />//>/ >> />//>//>/ >> />//>//>/ >> />/ > />//>/ >> />/-- />//>/ >> />//>//>/ >> />/ > />//>/ >> />//>//>/ >> />//>//>/ >> />/ > />//>/ >> />/Thanks, />//>/ >> />//>//>/ >> />/ > />//>/ >> />/Shailja />//>/ >> />//>//>/ >> />/ > />//>/ >> />//>//>/ >> />//>//>/ >> />//>//>/ >> />/Regards, />//>/ >> />/Keith />//>/ >> />//>//>/ >> />/-- />//>/ >> />//>//>/ >> />/Thanks, />//>/ >> />/Shailja />//>/ >> />//>//>/ > />//>//>/ > />/-- />//>/ > />//>//>/ > />/Thanks, />//>/ > />/Shailja />//>/ > />//>//>//>/Regards, />/Keith />//>/-- />//>/Thanks, />/Shailja />//
Regards,
Keith

-- 

Thanks,
Shailja

-- 

Thanks,
Shailja

Re: [dpdk-dev] DPDK-pktgen not giving expected throughput

[Wiles, Keith]

> On Jan 25, 2018, at 1:56 AM, Shailja Pandey <csz168117@iitd.ac.in> wrote:
> 
> Hi,
> 
> I tried this command line and blacklisted 04:00.0 and 04:00.1-
> 
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -b 04:00.0 -b 04:00.1 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
> But I am still not able to see the ports using comma separated cores. I have also tried the following configuration but not getting 100 Gbps for packet size of 64 bytes and 1500 bytes:
> 
>  make; ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffffffff -n 4 -b 04:00.0 -b 04:00.1 -- -T -P -m "[2-6:7-11].0,[12-16:17-21].1”


I guess I am not completely following the problem are you stating the comma list of cores does not work or just not able to get the performance you want?


Many factors effect the performance like the PCI slot width, location of PCI slot relative to the CPU socket, cores being used are not on the same CPU socket as the PCI slot/card, not setting up the range command to allow RSS to distribute the packets across the RX ports.

We need to isolate and verify these possible problems could be the issue.

> 
> Any help would be highly appreciated.
> > On Jan 13, 2018, at 12:51 PM, Shailja Pandey <csz168117 at iitd.ac.in
> > wrote:
> 
> >
>  
> 
> >
>  Hi,
> 
> >
>  
> 
> >
>  Output of command ‘lspci | grep Ether’ is-
> 
> >
>  
> 
> >
>  04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
> 
> >
>  04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
> 
> >
>  05:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
> 
> >
>  05:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
> 
> >
>  
> 
> >
>  I tried running pktgen with this command 
> 
> >
>  ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
> 
> 
> Need to blacklist 04:00.0 and 04:00.1 using -b 04:00.0 -b 04:00.1 just before the  '--'
> 
> >
>  
> 
> >
>  but I am unable to see the ports while using comma separated cores like 2,4,6. However when I tried with multiple CPUs, the result is still the same.
> 
> >
>  
> 
> >
>  After allocating multiple cores, I also tried the range commands to distribute the packets on different cores(RSS).
> 
> >
>  
> 
> >
>   
> 
> >
>  >
> 
> >
>   On Jan 11, 2018, at 12:45 AM, Shailja Pandey <csz168117 at iitd.ac.in
> 
> >
>  > wrote:
> 
> >
>  
> 
> >
>  >
> 
> >
>   
> 
> >
>  
> 
> >
>  >
> 
> >
>   I have also tried with multiple cpus using the command line-
> 
> >
>  
> 
> >
>  >
> 
> >
>   
> 
> >
>  
> 
> >
>  >
> 
> >
>   ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m "[0:4].0,[5:8].1,[9-12].2,[13-16].3” 
> 
> >
>  
> 
> >
>  
> 
> >
>  What is port 0 and 1, I guess that port 2 and 3 are the 2x40 card. I guess I need the ‘lspci | grep Ether’ output too.
> 
> >
>  
> 
> >
>  Lets assume port 2 and 3 are the 2 ports of 40G and the NIC is on PCI bus0 attached to NUMA node 0.
> 
> >
>  
> 
> >
>  May need to blacklist the first two port 0 and 1 to remove them from being used.
> 
> >
>  
> 
> >
>  ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
> 
> >
>  
> 
> >
>  Now you have 6 cores per port 3 on TX and 3 on RX on each port. You maybe be able to get away with 4 per port with the speed of the machine you have.
> 
> >
>  
> 
> >
>  Next we have to use the range command to make sure the packets are transmitted with a varying set of 5 tuples.
> 
> >
>  
> 
> >
>  Do a help command an look at the range commands, there is an example in the test directory.
> 
> >
>  
> 
> >
>  To see the range information use ‘page range’ and then you can modify each port.
> 
> >
>  
> 
> >
>  After you are happy with the configuration use the ‘save <filename>’ command to save the configuration. Then you can use ‘load <filename>’ or add it to the command line with the -f <filename> option.
> 
> >
>  
> 
> >
>  
> 
> >
>  
> 
> >
>  
> 
> >
>  >
> 
> >
>   
> 
> >
>  
> 
> >
>  >
> 
> >
>   On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote:
> 
> >
>  
> 
> >
>  >>
> 
> >
>   The command line for Pktgen is-
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m "[2].0,[4].1,[8].2,[10].3"
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Configuration of the machine is-
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Architecture:          x86_64
> 
> >
>  
> 
> >
>  >>
> 
> >
>   CPU op-mode(s):        32-bit, 64-bit
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Byte Order:            Little Endian
> 
> >
>  
> 
> >
>  >>
> 
> >
>   CPU(s):                32
> 
> >
>  
> 
> >
>  >>
> 
> >
>   On-line CPU(s) list:   0-31
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Thread(s) per core:    2
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Core(s) per socket:    8
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Socket(s):             2
> 
> >
>  
> 
> >
>  >>
> 
> >
>   NUMA node(s):          2
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Vendor ID:             GenuineIntel
> 
> >
>  
> 
> >
>  >>
> 
> >
>   CPU family:            6
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Model:                 63
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Model name:            Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Stepping:              2
> 
> >
>  
> 
> >
>  >>
> 
> >
>   CPU MHz:               1204.226
> 
> >
>  
> 
> >
>  >>
> 
> >
>   CPU max MHz:           3400.0000
> 
> >
>  
> 
> >
>  >>
> 
> >
>   CPU min MHz:           1200.0000
> 
> >
>  
> 
> >
>  >>
> 
> >
>   BogoMIPS:              5195.08
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Virtualization:        VT-x
> 
> >
>  
> 
> >
>  >>
> 
> >
>   L1d cache:             32K
> 
> >
>  
> 
> >
>  >>
> 
> >
>   L1i cache:             32K
> 
> >
>  
> 
> >
>  >>
> 
> >
>   L2 cache:              256K
> 
> >
>  
> 
> >
>  >>
> 
> >
>   L3 cache:              20480K
> 
> >
>  
> 
> >
>  >>
> 
> >
>   NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
> 
> >
>  
> 
> >
>  >>
> 
> >
>   NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Linux XeonE5 4.4.0-93-generic
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Thanks!
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in
> 
> >
>  
> 
> >
>  >>
> 
> >
>   > wrote:
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    Hi,
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    We are performing experiments on Dell Poweredge R430 server, which is based on Haswell architecture based xeon-2640 v3 processor. We have attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet generation per port. But pktgen is able to generate only 21 Mpps per port and we are not sure about the problem.
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there any way to generate packets from pktgen at 118 Mpps ?
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   What is the command line for Pktgen?
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   I need to know the command line and the configuration of the machine to help.
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    -- 
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    Thanks,
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    Shailja
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   >
> 
> >
>  
> 
> >
>  >>
> 
> >
>    
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Regards,
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Keith
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   -- 
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Thanks,
> 
> >
>  
> 
> >
>  >>
> 
> >
>   Shailja
> 
> >
>  
> 
> >
>  >>
> 
> >
>   
> 
> >
>  
> 
> >
>  >
> 
> >
>   
> 
> >
>  
> 
> >
>  >
> 
> >
>   -- 
> 
> >
>  
> 
> >
>  >
> 
> >
>   
> 
> >
>  
> 
> >
>  >
> 
> >
>   Thanks,
> 
> >
>  
> 
> >
>  >
> 
> >
>   Shailja
> 
> >
>  
> 
> >
>  >
> 
> >
>   
> 
> >
>  
> 
> >
>  
> 
> >
>  Regards,
> 
> >
>  Keith
> 
> >
>  
> 
> >
>  -- 
> 
> >
>  
> 
> >
>  Thanks,
> 
> >
>  Shailja
> 
> >
>  
> 
> 
> Regards,
> Keith
> 
> -- 
> 
> Thanks,
> Shailja
> 
> -- 
> 
> Thanks,
> Shailja
> 

Regards,
Keith

Re: [dpdk-dev] DPDK-pktgen not giving expected throughput

[Wiles, Keith]

> On Jan 13, 2018, at 12:51 PM, Shailja Pandey <csz168117@iitd.ac.in> wrote:
> 
> Hi,
> 
> Output of command ‘lspci | grep Ether’ is-
> 
> 04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
> 04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
> 05:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
> 05:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
> 
> I tried running pktgen with this command 
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”

Need to blacklist 04:00.0 and 04:00.1 using -b 04:00.0 -b 04:00.1 just before the  '--'

> 
> but I am unable to see the ports while using comma separated cores like 2,4,6. However when I tried with multiple CPUs, the result is still the same.
> 
> After allocating multiple cores, I also tried the range commands to distribute the packets on different cores(RSS).
> 
>  
> >
>  On Jan 11, 2018, at 12:45 AM, Shailja Pandey <csz168117 at iitd.ac.in
> > wrote:
> 
> >
>  
> 
> >
>  I have also tried with multiple cpus using the command line-
> 
> >
>  
> 
> >
>  ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m "[0:4].0,[5:8].1,[9-12].2,[13-16].3” 
> 
> 
> What is port 0 and 1, I guess that port 2 and 3 are the 2x40 card. I guess I need the ‘lspci | grep Ether’ output too.
> 
> Lets assume port 2 and 3 are the 2 ports of 40G and the NIC is on PCI bus0 attached to NUMA node 0.
> 
> May need to blacklist the first two port 0 and 1 to remove them from being used.
> 
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
> 
> Now you have 6 cores per port 3 on TX and 3 on RX on each port. You maybe be able to get away with 4 per port with the speed of the machine you have.
> 
> Next we have to use the range command to make sure the packets are transmitted with a varying set of 5 tuples.
> 
> Do a help command an look at the range commands, there is an example in the test directory.
> 
> To see the range information use ‘page range’ and then you can modify each port.
> 
> After you are happy with the configuration use the ‘save <filename>’ command to save the configuration. Then you can use ‘load <filename>’ or add it to the command line with the -f <filename> option.
> 
> 
> 
> 
> >
>  
> 
> >
>  On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote:
> 
> >>
>  The command line for Pktgen is-
> 
> >>
>  
> 
> >>
>  ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m "[2].0,[4].1,[8].2,[10].3"
> 
> >>
>  
> 
> >>
>  Configuration of the machine is-
> 
> >>
>  
> 
> >>
>  Architecture:          x86_64
> 
> >>
>  CPU op-mode(s):        32-bit, 64-bit
> 
> >>
>  Byte Order:            Little Endian
> 
> >>
>  CPU(s):                32
> 
> >>
>  On-line CPU(s) list:   0-31
> 
> >>
>  Thread(s) per core:    2
> 
> >>
>  Core(s) per socket:    8
> 
> >>
>  Socket(s):             2
> 
> >>
>  NUMA node(s):          2
> 
> >>
>  Vendor ID:             GenuineIntel
> 
> >>
>  CPU family:            6
> 
> >>
>  Model:                 63
> 
> >>
>  Model name:            Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
> 
> >>
>  Stepping:              2
> 
> >>
>  CPU MHz:               1204.226
> 
> >>
>  CPU max MHz:           3400.0000
> 
> >>
>  CPU min MHz:           1200.0000
> 
> >>
>  BogoMIPS:              5195.08
> 
> >>
>  Virtualization:        VT-x
> 
> >>
>  L1d cache:             32K
> 
> >>
>  L1i cache:             32K
> 
> >>
>  L2 cache:              256K
> 
> >>
>  L3 cache:              20480K
> 
> >>
>  NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
> 
> >>
>  NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31
> 
> >>
>  
> 
> >>
>  Linux XeonE5 4.4.0-93-generic
> 
> >>
>  
> 
> >>
>  Thanks!
> 
> >>
>  
> 
> >>
>  
> 
> >>
>  
> 
> >>
>  
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in
> 
> >>
>  > wrote:
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   Hi,
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   We are performing experiments on Dell Poweredge R430 server, which is based on Haswell architecture based xeon-2640 v3 processor. We have attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet generation per port. But pktgen is able to generate only 21 Mpps per port and we are not sure about the problem.
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there any way to generate packets from pktgen at 118 Mpps ?
> 
> >>
>  
> 
> >>
>  
> 
> >>
>  What is the command line for Pktgen?
> 
> >>
>  
> 
> >>
>  Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.
> 
> >>
>  
> 
> >>
>  Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.
> 
> >>
>  
> 
> >>
>  I need to know the command line and the configuration of the machine to help.
> 
> >>
>  >
> 
> >>
>   
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   -- 
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   Thanks,
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   Shailja
> 
> >>
>  
> 
> >>
>  >
> 
> >>
>   
> 
> >>
>  
> 
> >>
>  
> 
> >>
>  Regards,
> 
> >>
>  Keith
> 
> >>
>  
> 
> >>
>  -- 
> 
> >>
>  
> 
> >>
>  Thanks,
> 
> >>
>  Shailja
> 
> >>
>  
> 
> >
>  
> 
> >
>  -- 
> 
> >
>  
> 
> >
>  Thanks,
> 
> >
>  Shailja
> 
> >
>  
> 
> 
> Regards,
> Keith
> 
> -- 
> 
> Thanks,
> Shailja
> 

Regards,
Keith

Re: [dpdk-dev] DPDK-pktgen not giving expected throughput

[Shailja Pandey]

Hi,

Output of command ‘lspci | grep Ether’ is-

04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
05:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
05:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)

I tried running pktgen with this command
./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”

but I am unable to see the ports while using comma separated cores like 2,4,6. However when I tried with multiple CPUs, the result is still the same.

After allocating multiple cores, I also tried the range commands to distribute the packets on different cores(RSS).

  
>/On Jan 11, 2018, at 12:45 AM, Shailja Pandey <csz168117 at iitd.ac.in 
<https://dpdk.org/ml/listinfo/dev>> wrote: />//>/I have also tried with multiple cpus using the command line- />//>/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m 
"[0:4].0,[5:8].1,[9-12].2,[13-16].3” /
What is port 0 and 1, I guess that port 2 and 3 are the 2x40 card. I guess I need the ‘lspci | grep Ether’ output too.

Lets assume port 2 and 3 are the 2 ports of 40G and the NIC is on PCI bus0 attached to NUMA node 0.

May need to blacklist the first two port 0 and 1 to remove them from being used.

./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”

Now you have 6 cores per port 3 on TX and 3 on RX on each port. You maybe be able to get away with 4 per port with the speed of the machine you have.

Next we have to use the range command to make sure the packets are transmitted with a varying set of 5 tuples.

Do a help command an look at the range commands, there is an example in the test directory.

To see the range information use ‘page range’ and then you can modify each port.

After you are happy with the configuration use the ‘save <filename>’ command to save the configuration. Then you can use ‘load <filename>’ or add it to the command line with the -f <filename> option.




>//>/On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote: />>/The command line for Pktgen is- />>//>>/./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m 
"[2].0,[4].1,[8].2,[10].3" />>//>>/Configuration of the machine is- />>//>>/Architecture: x86_64 />>/CPU op-mode(s): 32-bit, 64-bit />>/Byte Order: Little Endian />>/CPU(s): 32 />>/On-line CPU(s) list: 0-31 />>/Thread(s) per core: 2 />>/Core(s) per socket: 8 />>/Socket(s): 2 />>/NUMA node(s): 2 />>/Vendor ID: GenuineIntel />>/CPU family: 6 />>/Model: 63 />>/Model name: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz />>/Stepping: 2 />>/CPU MHz: 1204.226 />>/CPU max MHz: 3400.0000 />>/CPU min MHz: 1200.0000 />>/BogoMIPS: 5195.08 />>/Virtualization: VT-x />>/L1d cache: 32K />>/L1i cache: 32K />>/L2 cache: 256K />>/L3 cache: 20480K />>/NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 />>/NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 />>//>>/Linux XeonE5 4.4.0-93-generic />>//>>/Thanks! />>//>>//>>//>>//>>//>>/ > />>/On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in />>/ > wrote: />>//>>/ > />>//>>//>>/ > />>/Hi, />>//>>/ > />>//>>//>>/ > />>/We are performing experiments on Dell Poweredge R430 server, which is 
based on Haswell architecture based xeon-2640 v3 processor. We have 
attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet 
generation per port. But pktgen is able to generate only 21 Mpps per 
port and we are not sure about the problem. />>//>>/ > />>//>>//>>/ > />>/We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there 
any way to generate packets from pktgen at 118 Mpps ? />>//>>//>>/What is the command line for Pktgen? />>//>>/Using a single CPU you should be able to generate about 21Mpps, which 
tells me the configuration is not correct. />>//>>/Pktgen needs N number of TX and RX cores to generate more traffic, plus 
the packets need to be spread across the cores using RSS. This means 
pktgen needs to be setup to generate packets with the 5 tuple difference 
to get all of the cores to receive the packets. />>//>>/I need to know the command line and the configuration of the machine to 
help. />>/ > />>//>>//>>/ > />>/-- />>//>>/ > />>//>>//>>/ > />>/Thanks, />>//>>/ > />>/Shailja />>//>>/ > />>//>>//>>//>>/Regards, />>/Keith />>//>>/-- />>//>>/Thanks, />>/Shailja />>//>//>/-- />//>/Thanks, />/Shailja />//
Regards,
Keith

-- 

Thanks,
Shailja

Re: [dpdk-dev] DPDK-pktgen not giving expected throughput

[Wiles, Keith]

> On Jan 11, 2018, at 12:45 AM, Shailja Pandey <csz168117@iitd.ac.in> wrote:
> 
> I have also tried with multiple cpus using the command line-
> 
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m "[0:4].0,[5:8].1,[9-12].2,[13-16].3” 

What is port 0 and 1, I guess that port 2 and 3 are the 2x40 card. I guess I need the ‘lspci | grep Ether’ output too.

Lets assume port 2 and 3 are the 2 ports of 40G and the NIC is on PCI bus0 attached to NUMA node 0.

May need to blacklist the first two port 0 and 1 to remove them from being used.

./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”

Now you have 6 cores per port 3 on TX and 3 on RX on each port. You maybe be able to get away with 4 per port with the speed of the machine you have.

Next we have to use the range command to make sure the packets are transmitted with a varying set of 5 tuples.

Do a help command an look at the range commands, there is an example in the test directory.

To see the range information use ‘page range’ and then you can modify each port.

After you are happy with the configuration use the ‘save <filename>’ command to save the configuration. Then you can use ‘load <filename>’ or add it to the command line with the -f <filename> option.




> 
> On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote:
>> The command line for Pktgen is-
>> 
>> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m "[2].0,[4].1,[8].2,[10].3"
>> 
>> Configuration of the machine is-
>> 
>> Architecture:          x86_64
>> CPU op-mode(s):        32-bit, 64-bit
>> Byte Order:            Little Endian
>> CPU(s):                32
>> On-line CPU(s) list:   0-31
>> Thread(s) per core:    2
>> Core(s) per socket:    8
>> Socket(s):             2
>> NUMA node(s):          2
>> Vendor ID:             GenuineIntel
>> CPU family:            6
>> Model:                 63
>> Model name:            Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
>> Stepping:              2
>> CPU MHz:               1204.226
>> CPU max MHz:           3400.0000
>> CPU min MHz:           1200.0000
>> BogoMIPS:              5195.08
>> Virtualization:        VT-x
>> L1d cache:             32K
>> L1i cache:             32K
>> L2 cache:              256K
>> L3 cache:              20480K
>> NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
>> NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31
>> 
>> Linux XeonE5 4.4.0-93-generic
>> 
>> Thanks!
>> 
>> 
>> 
>> 
>> 
>> >
>>  On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in
>> > wrote:
>> 
>> >
>>  
>> 
>> >
>>  Hi,
>> 
>> >
>>  
>> 
>> >
>>  We are performing experiments on Dell Poweredge R430 server, which is based on Haswell architecture based xeon-2640 v3 processor. We have attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet generation per port. But pktgen is able to generate only 21 Mpps per port and we are not sure about the problem.
>> 
>> >
>>  
>> 
>> >
>>  We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there any way to generate packets from pktgen at 118 Mpps ?
>> 
>> 
>> What is the command line for Pktgen?
>> 
>> Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.
>> 
>> Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.
>> 
>> I need to know the command line and the configuration of the machine to help.
>> >
>>  
>> 
>> >
>>  -- 
>> 
>> >
>>  
>> 
>> >
>>  Thanks,
>> 
>> >
>>  Shailja
>> 
>> >
>>  
>> 
>> 
>> Regards,
>> Keith
>> 
>> -- 
>> 
>> Thanks,
>> Shailja
>> 
> 
> -- 
> 
> Thanks,
> Shailja
> 

Regards,
Keith

Re: [dpdk-dev] DPDK-pktgen not giving expected throughput

[Shailja Pandey]

I have also tried with multiple cpus using the command line-

./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m 
"[0:4].0,[5:8].1,[9-12].2,[13-16].3"


On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote:
> The command line for Pktgen is-
>
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m "[2].0,[4].1,[8].2,[10].3"
>
> Configuration of the machine is-
>
> Architecture:          x86_64
> CPU op-mode(s):        32-bit, 64-bit
> Byte Order:            Little Endian
> CPU(s):                32
> On-line CPU(s) list:   0-31
> Thread(s) per core:    2
> Core(s) per socket:    8
> Socket(s):             2
> NUMA node(s):          2
> Vendor ID:             GenuineIntel
> CPU family:            6
> Model:                 63
> Model name:            Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
> Stepping:              2
> CPU MHz:               1204.226
> CPU max MHz:           3400.0000
> CPU min MHz:           1200.0000
> BogoMIPS:              5195.08
> Virtualization:        VT-x
> L1d cache:             32K
> L1i cache:             32K
> L2 cache:              256K
> L3 cache:              20480K
> NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
> NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31
>
> Linux XeonE5 4.4.0-93-generic
>
> Thanks!
>
>
>
>
>
> >/On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in 
> <https://dpdk.org/ml/listinfo/dev>> wrote: />//>/Hi, />//>/We are performing experiments on Dell Poweredge R430 server, which is 
> based on Haswell architecture based xeon-2640 v3 processor. We have 
> attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet 
> generation per port. But pktgen is able to generate only 21 Mpps per 
> port and we are not sure about the problem. />//>/We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is 
> there any way to generate packets from pktgen at 118 Mpps ? /
> What is the command line for Pktgen?
>
> Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.
>
> Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.
>
> I need to know the command line and the configuration of the machine to help.
> >//>/-- />//>/Thanks, />/Shailja />//
> Regards,
> Keith
> -- 
>
> Thanks,
> Shailja

-- 

Thanks,
Shailja

[dpdk-dev] DPDK-pktgen not giving expected throughput

[Shailja Pandey]

The command line for Pktgen is-

./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m "[2].0,[4].1,[8].2,[10].3"

Configuration of the machine is-

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Model name:            Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
Stepping:              2
CPU MHz:               1204.226
CPU max MHz:           3400.0000
CPU min MHz:           1200.0000
BogoMIPS:              5195.08
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31

Linux XeonE5 4.4.0-93-generic

Thanks!





>/On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in 
<https://dpdk.org/ml/listinfo/dev>> wrote: />//>/Hi, />//>/We are performing experiments on Dell Poweredge R430 server, which is 
based on Haswell architecture based xeon-2640 v3 processor. We have 
attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet 
generation per port. But pktgen is able to generate only 21 Mpps per 
port and we are not sure about the problem. />//>/We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there 
any way to generate packets from pktgen at 118 Mpps ? /
What is the command line for Pktgen?

Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.

Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.

I need to know the command line and the configuration of the machine to help.
>//>/-- />//>/Thanks, />/Shailja />//
Regards,
Keith

-- 

Thanks,
Shailja

Re: [dpdk-dev] DPDK-pktgen not giving expected throughput

[Wiles, Keith]

> On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117@iitd.ac.in> wrote:
> 
> Hi,
> 
> We are performing experiments on Dell Poweredge R430 server, which is based on Haswell architecture based xeon-2640 v3 processor. We have attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet generation per port. But pktgen is able to generate only 21 Mpps per port and we are not sure about the problem.
> 
> We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there any way to generate packets from pktgen at 118 Mpps ?

What is the command line for Pktgen?

Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.

Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.

I need to know the command line and the configuration of the machine to help.
> 
> -- 
> 
> Thanks,
> Shailja
> 

Regards,
Keith

[dpdk-dev] DPDK-pktgen not giving expected throughput

[Shailja Pandey]

Hi,

We are performing experiments on Dell Poweredge R430 server, which is 
based on Haswell architecture based xeon-2640 v3 processor. We have 
attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet 
generation per port. But pktgen is able to generate only 21 Mpps per 
port and we are not sure about the problem.

We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there 
any way to generate packets from pktgen at 118 Mpps ?

-- 

Thanks,
Shailja