DPDK usage discussions
 help / color / Atom feed
* [dpdk-users] L3fwd-power queue utilization issue
@ 2020-02-14 12:05 Soumen Chandra Laha
  2020-02-15  8:51 ` [dpdk-users] [dpdk-dev] " Harman Kalra
  0 siblings, 1 reply; 5+ messages in thread
From: Soumen Chandra Laha @ 2020-02-14 12:05 UTC (permalink / raw)
  To: users, users, dev

Hi,

I'm trying to run l3fwd-power application for 2 Port with queue. Always
only queue 1 of port 0 and port 1 are utilized.
CPU C-state, hpet are enabled,  ports are binded to igb_uio.
Tried with NIC's x772 (10 GbE) and XXV710(25 GbE), Observed same behavior.

I am observing the below output:
*/root/comms/dpdk/examples/l3fwd-power/build/l3fwd-power  -l 2,3,4,5 -n 6
--socket-mem 512,0  -w 17:00.0 -w 17:00.1  -- -p 0x3 --config
'(0,0,2),(0,1,3),(1,0,4),(1,1,5)'  -P*
EAL: Detected 48 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/l3fwd/mp_socket
EAL: Probing VFIO support...
EAL: PCI device 0000:17:00.0 on NUMA socket 0
EAL:   probe driver: 8086:158b net_i40e
EAL: PCI device 0000:17:00.1 on NUMA socket 0
EAL:   probe driver: 8086:158b net_i40e
Promiscuous mode selected
POWER: Attempting to initialise ACPI cpufreq power management...
POWER: Power management governor of lcore 2 has been set to user space
successfully
POWER: Initialized successfully for lcore 2 power management
POWER: Power management governor of lcore 3 has been set to user space
successfully
POWER: Initialized successfully for lcore 3 power management
POWER: Power management governor of lcore 4 has been set to user space
successfully
POWER: Initialized successfully for lcore 4 power management
POWER: Power management governor of lcore 5 has been set to user space
successfully
POWER: Initialized successfully for lcore 5 power management
Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4... Port 0
modified RSS hash function based on hardware support,requested:0x20820
configured:0x820
 Address:00:E0:ED:96:2B:AA, Allocated mbuf pool on socket 0
Initializing rx queues on lcore 2 ... rxq=0,0,0
Initializing rx queues on lcore 3 ... rxq=0,1,0
Initializing rx queues on lcore 4 ... rxq=1,0,0
Initializing rx queues on lcore 5 ... rxq=1,1,0

Checking link status.....done
Port 0 Link Up - speed 25000 Mbps - full-duplex
Port 1 Link Up - speed 25000 Mbps - full-duplex
L3FWD_POWER: entering main loop on lcore 3
L3FWD_POWER:  -- lcoreid=3 portid=0 rxqueueid=1
L3FWD_POWER: entering main loop on lcore 4
L3FWD_POWER:  -- lcoreid=4 portid=1 rxqueueid=0
L3FWD_POWER: entering main loop on lcore 5
L3FWD_POWER:  -- lcoreid=5 portid=1 rxqueueid=1
EAL: Event already been added.
L3FWD_POWER: entering main loop on lcore 2
L3FWD_POWER:  -- lcoreid=2 portid=0 rxqueueid=0
EAL: Event already been added.
L3FWD_POWER: lcore 3 sleeps until interrupt triggers
L3FWD_POWER: lcore 4 sleeps until interrupt triggers
L3FWD_POWER: lcore 5 sleeps until interrupt triggers
L3FWD_POWER: lcore 2 sleeps until interrupt triggers
L3FWD_POWER: lcore 3 is waked up from rx interrupt on port 0 queue 1
L3FWD_POWER: lcore 3 sleeps until interrupt triggers
L3FWD_POWER: lcore 5 is waked up from rx interrupt on port 1 queue 1
L3FWD_POWER: lcore 5 sleeps until interrupt triggers

I would appreciate it if some expert can shade a light on this.

Thanks & Regards,
Soumen

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] [dpdk-dev] L3fwd-power queue utilization issue
  2020-02-14 12:05 [dpdk-users] L3fwd-power queue utilization issue Soumen Chandra Laha
@ 2020-02-15  8:51 ` " Harman Kalra
  0 siblings, 0 replies; 5+ messages in thread
From: Harman Kalra @ 2020-02-15  8:51 UTC (permalink / raw)
  To: Soumen Chandra Laha; +Cc: users, users, dev

On Fri, Feb 14, 2020 at 05:35:53PM +0530, Soumen Chandra Laha wrote:
> Hi,
> 
> I'm trying to run l3fwd-power application for 2 Port with queue. Always
> only queue 1 of port 0 and port 1 are utilized.
> CPU C-state, hpet are enabled,  ports are binded to igb_uio.
> Tried with NIC's x772 (10 GbE) and XXV710(25 GbE), Observed same behavior.
>

We have tested l3fwd-power with octeontx2 PMD, l3fwd-power disables
queue interrupt even if a single packet is received on the queue.
I assume you are pumping multple flows to the nic.

> I am observing the below output:
> */root/comms/dpdk/examples/l3fwd-power/build/l3fwd-power  -l 2,3,4,5 -n 6
> --socket-mem 512,0  -w 17:00.0 -w 17:00.1  -- -p 0x3 --config
> '(0,0,2),(0,1,3),(1,0,4),(1,1,5)'  -P*
> EAL: Detected 48 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/l3fwd/mp_socket
> EAL: Probing VFIO support...
> EAL: PCI device 0000:17:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:158b net_i40e
> EAL: PCI device 0000:17:00.1 on NUMA socket 0
> EAL:   probe driver: 8086:158b net_i40e
> Promiscuous mode selected
> POWER: Attempting to initialise ACPI cpufreq power management...
> POWER: Power management governor of lcore 2 has been set to user space
> successfully
> POWER: Initialized successfully for lcore 2 power management
> POWER: Power management governor of lcore 3 has been set to user space
> successfully
> POWER: Initialized successfully for lcore 3 power management
> POWER: Power management governor of lcore 4 has been set to user space
> successfully
> POWER: Initialized successfully for lcore 4 power management
> POWER: Power management governor of lcore 5 has been set to user space
> successfully
> POWER: Initialized successfully for lcore 5 power management
> Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4... Port 0
> modified RSS hash function based on hardware support,requested:0x20820
> configured:0x820
>  Address:00:E0:ED:96:2B:AA, Allocated mbuf pool on socket 0
> Initializing rx queues on lcore 2 ... rxq=0,0,0
> Initializing rx queues on lcore 3 ... rxq=0,1,0
> Initializing rx queues on lcore 4 ... rxq=1,0,0
> Initializing rx queues on lcore 5 ... rxq=1,1,0
> 
> Checking link status.....done
> Port 0 Link Up - speed 25000 Mbps - full-duplex
> Port 1 Link Up - speed 25000 Mbps - full-duplex
> L3FWD_POWER: entering main loop on lcore 3
> L3FWD_POWER:  -- lcoreid=3 portid=0 rxqueueid=1
> L3FWD_POWER: entering main loop on lcore 4
> L3FWD_POWER:  -- lcoreid=4 portid=1 rxqueueid=0
> L3FWD_POWER: entering main loop on lcore 5
> L3FWD_POWER:  -- lcoreid=5 portid=1 rxqueueid=1
> EAL: Event already been added.
> L3FWD_POWER: entering main loop on lcore 2
> L3FWD_POWER:  -- lcoreid=2 portid=0 rxqueueid=0
> EAL: Event already been added.
> L3FWD_POWER: lcore 3 sleeps until interrupt triggers
> L3FWD_POWER: lcore 4 sleeps until interrupt triggers
> L3FWD_POWER: lcore 5 sleeps until interrupt triggers
> L3FWD_POWER: lcore 2 sleeps until interrupt triggers
> L3FWD_POWER: lcore 3 is waked up from rx interrupt on port 0 queue 1
> L3FWD_POWER: lcore 3 sleeps until interrupt triggers
> L3FWD_POWER: lcore 5 is waked up from rx interrupt on port 1 queue 1
> L3FWD_POWER: lcore 5 sleeps until interrupt triggers
> 
> I would appreciate it if some expert can shade a light on this.
> 
> Thanks & Regards,
> Soumen

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] L3fwd-power queue utilization issue.
  2020-02-16 22:45 ` Stephen Hemminger
@ 2020-02-17  5:57   ` Soumen Chandra Laha
  0 siblings, 0 replies; 5+ messages in thread
From: Soumen Chandra Laha @ 2020-02-17  5:57 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: users

Hi Stephen,
Thanks for your email.

Tried binding the NIC's to vfio-pci. Observed still queue 0 of Port 0, Port
1 has been utilized. No interrupts for queue 1.

./dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:17:00.0 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=vfio-pci
unused=i40e,igb_uio
0000:17:00.1 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=vfio-pci
unused=i40e,igb_uio

------------------------------------------------------------------------------
./l3fwd-power  -l 2,3,4,5 -n 6 --socket-mem 512,0 --file-prefix l3fwd  -w
17:00.0 -w 17:00.1  -- -p 0x3 --config '(0,0,2),(0,1,3),(1,0,4),(1,1,5)'  -P
EAL: Detected 48 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/l3fwd/mp_socket
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:17:00.0 on NUMA socket 0
EAL:   probe driver: 8086:158b net_i40e
EAL:   using IOMMU type 1 (Type 1)
EAL: PCI device 0000:17:00.1 on NUMA socket 0
EAL:   probe driver: 8086:158b net_i40e
Promiscuous mode selected
POWER: Attempting to initialise ACPI cpufreq power management...
POWER: Power management governor of lcore 2 has been set to user space
successfully
POWER: Initialized successfully for lcore 2 power management
POWER: Power management governor of lcore 3 has been set to user space
successfully
POWER: Initialized successfully for lcore 3 power management
POWER: Power management governor of lcore 4 has been set to user space
successfully
POWER: Initialized successfully for lcore 4 power management
POWER: Power management governor of lcore 5 has been set to user space
successfully
POWER: Initialized successfully for lcore 5 power management
Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4... Port 0
modified RSS hash function based on hardware support,requested:0x20820
configured:0x820
 Address:00:E0:ED:96:2B:AA, Allocated mbuf pool on socket 0
LPM: Adding route 0x01010100 / 24 (0)
LPM: Adding route 0x02010100 / 24 (1)
LPM: Adding route 0x03010100 / 24 (2)
LPM: Adding route 0x04010100 / 24 (3)
LPM: Adding route 0x05010100 / 24 (4)
LPM: Adding route 0x06010100 / 24 (5)
LPM: Adding route 0x07010100 / 24 (6)
LPM: Adding route 0x08010100 / 24 (7)
txq=2,0,0 txq=3,1,0 txq=4,2,0 txq=5,3,0
Initializing port 1 ... Creating queues: nb_rxq=2 nb_txq=4... Port 1
modified RSS hash function based on hardware support,requested:0x20820
configured:0x820
 Address:00:E0:ED:96:2B:AB, txq=2,0,0 txq=3,1,0 txq=4,2,0 txq=5,3,0

Initializing rx queues on lcore 2 ... rxq=0,0,0
Initializing rx queues on lcore 3 ... rxq=0,1,0
Initializing rx queues on lcore 4 ... rxq=1,0,0
Initializing rx queues on lcore 5 ... rxq=1,1,0

Checking link status.....done
Port 0 Link Up - speed 25000 Mbps - full-duplex
Port 1 Link Up - speed 25000 Mbps - full-duplex
L3FWD_POWER: entering main loop on lcore 3
L3FWD_POWER:  -- lcoreid=3 portid=0 rxqueueid=1
L3FWD_POWER: entering main loop on lcore 4
L3FWD_POWER: entering main loop on lcore 5
L3FWD_POWER: entering main loop on lcore 2
L3FWD_POWER:  -- lcoreid=5 portid=1 rxqueueid=1
L3FWD_POWER:  -- lcoreid=4 portid=1 rxqueueid=0
L3FWD_POWER:  -- lcoreid=2 portid=0 rxqueueid=0
L3FWD_POWER: lcore 5 sleeps until interrupt triggers
L3FWD_POWER: lcore 4 sleeps until interrupt triggers
L3FWD_POWER: lcore 2 sleeps until interrupt triggers
L3FWD_POWER: lcore 3 sleeps until interrupt triggers
L3FWD_POWER: lcore 4 is waked up from rx interrupt on port 1 queue 0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port 0 queue 0
L3FWD_POWER: lcore 2 sleeps until interrupt triggers
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port 0 queue 0
L3FWD_POWER: lcore 2 sleeps until interrupt triggers
L3FWD_POWER: lcore 4 sleeps until interrupt triggers
L3FWD_POWER: lcore 4 is waked up from rx interrupt on port 1 queue 0
L3FWD_POWER: lcore 4 sleeps until interrupt triggers

Does anybody observed similar behavior, any help & suggestions will be
appreciated.

Thanks,
Soumen Chandra Laha



On Mon, Feb 17, 2020 at 4:15 AM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Fri, 14 Feb 2020 16:36:04 +0530
> Soumen Chandra Laha <soumen.laha32@gmail.com> wrote:
>
> > Hi,
> >
> > I'm trying to run l3fwd-power application for 2 Port with queue. Always
> > only queue 1 of port 0 and port 1 are utilized.
> > CPU C-state, hpet are enabled,  ports are binded to igb_uio.
> > Tried with NIC's x772 (10 GbE) and XXV710(25 GbE), Observed same
> behavior.
> >
>
> Igb_uio does not support per-queue interrupts. That needs MSI-X which is
> only supported by VFIO-pci.
>
>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] L3fwd-power queue utilization issue.
  2020-02-14 11:06 [dpdk-users] " Soumen Chandra Laha
@ 2020-02-16 22:45 ` Stephen Hemminger
  2020-02-17  5:57   ` Soumen Chandra Laha
  0 siblings, 1 reply; 5+ messages in thread
From: Stephen Hemminger @ 2020-02-16 22:45 UTC (permalink / raw)
  To: Soumen Chandra Laha; +Cc: users

On Fri, 14 Feb 2020 16:36:04 +0530
Soumen Chandra Laha <soumen.laha32@gmail.com> wrote:

> Hi,
> 
> I'm trying to run l3fwd-power application for 2 Port with queue. Always
> only queue 1 of port 0 and port 1 are utilized.
> CPU C-state, hpet are enabled,  ports are binded to igb_uio.
> Tried with NIC's x772 (10 GbE) and XXV710(25 GbE), Observed same behavior.
> 

Igb_uio does not support per-queue interrupts. That needs MSI-X which is
only supported by VFIO-pci.


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [dpdk-users] L3fwd-power queue utilization issue.
@ 2020-02-14 11:06 " Soumen Chandra Laha
  2020-02-16 22:45 ` Stephen Hemminger
  0 siblings, 1 reply; 5+ messages in thread
From: Soumen Chandra Laha @ 2020-02-14 11:06 UTC (permalink / raw)
  To: users

Hi,

I'm trying to run l3fwd-power application for 2 Port with queue. Always
only queue 1 of port 0 and port 1 are utilized.
CPU C-state, hpet are enabled,  ports are binded to igb_uio.
Tried with NIC's x772 (10 GbE) and XXV710(25 GbE), Observed same behavior.

I am observing the below output:
*/root/comms/dpdk/examples/l3fwd-power/build/l3fwd-power  -l 2,3,4,5 -n 6
--socket-mem 512,0  -w 17:00.0 -w 17:00.1  -- -p 0x3 --config
'(0,0,2),(0,1,3),(1,0,4),(1,1,5)'  -P*
EAL: Detected 48 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/l3fwd/mp_socket
EAL: Probing VFIO support...
EAL: PCI device 0000:17:00.0 on NUMA socket 0
EAL:   probe driver: 8086:158b net_i40e
EAL: PCI device 0000:17:00.1 on NUMA socket 0
EAL:   probe driver: 8086:158b net_i40e
Promiscuous mode selected
POWER: Attempting to initialise ACPI cpufreq power management...
POWER: Power management governor of lcore 2 has been set to user space
successfully
POWER: Initialized successfully for lcore 2 power management
POWER: Power management governor of lcore 3 has been set to user space
successfully
POWER: Initialized successfully for lcore 3 power management
POWER: Power management governor of lcore 4 has been set to user space
successfully
POWER: Initialized successfully for lcore 4 power management
POWER: Power management governor of lcore 5 has been set to user space
successfully
POWER: Initialized successfully for lcore 5 power management
Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4... Port 0
modified RSS hash function based on hardware support,requested:0x20820
configured:0x820
 Address:00:E0:ED:96:2B:AA, Allocated mbuf pool on socket 0
Initializing rx queues on lcore 2 ... rxq=0,0,0
Initializing rx queues on lcore 3 ... rxq=0,1,0
Initializing rx queues on lcore 4 ... rxq=1,0,0
Initializing rx queues on lcore 5 ... rxq=1,1,0

Checking link status.....done
Port 0 Link Up - speed 25000 Mbps - full-duplex
Port 1 Link Up - speed 25000 Mbps - full-duplex
L3FWD_POWER: entering main loop on lcore 3
L3FWD_POWER:  -- lcoreid=3 portid=0 rxqueueid=1
L3FWD_POWER: entering main loop on lcore 4
L3FWD_POWER:  -- lcoreid=4 portid=1 rxqueueid=0
L3FWD_POWER: entering main loop on lcore 5
L3FWD_POWER:  -- lcoreid=5 portid=1 rxqueueid=1
EAL: Event already been added.
L3FWD_POWER: entering main loop on lcore 2
L3FWD_POWER:  -- lcoreid=2 portid=0 rxqueueid=0
EAL: Event already been added.
L3FWD_POWER: lcore 3 sleeps until interrupt triggers
L3FWD_POWER: lcore 4 sleeps until interrupt triggers
L3FWD_POWER: lcore 5 sleeps until interrupt triggers
L3FWD_POWER: lcore 2 sleeps until interrupt triggers
L3FWD_POWER: lcore 3 is waked up from rx interrupt on port 0 queue 1
L3FWD_POWER: lcore 3 sleeps until interrupt triggers
L3FWD_POWER: lcore 5 is waked up from rx interrupt on port 1 queue 1
L3FWD_POWER: lcore 5 sleeps until interrupt triggers

I would appreciate it if some expert can shade a light on this.

Thanks & Regards,
Soumen

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-14 12:05 [dpdk-users] L3fwd-power queue utilization issue Soumen Chandra Laha
2020-02-15  8:51 ` [dpdk-users] [dpdk-dev] " Harman Kalra
  -- strict thread matches above, loose matches on Subject: below --
2020-02-14 11:06 [dpdk-users] " Soumen Chandra Laha
2020-02-16 22:45 ` Stephen Hemminger
2020-02-17  5:57   ` Soumen Chandra Laha

DPDK usage discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox