* [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G @ 2015-12-27 18:31 Laurent GUERBY 2015-12-27 19:43 ` Wiles, Keith 0 siblings, 1 reply; 11+ messages in thread From: Laurent GUERBY @ 2015-12-27 18:31 UTC (permalink / raw) To: dev Hi, I reported today an issue when using Pktgen-DPDK: https://github.com/pktgen/Pktgen-DPDK/issues/52 But I think it's more in DPDK than pktgen two identical machines with SFP+ DA cable between them DPDK 2.2.0 from tarball Pktgen-DPDK from git two identical machines: core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS ASUS P8H67-M PRO BIOS 3904 (latest available) Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection (rev 01) 01:00.0 0200: 8086:10f1 (rev 01) Subsystem: 8086:a21f boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 When launching pktgen even with no TX asked the Errors RX counters keeps going up by about 7.4 millions per second: Errors Rx/Tx : 7471857054/0 In the log I get "Could not enable Rx Queue", might be the source of the issue? PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 1 When sending traffic single UDP src/dst/IP/MAC the setup reaches 14204188 pps 64 bytes, the error counter is also increasing. Any idea what to look for? Laurent ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2015-12-27 18:31 [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G Laurent GUERBY @ 2015-12-27 19:43 ` Wiles, Keith 2015-12-27 20:09 ` Laurent GUERBY 0 siblings, 1 reply; 11+ messages in thread From: Wiles, Keith @ 2015-12-27 19:43 UTC (permalink / raw) To: Laurent GUERBY, dev On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: >Hi, > >I reported today an issue when using Pktgen-DPDK: >https://github.com/pktgen/Pktgen-DPDK/issues/52 > >But I think it's more in DPDK than pktgen > >two identical machines with SFP+ DA cable between them >DPDK 2.2.0 from tarball >Pktgen-DPDK from git >two identical machines: >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS >ASUS P8H67-M PRO BIOS 3904 (latest available) >Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port >Network Connection (rev 01) >01:00.0 0200: 8086:10f1 (rev 01) > Subsystem: 8086:a21f >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 > >When launching pktgen even with no TX asked the Errors RX counters keeps >going up by about 7.4 millions per second: > >Errors Rx/Tx : 7471857054/0 > >In the log I get "Could not enable Rx Queue", might be the >source of the issue? > >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 1 > >When sending traffic single UDP src/dst/IP/MAC the setup >reaches 14204188 pps 64 bytes, the error counter is also >increasing. > >Any idea what to look for? One more suggestion is to run test_pmd on one machine and something like iperf on the other to verify the DPDK is working correct, which I assume will be true. Not sure the RX errors are reported in the test_pmd or you could use the l3fwd application too. Please also send me the 'lspci | grep Ethernet’ output. Also send me the command line. > >Laurent > > Regards, Keith ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2015-12-27 19:43 ` Wiles, Keith @ 2015-12-27 20:09 ` Laurent GUERBY 2015-12-27 20:28 ` Wiles, Keith 0 siblings, 1 reply; 11+ messages in thread From: Laurent GUERBY @ 2015-12-27 20:09 UTC (permalink / raw) To: Wiles, Keith; +Cc: dev On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: > On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: > > >Hi, > > > >I reported today an issue when using Pktgen-DPDK: > >https://github.com/pktgen/Pktgen-DPDK/issues/52 > > > >But I think it's more in DPDK than pktgen > > > >two identical machines with SFP+ DA cable between them > >DPDK 2.2.0 from tarball > >Pktgen-DPDK from git > >two identical machines: > >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS > >ASUS P8H67-M PRO BIOS 3904 (latest available) > >Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port > >Network Connection (rev 01) > >01:00.0 0200: 8086:10f1 (rev 01) > > Subsystem: 8086:a21f > >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 > > > >When launching pktgen even with no TX asked the Errors RX counters keeps > >going up by about 7.4 millions per second: > > > >Errors Rx/Tx : 7471857054/0 > > > >In the log I get "Could not enable Rx Queue", might be the > >source of the issue? > > > >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 1 > > > >When sending traffic single UDP src/dst/IP/MAC the setup > >reaches 14204188 pps 64 bytes, the error counter is also > >increasing. > > > >Any idea what to look for? > > One more suggestion is to run test_pmd on one machine and something > like iperf on the other to verify the DPDK is working correct, which I > assume will be true. Not sure the RX errors are reported in the > test_pmd or you could use the l3fwd application too. Ok, I will check the test_pmd documentation and try to do this test: I'm just starting on DPDK :). > Please also send me the 'lspci | grep Ethernet’ output. I included one line in my original email above (plus extract of lspci -vn), here is the full output of the command: 01:00.0 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection (rev 01) 05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) (The realtek is used only for internet connectivity). > Also send me the command line. On the first machine t1 : root@t1:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ -c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N And on the other machine t3: root@t3:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ -c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N The two "-f" pkt files are attached to this email, I do "start 1" manually at the pktgen prompt. Thanks for your time, Sincerely, Laurent > > > >Laurent > > > > > > > Regards, > Keith > > > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2015-12-27 20:09 ` Laurent GUERBY @ 2015-12-27 20:28 ` Wiles, Keith 2016-01-27 6:50 ` Moon-Sang Lee 0 siblings, 1 reply; 11+ messages in thread From: Wiles, Keith @ 2015-12-27 20:28 UTC (permalink / raw) To: Laurent GUERBY; +Cc: dev On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net> wrote: >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: >> >> >Hi, >> > >> >I reported today an issue when using Pktgen-DPDK: >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 >> > >> >But I think it's more in DPDK than pktgen >> > >> >two identical machines with SFP+ DA cable between them >> >DPDK 2.2.0 from tarball >> >Pktgen-DPDK from git >> >two identical machines: >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS >> >ASUS P8H67-M PRO BIOS 3904 (latest available) >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port >> >Network Connection (rev 01) >> >01:00.0 0200: 8086:10f1 (rev 01) >> > Subsystem: 8086:a21f >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 >> > >> >When launching pktgen even with no TX asked the Errors RX counters keeps >> >going up by about 7.4 millions per second: >> > >> >Errors Rx/Tx : 7471857054/0 >> > >> >In the log I get "Could not enable Rx Queue", might be the >> >source of the issue? >> > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 1 >> > >> >When sending traffic single UDP src/dst/IP/MAC the setup >> >reaches 14204188 pps 64 bytes, the error counter is also >> >increasing. >> > >> >Any idea what to look for? >> >> One more suggestion is to run test_pmd on one machine and something >> like iperf on the other to verify the DPDK is working correct, which I >> assume will be true. Not sure the RX errors are reported in the >> test_pmd or you could use the l3fwd application too. > >Ok, I will check the test_pmd documentation and try to do this test: I'm >just starting on DPDK :). > >> Please also send me the 'lspci | grep Ethernet’ output. > >I included one line in my original email above (plus extract of lspci >-vn), here is the full output of the command: > >01:00.0 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF >Dual Port Network Connection (rev 01) >01:00.1 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF >Dual Port Network Connection (rev 01) >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) > >(The realtek is used only for internet connectivity). > >> Also send me the command line. > >On the first machine t1 : >root@t1:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N > >And on the other machine t3: >root@t3:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N You need to always start with port numbering for Pktgen with zero. Change the [2:3].1 to [2:3].0 the reason is you removed one of the two ports and Pktgen starts port counting from zero for the first available port :-) Sorry, I did not spot that sooner. > >The two "-f" pkt files are attached to this email, I do "start 1" >manually at the pktgen prompt. > >Thanks for your time, > >Sincerely, > >Laurent > > >> > >> >Laurent >> > >> > >> >> >> Regards, >> Keith >> >> >> >> > > Regards, Keith ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2015-12-27 20:28 ` Wiles, Keith @ 2016-01-27 6:50 ` Moon-Sang Lee 2016-01-27 7:15 ` Laurent GUERBY 2016-01-30 1:23 ` Wiles, Keith 0 siblings, 2 replies; 11+ messages in thread From: Moon-Sang Lee @ 2016-01-27 6:50 UTC (permalink / raw) To: Wiles, Keith; +Cc: dev Laurent, have you resolved this problem? I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced the same problem as you. Here is parts of my log and it says that PMD cannot enable RX queue for my NIC. I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling rte_eth_rx_queue_setup(). (i.e. 'null' parameter provides the default rx_conf value.) Thanks. APP: initialising port 0 ... PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27258040 sw_sc_ring=0x7f5f27257b00 hw_ring=0x7f5f27258580 dma_addr=0x41f458580 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f27245940 hw_ring=0x7f5f27247980 dma_addr=0x41f447980 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272337c0 hw_ring=0x7f5f27235800 dma_addr=0x41f435800 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=0). *PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0* APP: port 0 has started APP: port 0 has entered in promiscuous mode APP: port 0 initialization is done. KNI: pci: 09:00:00 8086:10c7 APP: kni allocation is done for port 0. APP: initialising port 1 ... PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27222dc0 sw_sc_ring=0x7f5f27222880 hw_ring=0x7f5f27223300 dma_addr=0x41f423300 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272106c0 hw_ring=0x7f5f27212700 dma_addr=0x41f412700 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f271fe540 hw_ring=0x7f5f27200580 dma_addr=0x41f400580 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=1). *PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0* APP: port 1 has started APP: port 1 has entered in promiscuous mode APP: port 1 initialization is done. KNI: pci: 0a:00:00 8086:10c7 APP: kni allocation is done for port 1. checking link status .done Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed 10000 Mbps - full-duplex On Mon, Dec 28, 2015 at 5:28 AM, Wiles, Keith <keith.wiles@intel.com> wrote: > On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net> wrote: > > >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: > >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" < > dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: > >> > >> >Hi, > >> > > >> >I reported today an issue when using Pktgen-DPDK: > >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 > >> > > >> >But I think it's more in DPDK than pktgen > >> > > >> >two identical machines with SFP+ DA cable between them > >> >DPDK 2.2.0 from tarball > >> >Pktgen-DPDK from git > >> >two identical machines: > >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS > >> >ASUS P8H67-M PRO BIOS 3904 (latest available) > >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port > >> >Network Connection (rev 01) > >> >01:00.0 0200: 8086:10f1 (rev 01) > >> > Subsystem: 8086:a21f > >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 > >> > > >> >When launching pktgen even with no TX asked the Errors RX counters > keeps > >> >going up by about 7.4 millions per second: > >> > > >> >Errors Rx/Tx : 7471857054/0 > >> > > >> >In the log I get "Could not enable Rx Queue", might be the > >> >source of the issue? > >> > > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 1 > >> > > >> >When sending traffic single UDP src/dst/IP/MAC the setup > >> >reaches 14204188 pps 64 bytes, the error counter is also > >> >increasing. > >> > > >> >Any idea what to look for? > >> > >> One more suggestion is to run test_pmd on one machine and something > >> like iperf on the other to verify the DPDK is working correct, which I > >> assume will be true. Not sure the RX errors are reported in the > >> test_pmd or you could use the l3fwd application too. > > > >Ok, I will check the test_pmd documentation and try to do this test: I'm > >just starting on DPDK :). > > > >> Please also send me the 'lspci | grep Ethernet’ output. > > > >I included one line in my original email above (plus extract of lspci > >-vn), here is the full output of the command: > > > >01:00.0 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF > >Dual Port Network Connection (rev 01) > >01:00.1 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF > >Dual Port Network Connection (rev 01) > >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) > > > >(The realtek is used only for internet connectivity). > > > >> Also send me the command line. > > > >On the first machine t1 : > >root@t1:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N > > > >And on the other machine t3: > >root@t3:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N > > You need to always start with port numbering for Pktgen with zero. > > Change the [2:3].1 to [2:3].0 the reason is you removed one of the two > ports and Pktgen starts port counting from zero for the first available > port :-) > > Sorry, I did not spot that sooner. > > > >The two "-f" pkt files are attached to this email, I do "start 1" > >manually at the pktgen prompt. > > > >Thanks for your time, > > > >Sincerely, > > > >Laurent > > > > > >> > > >> >Laurent > >> > > >> > > >> > >> > >> Regards, > >> Keith > >> > >> > >> > >> > > > > > > > Regards, > Keith > > > > > -- Moon-Sang Lee, SW Engineer Email: sang0627@gmail.com Wisdom begins in wonder. *Socrates* ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2016-01-27 6:50 ` Moon-Sang Lee @ 2016-01-27 7:15 ` Laurent GUERBY 2016-01-27 7:23 ` Zhang, Helin 2016-01-30 1:23 ` Wiles, Keith 1 sibling, 1 reply; 11+ messages in thread From: Laurent GUERBY @ 2016-01-27 7:15 UTC (permalink / raw) To: Moon-Sang Lee; +Cc: dev On Wed, 2016-01-27 at 15:50 +0900, Moon-Sang Lee wrote: > > > Laurent, have you resolved this problem? > I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced > the same problem as you. > Here is parts of my log and it says that PMD cannot enable RX queue > for my NIC. > I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling > rte_eth_rx_queue_setup(). > (i.e. 'null' parameter provides the default rx_conf value.) Hi, I had to reuse my DPDK machines for another task, I will go back to it after FOSDEM. The error you get is the same as mine. Sincerely, Laurent > > Thanks. > > > > > > APP: initialising port 0 ... > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27258040 > sw_sc_ring=0x7f5f27257b00 hw_ring=0x7f5f27258580 dma_addr=0x41f458580 > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f27245940 > hw_ring=0x7f5f27247980 dma_addr=0x41f447980 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272337c0 > hw_ring=0x7f5f27235800 dma_addr=0x41f435800 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX > burst size no less than 4 (port=0). > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > APP: port 0 has started > APP: port 0 has entered in promiscuous mode > APP: port 0 initialization is done. > KNI: pci: 09:00:00 8086:10c7 > APP: kni allocation is done for port 0. > APP: initialising port 1 ... > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27222dc0 > sw_sc_ring=0x7f5f27222880 hw_ring=0x7f5f27223300 dma_addr=0x41f423300 > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272106c0 > hw_ring=0x7f5f27212700 dma_addr=0x41f412700 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f271fe540 > hw_ring=0x7f5f27200580 dma_addr=0x41f400580 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX > burst size no less than 4 (port=1). > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > APP: port 1 has started > APP: port 1 has entered in promiscuous mode > APP: port 1 initialization is done. > KNI: pci: 0a:00:00 8086:10c7 > APP: kni allocation is done for port 1. > > > checking link status > .done > Port 0 Link Up - speed 10000 Mbps - full-duplex > Port 1 Link Up - speed 10000 Mbps - full-duplex > > > > On Mon, Dec 28, 2015 at 5:28 AM, Wiles, Keith <keith.wiles@intel.com> > wrote: > On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net> > wrote: > > >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: > >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" > <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: > >> > >> >Hi, > >> > > >> >I reported today an issue when using Pktgen-DPDK: > >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 > >> > > >> >But I think it's more in DPDK than pktgen > >> > > >> >two identical machines with SFP+ DA cable between them > >> >DPDK 2.2.0 from tarball > >> >Pktgen-DPDK from git > >> >two identical machines: > >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS > >> >ASUS P8H67-M PRO BIOS 3904 (latest available) > >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit > AF Dual Port > >> >Network Connection (rev 01) > >> >01:00.0 0200: 8086:10f1 (rev 01) > >> > Subsystem: 8086:a21f > >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 > >> > > >> >When launching pktgen even with no TX asked the Errors RX > counters keeps > >> >going up by about 7.4 millions per second: > >> > > >> >Errors Rx/Tx : 7471857054/0 > >> > > >> >In the log I get "Could not enable Rx Queue", might be the > >> >source of the issue? > >> > > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue > 0 > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue > 1 > >> > > >> >When sending traffic single UDP src/dst/IP/MAC the setup > >> >reaches 14204188 pps 64 bytes, the error counter is also > >> >increasing. > >> > > >> >Any idea what to look for? > >> > >> One more suggestion is to run test_pmd on one machine and > something > >> like iperf on the other to verify the DPDK is working > correct, which I > >> assume will be true. Not sure the RX errors are reported in > the > >> test_pmd or you could use the l3fwd application too. > > > >Ok, I will check the test_pmd documentation and try to do > this test: I'm > >just starting on DPDK :). > > > >> Please also send me the 'lspci | grep Ethernet’ output. > > > >I included one line in my original email above (plus extract > of lspci > >-vn), here is the full output of the command: > > > >01:00.0 Ethernet controller: Intel Corporation 82598EB > 10-Gigabit AF > >Dual Port Network Connection (rev 01) > >01:00.1 Ethernet controller: Intel Corporation 82598EB > 10-Gigabit AF > >Dual Port Network Connection (rev 01) > >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller > (rev 06) > > > >(The realtek is used only for internet connectivity). > > > >> Also send me the command line. > > > >On the first machine t1 : > >root@t1:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N > > > >And on the other machine t3: > >root@t3:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N > > > You need to always start with port numbering for Pktgen with > zero. > > Change the [2:3].1 to [2:3].0 the reason is you removed one of > the two ports and Pktgen starts port counting from zero for > the first available port :-) > > Sorry, I did not spot that sooner. > > > >The two "-f" pkt files are attached to this email, I do > "start 1" > >manually at the pktgen prompt. > > > >Thanks for your time, > > > >Sincerely, > > > >Laurent > > > > > >> > > >> >Laurent > >> > > >> > > >> > >> > >> Regards, > >> Keith > >> > >> > >> > >> > > > > > > > Regards, > Keith > > > > > > > > > -- > Moon-Sang Lee, SW Engineer > Email: sang0627@gmail.com > Wisdom begins in wonder. *Socrates* ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2016-01-27 7:15 ` Laurent GUERBY @ 2016-01-27 7:23 ` Zhang, Helin 2016-01-28 0:57 ` Moon-Sang Lee 0 siblings, 1 reply; 11+ messages in thread From: Zhang, Helin @ 2016-01-27 7:23 UTC (permalink / raw) To: Laurent GUERBY, Moon-Sang Lee; +Cc: dev Moon-Sang Were you using pktgen or else application? Could you help to share with me the detailed steps of your reproducing that issue? We will find time on that soon later. Thanks! Regards, Helin -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Laurent GUERBY Sent: Wednesday, January 27, 2016 3:16 PM To: Moon-Sang Lee <sang0627@gmail.com> Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G On Wed, 2016-01-27 at 15:50 +0900, Moon-Sang Lee wrote: > > > Laurent, have you resolved this problem? > I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced > the same problem as you. > Here is parts of my log and it says that PMD cannot enable RX queue > for my NIC. > I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling > rte_eth_rx_queue_setup(). > (i.e. 'null' parameter provides the default rx_conf value.) Hi, I had to reuse my DPDK machines for another task, I will go back to it after FOSDEM. The error you get is the same as mine. Sincerely, Laurent > > Thanks. > > > > > > APP: initialising port 0 ... > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27258040 > sw_sc_ring=0x7f5f27257b00 hw_ring=0x7f5f27258580 dma_addr=0x41f458580 > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f27245940 > hw_ring=0x7f5f27247980 dma_addr=0x41f447980 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272337c0 > hw_ring=0x7f5f27235800 dma_addr=0x41f435800 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX > burst size no less than 4 (port=0). > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > APP: port 0 has started > APP: port 0 has entered in promiscuous mode > APP: port 0 initialization is done. > KNI: pci: 09:00:00 8086:10c7 > APP: kni allocation is done for port 0. > APP: initialising port 1 ... > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27222dc0 > sw_sc_ring=0x7f5f27222880 hw_ring=0x7f5f27223300 dma_addr=0x41f423300 > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272106c0 > hw_ring=0x7f5f27212700 dma_addr=0x41f412700 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f271fe540 > hw_ring=0x7f5f27200580 dma_addr=0x41f400580 > PMD: ixgbe_set_tx_function(): Using simple tx code path > PMD: ixgbe_set_tx_function(): Vector tx enabled. > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX > burst size no less than 4 (port=1). > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > APP: port 1 has started > APP: port 1 has entered in promiscuous mode > APP: port 1 initialization is done. > KNI: pci: 0a:00:00 8086:10c7 > APP: kni allocation is done for port 1. > > > checking link status > .done > Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed > 10000 Mbps - full-duplex > > > > On Mon, Dec 28, 2015 at 5:28 AM, Wiles, Keith <keith.wiles@intel.com> > wrote: > On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net> > wrote: > > >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: > >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" > <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: > >> > >> >Hi, > >> > > >> >I reported today an issue when using Pktgen-DPDK: > >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 > >> > > >> >But I think it's more in DPDK than pktgen > >> > > >> >two identical machines with SFP+ DA cable between them > >> >DPDK 2.2.0 from tarball > >> >Pktgen-DPDK from git > >> >two identical machines: > >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS > >> >ASUS P8H67-M PRO BIOS 3904 (latest available) > >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit > AF Dual Port > >> >Network Connection (rev 01) > >> >01:00.0 0200: 8086:10f1 (rev 01) > >> > Subsystem: 8086:a21f > >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 > >> > > >> >When launching pktgen even with no TX asked the Errors RX > counters keeps > >> >going up by about 7.4 millions per second: > >> > > >> >Errors Rx/Tx : 7471857054/0 > >> > > >> >In the log I get "Could not enable Rx Queue", might be the > >> >source of the issue? > >> > > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue > 0 > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue > 1 > >> > > >> >When sending traffic single UDP src/dst/IP/MAC the setup > >> >reaches 14204188 pps 64 bytes, the error counter is also > >> >increasing. > >> > > >> >Any idea what to look for? > >> > >> One more suggestion is to run test_pmd on one machine and > something > >> like iperf on the other to verify the DPDK is working > correct, which I > >> assume will be true. Not sure the RX errors are reported in > the > >> test_pmd or you could use the l3fwd application too. > > > >Ok, I will check the test_pmd documentation and try to do > this test: I'm > >just starting on DPDK :). > > > >> Please also send me the 'lspci | grep Ethernet’ output. > > > >I included one line in my original email above (plus extract > of lspci > >-vn), here is the full output of the command: > > > >01:00.0 Ethernet controller: Intel Corporation 82598EB > 10-Gigabit AF > >Dual Port Network Connection (rev 01) > >01:00.1 Ethernet controller: Intel Corporation 82598EB > 10-Gigabit AF > >Dual Port Network Connection (rev 01) > >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller > (rev 06) > > > >(The realtek is used only for internet connectivity). > > > >> Also send me the command line. > > > >On the first machine t1 : > >root@t1:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N > > > >And on the other machine t3: > >root@t3:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N > > > You need to always start with port numbering for Pktgen with > zero. > > Change the [2:3].1 to [2:3].0 the reason is you removed one of > the two ports and Pktgen starts port counting from zero for > the first available port :-) > > Sorry, I did not spot that sooner. > > > >The two "-f" pkt files are attached to this email, I do > "start 1" > >manually at the pktgen prompt. > > > >Thanks for your time, > > > >Sincerely, > > > >Laurent > > > > > >> > > >> >Laurent > >> > > >> > > >> > >> > >> Regards, > >> Keith > >> > >> > >> > >> > > > > > > > Regards, > Keith > > > > > > > > > -- > Moon-Sang Lee, SW Engineer > Email: sang0627@gmail.com > Wisdom begins in wonder. *Socrates* ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2016-01-27 7:23 ` Zhang, Helin @ 2016-01-28 0:57 ` Moon-Sang Lee 2016-01-29 10:02 ` Moon-Sang Lee 0 siblings, 1 reply; 11+ messages in thread From: Moon-Sang Lee @ 2016-01-28 0:57 UTC (permalink / raw) To: Zhang, Helin; +Cc: dev Helin, I implemented my own sample application that is a kind of carrier grade NAT server. It works fine on 1G NIC (i.e. Intel Corporation 82576 Gigabit Network Connection (rev 01)) But, it does not receive packets on 10G NIC (i.e. Intel Corporation 82598EB 10-Gigabit AF Network Connection (rev 01)) as described in the previous email. According to my log messages, it seems that control register for RX DMA is not enabled. Here is some information about my environment. 1. HW & OS [mslee@centos7 ~]$ uname -a Linux centos7 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux [mslee@centos7 ~]$ more /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 microcode : 0x19 cpu MHz : 2262.000 cache size : 8192 KB physical id : 1 siblings : 8 core id : 0 cpu cores : 4 apicid : 16 initial apicid : 16 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm c onstant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 po pcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid bogomips : 4521.93 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ... 2. port ocnfigure parameter for rte_eth_dev_configure(): ret = rte_eth_dev_configure(port, NB_RXQ, NB_TXQ, &port_conf); where NB_RXQ=1, NB_TXQ=2, and struct rte_eth_conf port_conf = { .rxmode = { //.mq_mode = ETH_MQ_RX_RSS, .mq_mode = ETH_MQ_RX_NONE, // for 10G NIC .max_rx_pkt_len = ETHER_MAX_LEN, .split_hdr_size = 0, .header_split = 0, // Header Split disabled .hw_ip_checksum = 0, // IP checksum offload enabled .hw_vlan_filter = 0, // VLAN filtering disabled .jumbo_frame = 0, // Jumbo Frame Support disabled .hw_strip_crc = 0, // CRC stripped by hardware }, .rx_adv_conf = { .rss_conf = { .rss_key = NULL, .rss_hf = ETH_RSS_IP, }, }, .txmode = { .mq_mode = ETH_MQ_TX_NONE, }, }; 3. rx queue setup parameter ret = rte_eth_rx_queue_setup(port, RXQ_ID, NB_RXD, socket_id, NULL, pktmbuf_pool[socket_id]) where RXQ_ID = 0, NB_RXD = 128 4. config parameters in config/common_linuxapp # # Compile burst-oriented IXGBE PMD driver # CONFIG_RTE_LIBRTE_IXGBE_PMD=y CONFIG_RTE_LIBRTE_IXGBE_DEBUG_INIT=n CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX=n CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX=n CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX_FREE=n CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n CONFIG_RTE_IXGBE_INC_VECTOR=y CONFIG_RTE_IXGBE_RX_OLFLAGS_ENABLE=y 5. where log message is printed dpdk-2.2.0/drivers/net/ixgbe/ixgbe_rxtx.c: /* Allocate buffers for descriptor rings */ if (ixgbe_alloc_rx_queue_mbufs(rxq) != 0) { PMD_INIT_LOG(ERR, "Could not alloc mbuf for queue:%d", rx_queue_id); return -1; } rxdctl = IXGBE_READ_REG(hw, IXGBE_RXDCTL(rxq->reg_idx)); rxdctl |= IXGBE_RXDCTL_ENABLE; IXGBE_WRITE_REG(hw, IXGBE_RXDCTL(rxq->reg_idx), rxdctl); /* Wait until RX Enable ready */ poll_ms = RTE_IXGBE_REGISTER_POLL_WAIT_10_MS; do { rte_delay_ms(1); rxdctl = IXGBE_READ_REG(hw, IXGBE_RXDCTL(rxq->reg_idx)); } while (--poll_ms && !(rxdctl & IXGBE_RXDCTL_ENABLE)); if (!poll_ms) PMD_INIT_LOG(ERR, "*Could not enable Rx Queue %d*", rx_queue_id); I'm going to update firmware of my NIC, but I'm not sure it helps. I appreciate any comment. On Wed, Jan 27, 2016 at 4:23 PM, Zhang, Helin <helin.zhang@intel.com> wrote: > Moon-Sang > > Were you using pktgen or else application? > Could you help to share with me the detailed steps of your reproducing > that issue? > We will find time on that soon later. Thanks! > > Regards, > Helin > > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Laurent GUERBY > Sent: Wednesday, January 27, 2016 3:16 PM > To: Moon-Sang Lee <sang0627@gmail.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] Errors Rx count increasing while pktgen doing > nothing on Intel 82598EB 10G > > On Wed, 2016-01-27 at 15:50 +0900, Moon-Sang Lee wrote: > > > > > > Laurent, have you resolved this problem? > > I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced > > the same problem as you. > > Here is parts of my log and it says that PMD cannot enable RX queue > > for my NIC. > > I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling > > rte_eth_rx_queue_setup(). > > (i.e. 'null' parameter provides the default rx_conf value.) > > Hi, > > I had to reuse my DPDK machines for another task, I will go back to it > after FOSDEM. > > The error you get is the same as mine. > > Sincerely, > > Laurent > > > > > Thanks. > > > > > > > > > > > > APP: initialising port 0 ... > > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27258040 > > sw_sc_ring=0x7f5f27257b00 hw_ring=0x7f5f27258580 dma_addr=0x41f458580 > > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f27245940 > > hw_ring=0x7f5f27247980 dma_addr=0x41f447980 > > PMD: ixgbe_set_tx_function(): Using simple tx code path > > PMD: ixgbe_set_tx_function(): Vector tx enabled. > > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272337c0 > > hw_ring=0x7f5f27235800 dma_addr=0x41f435800 > > PMD: ixgbe_set_tx_function(): Using simple tx code path > > PMD: ixgbe_set_tx_function(): Vector tx enabled. > > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX > > burst size no less than 4 (port=0). > > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > > APP: port 0 has started > > APP: port 0 has entered in promiscuous mode > > APP: port 0 initialization is done. > > KNI: pci: 09:00:00 8086:10c7 > > APP: kni allocation is done for port 0. > > APP: initialising port 1 ... > > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27222dc0 > > sw_sc_ring=0x7f5f27222880 hw_ring=0x7f5f27223300 dma_addr=0x41f423300 > > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272106c0 > > hw_ring=0x7f5f27212700 dma_addr=0x41f412700 > > PMD: ixgbe_set_tx_function(): Using simple tx code path > > PMD: ixgbe_set_tx_function(): Vector tx enabled. > > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f271fe540 > > hw_ring=0x7f5f27200580 dma_addr=0x41f400580 > > PMD: ixgbe_set_tx_function(): Using simple tx code path > > PMD: ixgbe_set_tx_function(): Vector tx enabled. > > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX > > burst size no less than 4 (port=1). > > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 > > APP: port 1 has started > > APP: port 1 has entered in promiscuous mode > > APP: port 1 initialization is done. > > KNI: pci: 0a:00:00 8086:10c7 > > APP: kni allocation is done for port 1. > > > > > > checking link status > > .done > > Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed > > 10000 Mbps - full-duplex > > > > > > > > On Mon, Dec 28, 2015 at 5:28 AM, Wiles, Keith <keith.wiles@intel.com> > > wrote: > > On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net> > > wrote: > > > > >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: > > >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" > > <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: > > >> > > >> >Hi, > > >> > > > >> >I reported today an issue when using Pktgen-DPDK: > > >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 > > >> > > > >> >But I think it's more in DPDK than pktgen > > >> > > > >> >two identical machines with SFP+ DA cable between them > > >> >DPDK 2.2.0 from tarball > > >> >Pktgen-DPDK from git > > >> >two identical machines: > > >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS > > >> >ASUS P8H67-M PRO BIOS 3904 (latest available) > > >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit > > AF Dual Port > > >> >Network Connection (rev 01) > > >> >01:00.0 0200: 8086:10f1 (rev 01) > > >> > Subsystem: 8086:a21f > > >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 > > >> > > > >> >When launching pktgen even with no TX asked the Errors RX > > counters keeps > > >> >going up by about 7.4 millions per second: > > >> > > > >> >Errors Rx/Tx : 7471857054/0 > > >> > > > >> >In the log I get "Could not enable Rx Queue", might be the > > >> >source of the issue? > > >> > > > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue > > 0 > > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue > > 1 > > >> > > > >> >When sending traffic single UDP src/dst/IP/MAC the setup > > >> >reaches 14204188 pps 64 bytes, the error counter is also > > >> >increasing. > > >> > > > >> >Any idea what to look for? > > >> > > >> One more suggestion is to run test_pmd on one machine and > > something > > >> like iperf on the other to verify the DPDK is working > > correct, which I > > >> assume will be true. Not sure the RX errors are reported in > > the > > >> test_pmd or you could use the l3fwd application too. > > > > > >Ok, I will check the test_pmd documentation and try to do > > this test: I'm > > >just starting on DPDK :). > > > > > >> Please also send me the 'lspci | grep Ethernet’ output. > > > > > >I included one line in my original email above (plus extract > > of lspci > > >-vn), here is the full output of the command: > > > > > >01:00.0 Ethernet controller: Intel Corporation 82598EB > > 10-Gigabit AF > > >Dual Port Network Connection (rev 01) > > >01:00.1 Ethernet controller: Intel Corporation 82598EB > > 10-Gigabit AF > > >Dual Port Network Connection (rev 01) > > >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. > > >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller > > (rev 06) > > > > > >(The realtek is used only for internet connectivity). > > > > > >> Also send me the command line. > > > > > >On the first machine t1 : > > >root@t1:~/pktgen-dpdk# > ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N > > > > > >And on the other machine t3: > > >root@t3:~/pktgen-dpdk# > ./app/app/x86_64-native-linuxapp-gcc/pktgen \ > > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N > > > > > > You need to always start with port numbering for Pktgen with > > zero. > > > > Change the [2:3].1 to [2:3].0 the reason is you removed one of > > the two ports and Pktgen starts port counting from zero for > > the first available port :-) > > > > Sorry, I did not spot that sooner. > > > > > >The two "-f" pkt files are attached to this email, I do > > "start 1" > > >manually at the pktgen prompt. > > > > > >Thanks for your time, > > > > > >Sincerely, > > > > > >Laurent > > > > > > > > >> > > > >> >Laurent > > >> > > > >> > > > >> > > >> > > >> Regards, > > >> Keith > > >> > > >> > > >> > > >> > > > > > > > > > > > > Regards, > > Keith > > > > > > > > > > > > > > > > > > -- > > Moon-Sang Lee, SW Engineer > > Email: sang0627@gmail.com > > Wisdom begins in wonder. *Socrates* > > > -- Moon-Sang Lee, SW Engineer Email: sang0627@gmail.com Wisdom begins in wonder. *Socrates* ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2016-01-28 0:57 ` Moon-Sang Lee @ 2016-01-29 10:02 ` Moon-Sang Lee 0 siblings, 0 replies; 11+ messages in thread From: Moon-Sang Lee @ 2016-01-29 10:02 UTC (permalink / raw) To: Zhang, Helin; +Cc: dev Let me share one more information about this issue. I tried to run dpdk-2.2.0/examples/ethtool, which showed the same problem. (i.e. I guess other examples will also fail.) [root@centos7 app]# ./ethtool EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 1 EAL: Detected lcore 2 as core 1 on socket 0 EAL: Detected lcore 3 as core 1 on socket 1 EAL: Detected lcore 4 as core 2 on socket 0 EAL: Detected lcore 5 as core 2 on socket 1 EAL: Detected lcore 6 as core 3 on socket 0 EAL: Detected lcore 7 as core 3 on socket 1 EAL: Detected lcore 8 as core 0 on socket 0 EAL: Detected lcore 9 as core 0 on socket 1 EAL: Detected lcore 10 as core 1 on socket 0 EAL: Detected lcore 11 as core 1 on socket 1 EAL: Detected lcore 12 as core 2 on socket 0 EAL: Detected lcore 13 as core 2 on socket 1 EAL: Detected lcore 14 as core 3 on socket 0 EAL: Detected lcore 15 as core 3 on socket 1 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 16 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up physically contiguous memory... EAL: Ask a virtual area of 0x1400000 bytes EAL: Virtual area found at 0x7f79fa800000 (size = 0x1400000) EAL: Ask a virtual area of 0x7ec00000 bytes EAL: Virtual area found at 0x7f797ba00000 (size = 0x7ec00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f797b600000 (size = 0x200000) EAL: Ask a virtual area of 0x7fc00000 bytes EAL: Virtual area found at 0x7f78fb800000 (size = 0x7fc00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f78fb400000 (size = 0x200000) EAL: Ask a virtual area of 0x100000000 bytes EAL: Virtual area found at 0x7f77fb200000 (size = 0x100000000) EAL: Requesting 2048 pages of size 2MB from socket 0 EAL: Requesting 2048 pages of size 2MB from socket 1 EAL: TSC frequency is ~2260984 KHz EAL: Master lcore 0 is ready (tid=fdc578c0;cpuset=[0]) EAL: lcore 6 is ready (tid=f81cb700;cpuset=[6]) EAL: lcore 4 is ready (tid=f91cd700;cpuset=[4]) EAL: lcore 12 is ready (tid=f51c5700;cpuset=[12]) EAL: lcore 9 is ready (tid=f69c8700;cpuset=[9]) EAL: lcore 8 is ready (tid=f71c9700;cpuset=[8]) EAL: lcore 14 is ready (tid=f41c3700;cpuset=[14]) EAL: lcore 3 is ready (tid=f99ce700;cpuset=[3]) EAL: lcore 13 is ready (tid=f49c4700;cpuset=[13]) EAL: lcore 7 is ready (tid=f79ca700;cpuset=[7]) EAL: lcore 10 is ready (tid=f61c7700;cpuset=[10]) EAL: lcore 11 is ready (tid=f59c6700;cpuset=[11]) EAL: lcore 15 is ready (tid=f39c2700;cpuset=[15]) EAL: lcore 5 is ready (tid=f89cc700;cpuset=[5]) EAL: lcore 2 is ready (tid=fa1cf700;cpuset=[2]) EAL: lcore 1 is ready (tid=fc562700;cpuset=[1]) EAL: PCI device 0000:06:00.0 on NUMA socket -1 EAL: probe driver: 8086:10e8 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:06:00.1 on NUMA socket -1 EAL: probe driver: 8086:10e8 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:07:00.0 on NUMA socket -1 EAL: probe driver: 8086:10e8 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:07:00.1 on NUMA socket -1 EAL: probe driver: 8086:10e8 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:09:00.0 on NUMA socket -1 EAL: probe driver: 8086:10c7 rte_ixgbe_pmd EAL: PCI memory mapped at 0x7f79fbc00000 EAL: PCI memory mapped at 0x7f79fbc20000 EAL: PCI memory mapped at 0x7f79fbc60000 PMD: eth_ixgbe_dev_init(): MAC: 1, PHY: 8 PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x10c7 EAL: PCI device 0000:0a:00.0 on NUMA socket -1 EAL: probe driver: 8086:10c7 rte_ixgbe_pmd EAL: PCI memory mapped at 0x7f79fbc64000 EAL: PCI memory mapped at 0x7f79fbc84000 EAL: PCI memory mapped at 0x7f79fbcc4000 PMD: eth_ixgbe_dev_init(): MAC: 1, PHY: 8 PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10c7 Number of NICs: 2 Init port 0.. PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f78f99d7140 sw_sc_ring=0x7f78f99d6c00 hw_ring=0x7f78f99d7680 dma_addr=0x4207d7680 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f78f99c5a40 hw_ring=0x7f78f99c6a80 dma_addr=0x4207c6a80 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=0). PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 Init port 1.. PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f78f846ad40 sw_sc_ring=0x7f78f846a800 hw_ring=0x7f78f846b280 dma_addr=0x41f26b280 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f78f8459640 hw_ring=0x7f78f845a680 dma_addr=0x41f25a680 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=1). PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 EthApp> pause [UINT16]: pause <port_id> Print port pause state [UINT16]: pause <port_id> <all|tx|rx|none> Pause/unpause port EthApp> pause 0 Port 0: Rx Paused EthApp> pause 0 none PMD: ixgbe_flow_ctrl_set(): Rx packet buffer size = 0x80000 Port 0: Tx & Rx not paused EthApp> pause 0 Port 0: Rx Paused EthApp> quit [root@centos7 app]# pwd On Thu, Jan 28, 2016 at 9:57 AM, Moon-Sang Lee <sang0627@gmail.com> wrote: > > Helin, I implemented my own sample application that is a kind of carrier > grade NAT server. > It works fine on 1G NIC (i.e. Intel Corporation 82576 Gigabit Network > Connection (rev 01)) > But, it does not receive packets on 10G NIC (i.e. Intel Corporation > 82598EB 10-Gigabit AF Network Connection (rev 01)) as described in the > previous email. > According to my log messages, it seems that control register for RX DMA is > not enabled. > > Here is some information about my environment. > > 1. HW & OS > [mslee@centos7 ~]$ uname -a > Linux centos7 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 > x86_64 x86_64 x86_64 GNU/Linux > [mslee@centos7 ~]$ more /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 26 > model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz > stepping : 5 > microcode : 0x19 > cpu MHz : 2262.000 > cache size : 8192 KB > physical id : 1 > siblings : 8 > core id : 0 > cpu cores : 4 > apicid : 16 > initial apicid : 16 > fpu : yes > fpu_exception : yes > cpuid level : 11 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov > pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx > rdtscp lm c > onstant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc > aperfmperf > pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 > sse4_2 po > pcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid > bogomips : 4521.93 > clflush size : 64 > cache_alignment : 64 > address sizes : 40 bits physical, 48 bits virtual > power management: > ... > > > > 2. port ocnfigure parameter for rte_eth_dev_configure(): > ret = rte_eth_dev_configure(port, NB_RXQ, NB_TXQ, &port_conf); > where NB_RXQ=1, NB_TXQ=2, and > struct rte_eth_conf port_conf = { > .rxmode = { > //.mq_mode = ETH_MQ_RX_RSS, > .mq_mode = ETH_MQ_RX_NONE, // for 10G NIC > .max_rx_pkt_len = ETHER_MAX_LEN, > .split_hdr_size = 0, > .header_split = 0, // Header Split disabled > .hw_ip_checksum = 0, // IP checksum offload enabled > .hw_vlan_filter = 0, // VLAN filtering disabled > .jumbo_frame = 0, // Jumbo Frame Support disabled > .hw_strip_crc = 0, // CRC stripped by hardware > }, > .rx_adv_conf = { > .rss_conf = { > .rss_key = NULL, > .rss_hf = ETH_RSS_IP, > }, > }, > .txmode = { > .mq_mode = ETH_MQ_TX_NONE, > }, > }; > > > > 3. rx queue setup parameter > ret = rte_eth_rx_queue_setup(port, RXQ_ID, NB_RXD, socket_id, NULL, > pktmbuf_pool[socket_id]) > where RXQ_ID = 0, NB_RXD = 128 > > > > 4. config parameters in config/common_linuxapp > # > # Compile burst-oriented IXGBE PMD driver > # > CONFIG_RTE_LIBRTE_IXGBE_PMD=y > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_INIT=n > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX=n > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX=n > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX_FREE=n > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n > CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n > CONFIG_RTE_IXGBE_INC_VECTOR=y > CONFIG_RTE_IXGBE_RX_OLFLAGS_ENABLE=y > > > > 5. where log message is printed > > dpdk-2.2.0/drivers/net/ixgbe/ixgbe_rxtx.c: > > /* Allocate buffers for descriptor rings */ > if (ixgbe_alloc_rx_queue_mbufs(rxq) != 0) { > PMD_INIT_LOG(ERR, "Could not alloc mbuf for queue:%d", > rx_queue_id); > return -1; > } > rxdctl = IXGBE_READ_REG(hw, IXGBE_RXDCTL(rxq->reg_idx)); > rxdctl |= IXGBE_RXDCTL_ENABLE; > IXGBE_WRITE_REG(hw, IXGBE_RXDCTL(rxq->reg_idx), rxdctl); > > /* Wait until RX Enable ready */ > poll_ms = RTE_IXGBE_REGISTER_POLL_WAIT_10_MS; > do { > rte_delay_ms(1); > rxdctl = IXGBE_READ_REG(hw, IXGBE_RXDCTL(rxq->reg_idx)); > } while (--poll_ms && !(rxdctl & IXGBE_RXDCTL_ENABLE)); > if (!poll_ms) > PMD_INIT_LOG(ERR, "*Could not enable Rx Queue %d*", > rx_queue_id); > > > I'm going to update firmware of my NIC, but I'm not sure it helps. > I appreciate any comment. > > > > On Wed, Jan 27, 2016 at 4:23 PM, Zhang, Helin <helin.zhang@intel.com> > wrote: > >> Moon-Sang >> >> Were you using pktgen or else application? >> Could you help to share with me the detailed steps of your reproducing >> that issue? >> We will find time on that soon later. Thanks! >> >> Regards, >> Helin >> >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Laurent GUERBY >> Sent: Wednesday, January 27, 2016 3:16 PM >> To: Moon-Sang Lee <sang0627@gmail.com> >> Cc: dev@dpdk.org >> Subject: Re: [dpdk-dev] Errors Rx count increasing while pktgen doing >> nothing on Intel 82598EB 10G >> >> On Wed, 2016-01-27 at 15:50 +0900, Moon-Sang Lee wrote: >> > >> > >> > Laurent, have you resolved this problem? >> > I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced >> > the same problem as you. >> > Here is parts of my log and it says that PMD cannot enable RX queue >> > for my NIC. >> > I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling >> > rte_eth_rx_queue_setup(). >> > (i.e. 'null' parameter provides the default rx_conf value.) >> >> Hi, >> >> I had to reuse my DPDK machines for another task, I will go back to it >> after FOSDEM. >> >> The error you get is the same as mine. >> >> Sincerely, >> >> Laurent >> >> > >> > Thanks. >> > >> > >> > >> > >> > >> > APP: initialising port 0 ... >> > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27258040 >> > sw_sc_ring=0x7f5f27257b00 hw_ring=0x7f5f27258580 dma_addr=0x41f458580 >> > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f27245940 >> > hw_ring=0x7f5f27247980 dma_addr=0x41f447980 >> > PMD: ixgbe_set_tx_function(): Using simple tx code path >> > PMD: ixgbe_set_tx_function(): Vector tx enabled. >> > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272337c0 >> > hw_ring=0x7f5f27235800 dma_addr=0x41f435800 >> > PMD: ixgbe_set_tx_function(): Using simple tx code path >> > PMD: ixgbe_set_tx_function(): Vector tx enabled. >> > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX >> > burst size no less than 4 (port=0). >> > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 >> > APP: port 0 has started >> > APP: port 0 has entered in promiscuous mode >> > APP: port 0 initialization is done. >> > KNI: pci: 09:00:00 8086:10c7 >> > APP: kni allocation is done for port 0. >> > APP: initialising port 1 ... >> > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27222dc0 >> > sw_sc_ring=0x7f5f27222880 hw_ring=0x7f5f27223300 dma_addr=0x41f423300 >> > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272106c0 >> > hw_ring=0x7f5f27212700 dma_addr=0x41f412700 >> > PMD: ixgbe_set_tx_function(): Using simple tx code path >> > PMD: ixgbe_set_tx_function(): Vector tx enabled. >> > PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f271fe540 >> > hw_ring=0x7f5f27200580 dma_addr=0x41f400580 >> > PMD: ixgbe_set_tx_function(): Using simple tx code path >> > PMD: ixgbe_set_tx_function(): Vector tx enabled. >> > PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX >> > burst size no less than 4 (port=1). >> > PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 >> > APP: port 1 has started >> > APP: port 1 has entered in promiscuous mode >> > APP: port 1 initialization is done. >> > KNI: pci: 0a:00:00 8086:10c7 >> > APP: kni allocation is done for port 1. >> > >> > >> > checking link status >> > .done >> > Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed >> > 10000 Mbps - full-duplex >> > >> > >> > >> > On Mon, Dec 28, 2015 at 5:28 AM, Wiles, Keith <keith.wiles@intel.com> >> > wrote: >> > On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net> >> > wrote: >> > >> > >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: >> > >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" >> > <dev-bounces@dpdk.org on behalf of laurent@guerby.net> wrote: >> > >> >> > >> >Hi, >> > >> > >> > >> >I reported today an issue when using Pktgen-DPDK: >> > >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 >> > >> > >> > >> >But I think it's more in DPDK than pktgen >> > >> > >> > >> >two identical machines with SFP+ DA cable between them >> > >> >DPDK 2.2.0 from tarball >> > >> >Pktgen-DPDK from git >> > >> >two identical machines: >> > >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS >> > >> >ASUS P8H67-M PRO BIOS 3904 (latest available) >> > >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit >> > AF Dual Port >> > >> >Network Connection (rev 01) >> > >> >01:00.0 0200: 8086:10f1 (rev 01) >> > >> > Subsystem: 8086:a21f >> > >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 >> > >> > >> > >> >When launching pktgen even with no TX asked the Errors RX >> > counters keeps >> > >> >going up by about 7.4 millions per second: >> > >> > >> > >> >Errors Rx/Tx : 7471857054/0 >> > >> > >> > >> >In the log I get "Could not enable Rx Queue", might be the >> > >> >source of the issue? >> > >> > >> > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue >> > 0 >> > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue >> > 1 >> > >> > >> > >> >When sending traffic single UDP src/dst/IP/MAC the setup >> > >> >reaches 14204188 pps 64 bytes, the error counter is also >> > >> >increasing. >> > >> > >> > >> >Any idea what to look for? >> > >> >> > >> One more suggestion is to run test_pmd on one machine and >> > something >> > >> like iperf on the other to verify the DPDK is working >> > correct, which I >> > >> assume will be true. Not sure the RX errors are reported in >> > the >> > >> test_pmd or you could use the l3fwd application too. >> > > >> > >Ok, I will check the test_pmd documentation and try to do >> > this test: I'm >> > >just starting on DPDK :). >> > > >> > >> Please also send me the 'lspci | grep Ethernet’ output. >> > > >> > >I included one line in my original email above (plus extract >> > of lspci >> > >-vn), here is the full output of the command: >> > > >> > >01:00.0 Ethernet controller: Intel Corporation 82598EB >> > 10-Gigabit AF >> > >Dual Port Network Connection (rev 01) >> > >01:00.1 Ethernet controller: Intel Corporation 82598EB >> > 10-Gigabit AF >> > >Dual Port Network Connection (rev 01) >> > >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >> > >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller >> > (rev 06) >> > > >> > >(The realtek is used only for internet connectivity). >> > > >> > >> Also send me the command line. >> > > >> > >On the first machine t1 : >> > >root@t1:~/pktgen-dpdk# >> ./app/app/x86_64-native-linuxapp-gcc/pktgen \ >> > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N >> > > >> > >And on the other machine t3: >> > >root@t3:~/pktgen-dpdk# >> ./app/app/x86_64-native-linuxapp-gcc/pktgen \ >> > >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N >> > >> > >> > You need to always start with port numbering for Pktgen with >> > zero. >> > >> > Change the [2:3].1 to [2:3].0 the reason is you removed one of >> > the two ports and Pktgen starts port counting from zero for >> > the first available port :-) >> > >> > Sorry, I did not spot that sooner. >> > > >> > >The two "-f" pkt files are attached to this email, I do >> > "start 1" >> > >manually at the pktgen prompt. >> > > >> > >Thanks for your time, >> > > >> > >Sincerely, >> > > >> > >Laurent >> > > >> > > >> > >> > >> > >> >Laurent >> > >> > >> > >> > >> > >> >> > >> >> > >> Regards, >> > >> Keith >> > >> >> > >> >> > >> >> > >> >> > > >> > > >> > >> > >> > Regards, >> > Keith >> > >> > >> > >> > >> > >> > >> > >> > >> > -- >> > Moon-Sang Lee, SW Engineer >> > Email: sang0627@gmail.com >> > Wisdom begins in wonder. *Socrates* >> >> >> > > > -- > Moon-Sang Lee, SW Engineer > Email: sang0627@gmail.com > Wisdom begins in wonder. *Socrates* > -- Moon-Sang Lee, SW Engineer Email: sang0627@gmail.com Wisdom begins in wonder. *Socrates* ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2016-01-27 6:50 ` Moon-Sang Lee 2016-01-27 7:15 ` Laurent GUERBY @ 2016-01-30 1:23 ` Wiles, Keith 2016-01-30 1:55 ` Wiles, Keith 1 sibling, 1 reply; 11+ messages in thread From: Wiles, Keith @ 2016-01-30 1:23 UTC (permalink / raw) To: Moon-Sang Lee; +Cc: dev Regards, Keith From: Moon-Sang Lee <sang0627@gmail.com<mailto:sang0627@gmail.com>> Date: Wednesday, January 27, 2016 at 1:50 AM To: "Keith Wiles (Intel)" <keith.wiles@intel.com<mailto:keith.wiles@intel.com>> Cc: Laurent GUERBY <laurent@guerby.net<mailto:laurent@guerby.net>>, "dev@dpdk.org<mailto:dev@dpdk.org>" <dev@dpdk.org<mailto:dev@dpdk.org>> Subject: Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G Laurent, have you resolved this problem? I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced the same problem as you. Here is parts of my log and it says that PMD cannot enable RX queue for my NIC. I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling rte_eth_rx_queue_setup(). (i.e. 'null' parameter provides the default rx_conf value.) Thanks. The problem I found with the increasing RX errors is because a stats register was being read that does not exist in that given chipset and removing that register read would eliminate the false errors you were seeing. Here is part of the email I sent to Laurent. Hi Laurent, Well, it appears the problem is on the ixgbe driver the fccrc and fclast hw stats are not supported on 82598 only the 82599+ devices. On t3 you can look at the ixgbe_ethdev.c code I modified to remove these two stats, which caused ierrors to increase. You can also revert any changes in Pktgen to master as they did not help fix any thing. Sorry is it 6 am and I need to sleep some, I could not sleep tonight, so you I was able to debug the problem. Please remove the debug code in the ixgbe driver if you like as well and move that change to t1 machine. I will report this to the list later. The performance of pktgen/DPDK was not being effected by these counters. Thanks ++Keith APP: initialising port 0 ... PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27258040 sw_sc_ring=0x7f5f27257b00 hw_ring=0x7f5f27258580 dma_addr=0x41f458580 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f27245940 hw_ring=0x7f5f27247980 dma_addr=0x41f447980 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272337c0 hw_ring=0x7f5f27235800 dma_addr=0x41f435800 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=0). PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 APP: port 0 has started APP: port 0 has entered in promiscuous mode APP: port 0 initialization is done. KNI: pci: 09:00:00 8086:10c7 APP: kni allocation is done for port 0. APP: initialising port 1 ... PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5f27222dc0 sw_sc_ring=0x7f5f27222880 hw_ring=0x7f5f27223300 dma_addr=0x41f423300 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f272106c0 hw_ring=0x7f5f27212700 dma_addr=0x41f412700 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5f271fe540 hw_ring=0x7f5f27200580 dma_addr=0x41f400580 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 4 (port=1). PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 APP: port 1 has started APP: port 1 has entered in promiscuous mode APP: port 1 initialization is done. KNI: pci: 0a:00:00 8086:10c7 APP: kni allocation is done for port 1. checking link status .done Port 0 Link Up - speed 10000 Mbps - full-duplex Port 1 Link Up - speed 10000 Mbps - full-duplex On Mon, Dec 28, 2015 at 5:28 AM, Wiles, Keith <keith.wiles@intel.com<mailto:keith.wiles@intel.com>> wrote: On 12/27/15, 2:09 PM, "Laurent GUERBY" <laurent@guerby.net<mailto:laurent@guerby.net>> wrote: >On Sun, 2015-12-27 at 19:43 +0000, Wiles, Keith wrote: >> On 12/27/15, 12:31 PM, "dev on behalf of Laurent GUERBY" <dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org> on behalf of laurent@guerby.net<mailto:laurent@guerby.net>> wrote: >> >> >Hi, >> > >> >I reported today an issue when using Pktgen-DPDK: >> >https://github.com/pktgen/Pktgen-DPDK/issues/52 >> > >> >But I think it's more in DPDK than pktgen >> > >> >two identical machines with SFP+ DA cable between them >> >DPDK 2.2.0 from tarball >> >Pktgen-DPDK from git >> >two identical machines: >> >core i7 2600 (sandy bridge 4C/8T), HT disabled in the BIOS >> >ASUS P8H67-M PRO BIOS 3904 (latest available) >> >Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF Dual Port >> >Network Connection (rev 01) >> >01:00.0 0200: 8086:10f1 (rev 01) >> > Subsystem: 8086:a21f >> >boot kernel 3.16 unbutu 14.04 with isolcpus=2,3,4 >> > >> >When launching pktgen even with no TX asked the Errors RX counters keeps >> >going up by about 7.4 millions per second: >> > >> >Errors Rx/Tx : 7471857054/0 >> > >> >In the log I get "Could not enable Rx Queue", might be the >> >source of the issue? >> > >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 0 >> >PMD: ixgbe_dev_rx_queue_start(): Could not enable Rx Queue 1 >> > >> >When sending traffic single UDP src/dst/IP/MAC the setup >> >reaches 14204188 pps 64 bytes, the error counter is also >> >increasing. >> > >> >Any idea what to look for? >> >> One more suggestion is to run test_pmd on one machine and something >> like iperf on the other to verify the DPDK is working correct, which I >> assume will be true. Not sure the RX errors are reported in the >> test_pmd or you could use the l3fwd application too. > >Ok, I will check the test_pmd documentation and try to do this test: I'm >just starting on DPDK :). > >> Please also send me the 'lspci | grep Ethernet’ output. > >I included one line in my original email above (plus extract of lspci >-vn), here is the full output of the command: > >01:00.0 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF >Dual Port Network Connection (rev 01) >01:00.1 Ethernet controller: Intel Corporation 82598EB 10-Gigabit AF >Dual Port Network Connection (rev 01) >05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. >RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06) > >(The realtek is used only for internet connectivity). > >> Also send me the command line. > >On the first machine t1 : >root@t1:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t1-t3.pkt -N > >And on the other machine t3: >root@t3:~/pktgen-dpdk# ./app/app/x86_64-native-linuxapp-gcc/pktgen \ >-c e -n 1 --proc-type auto -- -m '[2:3].1' -P -f t3-t1.pkt -N You need to always start with port numbering for Pktgen with zero. Change the [2:3].1 to [2:3].0 the reason is you removed one of the two ports and Pktgen starts port counting from zero for the first available port :-) Sorry, I did not spot that sooner. > >The two "-f" pkt files are attached to this email, I do "start 1" >manually at the pktgen prompt. > >Thanks for your time, > >Sincerely, > >Laurent > > >> > >> >Laurent >> > >> > >> >> >> Regards, >> Keith >> >> >> >> > > Regards, Keith -- Moon-Sang Lee, SW Engineer Email: sang0627@gmail.com<mailto:sang0627@gmail.com> Wisdom begins in wonder. *Socrates* ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G 2016-01-30 1:23 ` Wiles, Keith @ 2016-01-30 1:55 ` Wiles, Keith 0 siblings, 0 replies; 11+ messages in thread From: Wiles, Keith @ 2016-01-30 1:55 UTC (permalink / raw) To: Moon-Sang Lee; +Cc: dev > >From: Moon-Sang Lee <sang0627@gmail.com<mailto:sang0627@gmail.com>> >Date: Wednesday, January 27, 2016 at 1:50 AM >To: "Keith Wiles (Intel)" <keith.wiles@intel.com<mailto:keith.wiles@intel.com>> >Cc: Laurent GUERBY <laurent@guerby.net<mailto:laurent@guerby.net>>, "dev@dpdk.org<mailto:dev@dpdk.org>" <dev@dpdk.org<mailto:dev@dpdk.org>> >Subject: Re: [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G > > >Laurent, have you resolved this problem? >I'm using the same NIC as yours (i.e. Intel 82598EB 10G NIC) and faced the same problem as you. >Here is parts of my log and it says that PMD cannot enable RX queue for my NIC. >I'm using DPDK 2.2.0 and used 'null' for the 4th parameter in calling rte_eth_rx_queue_setup(). >(i.e. 'null' parameter provides the default rx_conf value.) > >Thanks. > >The problem I found with the increasing RX errors is because a stats register was being read that does not exist in that given chipset and removing that register read would eliminate the false errors you were seeing. Here is part of the email I sent to Laurent. The problem I found with the increasing RX errors is because a stats register was being read that does not exist in that given chipset and removing that register read would eliminate the false errors you were seeing. Here is part of the email I sent to Laurent. Hi Laurent, Well, it appears the problem is on the ixgbe driver the fccrc and fclast hw stats are not supported on 82598 only the 82599+ devices. On t3 you can look at the ixgbe_ethdev.c code I modified to remove these two stats, which caused ierrors to increase. You can also revert any changes in Pktgen to master as they did not help fix any thing. Sorry is it 6 am and I need to sleep some, I could not sleep tonight, so you I was able to debug the problem. Please remove the debug code in the ixgbe driver if you like as well and move that change to t1 machine. I will report this to the list later. The performance of pktgen/DPDK was not being effected by these counters. Thanks ++Keith ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-01-30 1:55 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-12-27 18:31 [dpdk-dev] Errors Rx count increasing while pktgen doing nothing on Intel 82598EB 10G Laurent GUERBY 2015-12-27 19:43 ` Wiles, Keith 2015-12-27 20:09 ` Laurent GUERBY 2015-12-27 20:28 ` Wiles, Keith 2016-01-27 6:50 ` Moon-Sang Lee 2016-01-27 7:15 ` Laurent GUERBY 2016-01-27 7:23 ` Zhang, Helin 2016-01-28 0:57 ` Moon-Sang Lee 2016-01-29 10:02 ` Moon-Sang Lee 2016-01-30 1:23 ` Wiles, Keith 2016-01-30 1:55 ` Wiles, Keith
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).