* [dpdk-dev] virtio pmd failed in pci probing @ 2016-05-12 16:34 Vincent Li 2016-05-12 20:10 ` Vincent Li 0 siblings, 1 reply; 3+ messages in thread From: Vincent Li @ 2016-05-12 16:34 UTC (permalink / raw) To: dev Hi I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04 on KVM guest and it appears the virtio pmd fail to load, detail info below: # ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ 0000:00:07.0 'Virtio network device' drv=igb_uio unused= Network devices using kernel driver =================================== 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio 0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it easy to test for me): sprintf(whitelist, "%s", "0000:00:07.0"); /* initialize the rte env first, what a waste of implementation effort! */ char *argv[] = {"", "-c", cpumaskbuf, "-w", whitelist, "-n", mem_channels, "--proc-type=auto", "" }; const int argc = 8; /* * re-set getopt extern variable optind. * this issue was a bitch to debug * rte_eal_init() internally uses getopt() syscall * mtcp applications that also use an `external' getopt * will cause a violent crash if optind is not reset to zero * prior to calling the func below... * see man getopt(3) for more details */ optind = 0; /* initialize the dpdk eal env */ ret = rte_eal_init(argc, argv); if (ret < 0) rte_exit(EXIT_FAILURE, "Invalid EAL args!\n"); /* give me the count of 'detected' ethernet ports */ num_devices = rte_eth_dev_count(); if (num_devices == 0) { rte_exit(EXIT_FAILURE, "No Ethernet port!\n"); } in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c rte_eal_pci_probe_one_driver, I added debug line below: /* * If vendor/device ID match, call the devinit() function of the * driver. */ static int rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct rte_pci_device *dev) { int ret; const struct rte_pci_id *id_table; RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id dr->name %x:%x %s\n", dev->id.vendor_id, dev->id.device_id, dr->name); when I run mTCP app as below, it says "No Ethernet port!", please note that the debug line did not show rte_virtio_pmd, why? is it because virtio pmd lack of vendor_id/device_id implementation? EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0]) EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted EAL: Error - exiting with code: 1 Cause: No Ethernet port! dpdk testpmd app detects the ethernet port fine, here is the testpmd output (Note here the debug line shows rte_virtio_pmd): ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0]) EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3]) EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2]) EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1]) EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_virtio_pmd EAL: PCI device 0000:00:07.0 on NUMA socket -1 EAL: probe driver: 1af4:1000 rte_virtio_pmd EAL: PCI memory mapped at 0x7fabe8a00000 I am able to workaround this by compiling dpdk as shared library for mTCP and use '-d' to load librte_pmd_virtio.so explicitly. it looks to me there isn't much difference between how testpmd and mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and rte_eth_dev_count() is there any potential bug in the implementation of virtio pmd in pci probing ? Thanks! ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] virtio pmd failed in pci probing 2016-05-12 16:34 [dpdk-dev] virtio pmd failed in pci probing Vincent Li @ 2016-05-12 20:10 ` Vincent Li 2016-05-13 17:18 ` Vincent Li 0 siblings, 1 reply; 3+ messages in thread From: Vincent Li @ 2016-05-12 20:10 UTC (permalink / raw) To: dev I add a debug log line in dpdk-16.04/lib/librte_ether/rte_ethdev.c rte_eth_driver_register void rte_eth_driver_register(struct eth_driver *eth_drv) { eth_drv->pci_drv.devinit = rte_eth_dev_init; eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; rte_eal_pci_register(ð_drv->pci_drv); RTE_LOG(DEBUG, EAL, " register pmd driver %s\n", eth_drv->pci_drv.name); } then run the mTCP app, rte_virtio_pmd is missing: EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: Master lcore 0 is ready (tid=6ec31900;cpuset=[0]) EAL: register pmd driver rte_vmxnet3_pmd EAL: register pmd driver rte_i40e_pmd EAL: register pmd driver rte_i40evf_pmd EAL: register pmd driver rte_ixgbe_pmd EAL: register pmd driver rte_ixgbevf_pmd EAL: register pmd driver rte_igb_pmd EAL: register pmd driver rte_igbvf_pmd EAL: register pmd driver rte_em_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted EAL: Error - exiting with code: 1 Cause: No Ethernet port! any clue? On Thu, May 12, 2016 at 9:34 AM, Vincent Li <vincent.mc.li@gmail.com> wrote: > Hi > > I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04 > on KVM guest and it appears the virtio pmd fail to load, detail info > below: > > > # ./tools/dpdk_nic_bind.py --status > > Network devices using DPDK-compatible driver > ============================================ > 0000:00:07.0 'Virtio network device' drv=igb_uio unused= > > Network devices using kernel driver > =================================== > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio > 0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio > > in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it > easy to test for me): > > > > sprintf(whitelist, "%s", "0000:00:07.0"); > /* initialize the rte env first, what a waste of > implementation effort! */ > char *argv[] = {"", > "-c", > cpumaskbuf, > "-w", > whitelist, > "-n", > mem_channels, > "--proc-type=auto", > "" > }; > const int argc = 8; > > /* > * re-set getopt extern variable optind. > * this issue was a bitch to debug > * rte_eal_init() internally uses getopt() syscall > * mtcp applications that also use an `external' getopt > * will cause a violent crash if optind is not reset to zero > * prior to calling the func below... > * see man getopt(3) for more details > */ > optind = 0; > > /* initialize the dpdk eal env */ > ret = rte_eal_init(argc, argv); > if (ret < 0) > rte_exit(EXIT_FAILURE, "Invalid EAL args!\n"); > /* give me the count of 'detected' ethernet ports */ > num_devices = rte_eth_dev_count(); > if (num_devices == 0) { > rte_exit(EXIT_FAILURE, "No Ethernet port!\n"); > } > > in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c > rte_eal_pci_probe_one_driver, I added debug line below: > > /* > * If vendor/device ID match, call the devinit() function of the > * driver. > */ > static int > rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct > rte_pci_device *dev) > { > int ret; > const struct rte_pci_id *id_table; > > RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id > dr->name %x:%x %s\n", dev->id.vendor_id, > dev->id.device_id, dr->name); > > > when I run mTCP app as below, it says "No Ethernet port!", please note > that the debug line did not show rte_virtio_pmd, why? is it because > virtio pmd lack of vendor_id/device_id implementation? > > > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > unreliable clock cycles ! > EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0]) > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd > pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted > EAL: Error - exiting with code: 1 > Cause: No Ethernet port! > > > dpdk testpmd app detects the ethernet port fine, here is the testpmd > output (Note here the debug line shows rte_virtio_pmd): > > ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i > > EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0]) > EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3]) > EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2]) > EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1]) > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 > rte_virtio_pmd > EAL: PCI device 0000:00:07.0 on NUMA socket -1 > EAL: probe driver: 1af4:1000 rte_virtio_pmd > EAL: PCI memory mapped at 0x7fabe8a00000 > > > I am able to workaround this by compiling dpdk as shared library for > mTCP and use '-d' to load librte_pmd_virtio.so explicitly. > > it looks to me there isn't much difference between how testpmd and > mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and > rte_eth_dev_count() > > is there any potential bug in the implementation of virtio pmd in pci probing ? > > Thanks! ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] virtio pmd failed in pci probing 2016-05-12 20:10 ` Vincent Li @ 2016-05-13 17:18 ` Vincent Li 0 siblings, 0 replies; 3+ messages in thread From: Vincent Li @ 2016-05-13 17:18 UTC (permalink / raw) To: dev sorry for the noise, it turned out I need to add the virtio pmd in the mTCP app Makefile as below LIBS += -m64 -g -O3 -pthread -lrt -march=native -Wl,-export-dynamic ${MTCP_FLD}/lib/libmtcp.a -L../../dpdk/lib -Wl,-lnuma -Wl,-lmtcp -Wl,-lpthread -Wl,-lrt -Wl,-ldl -Wl,--whole-archive -Wl,-lrte_distributor -Wl,-lrte_kni -Wl,-lrte_pipeline -Wl,-lrte_table -Wl,-lrte_port -Wl,-lrte_timer -Wl,-lrte_hash -Wl,-lrte_lpm -Wl,-lrte_power -Wl,-lrte_acl -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm -Wl,-lrt -Wl,--start-group -Wl,-lrte_kvargs -Wl,-lrte_mbuf -Wl,-lrte_ip_frag -Wl,-lethdev -Wl,-lrte_mempool -Wl,-lrte_ring -Wl,-lrte_eal -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond -Wl,-lrte_pmd_vmxnet3_uio -Wl,-lrte_pmd_i40e -Wl,-lrte_pmd_ixgbe -Wl,-lrte_pmd_e1000 -Wl,-lrte_pmd_virtio -Wl,-lrte_pmd_ring -Wl,-lrt -Wl,-lm -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive On Thu, May 12, 2016 at 1:10 PM, Vincent Li <vincent.mc.li@gmail.com> wrote: > I add a debug log line in dpdk-16.04/lib/librte_ether/rte_ethdev.c > rte_eth_driver_register > > void > rte_eth_driver_register(struct eth_driver *eth_drv) > { > eth_drv->pci_drv.devinit = rte_eth_dev_init; > eth_drv->pci_drv.devuninit = rte_eth_dev_uninit; > rte_eal_pci_register(ð_drv->pci_drv); > RTE_LOG(DEBUG, EAL, " register pmd driver %s\n", > eth_drv->pci_drv.name); > } > > then run the mTCP app, rte_virtio_pmd is missing: > > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > unreliable clock cycles ! > EAL: Master lcore 0 is ready (tid=6ec31900;cpuset=[0]) > EAL: register pmd driver rte_vmxnet3_pmd > EAL: register pmd driver rte_i40e_pmd > EAL: register pmd driver rte_i40evf_pmd > EAL: register pmd driver rte_ixgbe_pmd > EAL: register pmd driver rte_ixgbevf_pmd > EAL: register pmd driver rte_igb_pmd > EAL: register pmd driver rte_igbvf_pmd > EAL: register pmd driver rte_em_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd > EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd > pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted > EAL: Error - exiting with code: 1 > Cause: No Ethernet port! > > any clue? > > > On Thu, May 12, 2016 at 9:34 AM, Vincent Li <vincent.mc.li@gmail.com> wrote: >> Hi >> >> I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04 >> on KVM guest and it appears the virtio pmd fail to load, detail info >> below: >> >> >> # ./tools/dpdk_nic_bind.py --status >> >> Network devices using DPDK-compatible driver >> ============================================ >> 0000:00:07.0 'Virtio network device' drv=igb_uio unused= >> >> Network devices using kernel driver >> =================================== >> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio >> 0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio >> >> in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it >> easy to test for me): >> >> >> >> sprintf(whitelist, "%s", "0000:00:07.0"); >> /* initialize the rte env first, what a waste of >> implementation effort! */ >> char *argv[] = {"", >> "-c", >> cpumaskbuf, >> "-w", >> whitelist, >> "-n", >> mem_channels, >> "--proc-type=auto", >> "" >> }; >> const int argc = 8; >> >> /* >> * re-set getopt extern variable optind. >> * this issue was a bitch to debug >> * rte_eal_init() internally uses getopt() syscall >> * mtcp applications that also use an `external' getopt >> * will cause a violent crash if optind is not reset to zero >> * prior to calling the func below... >> * see man getopt(3) for more details >> */ >> optind = 0; >> >> /* initialize the dpdk eal env */ >> ret = rte_eal_init(argc, argv); >> if (ret < 0) >> rte_exit(EXIT_FAILURE, "Invalid EAL args!\n"); >> /* give me the count of 'detected' ethernet ports */ >> num_devices = rte_eth_dev_count(); >> if (num_devices == 0) { >> rte_exit(EXIT_FAILURE, "No Ethernet port!\n"); >> } >> >> in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c >> rte_eal_pci_probe_one_driver, I added debug line below: >> >> /* >> * If vendor/device ID match, call the devinit() function of the >> * driver. >> */ >> static int >> rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct >> rte_pci_device *dev) >> { >> int ret; >> const struct rte_pci_id *id_table; >> >> RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id >> dr->name %x:%x %s\n", dev->id.vendor_id, >> dev->id.device_id, dr->name); >> >> >> when I run mTCP app as below, it says "No Ethernet port!", please note >> that the debug line did not show rte_virtio_pmd, why? is it because >> virtio pmd lack of vendor_id/device_id implementation? >> >> >> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using >> unreliable clock cycles ! >> EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0]) >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd >> pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted >> EAL: Error - exiting with code: 1 >> Cause: No Ethernet port! >> >> >> dpdk testpmd app detects the ethernet port fine, here is the testpmd >> output (Note here the debug line shows rte_virtio_pmd): >> >> ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i >> >> EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0]) >> EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3]) >> EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2]) >> EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1]) >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd >> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 >> rte_virtio_pmd >> EAL: PCI device 0000:00:07.0 on NUMA socket -1 >> EAL: probe driver: 1af4:1000 rte_virtio_pmd >> EAL: PCI memory mapped at 0x7fabe8a00000 >> >> >> I am able to workaround this by compiling dpdk as shared library for >> mTCP and use '-d' to load librte_pmd_virtio.so explicitly. >> >> it looks to me there isn't much difference between how testpmd and >> mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and >> rte_eth_dev_count() >> >> is there any potential bug in the implementation of virtio pmd in pci probing ? >> >> Thanks! ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-05-13 17:18 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-05-12 16:34 [dpdk-dev] virtio pmd failed in pci probing Vincent Li 2016-05-12 20:10 ` Vincent Li 2016-05-13 17:18 ` Vincent Li
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).