* [dpdk-dev] virtio pmd failed in pci probing
@ 2016-05-12 16:34 Vincent Li
2016-05-12 20:10 ` Vincent Li
0 siblings, 1 reply; 3+ messages in thread
From: Vincent Li @ 2016-05-12 16:34 UTC (permalink / raw)
To: dev
Hi
I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04
on KVM guest and it appears the virtio pmd fail to load, detail info
below:
# ./tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
0000:00:07.0 'Virtio network device' drv=igb_uio unused=
Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it
easy to test for me):
sprintf(whitelist, "%s", "0000:00:07.0");
/* initialize the rte env first, what a waste of
implementation effort! */
char *argv[] = {"",
"-c",
cpumaskbuf,
"-w",
whitelist,
"-n",
mem_channels,
"--proc-type=auto",
""
};
const int argc = 8;
/*
* re-set getopt extern variable optind.
* this issue was a bitch to debug
* rte_eal_init() internally uses getopt() syscall
* mtcp applications that also use an `external' getopt
* will cause a violent crash if optind is not reset to zero
* prior to calling the func below...
* see man getopt(3) for more details
*/
optind = 0;
/* initialize the dpdk eal env */
ret = rte_eal_init(argc, argv);
if (ret < 0)
rte_exit(EXIT_FAILURE, "Invalid EAL args!\n");
/* give me the count of 'detected' ethernet ports */
num_devices = rte_eth_dev_count();
if (num_devices == 0) {
rte_exit(EXIT_FAILURE, "No Ethernet port!\n");
}
in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c
rte_eal_pci_probe_one_driver, I added debug line below:
/*
* If vendor/device ID match, call the devinit() function of the
* driver.
*/
static int
rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct
rte_pci_device *dev)
{
int ret;
const struct rte_pci_id *id_table;
RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id
dr->name %x:%x %s\n", dev->id.vendor_id,
dev->id.device_id, dr->name);
when I run mTCP app as below, it says "No Ethernet port!", please note
that the debug line did not show rte_virtio_pmd, why? is it because
virtio pmd lack of vendor_id/device_id implementation?
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
unreliable clock cycles !
EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0])
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd
pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted
EAL: Error - exiting with code: 1
Cause: No Ethernet port!
dpdk testpmd app detects the ethernet port fine, here is the testpmd
output (Note here the debug line shows rte_virtio_pmd):
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i
EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0])
EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3])
EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2])
EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1])
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000
rte_virtio_pmd
EAL: PCI device 0000:00:07.0 on NUMA socket -1
EAL: probe driver: 1af4:1000 rte_virtio_pmd
EAL: PCI memory mapped at 0x7fabe8a00000
I am able to workaround this by compiling dpdk as shared library for
mTCP and use '-d' to load librte_pmd_virtio.so explicitly.
it looks to me there isn't much difference between how testpmd and
mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and
rte_eth_dev_count()
is there any potential bug in the implementation of virtio pmd in pci probing ?
Thanks!
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] virtio pmd failed in pci probing
2016-05-12 16:34 [dpdk-dev] virtio pmd failed in pci probing Vincent Li
@ 2016-05-12 20:10 ` Vincent Li
2016-05-13 17:18 ` Vincent Li
0 siblings, 1 reply; 3+ messages in thread
From: Vincent Li @ 2016-05-12 20:10 UTC (permalink / raw)
To: dev
I add a debug log line in dpdk-16.04/lib/librte_ether/rte_ethdev.c
rte_eth_driver_register
void
rte_eth_driver_register(struct eth_driver *eth_drv)
{
eth_drv->pci_drv.devinit = rte_eth_dev_init;
eth_drv->pci_drv.devuninit = rte_eth_dev_uninit;
rte_eal_pci_register(ð_drv->pci_drv);
RTE_LOG(DEBUG, EAL, " register pmd driver %s\n",
eth_drv->pci_drv.name);
}
then run the mTCP app, rte_virtio_pmd is missing:
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
unreliable clock cycles !
EAL: Master lcore 0 is ready (tid=6ec31900;cpuset=[0])
EAL: register pmd driver rte_vmxnet3_pmd
EAL: register pmd driver rte_i40e_pmd
EAL: register pmd driver rte_i40evf_pmd
EAL: register pmd driver rte_ixgbe_pmd
EAL: register pmd driver rte_ixgbevf_pmd
EAL: register pmd driver rte_igb_pmd
EAL: register pmd driver rte_igbvf_pmd
EAL: register pmd driver rte_em_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd
EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd
pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted
EAL: Error - exiting with code: 1
Cause: No Ethernet port!
any clue?
On Thu, May 12, 2016 at 9:34 AM, Vincent Li <vincent.mc.li@gmail.com> wrote:
> Hi
>
> I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04
> on KVM guest and it appears the virtio pmd fail to load, detail info
> below:
>
>
> # ./tools/dpdk_nic_bind.py --status
>
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:07.0 'Virtio network device' drv=igb_uio unused=
>
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> 0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>
> in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it
> easy to test for me):
>
>
>
> sprintf(whitelist, "%s", "0000:00:07.0");
> /* initialize the rte env first, what a waste of
> implementation effort! */
> char *argv[] = {"",
> "-c",
> cpumaskbuf,
> "-w",
> whitelist,
> "-n",
> mem_channels,
> "--proc-type=auto",
> ""
> };
> const int argc = 8;
>
> /*
> * re-set getopt extern variable optind.
> * this issue was a bitch to debug
> * rte_eal_init() internally uses getopt() syscall
> * mtcp applications that also use an `external' getopt
> * will cause a violent crash if optind is not reset to zero
> * prior to calling the func below...
> * see man getopt(3) for more details
> */
> optind = 0;
>
> /* initialize the dpdk eal env */
> ret = rte_eal_init(argc, argv);
> if (ret < 0)
> rte_exit(EXIT_FAILURE, "Invalid EAL args!\n");
> /* give me the count of 'detected' ethernet ports */
> num_devices = rte_eth_dev_count();
> if (num_devices == 0) {
> rte_exit(EXIT_FAILURE, "No Ethernet port!\n");
> }
>
> in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c
> rte_eal_pci_probe_one_driver, I added debug line below:
>
> /*
> * If vendor/device ID match, call the devinit() function of the
> * driver.
> */
> static int
> rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct
> rte_pci_device *dev)
> {
> int ret;
> const struct rte_pci_id *id_table;
>
> RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id
> dr->name %x:%x %s\n", dev->id.vendor_id,
> dev->id.device_id, dr->name);
>
>
> when I run mTCP app as below, it says "No Ethernet port!", please note
> that the debug line did not show rte_virtio_pmd, why? is it because
> virtio pmd lack of vendor_id/device_id implementation?
>
>
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0])
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd
> pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted
> EAL: Error - exiting with code: 1
> Cause: No Ethernet port!
>
>
> dpdk testpmd app detects the ethernet port fine, here is the testpmd
> output (Note here the debug line shows rte_virtio_pmd):
>
> ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i
>
> EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0])
> EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3])
> EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2])
> EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1])
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000
> rte_virtio_pmd
> EAL: PCI device 0000:00:07.0 on NUMA socket -1
> EAL: probe driver: 1af4:1000 rte_virtio_pmd
> EAL: PCI memory mapped at 0x7fabe8a00000
>
>
> I am able to workaround this by compiling dpdk as shared library for
> mTCP and use '-d' to load librte_pmd_virtio.so explicitly.
>
> it looks to me there isn't much difference between how testpmd and
> mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and
> rte_eth_dev_count()
>
> is there any potential bug in the implementation of virtio pmd in pci probing ?
>
> Thanks!
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] virtio pmd failed in pci probing
2016-05-12 20:10 ` Vincent Li
@ 2016-05-13 17:18 ` Vincent Li
0 siblings, 0 replies; 3+ messages in thread
From: Vincent Li @ 2016-05-13 17:18 UTC (permalink / raw)
To: dev
sorry for the noise, it turned out I need to add the virtio pmd in the
mTCP app Makefile as below
LIBS += -m64 -g -O3 -pthread -lrt -march=native -Wl,-export-dynamic
${MTCP_FLD}/lib/libmtcp.a -L../../dpdk/lib -Wl,-lnuma -Wl,-lmtcp
-Wl,-lpthread -Wl,-lrt -Wl,-ldl -Wl,--whole-archive
-Wl,-lrte_distributor -Wl,-lrte_kni -Wl,-lrte_pipeline -Wl,-lrte_table
-Wl,-lrte_port -Wl,-lrte_timer -Wl,-lrte_hash -Wl,-lrte_lpm
-Wl,-lrte_power -Wl,-lrte_acl -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm
-Wl,-lrt -Wl,--start-group -Wl,-lrte_kvargs -Wl,-lrte_mbuf
-Wl,-lrte_ip_frag -Wl,-lethdev -Wl,-lrte_mempool -Wl,-lrte_ring
-Wl,-lrte_eal -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond
-Wl,-lrte_pmd_vmxnet3_uio -Wl,-lrte_pmd_i40e -Wl,-lrte_pmd_ixgbe
-Wl,-lrte_pmd_e1000 -Wl,-lrte_pmd_virtio -Wl,-lrte_pmd_ring -Wl,-lrt
-Wl,-lm -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive
On Thu, May 12, 2016 at 1:10 PM, Vincent Li <vincent.mc.li@gmail.com> wrote:
> I add a debug log line in dpdk-16.04/lib/librte_ether/rte_ethdev.c
> rte_eth_driver_register
>
> void
> rte_eth_driver_register(struct eth_driver *eth_drv)
> {
> eth_drv->pci_drv.devinit = rte_eth_dev_init;
> eth_drv->pci_drv.devuninit = rte_eth_dev_uninit;
> rte_eal_pci_register(ð_drv->pci_drv);
> RTE_LOG(DEBUG, EAL, " register pmd driver %s\n",
> eth_drv->pci_drv.name);
> }
>
> then run the mTCP app, rte_virtio_pmd is missing:
>
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: Master lcore 0 is ready (tid=6ec31900;cpuset=[0])
> EAL: register pmd driver rte_vmxnet3_pmd
> EAL: register pmd driver rte_i40e_pmd
> EAL: register pmd driver rte_i40evf_pmd
> EAL: register pmd driver rte_ixgbe_pmd
> EAL: register pmd driver rte_ixgbevf_pmd
> EAL: register pmd driver rte_igb_pmd
> EAL: register pmd driver rte_igbvf_pmd
> EAL: register pmd driver rte_em_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd
> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd
> pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted
> EAL: Error - exiting with code: 1
> Cause: No Ethernet port!
>
> any clue?
>
>
> On Thu, May 12, 2016 at 9:34 AM, Vincent Li <vincent.mc.li@gmail.com> wrote:
>> Hi
>>
>> I am testing mTCP https://github.com/eunyoung14/mtcp with dpdk-16.04
>> on KVM guest and it appears the virtio pmd fail to load, detail info
>> below:
>>
>>
>> # ./tools/dpdk_nic_bind.py --status
>>
>> Network devices using DPDK-compatible driver
>> ============================================
>> 0000:00:07.0 'Virtio network device' drv=igb_uio unused=
>>
>> Network devices using kernel driver
>> ===================================
>> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>> 0000:00:08.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>>
>> in mtcp/src/io_module.c, I have (I hard code the whiltelist to make it
>> easy to test for me):
>>
>>
>>
>> sprintf(whitelist, "%s", "0000:00:07.0");
>> /* initialize the rte env first, what a waste of
>> implementation effort! */
>> char *argv[] = {"",
>> "-c",
>> cpumaskbuf,
>> "-w",
>> whitelist,
>> "-n",
>> mem_channels,
>> "--proc-type=auto",
>> ""
>> };
>> const int argc = 8;
>>
>> /*
>> * re-set getopt extern variable optind.
>> * this issue was a bitch to debug
>> * rte_eal_init() internally uses getopt() syscall
>> * mtcp applications that also use an `external' getopt
>> * will cause a violent crash if optind is not reset to zero
>> * prior to calling the func below...
>> * see man getopt(3) for more details
>> */
>> optind = 0;
>>
>> /* initialize the dpdk eal env */
>> ret = rte_eal_init(argc, argv);
>> if (ret < 0)
>> rte_exit(EXIT_FAILURE, "Invalid EAL args!\n");
>> /* give me the count of 'detected' ethernet ports */
>> num_devices = rte_eth_dev_count();
>> if (num_devices == 0) {
>> rte_exit(EXIT_FAILURE, "No Ethernet port!\n");
>> }
>>
>> in dpdk-16.04/lib/librte_eal/common/eal_common_pci.c
>> rte_eal_pci_probe_one_driver, I added debug line below:
>>
>> /*
>> * If vendor/device ID match, call the devinit() function of the
>> * driver.
>> */
>> static int
>> rte_eal_pci_probe_one_driver(struct rte_pci_driver *dr, struct
>> rte_pci_device *dev)
>> {
>> int ret;
>> const struct rte_pci_id *id_table;
>>
>> RTE_LOG(DEBUG, EAL, " dev->id.vendor_id:dev->id.device_id
>> dr->name %x:%x %s\n", dev->id.vendor_id,
>> dev->id.device_id, dr->name);
>>
>>
>> when I run mTCP app as below, it says "No Ethernet port!", please note
>> that the debug line did not show rte_virtio_pmd, why? is it because
>> virtio pmd lack of vendor_id/device_id implementation?
>>
>>
>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>> unreliable clock cycles !
>> EAL: Master lcore 0 is ready (tid=dca0e900;cpuset=[0])
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40e_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_i40evf_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbe_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_ixgbevf_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igb_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_igbvf_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_em_pmd
>> pci_probe_all_drivers ret: 1 devargs not NULL and whitelisted
>> EAL: Error - exiting with code: 1
>> Cause: No Ethernet port!
>>
>>
>> dpdk testpmd app detects the ethernet port fine, here is the testpmd
>> output (Note here the debug line shows rte_virtio_pmd):
>>
>> ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:00:07.0 -- -i
>>
>> EAL: Master lcore 0 is ready (tid=e9c98900;cpuset=[0])
>> EAL: lcore 3 is ready (tid=a69f6700;cpuset=[3])
>> EAL: lcore 2 is ready (tid=a71f7700;cpuset=[2])
>> EAL: lcore 1 is ready (tid=a79f8700;cpuset=[1])
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000 rte_vmxnet3_pmd
>> EAL: dev->id.vendor_id:dev->id.device_id dr->name 1af4:1000
>> rte_virtio_pmd
>> EAL: PCI device 0000:00:07.0 on NUMA socket -1
>> EAL: probe driver: 1af4:1000 rte_virtio_pmd
>> EAL: PCI memory mapped at 0x7fabe8a00000
>>
>>
>> I am able to workaround this by compiling dpdk as shared library for
>> mTCP and use '-d' to load librte_pmd_virtio.so explicitly.
>>
>> it looks to me there isn't much difference between how testpmd and
>> mTCP invokes dpdk, it is straightforward rte_eal_init(argc, argv) and
>> rte_eth_dev_count()
>>
>> is there any potential bug in the implementation of virtio pmd in pci probing ?
>>
>> Thanks!
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-05-13 17:18 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-05-12 16:34 [dpdk-dev] virtio pmd failed in pci probing Vincent Li
2016-05-12 20:10 ` Vincent Li
2016-05-13 17:18 ` Vincent Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).