* Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment
[not found] <CAC1b25pX3R2y_Cjp5UAdL6Bozrf01OkG6U6LGQvJ2aUTDXGvSA@mail.gmail.com>
@ 2014-11-05 10:12 ` Bruce Richardson
2014-11-07 15:01 ` Manoj Viswanath
0 siblings, 1 reply; 6+ messages in thread
From: Bruce Richardson @ 2014-11-05 10:12 UTC (permalink / raw)
To: Manoj Viswanath; +Cc: dev
On Wed, Nov 05, 2014 at 03:28:13PM +0530, Manoj Viswanath wrote:
> Hi,
>
> I have a DPDK application running on QEMU-KVM environment using DPDK 1.6.0.
> I am trying to port the same to DPDK version 1.7.1.
>
> I am using Virt-manager GUI to assign e1000 emulated port to the VM. This
> works fine in DPDK 1.6.0. The device is identified by IGB_UIO and
> initialized by my application as expected.
>
> However in case of DPDK 1.7.1, the emulated e1000 devices do not seem to be
> recognized.
> Following is my analysis:
>
> 1. The API pci_get_uio_dev() is returning ERROR. This is called from
> pci_uio_map_resource() in the flow of PCI PROBE [rte_eal_pci_probe()].
>
> 2. Due to this, the PCI device is not getting mapped to the correct driver
> (EM
> Driver
> ).
>
> 3. The reason for the error in [1] appears to be that "uio" sub-directory
> doesn't seem to be correctly created for interfaces assigned to this VM.
>
> 4. Upon further analysis i found that IGB_UIO probe function
> ["igbuio_pci_probe()"] is not getting triggered indicating the port has *not
> been assigned* to the IGB_UIO.
>
> Kindly refer to the attachments:-
> - "Output of sys-bus-pci-devices" - indicating "uio" subdirectory not
> created for PCI devices in case of DPDK 1.7.1
> - "Output of lspci -v" - indicating device not bound to driver in case of
> DPDK 1.7.1
> - IGB_UIO init log snippet - indicating PCI devices not detected and
> initialized by IGB_UIO in case of DPDK 1.7.1
> - CONFIG file used for DPDK compilation
>
> Not sure what has changed between 1.6.0 and 1.7.1 which is impacting this.
>
> Could someone throw light in this regard as to what i may be missing ?
>
> Thanks in advance.
>
> Regards,
> Manoj
Hi Manoj,
can you perhaps give some details on how you were binding the device to the uio
module both for 1.6 and for 1.7?
/Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment
2014-11-05 10:12 ` [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment Bruce Richardson
@ 2014-11-07 15:01 ` Manoj Viswanath
[not found] ` <20141107153053.GA10376@bricha3-MOBL3>
0 siblings, 1 reply; 6+ messages in thread
From: Manoj Viswanath @ 2014-11-07 15:01 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
Hi Bruce,
I was not doing anything specific for binding the NICs to IGB_UIO (like
invoking "dpdk_nic_bind.py" script explicitly) when using my application
with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM were
automatically getting picked up and initialized by IGB_UIO within each VM.
The same is not working with DPDK 1.7.1 now.
I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
1.7.1) and tried to check the status. The emulated devices were shown as
neither bound to kernel nor to IGB_UIO as evident from below output:-
<--------------------------------------------------------------------------------------------------->
Network devices using DPDK-compatible driver
============================================
<none>
Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
Other network devices
=====================
0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
<--------------------------------------------------------------------------------------------------->
When i tried to forcefully bind the NICs using the "--bind=igb_uio" option
of the script, i am getting the following sequence of error log
continuously:-
<--------------------------------------------------------------------------------------------------->
........
........
EAL: Error reading from file descriptor 31: Input/output error
EAL: Error reading from file descriptor 32: Input/output error
........
........
<--------------------------------------------------------------------------------------------------->
Kindly advise.
Thanks in advance !
Regards,
Manoj
On Wed, Nov 5, 2014 at 3:42 PM, Bruce Richardson <bruce.richardson@intel.com
> wrote:
> On Wed, Nov 05, 2014 at 03:28:13PM +0530, Manoj Viswanath wrote:
> > Hi,
> >
> > I have a DPDK application running on QEMU-KVM environment using DPDK
> 1.6.0.
> > I am trying to port the same to DPDK version 1.7.1.
> >
> > I am using Virt-manager GUI to assign e1000 emulated port to the VM. This
> > works fine in DPDK 1.6.0. The device is identified by IGB_UIO and
> > initialized by my application as expected.
> >
> > However in case of DPDK 1.7.1, the emulated e1000 devices do not seem to
> be
> > recognized.
> > Following is my analysis:
> >
> > 1. The API pci_get_uio_dev() is returning ERROR. This is called from
> > pci_uio_map_resource() in the flow of PCI PROBE [rte_eal_pci_probe()].
> >
> > 2. Due to this, the PCI device is not getting mapped to the correct
> driver
> > (EM
> > Driver
> > ).
> >
> > 3. The reason for the error in [1] appears to be that "uio" sub-directory
> > doesn't seem to be correctly created for interfaces assigned to this VM.
> >
> > 4. Upon further analysis i found that IGB_UIO probe function
> > ["igbuio_pci_probe()"] is not getting triggered indicating the port has
> *not
> > been assigned* to the IGB_UIO.
> >
> > Kindly refer to the attachments:-
> > - "Output of sys-bus-pci-devices" - indicating "uio" subdirectory not
> > created for PCI devices in case of DPDK 1.7.1
> > - "Output of lspci -v" - indicating device not bound to driver in case of
> > DPDK 1.7.1
> > - IGB_UIO init log snippet - indicating PCI devices not detected and
> > initialized by IGB_UIO in case of DPDK 1.7.1
> > - CONFIG file used for DPDK compilation
> >
> > Not sure what has changed between 1.6.0 and 1.7.1 which is impacting
> this.
> >
> > Could someone throw light in this regard as to what i may be missing ?
> >
> > Thanks in advance.
> >
> > Regards,
> > Manoj
>
> Hi Manoj,
>
> can you perhaps give some details on how you were binding the device to
> the uio
> module both for 1.6 and for 1.7?
>
> /Bruce
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment
[not found] ` <20141107153053.GA10376@bricha3-MOBL3>
@ 2014-11-07 17:56 ` Manoj Viswanath
2014-11-10 10:58 ` Bruce Richardson
0 siblings, 1 reply; 6+ messages in thread
From: Manoj Viswanath @ 2014-11-07 17:56 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
Hi Bruce,
Please find my comment in lined.
On Fri, Nov 7, 2014 at 9:00 PM, Bruce Richardson <bruce.richardson@intel.com
> wrote:
> On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> > Hi Bruce,
> >
> > I was not doing anything specific for binding the NICs to IGB_UIO (like
> > invoking "dpdk_nic_bind.py" script explicitly) when using my application
> > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM
> were
> > automatically getting picked up and initialized by IGB_UIO within each
> VM.
> >
> > The same is not working with DPDK 1.7.1 now.
> >
> > I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
> > 1.7.1) and tried to check the status. The emulated devices were shown as
> > neither bound to kernel nor to IGB_UIO as evident from below output:-
> >
> >
> <--------------------------------------------------------------------------------------------------->
> > Network devices using DPDK-compatible driver
> > ============================================
> > <none>
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> >
> > Other network devices
> > =====================
> > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> >
> <--------------------------------------------------------------------------------------------------->
> >
> > When i tried to forcefully bind the NICs using the "--bind=igb_uio"
> option
>
> Was there any output of the dpdk_nic_bind script? What does the output of
> it with --status show afterwards?
>
>
[MANOJ]
Yes. Please refer below output:-
********************************************************************
Network devices using DPDK-compatible driver
============================================
0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
0000:00:05.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
Other network devices
=====================
<none>
********************************************************************
However, when i start the DPDK application, i am getting the error log as
indicated in earlier mail.
The difference with DPDK 1.6.1 is that at the same stage IGB_UIO has
already bound the assigned devices without having to explicitly run the
"dpdk_nic_bind.py". Please find below the application log when run with
DPDK 1.6.0:-
********************************************************************
Network devices using DPDK-compatible driver
============================================
0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
Other network devices
=====================
<none>
********************************************************************
Kindly note that in both cases, logs have been taken after loading IGB_UIO
prior to starting DPDK application.
[/MANOJ]
Regards,
> /Bruce
>
>
Regards,
MANOJ
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment
2014-11-07 17:56 ` Manoj Viswanath
@ 2014-11-10 10:58 ` Bruce Richardson
2014-11-11 13:48 ` Manoj Viswanath
0 siblings, 1 reply; 6+ messages in thread
From: Bruce Richardson @ 2014-11-10 10:58 UTC (permalink / raw)
To: Manoj Viswanath; +Cc: dev
On Fri, Nov 07, 2014 at 11:26:08PM +0530, Manoj Viswanath wrote:
> Hi Bruce,
>
> Please find my comment in lined.
>
> On Fri, Nov 7, 2014 at 9:00 PM, Bruce Richardson <bruce.richardson@intel.com
> > wrote:
>
> > On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> > > Hi Bruce,
> > >
> > > I was not doing anything specific for binding the NICs to IGB_UIO (like
> > > invoking "dpdk_nic_bind.py" script explicitly) when using my application
> > > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the VM
> > were
> > > automatically getting picked up and initialized by IGB_UIO within each
> > VM.
> > >
> > > The same is not working with DPDK 1.7.1 now.
> > >
> > > I tried exporting the "dpdk_nic_bind.py" script into my VM (running DPDK
> > > 1.7.1) and tried to check the status. The emulated devices were shown as
> > > neither bound to kernel nor to IGB_UIO as evident from below output:-
> > >
> > >
> > <--------------------------------------------------------------------------------------------------->
> > > Network devices using DPDK-compatible driver
> > > ============================================
> > > <none>
> > >
> > > Network devices using kernel driver
> > > ===================================
> > > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> > >
> > > Other network devices
> > > =====================
> > > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > >
> > <--------------------------------------------------------------------------------------------------->
> > >
> > > When i tried to forcefully bind the NICs using the "--bind=igb_uio"
> > option
> >
> > Was there any output of the dpdk_nic_bind script? What does the output of
> > it with --status show afterwards?
> >
> >
> [MANOJ]
>
> Yes. Please refer below output:-
> ********************************************************************
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> 0000:00:05.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
>
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>
> Other network devices
> =====================
> <none>
> ********************************************************************
>
> However, when i start the DPDK application, i am getting the error log as
> indicated in earlier mail.
>
> The difference with DPDK 1.6.1 is that at the same stage IGB_UIO has
> already bound the assigned devices without having to explicitly run the
> "dpdk_nic_bind.py". Please find below the application log when run with
> DPDK 1.6.0:-
>
> ********************************************************************
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> 0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
>
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>
> Other network devices
> =====================
> <none>
> ********************************************************************
>
> Kindly note that in both cases, logs have been taken after loading IGB_UIO
> prior to starting DPDK application.
>
> [/MANOJ]
>
> Regards,
Ok, so it appears that after running dpdk_nic_bind to bind the devices to igb_uio
the differences between 1.6 and 1.7 are resolved for that part. The reason
why you explicitly need to bind the devices in 1.7 is due to this commit which
removes the pci id table from the igb_uio driver.
http://dpdk.org/browse/dpdk/commit/?id=629395b063e8278a05ea41908d1152fa68df098c
As for the other errors you are seeing, I'm not sure of the cause, though they
may be related to interrupt support for changes in link status. Can you perhaps
use a debugger and find out what the file descriptor in question refers to?
/Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment
2014-11-10 10:58 ` Bruce Richardson
@ 2014-11-11 13:48 ` Manoj Viswanath
2014-11-12 14:12 ` Bruce Richardson
0 siblings, 1 reply; 6+ messages in thread
From: Manoj Viswanath @ 2014-11-11 13:48 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
Bruce,
Thanks for the input.
Sure, will figure out the offending file behind this error and update this
thread.
Meanwhile, wanted to share one more observation regarding this issue:
The "file descriptor error" is NOT SEEN with DPDK 1.7.0 (dpdk-1.7.0.tar.gz
<http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.0.tar.gz>) when explicitly
binding NICs to IGB_UIO using the .py script.
The issue is only seen when using DPDK 1.7.1 ( dpdk-1.7.1.tar.gz
<http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.1.tar.gz>). I hope the
version i am using is the last official tag in the 1.7.x tree.
I see that this problem ("file descriptor error") in DPDK 1.7.1 and its
non-occurrence in DPDK 1.7.0 has already been reported in this group on Sep
11 by zimeiw <zimeiw@163.com> in a mail with following subject line:- "*There
are a lot of error log when run l3fwd of dpdk-1.7.1*". Couldn't find any
responses to that thread.
Regards,
Manoj
On Mon, Nov 10, 2014 at 4:28 PM, Bruce Richardson <
bruce.richardson@intel.com> wrote:
> On Fri, Nov 07, 2014 at 11:26:08PM +0530, Manoj Viswanath wrote:
> > Hi Bruce,
> >
> > Please find my comment in lined.
> >
> > On Fri, Nov 7, 2014 at 9:00 PM, Bruce Richardson <
> bruce.richardson@intel.com
> > > wrote:
> >
> > > On Fri, Nov 07, 2014 at 08:31:34PM +0530, Manoj Viswanath wrote:
> > > > Hi Bruce,
> > > >
> > > > I was not doing anything specific for binding the NICs to IGB_UIO
> (like
> > > > invoking "dpdk_nic_bind.py" script explicitly) when using my
> application
> > > > with DPDK 1.6.0. The e1000 devices assigned via virt-manager to the
> VM
> > > were
> > > > automatically getting picked up and initialized by IGB_UIO within
> each
> > > VM.
> > > >
> > > > The same is not working with DPDK 1.7.1 now.
> > > >
> > > > I tried exporting the "dpdk_nic_bind.py" script into my VM (running
> DPDK
> > > > 1.7.1) and tried to check the status. The emulated devices were
> shown as
> > > > neither bound to kernel nor to IGB_UIO as evident from below output:-
> > > >
> > > >
> > >
> <--------------------------------------------------------------------------------------------------->
> > > > Network devices using DPDK-compatible driver
> > > > ============================================
> > > > <none>
> > > >
> > > > Network devices using kernel driver
> > > > ===================================
> > > > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci
> unused=igb_uio
> > > >
> > > > Other network devices
> > > > =====================
> > > > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > > > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' unused=igb_uio
> > > >
> > >
> <--------------------------------------------------------------------------------------------------->
> > > >
> > > > When i tried to forcefully bind the NICs using the "--bind=igb_uio"
> > > option
> > >
> > > Was there any output of the dpdk_nic_bind script? What does the output
> of
> > > it with --status show afterwards?
> > >
> > >
> > [MANOJ]
> >
> > Yes. Please refer below output:-
> > ********************************************************************
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> > 0000:00:05.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> >
> > Other network devices
> > =====================
> > <none>
> > ********************************************************************
> >
> > However, when i start the DPDK application, i am getting the error log
> as
> > indicated in earlier mail.
> >
> > The difference with DPDK 1.6.1 is that at the same stage IGB_UIO has
> > already bound the assigned devices without having to explicitly run the
> > "dpdk_nic_bind.py". Please find below the application log when run with
> > DPDK 1.6.0:-
> >
> > ********************************************************************
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:00:04.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> > 0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> >
> > Other network devices
> > =====================
> > <none>
> > ********************************************************************
> >
> > Kindly note that in both cases, logs have been taken after loading
> IGB_UIO
> > prior to starting DPDK application.
> >
> > [/MANOJ]
> >
> > Regards,
>
> Ok, so it appears that after running dpdk_nic_bind to bind the devices to
> igb_uio
> the differences between 1.6 and 1.7 are resolved for that part. The reason
> why you explicitly need to bind the devices in 1.7 is due to this commit
> which
> removes the pci id table from the igb_uio driver.
>
>
> http://dpdk.org/browse/dpdk/commit/?id=629395b063e8278a05ea41908d1152fa68df098c
>
> As for the other errors you are seeing, I'm not sure of the cause, though
> they
> may be related to interrupt support for changes in link status. Can you
> perhaps
> use a debugger and find out what the file descriptor in question refers to?
>
> /Bruce
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment
2014-11-11 13:48 ` Manoj Viswanath
@ 2014-11-12 14:12 ` Bruce Richardson
0 siblings, 0 replies; 6+ messages in thread
From: Bruce Richardson @ 2014-11-12 14:12 UTC (permalink / raw)
To: Manoj Viswanath; +Cc: dev
On Tue, Nov 11, 2014 at 07:18:32PM +0530, Manoj Viswanath wrote:
> Bruce,
>
> Thanks for the input.
> Sure, will figure out the offending file behind this error and update this
> thread.
>
> Meanwhile, wanted to share one more observation regarding this issue:
> The "file descriptor error" is NOT SEEN with DPDK 1.7.0 (dpdk-1.7.0.tar.gz
> <http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.0.tar.gz>) when explicitly
> binding NICs to IGB_UIO using the .py script.
>
> The issue is only seen when using DPDK 1.7.1 ( dpdk-1.7.1.tar.gz
> <http://dpdk.org/browse/dpdk/snapshot/dpdk-1.7.1.tar.gz>). I hope the
> version i am using is the last official tag in the 1.7.x tree.
>
Given that it's narrowed down to a change between 1.7.0 and 1.7.1 would it also
be possible for you to use git bisect to track down the commit that introduces
the error?
/Bruce
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-11-12 14:02 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAC1b25pX3R2y_Cjp5UAdL6Bozrf01OkG6U6LGQvJ2aUTDXGvSA@mail.gmail.com>
2014-11-05 10:12 ` [dpdk-dev] Ports not detected by IGB_UIO in DPDK 1.7.1 in QEMU_KVM environment Bruce Richardson
2014-11-07 15:01 ` Manoj Viswanath
[not found] ` <20141107153053.GA10376@bricha3-MOBL3>
2014-11-07 17:56 ` Manoj Viswanath
2014-11-10 10:58 ` Bruce Richardson
2014-11-11 13:48 ` Manoj Viswanath
2014-11-12 14:12 ` Bruce Richardson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).