[dpdk-dev] [dpdk-virtio] DPDK stopped working on virtio

[dpdk-dev] [dpdk-virtio] DPDK stopped working on virtio

[Clarylin L]

I am running DPDK application (testpmd) within a VM based on virtio. With
the same hypervisor and same DPDK code, it used to work well. But it
stopped working since last week. The VM's port could not receive anything.
I ran tcpdump on host's physical port, bridge interface as well as vnet
interface, and did see packets coming in. However on VM's port there was
nothing.

I ran gdb trying to debug, it hit function virtio_recv_mergeable_pkts(),
but nb_used=VIRTQUEUE_NUSED(rxvq) always gave 0. I guess it's because the
queue was empty.

I enabled the PMD debug logging and the only thing that might be an issue
was the following part. Other than this I could not see any thing that
could indicate potential issues.

Thu Feb 18 19:20:10 2016^@PMD: get_uio_dev(): Could not find uio resource
Thu Feb 18 19:20:10 2016^@PMD: virtio_resource_init_by_ioports(): PCI Port
IO found start=0xc040 with size=0x40

If someone can give any pointers that I should further look into, that'd be
very helpful. Appreciate your help!

Re: [dpdk-dev] [dpdk-virtio] DPDK stopped working on virtio

[Yuanhan Liu]

On Mon, Feb 22, 2016 at 11:15:57AM -0800, Clarylin L wrote:
> I am running DPDK application (testpmd) within a VM based on virtio. With
> the same hypervisor and same DPDK code, it used to work well. But it
> stopped working since last week. The VM's port could not receive anything.
> I ran tcpdump on host's physical port, bridge interface as well as vnet
> interface, and did see packets coming in. However on VM's port there was
> nothing.
> 
> I ran gdb trying to debug, it hit function virtio_recv_mergeable_pkts(),
> but nb_used=VIRTQUEUE_NUSED(rxvq) always gave 0. I guess it's because the
> queue was empty.
> 
> I enabled the PMD debug logging and the only thing that might be an issue
> was the following part. Other than this I could not see any thing that
> could indicate potential issues.
> 
> Thu Feb 18 19:20:10 2016^@PMD: get_uio_dev(): Could not find uio resource
> Thu Feb 18 19:20:10 2016^@PMD: virtio_resource_init_by_ioports(): PCI Port
> IO found start=0xc040 with size=0x40

That could be normal, when you don't bind the driver to igb_uio.

> If someone can give any pointers that I should further look into, that'd be
> very helpful. Appreciate your help!

What's the last commit you are testing? And what are the steps
to reproduce it? I have a quick try with vhost-switch example,
with pkts injected by IXIA; it works fine here.

Or better, mind do a git bisect? There aren't too many commits
there. It should be a pretty fast bisect.

	--yliu

Re: [dpdk-dev] [dpdk-virtio] DPDK stopped working on virtio

[Clarylin L]

I am working with DPDK 2.0.

I guess it's not DPDK code issue , but more like an environment issue (as
same code has been working fine before. It's even working on another setup
now). Someone might have accidentally changed my setup, and I want to find
out what made dpdk-virtio stop working.

Is DPDK-virtio dependent on any specific modules on the hypervisor? or are
there any configurations on the hypervisor that would impact the
functionality of  virtio? or anything else I need to look into and check?
My VM is running on Ubuntu KVM.

On Mon, Feb 22, 2016 at 7:50 PM, Yuanhan Liu <yuanhan.liu@linux.intel.com>
wrote:

> On Mon, Feb 22, 2016 at 11:15:57AM -0800, Clarylin L wrote:
> > I am running DPDK application (testpmd) within a VM based on virtio. With
> > the same hypervisor and same DPDK code, it used to work well. But it
> > stopped working since last week. The VM's port could not receive
> anything.
> > I ran tcpdump on host's physical port, bridge interface as well as vnet
> > interface, and did see packets coming in. However on VM's port there was
> > nothing.
> >
> > I ran gdb trying to debug, it hit function virtio_recv_mergeable_pkts(),
> > but nb_used=VIRTQUEUE_NUSED(rxvq) always gave 0. I guess it's because the
> > queue was empty.
> >
> > I enabled the PMD debug logging and the only thing that might be an issue
> > was the following part. Other than this I could not see any thing that
> > could indicate potential issues.
> >
> > Thu Feb 18 19:20:10 2016^@PMD: get_uio_dev(): Could not find uio resource
> > Thu Feb 18 19:20:10 2016^@PMD: virtio_resource_init_by_ioports(): PCI
> Port
> > IO found start=0xc040 with size=0x40
>
> That could be normal, when you don't bind the driver to igb_uio.
>
> > If someone can give any pointers that I should further look into, that'd
> be
> > very helpful. Appreciate your help!
>
> What's the last commit you are testing? And what are the steps
> to reproduce it? I have a quick try with vhost-switch example,
> with pkts injected by IXIA; it works fine here.
>
> Or better, mind do a git bisect? There aren't too many commits
> there. It should be a pretty fast bisect.
>
>         --yliu
>

Re: [dpdk-dev] [dpdk-virtio] DPDK stopped working on virtio

[Yuanhan Liu]

On Mon, Feb 22, 2016 at 11:00:28PM -0800, Clarylin L wrote:
> I am working with DPDK 2.0. 
> 
> I guess it's not DPDK code issue , but more like an environment issue (as same
> code has been working fine before. It's even working on another setup now).
> Someone might have accidentally changed my setup, and I want to find out what
> made dpdk-virtio stop working. 
> 
> Is DPDK-virtio dependent on any specific modules on the hypervisor? or are
> there any configurations on the hypervisor that would impact the functionality
> of  virtio?

Nope, none that I can think of.

> or anything else I need to look into and check? My VM is running on
> Ubuntu KVM.

The vhost and virtio log?

	--yliu
> 
> On Mon, Feb 22, 2016 at 7:50 PM, Yuanhan Liu <yuanhan.liu@linux.intel.com>
> wrote:
> 
>     On Mon, Feb 22, 2016 at 11:15:57AM -0800, Clarylin L wrote:
>     > I am running DPDK application (testpmd) within a VM based on virtio. With
>     > the same hypervisor and same DPDK code, it used to work well. But it
>     > stopped working since last week. The VM's port could not receive
>     anything.
>     > I ran tcpdump on host's physical port, bridge interface as well as vnet
>     > interface, and did see packets coming in. However on VM's port there was
>     > nothing.
>     >
>     > I ran gdb trying to debug, it hit function virtio_recv_mergeable_pkts(),
>     > but nb_used=VIRTQUEUE_NUSED(rxvq) always gave 0. I guess it's because the
>     > queue was empty.
>     >
>     > I enabled the PMD debug logging and the only thing that might be an issue
>     > was the following part. Other than this I could not see any thing that
>     > could indicate potential issues.
>     >
>     > Thu Feb 18 19:20:10 2016^@PMD: get_uio_dev(): Could not find uio resource
>     > Thu Feb 18 19:20:10 2016^@PMD: virtio_resource_init_by_ioports(): PCI
>     Port
>     > IO found start=0xc040 with size=0x40
> 
>     That could be normal, when you don't bind the driver to igb_uio.
>    
>     > If someone can give any pointers that I should further look into, that'd
>     be
>     > very helpful. Appreciate your help!
> 
>     What's the last commit you are testing? And what are the steps
>     to reproduce it? I have a quick try with vhost-switch example,
>     with pkts injected by IXIA; it works fine here.
> 
>     Or better, mind do a git bisect? There aren't too many commits
>     there. It should be a pretty fast bisect.
> 
>             --yliu
> 
> 

Re: [dpdk-dev] [dpdk-virtio] DPDK stopped working on virtio

[Clarylin L]

Played with it a little bit more. It can receive packets only if
promiscuous mode is enabled in DPDK.

On Mon, Feb 22, 2016 at 11:00 PM, Clarylin L <clearasu@gmail.com> wrote:

> I am working with DPDK 2.0.
>
> I guess it's not DPDK code issue , but more like an environment issue (as
> same code has been working fine before. It's even working on another setup
> now). Someone might have accidentally changed my setup, and I want to find
> out what made dpdk-virtio stop working.
>
> Is DPDK-virtio dependent on any specific modules on the hypervisor? or are
> there any configurations on the hypervisor that would impact the
> functionality of  virtio? or anything else I need to look into and check?
> My VM is running on Ubuntu KVM.
>
> On Mon, Feb 22, 2016 at 7:50 PM, Yuanhan Liu <yuanhan.liu@linux.intel.com>
> wrote:
>
>> On Mon, Feb 22, 2016 at 11:15:57AM -0800, Clarylin L wrote:
>> > I am running DPDK application (testpmd) within a VM based on virtio.
>> With
>> > the same hypervisor and same DPDK code, it used to work well. But it
>> > stopped working since last week. The VM's port could not receive
>> anything.
>> > I ran tcpdump on host's physical port, bridge interface as well as vnet
>> > interface, and did see packets coming in. However on VM's port there was
>> > nothing.
>> >
>> > I ran gdb trying to debug, it hit function virtio_recv_mergeable_pkts(),
>> > but nb_used=VIRTQUEUE_NUSED(rxvq) always gave 0. I guess it's because
>> the
>> > queue was empty.
>> >
>> > I enabled the PMD debug logging and the only thing that might be an
>> issue
>> > was the following part. Other than this I could not see any thing that
>> > could indicate potential issues.
>> >
>> > Thu Feb 18 19:20:10 2016^@PMD: get_uio_dev(): Could not find uio
>> resource
>> > Thu Feb 18 19:20:10 2016^@PMD: virtio_resource_init_by_ioports(): PCI
>> Port
>> > IO found start=0xc040 with size=0x40
>>
>> That could be normal, when you don't bind the driver to igb_uio.
>>
>> > If someone can give any pointers that I should further look into,
>> that'd be
>> > very helpful. Appreciate your help!
>>
>> What's the last commit you are testing? And what are the steps
>> to reproduce it? I have a quick try with vhost-switch example,
>> with pkts injected by IXIA; it works fine here.
>>
>> Or better, mind do a git bisect? There aren't too many commits
>> there. It should be a pretty fast bisect.
>>
>>         --yliu
>>
>
>