[dpdk-dev] Crashing OVS+DPDK at the host, from inside of a KVM Guest

[dpdk-dev] Crashing OVS+DPDK at the host, from inside of a KVM Guest

[Martinx - ジェームズ]

Guys,

 If using OVS 2.5 with DPDK 2.2, on Ubuntu Xenial, it is possible to crash
the OVS running at the host, from inside of a KVM Guest.

 Basically, what I'm trying to do, is to run OVS+DPDK at the host, and
also, inside of a KVM Guest, with multi-queue, but it doesn't work and
crashes.

 Soon as you enable multi-queue at the guest, it crashes the OVS of the
host!

OVS+DPDK segfault at the host, after running "ovs-vsctl set Open_vSwitch .
other_config:n-dpdk-rxqs=4" within a KVM Guest:

https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1577088

Thanks!
Thiago

Re: [dpdk-dev] Crashing OVS+DPDK at the host, from inside of a KVM Guest

[Christian Ehrhardt]

Hi,
ping ...

Later on I want to look at it again once we upgraded to more recent
releases of the software components involved, but those have to be made
ready to use first :-/

But the description is good and I wonder if anybody else could reproduce
this and/or would have a hint on where this might come from or already
existing related fixes.

I mean in general nothing should be able to crash the host right?


P.S. yeah two list cross posting, but it is yet unclear which it belongs to
so I'll keep it

Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Sun, May 15, 2016 at 7:08 AM, Martinx - ジェームズ <thiagocmartinsc@gmail.com>
wrote:

> Guys,
>
>  If using OVS 2.5 with DPDK 2.2, on Ubuntu Xenial, it is possible to crash
> the OVS running at the host, from inside of a KVM Guest.
>
>  Basically, what I'm trying to do, is to run OVS+DPDK at the host, and
> also, inside of a KVM Guest, with multi-queue, but it doesn't work and
> crashes.
>
>  Soon as you enable multi-queue at the guest, it crashes the OVS of the
> host!
>
> OVS+DPDK segfault at the host, after running "ovs-vsctl set Open_vSwitch .
> other_config:n-dpdk-rxqs=4" within a KVM Guest:
>
> https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1577088
>
> Thanks!
> Thiago
>

Re: [dpdk-dev] Crashing OVS+DPDK at the host, from inside of a KVM Guest

[Traynor, Kevin]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Christian
> Ehrhardt
> Sent: Wednesday, May 25, 2016 7:06 AM
> To: Martinx - ジェームズ <thiagocmartinsc@gmail.com>
> Cc: <dev@openvswitch.org> <dev@openvswitch.org>; dev <dev@dpdk.org>
> Subject: Re: [dpdk-dev] Crashing OVS+DPDK at the host, from inside of
> a KVM Guest
> 
> Hi,
> ping ...
> 
> Later on I want to look at it again once we upgraded to more recent
> releases of the software components involved, but those have to be
> made
> ready to use first :-/
> 
> But the description is good and I wonder if anybody else could
> reproduce
> this and/or would have a hint on where this might come from or already
> existing related fixes.
> 
> I mean in general nothing should be able to crash the host right?

Hi, I don't know if they are related to the issue that is being seen,
but Yuanhan made some fixes in DPDK 16.04 regarding a malicious guest
affecting the host. rte_vhost_dequeue_burst() is showing in the stack
trace so it might worth testing with the latest code to see if it's the
same issue and has been fixed.

Kevin.

> 
> 
> P.S. yeah two list cross posting, but it is yet unclear which it
> belongs to
> so I'll keep it
> 
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd
> 
> On Sun, May 15, 2016 at 7:08 AM, Martinx - ジェームズ
> <thiagocmartinsc@gmail.com>
> wrote:
> 
> > Guys,
> >
> >  If using OVS 2.5 with DPDK 2.2, on Ubuntu Xenial, it is possible to
> crash
> > the OVS running at the host, from inside of a KVM Guest.
> >
> >  Basically, what I'm trying to do, is to run OVS+DPDK at the host,
> and
> > also, inside of a KVM Guest, with multi-queue, but it doesn't work
> and
> > crashes.
> >
> >  Soon as you enable multi-queue at the guest, it crashes the OVS of
> the
> > host!
> >
> > OVS+DPDK segfault at the host, after running "ovs-vsctl set
> Open_vSwitch .
> > other_config:n-dpdk-rxqs=4" within a KVM Guest:
> >
> > https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1577088
> >
> > Thanks!
> > Thiago
> >

Re: [dpdk-dev] Crashing OVS+DPDK at the host, from inside of a KVM Guest

[Xie, Huawei]

On 5/25/2016 2:06 PM, Christian Ehrhardt wrote:
> Hi,
> ping ...
>
> Later on I want to look at it again once we upgraded to more recent
> releases of the software components involved, but those have to be made
> ready to use first :-/
>
> But the description is good and I wonder if anybody else could reproduce
> this and/or would have a hint on where this might come from or already
> existing related fixes.
>
> I mean in general nothing should be able to crash the host right?

Yes, we are taking care of these issues to avoid malicious or buggy
guest driver to corrupt vhost.
We have fixed some issues, and would continue to check if there are
other potential issues.

>
>
> P.S. yeah two list cross posting, but it is yet unclear which it belongs to
> so I'll keep it
>
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd
>
> On Sun, May 15, 2016 at 7:08 AM, Martinx - ジェームズ <thiagocmartinsc@gmail.com>
> wrote:
>
>> Guys,
>>
>>  If using OVS 2.5 with DPDK 2.2, on Ubuntu Xenial, it is possible to crash
>> the OVS running at the host, from inside of a KVM Guest.
>>
>>  Basically, what I'm trying to do, is to run OVS+DPDK at the host, and
>> also, inside of a KVM Guest, with multi-queue, but it doesn't work and
>> crashes.
>>
>>  Soon as you enable multi-queue at the guest, it crashes the OVS of the
>> host!
>>
>> OVS+DPDK segfault at the host, after running "ovs-vsctl set Open_vSwitch .
>> other_config:n-dpdk-rxqs=4" within a KVM Guest:
>>
>> https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1577088
>>
>> Thanks!
>> Thiago
>>

Re: [dpdk-dev] Crashing OVS+DPDK at the host, from inside of a KVM Guest

[Yuanhan Liu]

On Wed, May 25, 2016 at 01:53:07PM +0000, Traynor, Kevin wrote:
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Christian
> > Ehrhardt
> > Sent: Wednesday, May 25, 2016 7:06 AM
> > To: Martinx - ジェームズ <thiagocmartinsc@gmail.com>
> > Cc: <dev@openvswitch.org> <dev@openvswitch.org>; dev <dev@dpdk.org>
> > Subject: Re: [dpdk-dev] Crashing OVS+DPDK at the host, from inside of
> > a KVM Guest
> > 
> > Hi,
> > ping ...
> > 
> > Later on I want to look at it again once we upgraded to more recent
> > releases of the software components involved, but those have to be
> > made
> > ready to use first :-/
> > 
> > But the description is good and I wonder if anybody else could
> > reproduce
> > this and/or would have a hint on where this might come from or already
> > existing related fixes.
> > 
> > I mean in general nothing should be able to crash the host right?
> 
> Hi, I don't know if they are related to the issue that is being seen,
> but Yuanhan made some fixes in DPDK 16.04 regarding a malicious guest
> affecting the host. rte_vhost_dequeue_burst() is showing in the stack
> trace so it might worth testing with the latest code to see if it's the
> same issue and has been fixed.

Hi,

Try backporting following 3 patches? It might/should fix your issues:

    a436f53 vhost: avoid dead loop chain
    c687b0b vhost: check for ring descriptors overflow
    623bc47 vhost: do sanity check for ring descriptor length

Note that it will not apply cleanly on top of v2.2, but it should
trivial to do backport, as they are simple changes.


	--yliu