From: Ilya Maximets <i.maximets@samsung.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Xie, Huawei" <huawei.xie@intel.com>,
Yuanhan Liu <yuanhan.liu@linux.intel.com>
Cc: Dyasly Sergey <s.dyasly@samsung.com>,
Heetae Ahn <heetae82.ahn@samsung.com>
Subject: Re: [dpdk-dev] [PATCH] vhost: fix segfault on bad descriptor address.
Date: Tue, 31 May 2016 12:10:46 +0300 [thread overview]
Message-ID: <574D5516.3010405@samsung.com> (raw)
In-Reply-To: <1bb65424-fda0-8e60-554d-66ccc19e7d90@intel.com>
On 31.05.2016 09:53, Tan, Jianfeng wrote:
> Hi,
>
>
> On 5/30/2016 8:24 PM, Ilya Maximets wrote:
>> On 30.05.2016 15:00, Tan, Jianfeng wrote:
>>> Hi,
>>>
>>>> -----Original Message-----
>>>> From: Ilya Maximets [mailto:i.maximets@samsung.com]
>>>> Sent: Friday, May 20, 2016 8:50 PM
>>>> To: dev@dpdk.org; Xie, Huawei; Yuanhan Liu
>>>> Cc: Dyasly Sergey; Heetae Ahn; Tan, Jianfeng; Ilya Maximets
>>>> Subject: [PATCH] vhost: fix segfault on bad descriptor address.
>>>>
>>>> In current implementation guest application can reinitialize vrings
>>>> by executing start after stop. In the same time host application
>>>> can still poll virtqueue while device stopped in guest and it will
>>>> crash with segmentation fault while vring reinitialization because
>>>> of dereferencing of bad descriptor addresses.
>>>>
>>>> OVS crash for example:
>>>> <------------------------------------------------------------------------>
>>>> [test-pmd inside guest VM]
>>>>
>>>> testpmd> port stop all
>>>> Stopping ports...
>>>> Checking link statuses...
>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>>> Done
>>>> testpmd> port config all rxq 2
>>>> testpmd> port config all txq 2
>>>> testpmd> port start all
>>>> Configuring Port 0 (socket 0)
>>>> Port 0: 52:54:00:CB:44:C8
>>>> Checking link statuses...
>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>>> Done
>>>>
>>>> [OVS on host]
>>>> Program received signal SIGSEGV, Segmentation fault.
>>>> rte_memcpy (n=2056, src=0xc, dst=0x7ff4d5247000) at
>>>> rte_memcpy.h
>>>>
>>>> (gdb) bt
>>>> #0 rte_memcpy (n=2056, src=0xc, dst=0x7ff4d5247000)
>>>> #1 copy_desc_to_mbuf
>>>> #2 rte_vhost_dequeue_burst
>>>> #3 netdev_dpdk_vhost_rxq_recv
>>>> ...
>>>>
>>>> (gdb) bt full
>>>> #0 rte_memcpy
>>>> ...
>>>> #1 copy_desc_to_mbuf
>>>> desc_addr = 0
>>>> mbuf_offset = 0
>>>> desc_offset = 12
>>>> ...
>>>> <------------------------------------------------------------------------>
>>>>
>>>> Fix that by checking addresses of descriptors before using them.
>>>>
>>>> Note: For mergeable buffers this patch checks only guest's address for
>>>> zero, but in non-meargeable case host's address checked. This is done
>>>> because checking of host's address in mergeable case requires additional
>>>> refactoring to keep virtqueue in consistent state in case of error.
>>>
>>> I agree with you that it should be fixed because malicious guest could launch
>>> DOS attack on vswitch with the current implementation.
>>>
>>> But I don't understand why you do not fix the mergable case in
>>> copy_mbuf_to_desc_mergable() on where gpa_to_vva() happens? And the change in
>>> fill_vec_buf(), checking !vq->desc[idx].addr, make any sense?
>>>
>>> Thanks,
>>> Jianfeng
>> Hi.
>> As I said inside commit-message, checking of host's address in mergeable case
>> requires additional refactoring to keep virtqueue in consistent state.
>>
>> There are few issues with checking inside copy_mbuf_to_desc_mergable() :
>>
>> 1. Ring elements already reserved and we must fill them with some
>> sane data before going out of virtio_dev_merge_rx().
>>
>> 2. copy_mbuf_to_desc_mergable() can't return an error in current
>> implementation (additional checking needed), otherwise used->idx
>> will be decremented (I think, it's bad).
>
> Yes, currently there is no way to return these invalid desc back to virtio because there's no invalid flag in virtio_net_hdr to indicate this desc contains no pkt. I see kernel just skips those descriptors with bad addr. I think it may rely on reset of the virtio device to improve such situation.
>
> Another thing is that, your patch only checks the desc->addr, but we should check desc->addr + desc->len too, right?
To do it fast we need to check whole range inside gpa_to_vva(), but even
more refactoring is required for that. Also, this can be a different
patch because checking of addr + len not required to fix original issue
with virtio reconfiguration.
>
> Thanks,
> Jianfeng
>
>>
>>
>> Checking !vq->desc[idx].addr inside fill_vec_buf() make sense in case of virtio
>> reinitialization, because guest's address will be zero (case described in
>> commit-message). Checking of guest's address will not help in case of bad and
>> not NULL address, but useful in this common case.
>> Also, we can't catch bad address what we able to map, so, malicious guest could
>> break vhost anyway.
>>
>> I agree, that checking of host's address is better, but this may be done later
>> together with resolving above issues.
>>
>> Best regards, Ilya Maximets.
next prev parent reply other threads:[~2016-05-31 9:10 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-20 12:50 Ilya Maximets
2016-05-23 10:57 ` Yuanhan Liu
2016-05-23 11:04 ` Ilya Maximets
2016-05-30 11:05 ` Ilya Maximets
2016-05-30 14:25 ` Yuanhan Liu
2016-05-31 9:12 ` Ilya Maximets
2016-05-30 12:00 ` Tan, Jianfeng
2016-05-30 12:24 ` Ilya Maximets
2016-05-31 6:53 ` Tan, Jianfeng
2016-05-31 9:10 ` Ilya Maximets [this message]
2016-05-31 22:06 ` Rich Lane
2016-06-02 10:46 ` Ilya Maximets
2016-06-02 16:22 ` Rich Lane
2016-06-03 6:01 ` Ilya Maximets
2016-07-01 7:35 ` Yuanhan Liu
2016-07-06 11:19 ` Ilya Maximets
2016-07-06 12:24 ` Yuanhan Liu
2016-07-08 11:48 ` Ilya Maximets
2016-07-10 13:17 ` Yuanhan Liu
2016-07-11 8:38 ` Yuanhan Liu
2016-07-11 9:50 ` Ilya Maximets
2016-07-11 11:05 ` Yuanhan Liu
2016-07-11 11:47 ` Ilya Maximets
2016-07-12 2:43 ` Yuanhan Liu
2016-07-12 5:53 ` Ilya Maximets
2016-07-13 7:34 ` Ilya Maximets
2016-07-13 8:47 ` Yuanhan Liu
2016-07-13 15:54 ` Rich Lane
2016-07-14 1:42 ` Yuanhan Liu
2016-07-14 4:38 ` Ilya Maximets
2016-07-14 8:18 ` [dpdk-dev] [PATCH v2] " Ilya Maximets
2016-07-15 6:17 ` Yuanhan Liu
2016-07-15 7:23 ` Ilya Maximets
2016-07-15 8:40 ` Yuanhan Liu
2016-07-15 11:15 ` [dpdk-dev] [PATCH v3 0/2] " Ilya Maximets
2016-07-15 11:15 ` [dpdk-dev] [PATCH v3 1/2] vhost: fix using of bad return value on mergeable enqueue Ilya Maximets
2016-07-15 11:15 ` [dpdk-dev] [PATCH v3 2/2] vhost: do sanity check for ring descriptor address Ilya Maximets
2016-07-15 12:14 ` [dpdk-dev] [PATCH v3 0/2] vhost: fix segfault on bad " Yuanhan Liu
2016-07-15 19:37 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=574D5516.3010405@samsung.com \
--to=i.maximets@samsung.com \
--cc=dev@dpdk.org \
--cc=heetae82.ahn@samsung.com \
--cc=huawei.xie@intel.com \
--cc=jianfeng.tan@intel.com \
--cc=s.dyasly@samsung.com \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).