* [PATCH] vhost: fix OOB access for invalid vid
@ 2023-02-27 10:59 David Marchand
2023-02-27 16:19 ` Maxime Coquelin
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: David Marchand @ 2023-02-27 10:59 UTC (permalink / raw)
To: dev; +Cc: stable, Maxime Coquelin, Chenbo Xia
The net/vhost pmd currently provides a -1 vid when disabling interrupt
after a virtio port got disconnected.
This can be caught when running with ASan.
First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
$ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
-a 0000:00:00.0 \
--vdev net_vhost0,iface=plop.sock,client=1\
-- \
-p 0x1 \
--interrupt-only \
--config '(0,0,1)' \
--parse-ptype 0
Then start testpmd with virtio-user.
$ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
-a 0000:00:00.0 \
--vdev net_virtio_user0,path=plop.sock,server=1 \
-- \
-i
Finally stop testpmd.
ASan then splats in dpdk-l3fwd-power:
=================================================================
==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
sp 0x7fddbd2eee18
READ of size 8 at 0x000005ed0778 thread T2
#0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
#1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
#2 0x3ac95cb in eth_rxq_intr_disable
.../drivers/net/vhost/rte_eth_vhost.c:647:8
#3 0x170e0bf in rte_eth_dev_rx_intr_disable
.../lib/ethdev/rte_ethdev.c:5443:25
#4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
#5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
#6 0x17f9292 in eal_thread_loop
.../lib/eal/common/eal_common_thread.c:210:9
#7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
#8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
#9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
0x000005ed0778 is located 8 bytes to the left of global variable
'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
(0x5ed0780) of size 8192
0x000005ed0778 is located 20 bytes to the right of global variable
'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
(0x5ed0760) of size 4
SUMMARY: AddressSanitizer: global-buffer-overflow
.../lib/vhost/vhost.h:801:27 in get_device
Shadow bytes around the buggy address:
0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
=>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Thread T2 created by T0 here:
#0 0xe98996 in __interceptor_pthread_create
(.examples/dpdk-l3fwd-power+0xe98996)
(BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
#1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
#2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
#3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
#4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
==3641005==ABORTING
More generally, any application passing an incorrect vid would trigger
such an OOB access.
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Note: even after this patch, reconnecting the virtio-user port with
dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
pmd keeps complaining about an issue with rx interrupt fds.
---
lib/vhost/vhost.h | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
index 5750f0c005..d9e97280fa 100644
--- a/lib/vhost/vhost.h
+++ b/lib/vhost/vhost.h
@@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
static __rte_always_inline struct virtio_net *
get_device(int vid)
{
- struct virtio_net *dev = vhost_devices[vid];
+ struct virtio_net *dev = NULL;
+
+ if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
+ dev = vhost_devices[vid];
if (unlikely(!dev)) {
VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
--
2.39.2
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] vhost: fix OOB access for invalid vid
2023-02-27 10:59 [PATCH] vhost: fix OOB access for invalid vid David Marchand
@ 2023-02-27 16:19 ` Maxime Coquelin
2023-02-28 8:27 ` David Marchand
2023-03-06 14:07 ` Maxime Coquelin
2023-03-06 14:22 ` Maxime Coquelin
2 siblings, 1 reply; 6+ messages in thread
From: Maxime Coquelin @ 2023-02-27 16:19 UTC (permalink / raw)
To: David Marchand, dev; +Cc: stable, Chenbo Xia
On 2/27/23 11:59, David Marchand wrote:
> The net/vhost pmd currently provides a -1 vid when disabling interrupt
> after a virtio port got disconnected.
>
> This can be caught when running with ASan.
>
> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
>
> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> -a 0000:00:00.0 \
> --vdev net_vhost0,iface=plop.sock,client=1\
> -- \
> -p 0x1 \
> --interrupt-only \
> --config '(0,0,1)' \
> --parse-ptype 0
>
> Then start testpmd with virtio-user.
>
> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> -a 0000:00:00.0 \
> --vdev net_virtio_user0,path=plop.sock,server=1 \
> -- \
> -i
>
> Finally stop testpmd.
> ASan then splats in dpdk-l3fwd-power:
>
> =================================================================
> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> 0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> sp 0x7fddbd2eee18
> READ of size 8 at 0x000005ed0778 thread T2
> #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
> #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
> #2 0x3ac95cb in eth_rxq_intr_disable
> .../drivers/net/vhost/rte_eth_vhost.c:647:8
> #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> .../lib/ethdev/rte_ethdev.c:5443:25
> #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
> #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
> #6 0x17f9292 in eal_thread_loop
> .../lib/eal/common/eal_common_thread.c:210:9
> #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
> #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>
> 0x000005ed0778 is located 8 bytes to the left of global variable
> 'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> (0x5ed0780) of size 8192
> 0x000005ed0778 is located 20 bytes to the right of global variable
> 'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> (0x5ed0760) of size 4
> SUMMARY: AddressSanitizer: global-buffer-overflow
> .../lib/vhost/vhost.h:801:27 in get_device
> Shadow bytes around the buggy address:
> 0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> 0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> 0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
> 0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
> 0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
> 0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone: cb
> Thread T2 created by T0 here:
> #0 0xe98996 in __interceptor_pthread_create
> (.examples/dpdk-l3fwd-power+0xe98996)
> (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
> #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
> #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
> #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
> #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>
> ==3641005==ABORTING
>
> More generally, any application passing an incorrect vid would trigger
> such an OOB access.
>
> Cc: stable@dpdk.org
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Note: even after this patch, reconnecting the virtio-user port with
> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> pmd keeps complaining about an issue with rx interrupt fds.
>
> ---
> lib/vhost/vhost.h | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 5750f0c005..d9e97280fa 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
> static __rte_always_inline struct virtio_net *
> get_device(int vid)
> {
> - struct virtio_net *dev = vhost_devices[vid];
> + struct virtio_net *dev = NULL;
> +
> + if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
Maybe use likely() to help the compiler?
> + dev = vhost_devices[vid];
>
> if (unlikely(!dev)) {
> VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] vhost: fix OOB access for invalid vid
2023-02-27 16:19 ` Maxime Coquelin
@ 2023-02-28 8:27 ` David Marchand
2023-02-28 9:24 ` Maxime Coquelin
0 siblings, 1 reply; 6+ messages in thread
From: David Marchand @ 2023-02-28 8:27 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, stable, Chenbo Xia
On Mon, Feb 27, 2023 at 5:19 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> On 2/27/23 11:59, David Marchand wrote:
> > The net/vhost pmd currently provides a -1 vid when disabling interrupt
> > after a virtio port got disconnected.
> >
> > This can be caught when running with ASan.
> >
> > First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
> >
> > $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> > -a 0000:00:00.0 \
> > --vdev net_vhost0,iface=plop.sock,client=1\
> > -- \
> > -p 0x1 \
> > --interrupt-only \
> > --config '(0,0,1)' \
> > --parse-ptype 0
> >
> > Then start testpmd with virtio-user.
> >
> > $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> > -a 0000:00:00.0 \
> > --vdev net_virtio_user0,path=plop.sock,server=1 \
> > -- \
> > -i
> >
> > Finally stop testpmd.
> > ASan then splats in dpdk-l3fwd-power:
> >
> > =================================================================
> > ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> > 0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> > sp 0x7fddbd2eee18
> > READ of size 8 at 0x000005ed0778 thread T2
> > #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
> > #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
> > #2 0x3ac95cb in eth_rxq_intr_disable
> > .../drivers/net/vhost/rte_eth_vhost.c:647:8
> > #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> > .../lib/ethdev/rte_ethdev.c:5443:25
> > #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
> > #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
> > #6 0x17f9292 in eal_thread_loop
> > .../lib/eal/common/eal_common_thread.c:210:9
> > #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
> > #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> > (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> > #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> > (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> >
> > 0x000005ed0778 is located 8 bytes to the left of global variable
> > 'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> > (0x5ed0780) of size 8192
> > 0x000005ed0778 is located 20 bytes to the right of global variable
> > 'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> > (0x5ed0760) of size 4
> > SUMMARY: AddressSanitizer: global-buffer-overflow
> > .../lib/vhost/vhost.h:801:27 in get_device
> > Shadow bytes around the buggy address:
> > 0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> > 0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> > 0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
> > 0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
> > 0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> > =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
> > 0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > Shadow byte legend (one shadow byte represents 8 application bytes):
> > Addressable: 00
> > Partially addressable: 01 02 03 04 05 06 07
> > Heap left redzone: fa
> > Freed heap region: fd
> > Stack left redzone: f1
> > Stack mid redzone: f2
> > Stack right redzone: f3
> > Stack after return: f5
> > Stack use after scope: f8
> > Global redzone: f9
> > Global init order: f6
> > Poisoned by user: f7
> > Container overflow: fc
> > Array cookie: ac
> > Intra object redzone: bb
> > ASan internal: fe
> > Left alloca redzone: ca
> > Right alloca redzone: cb
> > Thread T2 created by T0 here:
> > #0 0xe98996 in __interceptor_pthread_create
> > (.examples/dpdk-l3fwd-power+0xe98996)
> > (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
> > #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
> > #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
> > #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
> > #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> > (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> >
> > ==3641005==ABORTING
> >
> > More generally, any application passing an incorrect vid would trigger
> > such an OOB access.
> >
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> > Note: even after this patch, reconnecting the virtio-user port with
> > dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> > pmd keeps complaining about an issue with rx interrupt fds.
> >
> > ---
> > lib/vhost/vhost.h | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> > index 5750f0c005..d9e97280fa 100644
> > --- a/lib/vhost/vhost.h
> > +++ b/lib/vhost/vhost.h
> > @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
> > static __rte_always_inline struct virtio_net *
> > get_device(int vid)
> > {
> > - struct virtio_net *dev = vhost_devices[vid];
> > + struct virtio_net *dev = NULL;
> > +
> > + if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
>
> Maybe use likely() to help the compiler?
I see no difference in x86 generated code with gcc 12 but I don't mind
adding it (or you can add it while applying ? :-)).
>
> > + dev = vhost_devices[vid];
> >
> > if (unlikely(!dev)) {
> > VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
--
David Marchand
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] vhost: fix OOB access for invalid vid
2023-02-28 8:27 ` David Marchand
@ 2023-02-28 9:24 ` Maxime Coquelin
0 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2023-02-28 9:24 UTC (permalink / raw)
To: David Marchand; +Cc: dev, stable, Chenbo Xia
On 2/28/23 09:27, David Marchand wrote:
> On Mon, Feb 27, 2023 at 5:19 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> On 2/27/23 11:59, David Marchand wrote:
>>> The net/vhost pmd currently provides a -1 vid when disabling interrupt
>>> after a virtio port got disconnected.
>>>
>>> This can be caught when running with ASan.
>>>
>>> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
>>>
>>> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
>>> -a 0000:00:00.0 \
>>> --vdev net_vhost0,iface=plop.sock,client=1\
>>> -- \
>>> -p 0x1 \
>>> --interrupt-only \
>>> --config '(0,0,1)' \
>>> --parse-ptype 0
>>>
>>> Then start testpmd with virtio-user.
>>>
>>> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
>>> -a 0000:00:00.0 \
>>> --vdev net_virtio_user0,path=plop.sock,server=1 \
>>> -- \
>>> -i
>>>
>>> Finally stop testpmd.
>>> ASan then splats in dpdk-l3fwd-power:
>>>
>>> =================================================================
>>> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
>>> 0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
>>> sp 0x7fddbd2eee18
>>> READ of size 8 at 0x000005ed0778 thread T2
>>> #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
>>> #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
>>> #2 0x3ac95cb in eth_rxq_intr_disable
>>> .../drivers/net/vhost/rte_eth_vhost.c:647:8
>>> #3 0x170e0bf in rte_eth_dev_rx_intr_disable
>>> .../lib/ethdev/rte_ethdev.c:5443:25
>>> #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
>>> #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
>>> #6 0x17f9292 in eal_thread_loop
>>> .../lib/eal/common/eal_common_thread.c:210:9
>>> #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
>>> #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
>>> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>>> #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
>>> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>>>
>>> 0x000005ed0778 is located 8 bytes to the left of global variable
>>> 'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
>>> (0x5ed0780) of size 8192
>>> 0x000005ed0778 is located 20 bytes to the right of global variable
>>> 'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
>>> (0x5ed0760) of size 4
>>> SUMMARY: AddressSanitizer: global-buffer-overflow
>>> .../lib/vhost/vhost.h:801:27 in get_device
>>> Shadow bytes around the buggy address:
>>> 0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>>> 0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>>> 0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
>>> 0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
>>> 0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
>>> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
>>> 0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> Shadow byte legend (one shadow byte represents 8 application bytes):
>>> Addressable: 00
>>> Partially addressable: 01 02 03 04 05 06 07
>>> Heap left redzone: fa
>>> Freed heap region: fd
>>> Stack left redzone: f1
>>> Stack mid redzone: f2
>>> Stack right redzone: f3
>>> Stack after return: f5
>>> Stack use after scope: f8
>>> Global redzone: f9
>>> Global init order: f6
>>> Poisoned by user: f7
>>> Container overflow: fc
>>> Array cookie: ac
>>> Intra object redzone: bb
>>> ASan internal: fe
>>> Left alloca redzone: ca
>>> Right alloca redzone: cb
>>> Thread T2 created by T0 here:
>>> #0 0xe98996 in __interceptor_pthread_create
>>> (.examples/dpdk-l3fwd-power+0xe98996)
>>> (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
>>> #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
>>> #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
>>> #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
>>> #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
>>> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>>>
>>> ==3641005==ABORTING
>>>
>>> More generally, any application passing an incorrect vid would trigger
>>> such an OOB access.
>>>
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: David Marchand <david.marchand@redhat.com>
>>> ---
>>> Note: even after this patch, reconnecting the virtio-user port with
>>> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
>>> pmd keeps complaining about an issue with rx interrupt fds.
>>>
>>> ---
>>> lib/vhost/vhost.h | 5 ++++-
>>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
>>> index 5750f0c005..d9e97280fa 100644
>>> --- a/lib/vhost/vhost.h
>>> +++ b/lib/vhost/vhost.h
>>> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
>>> static __rte_always_inline struct virtio_net *
>>> get_device(int vid)
>>> {
>>> - struct virtio_net *dev = vhost_devices[vid];
>>> + struct virtio_net *dev = NULL;
>>> +
>>> + if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
>>
>> Maybe use likely() to help the compiler?
>
> I see no difference in x86 generated code with gcc 12 but I don't mind
> adding it (or you can add it while applying ? :-)).
I can add it while applying.
Thanks!
Maxime
>
>
>>
>>> + dev = vhost_devices[vid];
>>>
>>> if (unlikely(!dev)) {
>>> VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] vhost: fix OOB access for invalid vid
2023-02-27 10:59 [PATCH] vhost: fix OOB access for invalid vid David Marchand
2023-02-27 16:19 ` Maxime Coquelin
@ 2023-03-06 14:07 ` Maxime Coquelin
2023-03-06 14:22 ` Maxime Coquelin
2 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2023-03-06 14:07 UTC (permalink / raw)
To: David Marchand, dev; +Cc: stable, Chenbo Xia
On 2/27/23 11:59, David Marchand wrote:
> The net/vhost pmd currently provides a -1 vid when disabling interrupt
> after a virtio port got disconnected.
>
> This can be caught when running with ASan.
>
> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
>
> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> -a 0000:00:00.0 \
> --vdev net_vhost0,iface=plop.sock,client=1\
> -- \
> -p 0x1 \
> --interrupt-only \
> --config '(0,0,1)' \
> --parse-ptype 0
>
> Then start testpmd with virtio-user.
>
> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> -a 0000:00:00.0 \
> --vdev net_virtio_user0,path=plop.sock,server=1 \
> -- \
> -i
>
> Finally stop testpmd.
> ASan then splats in dpdk-l3fwd-power:
>
> =================================================================
> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> 0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> sp 0x7fddbd2eee18
> READ of size 8 at 0x000005ed0778 thread T2
> #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
> #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
> #2 0x3ac95cb in eth_rxq_intr_disable
> .../drivers/net/vhost/rte_eth_vhost.c:647:8
> #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> .../lib/ethdev/rte_ethdev.c:5443:25
> #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
> #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
> #6 0x17f9292 in eal_thread_loop
> .../lib/eal/common/eal_common_thread.c:210:9
> #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
> #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>
> 0x000005ed0778 is located 8 bytes to the left of global variable
> 'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> (0x5ed0780) of size 8192
> 0x000005ed0778 is located 20 bytes to the right of global variable
> 'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> (0x5ed0760) of size 4
> SUMMARY: AddressSanitizer: global-buffer-overflow
> .../lib/vhost/vhost.h:801:27 in get_device
> Shadow bytes around the buggy address:
> 0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> 0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> 0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
> 0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
> 0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
> 0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone: cb
> Thread T2 created by T0 here:
> #0 0xe98996 in __interceptor_pthread_create
> (.examples/dpdk-l3fwd-power+0xe98996)
> (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
> #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
> #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
> #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
> #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>
> ==3641005==ABORTING
>
> More generally, any application passing an incorrect vid would trigger
> such an OOB access.
>
> Cc: stable@dpdk.org
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Note: even after this patch, reconnecting the virtio-user port with
> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> pmd keeps complaining about an issue with rx interrupt fds.
>
> ---
> lib/vhost/vhost.h | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 5750f0c005..d9e97280fa 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
> static __rte_always_inline struct virtio_net *
> get_device(int vid)
> {
> - struct virtio_net *dev = vhost_devices[vid];
> + struct virtio_net *dev = NULL;
> +
> + if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
> + dev = vhost_devices[vid];
>
> if (unlikely(!dev)) {
> VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Thanks,
Maxime
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] vhost: fix OOB access for invalid vid
2023-02-27 10:59 [PATCH] vhost: fix OOB access for invalid vid David Marchand
2023-02-27 16:19 ` Maxime Coquelin
2023-03-06 14:07 ` Maxime Coquelin
@ 2023-03-06 14:22 ` Maxime Coquelin
2 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2023-03-06 14:22 UTC (permalink / raw)
To: David Marchand, dev; +Cc: stable, Chenbo Xia
On 2/27/23 11:59, David Marchand wrote:
> The net/vhost pmd currently provides a -1 vid when disabling interrupt
> after a virtio port got disconnected.
>
> This can be caught when running with ASan.
>
> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
>
> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> -a 0000:00:00.0 \
> --vdev net_vhost0,iface=plop.sock,client=1\
> -- \
> -p 0x1 \
> --interrupt-only \
> --config '(0,0,1)' \
> --parse-ptype 0
>
> Then start testpmd with virtio-user.
>
> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> -a 0000:00:00.0 \
> --vdev net_virtio_user0,path=plop.sock,server=1 \
> -- \
> -i
>
> Finally stop testpmd.
> ASan then splats in dpdk-l3fwd-power:
>
> =================================================================
> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> 0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> sp 0x7fddbd2eee18
> READ of size 8 at 0x000005ed0778 thread T2
> #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
> #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
> #2 0x3ac95cb in eth_rxq_intr_disable
> .../drivers/net/vhost/rte_eth_vhost.c:647:8
> #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> .../lib/ethdev/rte_ethdev.c:5443:25
> #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
> #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
> #6 0x17f9292 in eal_thread_loop
> .../lib/eal/common/eal_common_thread.c:210:9
> #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
> #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>
> 0x000005ed0778 is located 8 bytes to the left of global variable
> 'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> (0x5ed0780) of size 8192
> 0x000005ed0778 is located 20 bytes to the right of global variable
> 'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> (0x5ed0760) of size 4
> SUMMARY: AddressSanitizer: global-buffer-overflow
> .../lib/vhost/vhost.h:801:27 in get_device
> Shadow bytes around the buggy address:
> 0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> 0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> 0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
> 0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
> 0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
> 0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone: cb
> Thread T2 created by T0 here:
> #0 0xe98996 in __interceptor_pthread_create
> (.examples/dpdk-l3fwd-power+0xe98996)
> (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
> #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
> #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
> #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
> #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>
> ==3641005==ABORTING
>
> More generally, any application passing an incorrect vid would trigger
> such an OOB access.
>
> Cc: stable@dpdk.org
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Note: even after this patch, reconnecting the virtio-user port with
> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> pmd keeps complaining about an issue with rx interrupt fds.
>
> ---
> lib/vhost/vhost.h | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 5750f0c005..d9e97280fa 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
> static __rte_always_inline struct virtio_net *
> get_device(int vid)
> {
> - struct virtio_net *dev = vhost_devices[vid];
> + struct virtio_net *dev = NULL;
> +
> + if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
> + dev = vhost_devices[vid];
>
> if (unlikely(!dev)) {
> VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
Applied to dpdk-next-virtio/main.
Thanks,
Maxime
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-03-06 14:23 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-27 10:59 [PATCH] vhost: fix OOB access for invalid vid David Marchand
2023-02-27 16:19 ` Maxime Coquelin
2023-02-28 8:27 ` David Marchand
2023-02-28 9:24 ` Maxime Coquelin
2023-03-06 14:07 ` Maxime Coquelin
2023-03-06 14:22 ` Maxime Coquelin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).