* [dpdk-dev] Deadlock when start virtio_user + vhost_kernel
@ 2018-09-12 3:47 Jason Wang
2018-09-12 4:27 ` Tiwei Bie
0 siblings, 1 reply; 3+ messages in thread
From: Jason Wang @ 2018-09-12 3:47 UTC (permalink / raw)
To: dev; +Cc: Tiwei Bie, Maxime Coquelin
Hi:
Try to launch virtio_user + vhost_kernel with: testpmd
--vdev=virtio_user0,path=/dev/vhost-net -- -i
It seems we get a deadlock on
rte_rwlock_read_lock(&mcfg->memory_hotplug_lock)
calltrace:
Thread 1 "testpmd" received signal SIGINT, Interrupt.
rte_memseg_contig_walk (func=func@entry=0x555555a5e630
<add_memory_region>, arg=arg@entry=0x7fffffffcec0)
at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
469 rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
(gdb) bt
#0 rte_memseg_contig_walk (func=func@entry=0x555555a5e630
<add_memory_region>, arg=arg@entry=0x7fffffffcec0)
at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
#1 0x0000555555a5e9b1 in prepare_vhost_memory_kernel () at
/home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:118
#2 vhost_kernel_ioctl (dev=0x7ffbf5fb3300, req=<optimized out>,
arg=<optimized out>)
at
/home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:190
#3 0x0000555555a5f211 in virtio_user_mem_event_cb (type=<optimized
out>, addr=<optimized out>, len=<optimized out>, arg=0x7ffbf5fb3300)
at
/home/devel/git/dpdk/drivers/net/virtio/virtio_user/virtio_user_dev.c:297
#4 0x000055555574814b in eal_memalloc_mem_event_notify
(event=event@entry=RTE_MEM_EVENT_ALLOC, start=start@entry=0x7ffbf6000000,
len=len@entry=94371840) at
/home/devel/git/dpdk/lib/librte_eal/common/eal_common_memalloc.c:248
#5 0x00005555557563f6 in try_expand_heap_primary (contig=false,
bound=0, align=64, flags=0, socket=0, elt_size=0, pg_sz=<optimized out>,
heap=0x7ffff7ff667c) at
/home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:344
#6 try_expand_heap (heap=heap@entry=0x7ffff7ff667c, pg_sz=<optimized
out>, elt_size=elt_size@entry=92403968, socket=socket@entry=0,
flags=flags@entry=0, align=align@entry=64, bound=0, contig=false)
at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:426
#7 0x0000555555756928 in alloc_more_mem_on_socket
(heap=heap@entry=0x7ffff7ff667c, size=size@entry=92403968,
socket=socket@entry=0,
flags=flags@entry=0, align=align@entry=64, bound=bound@entry=0,
contig=false) at
/home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:554
#8 0x0000555555756e37 in heap_alloc_on_socket (contig=false, bound=0,
align=64, flags=0, socket=0, size=92403968, type=<optimized out>)
at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:590
#9 malloc_heap_alloc (type=<optimized out>, size=92403968,
socket_arg=<optimized out>, flags=0, align=<optimized out>, bound=0,
contig=false)
at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:626
#10 0x0000555555753fc1 in rte_zmalloc () at
/home/devel/git/dpdk/lib/librte_eal/common/rte_malloc.c:74
#11 0x00005555556192f9 in init_port () at
/home/devel/git/dpdk/app/test-pmd/testpmd.c:2645
#12 main (argc=4, argv=0x7fffffffdb18) at
/home/devel/git/dpdk/app/test-pmd/testpmd.c:2734
And I also get this warning:
vhost_kernel_ioctl(): VHOST_SET_OWNER failed: Device or resource busy
Both looks like a bug that needs to be fixed.
Thanks
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] Deadlock when start virtio_user + vhost_kernel
2018-09-12 3:47 [dpdk-dev] Deadlock when start virtio_user + vhost_kernel Jason Wang
@ 2018-09-12 4:27 ` Tiwei Bie
2018-09-12 8:14 ` Jason Wang
0 siblings, 1 reply; 3+ messages in thread
From: Tiwei Bie @ 2018-09-12 4:27 UTC (permalink / raw)
To: Jason Wang; +Cc: dev, Maxime Coquelin
On Wed, Sep 12, 2018 at 11:47:20AM +0800, Jason Wang wrote:
> Hi:
>
> Try to launch virtio_user + vhost_kernel with: testpmd
> --vdev=virtio_user0,path=/dev/vhost-net -- -i
>
> It seems we get a deadlock on
> rte_rwlock_read_lock(&mcfg->memory_hotplug_lock)
Yes, you're right. There is a deadlock here.
FYI, it can be fixed by below patch:
http://patches.dpdk.org/patch/44290/
>
> calltrace:
>
> Thread 1 "testpmd" received signal SIGINT, Interrupt.
> rte_memseg_contig_walk (func=func@entry=0x555555a5e630 <add_memory_region>,
> arg=arg@entry=0x7fffffffcec0)
> at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
> 469 rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
> (gdb) bt
> #0 rte_memseg_contig_walk (func=func@entry=0x555555a5e630
> <add_memory_region>, arg=arg@entry=0x7fffffffcec0)
> at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
> #1 0x0000555555a5e9b1 in prepare_vhost_memory_kernel () at
> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:118
> #2 vhost_kernel_ioctl (dev=0x7ffbf5fb3300, req=<optimized out>,
> arg=<optimized out>)
> at
> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:190
> #3 0x0000555555a5f211 in virtio_user_mem_event_cb (type=<optimized out>,
> addr=<optimized out>, len=<optimized out>, arg=0x7ffbf5fb3300)
> at
> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/virtio_user_dev.c:297
> #4 0x000055555574814b in eal_memalloc_mem_event_notify
> (event=event@entry=RTE_MEM_EVENT_ALLOC, start=start@entry=0x7ffbf6000000,
> len=len@entry=94371840) at
> /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memalloc.c:248
> #5 0x00005555557563f6 in try_expand_heap_primary (contig=false, bound=0,
> align=64, flags=0, socket=0, elt_size=0, pg_sz=<optimized out>,
> heap=0x7ffff7ff667c) at
> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:344
> #6 try_expand_heap (heap=heap@entry=0x7ffff7ff667c, pg_sz=<optimized out>,
> elt_size=elt_size@entry=92403968, socket=socket@entry=0,
> flags=flags@entry=0, align=align@entry=64, bound=0, contig=false) at
> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:426
> #7 0x0000555555756928 in alloc_more_mem_on_socket
> (heap=heap@entry=0x7ffff7ff667c, size=size@entry=92403968,
> socket=socket@entry=0,
> flags=flags@entry=0, align=align@entry=64, bound=bound@entry=0,
> contig=false) at
> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:554
> #8 0x0000555555756e37 in heap_alloc_on_socket (contig=false, bound=0,
> align=64, flags=0, socket=0, size=92403968, type=<optimized out>)
> at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:590
> #9 malloc_heap_alloc (type=<optimized out>, size=92403968,
> socket_arg=<optimized out>, flags=0, align=<optimized out>, bound=0,
> contig=false)
> at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:626
> #10 0x0000555555753fc1 in rte_zmalloc () at
> /home/devel/git/dpdk/lib/librte_eal/common/rte_malloc.c:74
> #11 0x00005555556192f9 in init_port () at
> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2645
> #12 main (argc=4, argv=0x7fffffffdb18) at
> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2734
>
> And I also get this warning:
>
> vhost_kernel_ioctl(): VHOST_SET_OWNER failed: Device or resource busy
Since below commit:
bce7e9050f9b ("net/virtio-user: fix start with kernel vhost")
https://github.com/DPDK/dpdk/commit/bce7e9050f9b
The vhost SET_OWNER will be done in virtio_user_start_device()
unconditionally. It caused above harmless but annoying warning.
>
> Both looks like a bug that needs to be fixed.
>
> Thanks
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-dev] Deadlock when start virtio_user + vhost_kernel
2018-09-12 4:27 ` Tiwei Bie
@ 2018-09-12 8:14 ` Jason Wang
0 siblings, 0 replies; 3+ messages in thread
From: Jason Wang @ 2018-09-12 8:14 UTC (permalink / raw)
To: Tiwei Bie; +Cc: dev, Maxime Coquelin
On 2018年09月12日 12:27, Tiwei Bie wrote:
> On Wed, Sep 12, 2018 at 11:47:20AM +0800, Jason Wang wrote:
>> Hi:
>>
>> Try to launch virtio_user + vhost_kernel with: testpmd
>> --vdev=virtio_user0,path=/dev/vhost-net -- -i
>>
>> It seems we get a deadlock on
>> rte_rwlock_read_lock(&mcfg->memory_hotplug_lock)
> Yes, you're right. There is a deadlock here.
> FYI, it can be fixed by below patch:
> http://patches.dpdk.org/patch/44290/
Thanks for the pointer, this fixes the deadlock.
>
>> calltrace:
>>
>> Thread 1 "testpmd" received signal SIGINT, Interrupt.
>> rte_memseg_contig_walk (func=func@entry=0x555555a5e630 <add_memory_region>,
>> arg=arg@entry=0x7fffffffcec0)
>> at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
>> 469 rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
>> (gdb) bt
>> #0 rte_memseg_contig_walk (func=func@entry=0x555555a5e630
>> <add_memory_region>, arg=arg@entry=0x7fffffffcec0)
>> at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
>> #1 0x0000555555a5e9b1 in prepare_vhost_memory_kernel () at
>> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:118
>> #2 vhost_kernel_ioctl (dev=0x7ffbf5fb3300, req=<optimized out>,
>> arg=<optimized out>)
>> at
>> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:190
>> #3 0x0000555555a5f211 in virtio_user_mem_event_cb (type=<optimized out>,
>> addr=<optimized out>, len=<optimized out>, arg=0x7ffbf5fb3300)
>> at
>> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/virtio_user_dev.c:297
>> #4 0x000055555574814b in eal_memalloc_mem_event_notify
>> (event=event@entry=RTE_MEM_EVENT_ALLOC, start=start@entry=0x7ffbf6000000,
>> len=len@entry=94371840) at
>> /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memalloc.c:248
>> #5 0x00005555557563f6 in try_expand_heap_primary (contig=false, bound=0,
>> align=64, flags=0, socket=0, elt_size=0, pg_sz=<optimized out>,
>> heap=0x7ffff7ff667c) at
>> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:344
>> #6 try_expand_heap (heap=heap@entry=0x7ffff7ff667c, pg_sz=<optimized out>,
>> elt_size=elt_size@entry=92403968, socket=socket@entry=0,
>> flags=flags@entry=0, align=align@entry=64, bound=0, contig=false) at
>> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:426
>> #7 0x0000555555756928 in alloc_more_mem_on_socket
>> (heap=heap@entry=0x7ffff7ff667c, size=size@entry=92403968,
>> socket=socket@entry=0,
>> flags=flags@entry=0, align=align@entry=64, bound=bound@entry=0,
>> contig=false) at
>> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:554
>> #8 0x0000555555756e37 in heap_alloc_on_socket (contig=false, bound=0,
>> align=64, flags=0, socket=0, size=92403968, type=<optimized out>)
>> at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:590
>> #9 malloc_heap_alloc (type=<optimized out>, size=92403968,
>> socket_arg=<optimized out>, flags=0, align=<optimized out>, bound=0,
>> contig=false)
>> at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:626
>> #10 0x0000555555753fc1 in rte_zmalloc () at
>> /home/devel/git/dpdk/lib/librte_eal/common/rte_malloc.c:74
>> #11 0x00005555556192f9 in init_port () at
>> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2645
>> #12 main (argc=4, argv=0x7fffffffdb18) at
>> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2734
>>
>> And I also get this warning:
>>
>> vhost_kernel_ioctl(): VHOST_SET_OWNER failed: Device or resource busy
> Since below commit:
> bce7e9050f9b ("net/virtio-user: fix start with kernel vhost")
> https://github.com/DPDK/dpdk/commit/bce7e9050f9b
>
> The vhost SET_OWNER will be done in virtio_user_start_device()
> unconditionally. It caused above harmless but annoying warning.
Ok, but it's better to remove it anyway in the future.
Thanks
>> Both looks like a bug that needs to be fixed.
>>
>> Thanks
>>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-09-12 8:15 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-12 3:47 [dpdk-dev] Deadlock when start virtio_user + vhost_kernel Jason Wang
2018-09-12 4:27 ` Tiwei Bie
2018-09-12 8:14 ` Jason Wang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).