DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev]  [4.4 kernel] kni lockup, kernel dump
@ 2016-04-13  0:19 ALeX Wang
  2016-04-13  0:28 ` ALeX Wang
  0 siblings, 1 reply; 6+ messages in thread
From: ALeX Wang @ 2016-04-13  0:19 UTC (permalink / raw)
  To: dev

Hi,

I recently upgraded my debian/jessie to 4.4 kernel, and my application uses
kni to
create test interface,

However, when doing 'rte_kni_alloc()', i observed the following log in
syslog...
If i go on trying setting interface via ifconfig, the execution locks up...

[  888.051427] BUG: unable to handle kernel paging request at
0000073b010bcb88
[  888.051972] IP: [<ffffffffa051a59a>] kni_net_rx_normal+0x3a/0x290
[rte_kni]
[  888.052371] PGD 0
[  888.052638] Oops: 0000 [#1] SMP
[  888.053041] Modules linked in: rte_kni(O) xt_conntrack ipt_MASQUERADE
nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat
nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data
dm_bio_prison dm_bufio libcrc32c crc32c_generic loop dm_mod nfsd
auth_rpcgss oid_registry nfs_acl nfs lockd grace fscache sunrpc
crct10dif_pclmul crc32_pclmul jitterentropy_rng sha256_ssse3 sha256_generic
hmac drbg ansi_cprng ppdev parport_pc i2c_piix4 aesni_intel aes_x86_64
parport acpi_cpufreq tpm_tis tpm lrw gf128mul glue_helper ablk_helper evdev
cryptd 8250_fintek psmouse processor serio_raw video battery button ac
pcspkr autofs4 ext4 crc16 mbcache jbd2 sg sd_mod ata_generic ahci
crc32c_intel libahci ata_piix fjes libata
[  888.065118]  e1000 scsi_mod
[  888.065476] CPU: 1 PID: 2226 Comm: kni_single Tainted: G           O
 4.4.0-0.bpo.1-amd64 #1 Debian 4.4.6-1~bpo8+1
[  888.065861] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
VirtualBox 12/01/2006
[  888.066183] task: ffff88011ab9e0c0 ti: ffff8800b2fb8000 task.ti:
ffff8800b2fb8000
[  888.066666] RIP: 0010:[<ffffffffa051a59a>]  [<ffffffffa051a59a>]
kni_net_rx_normal+0x3a/0x290 [rte_kni]
[  888.094268] RSP: 0018:ffff8800b2fbbd40  EFLAGS: 00010246
[  888.094879] RAX: ffff8800da073000 RBX: ffff8800d88bc2f8 RCX:
0000000000000000
[  888.095392] RDX: 0000073b010bcb80 RSI: 0000000000000282 RDI:
ffff8800da073840
[  888.095929] RBP: 00000000000003e8 R08: ffff8800b2fb8000 R09:
000000cec3fd1e2c
[  888.096425] R10: 0000000000000002 R11: 0000000000000000 R12:
ffff8800d88bc2c0
[  888.096913] R13: ffff8800d88bc2d0 R14: ffff8800da073840 R15:
ffff8800da073840
[  888.097382] FS:  0000000000000000(0000) GS:ffff88011fc80000(0000)
knlGS:0000000000000000
[  888.098052] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  888.098490] CR2: 0000073b010bcb88 CR3: 0000000000044000 CR4:
00000000000406e0
[  888.098958] Stack:
[  888.099294]  ffff8800b2fb8000 ffff88011fc95d80 ffff88011ab9e0c0
0000000000000000
[  888.100428]  ffffffff810bc311 0000000000000000 0000000000000002
000000cec3fd1e2c
[  888.101629]  ffff8800b2fb8000 ffff88011fc8df80 0000000000000282
0000000000000001
[  888.103134] Call Trace:
[  888.103607]  [<ffffffff810bc311>] ?
__raw_callee_save___pv_queued_spin_unlock+0x11/0x20
[  888.104553]  [<ffffffff810dc2c9>] ? try_to_del_timer_sync+0x59/0x80
[  888.105290]  [<ffffffff810dc334>] ? del_timer_sync+0x44/0x50
[  888.105722]  [<ffffffff81591839>] ? schedule_timeout+0x169/0x2d0
[  888.106159]  [<ffffffff810dbdd0>] ?
trace_event_raw_event_tick_stop+0x100/0x100
[  888.106816]  [<ffffffffa051a05c>] ? kni_thread_single+0x4c/0xa0 [rte_kni]
[  888.107279]  [<ffffffffa051a010>] ? kni_init_net+0x50/0x50 [rte_kni]
[  888.107754]  [<ffffffff81095baf>] ? kthread+0xdf/0x100
[  888.108177]  [<ffffffff81095ad0>] ? kthread_park+0x50/0x50
[  888.108600]  [<ffffffff81592adf>] ? ret_from_fork+0x3f/0x70
[  888.109026]  [<ffffffff81095ad0>] ? kthread_park+0x50/0x50
[  888.109448] Code: 54 55 53 48 81 ec 28 01 00 00 48 8b 97 78 01 00 00 65
48 8b 04 25 28 00 00 00 48 89 84 24 20 01 00 00 31 c0 48 8b 87 48 01 00 00
<44> 8b 72 08 48 89 04 24 8b 42 04 8b 0a 41 83 ee 01 83 e8 01 29
[  888.145305] RIP  [<ffffffffa051a59a>] kni_net_rx_normal+0x3a/0x290
[rte_kni]
[  888.145904]  RSP <ffff8800b2fbbd40>
[  888.146337] CR2: 0000073b010bcb88
[  888.146705] ---[ end trace ef3848430517129b ]---

-- 
Alex Wang,

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [4.4 kernel] kni lockup, kernel dump
  2016-04-13  0:19 [dpdk-dev] [4.4 kernel] kni lockup, kernel dump ALeX Wang
@ 2016-04-13  0:28 ` ALeX Wang
  2016-04-13 22:26   ` ALeX Wang
  0 siblings, 1 reply; 6+ messages in thread
From: ALeX Wang @ 2016-04-13  0:28 UTC (permalink / raw)
  To: dev

I tried compiling both dpdk-2.2 and dpdk-16.04, all have the issue,

Thanks,

On 12 April 2016 at 17:19, ALeX Wang <ee07b291@gmail.com> wrote:

> Hi,
>
> I recently upgraded my debian/jessie to 4.4 kernel, and my application
> uses kni to
> create test interface,
>
> However, when doing 'rte_kni_alloc()', i observed the following log in
> syslog...
> If i go on trying setting interface via ifconfig, the execution locks up...
>
> [  888.051427] BUG: unable to handle kernel paging request at
> 0000073b010bcb88
> [  888.051972] IP: [<ffffffffa051a59a>] kni_net_rx_normal+0x3a/0x290
> [rte_kni]
> [  888.052371] PGD 0
> [  888.052638] Oops: 0000 [#1] SMP
> [  888.053041] Modules linked in: rte_kni(O) xt_conntrack ipt_MASQUERADE
> nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
> nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat
> nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data
> dm_bio_prison dm_bufio libcrc32c crc32c_generic loop dm_mod nfsd
> auth_rpcgss oid_registry nfs_acl nfs lockd grace fscache sunrpc
> crct10dif_pclmul crc32_pclmul jitterentropy_rng sha256_ssse3 sha256_generic
> hmac drbg ansi_cprng ppdev parport_pc i2c_piix4 aesni_intel aes_x86_64
> parport acpi_cpufreq tpm_tis tpm lrw gf128mul glue_helper ablk_helper evdev
> cryptd 8250_fintek psmouse processor serio_raw video battery button ac
> pcspkr autofs4 ext4 crc16 mbcache jbd2 sg sd_mod ata_generic ahci
> crc32c_intel libahci ata_piix fjes libata
> [  888.065118]  e1000 scsi_mod
> [  888.065476] CPU: 1 PID: 2226 Comm: kni_single Tainted: G           O
>  4.4.0-0.bpo.1-amd64 #1 Debian 4.4.6-1~bpo8+1
> [  888.065861] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
> VirtualBox 12/01/2006
> [  888.066183] task: ffff88011ab9e0c0 ti: ffff8800b2fb8000 task.ti:
> ffff8800b2fb8000
> [  888.066666] RIP: 0010:[<ffffffffa051a59a>]  [<ffffffffa051a59a>]
> kni_net_rx_normal+0x3a/0x290 [rte_kni]
> [  888.094268] RSP: 0018:ffff8800b2fbbd40  EFLAGS: 00010246
> [  888.094879] RAX: ffff8800da073000 RBX: ffff8800d88bc2f8 RCX:
> 0000000000000000
> [  888.095392] RDX: 0000073b010bcb80 RSI: 0000000000000282 RDI:
> ffff8800da073840
> [  888.095929] RBP: 00000000000003e8 R08: ffff8800b2fb8000 R09:
> 000000cec3fd1e2c
> [  888.096425] R10: 0000000000000002 R11: 0000000000000000 R12:
> ffff8800d88bc2c0
> [  888.096913] R13: ffff8800d88bc2d0 R14: ffff8800da073840 R15:
> ffff8800da073840
> [  888.097382] FS:  0000000000000000(0000) GS:ffff88011fc80000(0000)
> knlGS:0000000000000000
> [  888.098052] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  888.098490] CR2: 0000073b010bcb88 CR3: 0000000000044000 CR4:
> 00000000000406e0
> [  888.098958] Stack:
> [  888.099294]  ffff8800b2fb8000 ffff88011fc95d80 ffff88011ab9e0c0
> 0000000000000000
> [  888.100428]  ffffffff810bc311 0000000000000000 0000000000000002
> 000000cec3fd1e2c
> [  888.101629]  ffff8800b2fb8000 ffff88011fc8df80 0000000000000282
> 0000000000000001
> [  888.103134] Call Trace:
> [  888.103607]  [<ffffffff810bc311>] ?
> __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
> [  888.104553]  [<ffffffff810dc2c9>] ? try_to_del_timer_sync+0x59/0x80
> [  888.105290]  [<ffffffff810dc334>] ? del_timer_sync+0x44/0x50
> [  888.105722]  [<ffffffff81591839>] ? schedule_timeout+0x169/0x2d0
> [  888.106159]  [<ffffffff810dbdd0>] ?
> trace_event_raw_event_tick_stop+0x100/0x100
> [  888.106816]  [<ffffffffa051a05c>] ? kni_thread_single+0x4c/0xa0
> [rte_kni]
> [  888.107279]  [<ffffffffa051a010>] ? kni_init_net+0x50/0x50 [rte_kni]
> [  888.107754]  [<ffffffff81095baf>] ? kthread+0xdf/0x100
> [  888.108177]  [<ffffffff81095ad0>] ? kthread_park+0x50/0x50
> [  888.108600]  [<ffffffff81592adf>] ? ret_from_fork+0x3f/0x70
> [  888.109026]  [<ffffffff81095ad0>] ? kthread_park+0x50/0x50
> [  888.109448] Code: 54 55 53 48 81 ec 28 01 00 00 48 8b 97 78 01 00 00 65
> 48 8b 04 25 28 00 00 00 48 89 84 24 20 01 00 00 31 c0 48 8b 87 48 01 00 00
> <44> 8b 72 08 48 89 04 24 8b 42 04 8b 0a 41 83 ee 01 83 e8 01 29
> [  888.145305] RIP  [<ffffffffa051a59a>] kni_net_rx_normal+0x3a/0x290
> [rte_kni]
> [  888.145904]  RSP <ffff8800b2fbbd40>
> [  888.146337] CR2: 0000073b010bcb88
> [  888.146705] ---[ end trace ef3848430517129b ]---
>
> --
> Alex Wang,
>
>


-- 
Alex Wang,
Open vSwitch developer

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [4.4 kernel] kni lockup, kernel dump
  2016-04-13  0:28 ` ALeX Wang
@ 2016-04-13 22:26   ` ALeX Wang
  2016-04-14 14:29     ` Ferruh Yigit
  0 siblings, 1 reply; 6+ messages in thread
From: ALeX Wang @ 2016-04-13 22:26 UTC (permalink / raw)
  To: dev

Did more experiment, found that it has nothing to do with the kernel
version,

It only happens when using kni module with '--no-huge' eal flag...

Is that expected?

Thanks,
Alex Wang,

On 12 April 2016 at 17:28, ALeX Wang <ee07b291@gmail.com> wrote:

> I tried compiling both dpdk-2.2 and dpdk-16.04, all have the issue,
>
> Thanks,
>
> On 12 April 2016 at 17:19, ALeX Wang <ee07b291@gmail.com> wrote:
>
>> Hi,
>>
>> I recently upgraded my debian/jessie to 4.4 kernel, and my application
>> uses kni to
>> create test interface,
>>
>> However, when doing 'rte_kni_alloc()', i observed the following log in
>> syslog...
>> If i go on trying setting interface via ifconfig, the execution locks
>> up...
>>
>> [  888.051427] BUG: unable to handle kernel paging request at
>> 0000073b010bcb88
>> [  888.051972] IP: [<ffffffffa051a59a>] kni_net_rx_normal+0x3a/0x290
>> [rte_kni]
>> [  888.052371] PGD 0
>> [  888.052638] Oops: 0000 [#1] SMP
>> [  888.053041] Modules linked in: rte_kni(O) xt_conntrack ipt_MASQUERADE
>> nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
>> nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat
>> nf_conntrack br_netfilter bridge stp llc dm_thin_pool dm_persistent_data
>> dm_bio_prison dm_bufio libcrc32c crc32c_generic loop dm_mod nfsd
>> auth_rpcgss oid_registry nfs_acl nfs lockd grace fscache sunrpc
>> crct10dif_pclmul crc32_pclmul jitterentropy_rng sha256_ssse3 sha256_generic
>> hmac drbg ansi_cprng ppdev parport_pc i2c_piix4 aesni_intel aes_x86_64
>> parport acpi_cpufreq tpm_tis tpm lrw gf128mul glue_helper ablk_helper evdev
>> cryptd 8250_fintek psmouse processor serio_raw video battery button ac
>> pcspkr autofs4 ext4 crc16 mbcache jbd2 sg sd_mod ata_generic ahci
>> crc32c_intel libahci ata_piix fjes libata
>> [  888.065118]  e1000 scsi_mod
>> [  888.065476] CPU: 1 PID: 2226 Comm: kni_single Tainted: G           O
>>  4.4.0-0.bpo.1-amd64 #1 Debian 4.4.6-1~bpo8+1
>> [  888.065861] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
>> VirtualBox 12/01/2006
>> [  888.066183] task: ffff88011ab9e0c0 ti: ffff8800b2fb8000 task.ti:
>> ffff8800b2fb8000
>> [  888.066666] RIP: 0010:[<ffffffffa051a59a>]  [<ffffffffa051a59a>]
>> kni_net_rx_normal+0x3a/0x290 [rte_kni]
>> [  888.094268] RSP: 0018:ffff8800b2fbbd40  EFLAGS: 00010246
>> [  888.094879] RAX: ffff8800da073000 RBX: ffff8800d88bc2f8 RCX:
>> 0000000000000000
>> [  888.095392] RDX: 0000073b010bcb80 RSI: 0000000000000282 RDI:
>> ffff8800da073840
>> [  888.095929] RBP: 00000000000003e8 R08: ffff8800b2fb8000 R09:
>> 000000cec3fd1e2c
>> [  888.096425] R10: 0000000000000002 R11: 0000000000000000 R12:
>> ffff8800d88bc2c0
>> [  888.096913] R13: ffff8800d88bc2d0 R14: ffff8800da073840 R15:
>> ffff8800da073840
>> [  888.097382] FS:  0000000000000000(0000) GS:ffff88011fc80000(0000)
>> knlGS:0000000000000000
>> [  888.098052] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [  888.098490] CR2: 0000073b010bcb88 CR3: 0000000000044000 CR4:
>> 00000000000406e0
>> [  888.098958] Stack:
>> [  888.099294]  ffff8800b2fb8000 ffff88011fc95d80 ffff88011ab9e0c0
>> 0000000000000000
>> [  888.100428]  ffffffff810bc311 0000000000000000 0000000000000002
>> 000000cec3fd1e2c
>> [  888.101629]  ffff8800b2fb8000 ffff88011fc8df80 0000000000000282
>> 0000000000000001
>> [  888.103134] Call Trace:
>> [  888.103607]  [<ffffffff810bc311>] ?
>> __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
>> [  888.104553]  [<ffffffff810dc2c9>] ? try_to_del_timer_sync+0x59/0x80
>> [  888.105290]  [<ffffffff810dc334>] ? del_timer_sync+0x44/0x50
>> [  888.105722]  [<ffffffff81591839>] ? schedule_timeout+0x169/0x2d0
>> [  888.106159]  [<ffffffff810dbdd0>] ?
>> trace_event_raw_event_tick_stop+0x100/0x100
>> [  888.106816]  [<ffffffffa051a05c>] ? kni_thread_single+0x4c/0xa0
>> [rte_kni]
>> [  888.107279]  [<ffffffffa051a010>] ? kni_init_net+0x50/0x50 [rte_kni]
>> [  888.107754]  [<ffffffff81095baf>] ? kthread+0xdf/0x100
>> [  888.108177]  [<ffffffff81095ad0>] ? kthread_park+0x50/0x50
>> [  888.108600]  [<ffffffff81592adf>] ? ret_from_fork+0x3f/0x70
>> [  888.109026]  [<ffffffff81095ad0>] ? kthread_park+0x50/0x50
>> [  888.109448] Code: 54 55 53 48 81 ec 28 01 00 00 48 8b 97 78 01 00 00
>> 65 48 8b 04 25 28 00 00 00 48 89 84 24 20 01 00 00 31 c0 48 8b 87 48 01 00
>> 00 <44> 8b 72 08 48 89 04 24 8b 42 04 8b 0a 41 83 ee 01 83 e8 01 29
>> [  888.145305] RIP  [<ffffffffa051a59a>] kni_net_rx_normal+0x3a/0x290
>> [rte_kni]
>> [  888.145904]  RSP <ffff8800b2fbbd40>
>> [  888.146337] CR2: 0000073b010bcb88
>> [  888.146705] ---[ end trace ef3848430517129b ]---
>>
>> --
>> Alex Wang,
>>
>>
>
>
> --
> Alex Wang,
> Open vSwitch developer
>



-- 
Alex Wang,
Open vSwitch developer

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [4.4 kernel] kni lockup, kernel dump
  2016-04-13 22:26   ` ALeX Wang
@ 2016-04-14 14:29     ` Ferruh Yigit
  2016-04-14 14:43       ` Thomas Monjalon
  0 siblings, 1 reply; 6+ messages in thread
From: Ferruh Yigit @ 2016-04-14 14:29 UTC (permalink / raw)
  To: ALeX Wang, dev

On 4/13/2016 11:26 PM, ALeX Wang wrote:
> Did more experiment, found that it has nothing to do with the kernel
> version,
> 
> It only happens when using kni module with '--no-huge' eal flag...
> 
> Is that expected?
> 

Yes.
KNI kernel module expects mempool is physically continuous, with
'--no-huge' flag this is no more true, and as a result KNI module can
access to incorrect address.

Regards,
ferruh

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [4.4 kernel] kni lockup, kernel dump
  2016-04-14 14:29     ` Ferruh Yigit
@ 2016-04-14 14:43       ` Thomas Monjalon
  2016-04-14 22:02         ` ALeX Wang
  0 siblings, 1 reply; 6+ messages in thread
From: Thomas Monjalon @ 2016-04-14 14:43 UTC (permalink / raw)
  To: Ferruh Yigit; +Cc: dev, ALeX Wang

2016-04-14 15:29, Ferruh Yigit:
> On 4/13/2016 11:26 PM, ALeX Wang wrote:
> > Did more experiment, found that it has nothing to do with the kernel
> > version,
> > 
> > It only happens when using kni module with '--no-huge' eal flag...
> > 
> > Is that expected?
> > 
> 
> Yes.
> KNI kernel module expects mempool is physically continuous, with
> '--no-huge' flag this is no more true, and as a result KNI module can
> access to incorrect address.

This is a bug.
The memory API should allow to explicit this restriction and return
an error if it cannot offer the requested continuous memory.
See this thread for discussion:
	http://dpdk.org/ml/archives/dev/2016-April/037444.html

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [4.4 kernel] kni lockup, kernel dump
  2016-04-14 14:43       ` Thomas Monjalon
@ 2016-04-14 22:02         ` ALeX Wang
  0 siblings, 0 replies; 6+ messages in thread
From: ALeX Wang @ 2016-04-14 22:02 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: Ferruh Yigit, dev

Thx, Ferruh and Thomas for the confirmation and pointer!

On 14 April 2016 at 07:43, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:

> 2016-04-14 15:29, Ferruh Yigit:
> > On 4/13/2016 11:26 PM, ALeX Wang wrote:
> > > Did more experiment, found that it has nothing to do with the kernel
> > > version,
> > >
> > > It only happens when using kni module with '--no-huge' eal flag...
> > >
> > > Is that expected?
> > >
> >
> > Yes.
> > KNI kernel module expects mempool is physically continuous, with
> > '--no-huge' flag this is no more true, and as a result KNI module can
> > access to incorrect address.
>
> This is a bug.
> The memory API should allow to explicit this restriction and return
> an error if it cannot offer the requested continuous memory.
> See this thread for discussion:
>         http://dpdk.org/ml/archives/dev/2016-April/037444.html
>



-- 
Alex Wang,
Open vSwitch developer

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2016-04-14 22:02 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-13  0:19 [dpdk-dev] [4.4 kernel] kni lockup, kernel dump ALeX Wang
2016-04-13  0:28 ` ALeX Wang
2016-04-13 22:26   ` ALeX Wang
2016-04-14 14:29     ` Ferruh Yigit
2016-04-14 14:43       ` Thomas Monjalon
2016-04-14 22:02         ` ALeX Wang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).