[dpdk-dev] struct malloc_elem overrun/corruption

[He Huang <hhuang@ddn.com>]

Hi,

I’ve been troubleshooting a possible memory allocator corruption:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffefdf0700 (LWP 1079)]
0x00000000004794ee in malloc_elem_free_list_insert (elem=0x7ff82d265000) at dpdk/lib/librte_eal/common/malloc_elem.c:292
292             LIST_INSERT_HEAD(&elem->heap->free_head[idx], elem, free_list);
(gdb) bt
#0  0x00000000004794ee in malloc_elem_free_list_insert (elem=0x7ff82d265000) at dpdk/lib/librte_eal/common/malloc_elem.c:292
#1  0x0000000000479971 in malloc_elem_free (elem=0x7ff82d265000) at dpdk/lib/librte_eal/common/malloc_elem.c:448
#2  0x000000000047b054 in malloc_heap_free (elem=0x7ff82d265fc0) at dpdk/lib/librte_eal/common/malloc_heap.c:628
#3  0x00000000004787f5 in rte_free (addr=0x7ff82d266000) at dpdk/lib/librte_eal/common/rte_malloc.c:32

Looked like the 1st field of struct malloc_elem (i.e. the heap pointer: struct malloc_heap *heap) was corrupted. Everything else looked good:
(gdb) p *elem
$2 = {
  heap = 0x9e0,
  prev = 0x7ff82d254fc0,
  next = 0x7ff84ce9a000,
  free_list = {
    le_next = 0x7ff873c89000,
    le_prev = 0x7ff82bcbf018
  },
  msl = 0x7ffff7f3d07c,
  state = ELEM_FREE,
  pad = 0,
  size = 532893696
}
(gdb) p *elem->prev
$3 = {
  heap = 0x7ffff7f3f67c,
  prev = 0x7ff82ce14000,
  next = 0x7ff82d265000,
  free_list = {
   le_next = 0x0,
    le_prev = 0x0
  },
  msl = 0x7ffff7f3d07c,
  state = ELEM_BUSY,
  pad = 0,
  size = 65600
}

I haven’t completely ruled out my own code had a buffer overrun and corrupted the first field of malloc_elem object yet, but I’m beginning to look at it as a possible DPDK internal corruption. The DPDK code isn’t the latest but it had malloc fixes up to commit 9554dbb50a8a22942128a0e5bcb52243a4f723ab.

Ideas/suggestions greatly appreciated! BTW it’s DMA memory so I couldn’t just use malloc/free and debug with standard memory debuggers.

Thanks,
Isaac