[dpdk-users] Mbuf Free Segfaults in Secondary application

[Kyle Ames <kyle.ames@FireEye.com>]

I'm running into an issue with primary/secondary DPDK processes. I am using DPDK 19.02.

I'm trying to explore a setup where one process pulls packets off the NIC, and then sends them over a rte_ring for additional processing. Unlike the client_server_mp example, I don't need to send the packets out a given port in the client. Once the client is done with them they can just go back into the mbuf mempool. In order to test this, I took the mp_client example and modified it immediately call rte_pktmbuf_free on the packet and not do anything else with it after receiving the packet over the shared ring.

This works fine for the first 1.5*N packets, where N is the value set for the per-lcore cache. Calling rte_pktmbuf_free on the next packet will segfault in bucket_enqueue. (backtrace from GDB below)

Program received signal SIGSEGV, Segmentation fault.
0x0000000000593822 in bucket_enqueue ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-196.el7_4.2.x86_64 libgcc-4.8.5-16.el7.x86_64 numactl-libs-2.0.9-6.el7_2.x86_64
(gdb) backtrace
#0  0x0000000000593822 in bucket_enqueue ()
#1  0x00000000004769f1 in rte_mempool_ops_enqueue_bulk (n=1, obj_table=0x7fffffffe398, mp=<optimized out>)
    at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mempool.h:704
#2  __mempool_generic_put (cache=<optimized out>, n=1, obj_table=0x7fffffffe398, mp=<optimized out>)
    at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mempool.h:1263
#3  rte_mempool_generic_put (cache=<optimized out>, n=1, obj_table=0x7fffffffe398, mp=<optimized out>)
    at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mempool.h:1285
#4  rte_mempool_put_bulk (n=1, obj_table=0x7fffffffe398, mp=<optimized out>) at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mempool.h:1308
#5  rte_mempool_put (obj=0x100800040, mp=<optimized out>) at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mempool.h:1326
#6  rte_mbuf_raw_free (m=0x100800040) at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mbuf.h:1185
#7  rte_pktmbuf_free_seg (m=<optimized out>) at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mbuf.h:1807
#8  rte_pktmbuf_free (m=0x100800040) at /home/kames/code/3rdparty/dpdk-hack/dpdk/build/include/rte_mbuf.h:1828
#9  main (argc=<optimized out>, argv=<optimized out>)
    at /home/kames/code/3rdparty/dpdk-hack/dpdk/examples/multi_process/client_server_mp/mp_client/client.c:90

I changed the size a few times, and the packet in the client that segfaults on free is always the 1.5N'th packet. This happens even if I set the cache_size to zero on mbuf pool creation. (The first mbuf free immediately segfaults)

I'm a bit stuck at the moment. There's clearly a pattern/interaction of some sort, but I don't know what it is or what to do about it. Is this even the right approach for such a scenario?

-Kyle Ames

This email and any attachments thereto may contain private, confidential, and/or privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments thereto) by others is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email and any attachments thereto.