Re: [dpdk-dev] multi-process shared memory on PPC

["Xueming(Steven) Li" <xuemingl@mellanox.com>]

It works for me after setting nr_overcommit_hugepages correctly, thanks again.

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xueming(Steven) Li
> Sent: Tuesday, September 12, 2017 4:13 PM
> To: Chao Zhu <chaozhu@linux.vnet.ibm.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] multi-process shared memory on PPC
> 
> Great, that might be something missing, will update later. Thanks!
> 
> > -----Original Message-----
> > From: Chao Zhu [mailto:chaozhu@linux.vnet.ibm.com]
> > Sent: Tuesday, September 12, 2017 4:02 PM
> > To: Xueming(Steven) Li <xuemingl@mellanox.com>
> > Cc: dev@dpdk.org
> > Subject: RE: multi-process shared memory on PPC
> >
> > Xueming,
> >
> > I tried it on my local server, it works fine.
> > Did you set this value
> > "/sys/kernel/mm/hugepages/hugepages-
> 16384kB/nr_overcommit_hugepages" ?
> > It should be the same value as "
> > /sys/kernel/mm/hugepages/hugepages-16384kB/nr_hugepages" on ppc64le.
> >
> > > -----Original Message-----
> > > From: Xueming(Steven) Li [mailto:xuemingl@mellanox.com]
> > > Sent: 2017年8月31日 22:40
> > > To: Chao Zhu <chaozhu@linux.vnet.ibm.com>
> > > Cc: dev@dpdk.org
> > > Subject: multi-process shared memory on PPC
> > >
> > > Hi all,
> > >
> > > I'm testing multi-process example on PowerPC system, looks like
> > > shared memory not working properly.
> > > Below are the error and debug info of the first multi-process
> > > example 'simple_mp' (works as expected on X86):
> > >
> > > // Build DPDK, master branch
> > > # make T=ppc_64-power8-linuxapp-gcc -j160 -C
> > > examples/multi_process/simple_mp/ #
> > > examples/multi_process/simple_mp/build/simple_mp  -w 0:0.0
> > > --proc-type=primary -l 1-2 --socket-mem
> > > 256,256,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256,256
> > > APP: Finished Process Init.
> > > Starting core 2
> > > simple_mp >
> > >
> > > // Open another terminal and run secondary process # gdb --args
> > > examples/multi_process/simple_mp/build/simple_mp -n 4 -w 0:0.0
> > > --proc-type=secondary -l 3-4 --socket-mem
> > > 256,256,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256,256
> > > GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1 Copyright (C) 2016
> > > Free Software Foundation, Inc.
> > > License GPLv3+: GNU GPL version 3 or later
> > >
> > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fur
> > ldef
> > ense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> >
> 3A__gnu.org_licenses_gpl&data=02%7C01%7Cxuemingl%40mellanox.com%7C
> >
> 9cd11fda7a7f48df7cbd08d4f9b486d8%7Ca652971c7d2e4d9ba6a4d149256f461
> >
> b%7C0%7C0%7C636408001138751301&sdata=bb3hwYz7ehCqt4QA6GGqTDMY
> > kY1qAJV%2BCMDBVbXmWzE%3D&reserved=0.
> > h
> > > tml&d=DwIFAg&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=4TYe4NpqlgVi0ArYpZ76FrWp7IE
> > > w8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwuuryoN7QPg9N-
> 6aTY_jEvDZtMZR0o&s
> > > =SbDQBN7qJ9z8fWCSePyvm5FFncp2P_9XZm-qs1lH5PE&e= > This is free
> > > software: you are free to change and redistribute it.
> > > There is NO WARRANTY, to the extent permitted by law.  Type "show
> > copying"
> > > and "show warranty" for details.
> > > This GDB was configured as "powerpc64le-linux-gnu".
> > > Type "show configuration" for configuration details.
> > > For bug reporting instructions, please see:
> > > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > ur
> > > ldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > 3A__www.gnu.org_softwar&
> > >
> >
> data=02%7C01%7Cxuemingl%40mellanox.com%7C9cd11fda7a7f48df7cbd08d4f
> > 9b48
> > >
> >
> 6d8%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C63640800113875
> > 1301&sd
> > >
> >
> ata=4DNDTyIcHpTIU60K7FM%2F1zuASSB1IdZDRyOaiVp1Y%2BY%3D&reserved=
> > 0
> > > e_gdb_bugs_&d=DwIFAg&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=4TYe4NpqlgVi0ArYpZ
> > > 76FrWp7IEw8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwuuryoN7QPg9N-
> > 6aTY_jEvDZt
> > > MZR0o&s=GAFKK3jSahAPFXMGcPGbVNTl0CesJy_qfs5TxXfGOX0&e= >.
> > > Find the GDB manual and other documentation resources online at:
> > > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > ur
> > > ldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > 3A__www.gnu.org_softwar&
> > >
> >
> data=02%7C01%7Cxuemingl%40mellanox.com%7C9cd11fda7a7f48df7cbd08d4f
> > 9b48
> > >
> >
> 6d8%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C63640800113875
> > 1301&sd
> > >
> >
> ata=4DNDTyIcHpTIU60K7FM%2F1zuASSB1IdZDRyOaiVp1Y%2BY%3D&reserved=
> > 0
> > > e_gdb_documentation_&d=DwIFAg&c=jf_iaSHvJObTbx-
> > siA1ZOg&r=4TYe4Npql
> > >
> gVi0ArYpZ76FrWp7IEw8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwuuryoN7QPg9N-
> > 6a
> > > TY_jEvDZtMZR0o&s=JdkMMjGGl-
> > 3Z3BGnjCkOFP5lI5BIbRNBxH4EVI70YFc&e= > .
> > > For help, type "help".
> > > Type "apropos word" to search for commands related to "word"...
> > > Reading symbols from
> > > examples/multi_process/simple_mp/build/simple_mp...done.
> > > (gdb) r
> > > Starting program:
> > > /home/xueming/mlnx-
> > dpdk.org/examples/multi_process/simple_mp/build/sim
> > > ple_mp -n 4 -w 0:0.0 --proc-type=secondary -l 3-4 --socket-mem
> > > 256,256,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256,256
> > > [Thread debugging using libthread_db enabled] Using host
> > > libthread_db
> > library
> > > "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
> > > EAL: Detected 144 lcore(s)
> > > EAL: Probing VFIO support...
> > > [New Thread 0x7ffff7a6eff0 (LWP 58811)] [New Thread 0x7ffff726eff0
> > > (LWP 58812)]
> > >
> > > Thread 1 "simple_mp" received signal SIGSEGV, Segmentation fault.
> > > 0x00007ffff7d2f3ec in ?? () from
> > > /lib/powerpc64le-linux-gnu/libc.so.6
> > >  (gdb) bt full
> > > #0  0x00007ffff7d2f3ec in ?? () from
> > > /lib/powerpc64le-linux-gnu/libc.so.6
> > > No symbol table info available.
> > > #1  0x0000000010077d54 in rte_ring_lookup (name=0x102a4e28
> > > "PRI_2_SEC")
> > >     at /home/xueming/mlnx-dpdk.org/lib/librte_ring/rte_ring.c:326
> > >         te = 0x7ef3eefd5e00
> > >         r = 0x0
> > >         ring_list = 0x7ffff7aa250c
> > > #2  0x0000000010032640 in main (argc=10, argv=0x7ffffffff3e8)
> > >     at
> > > /home/xueming/mlnx-
> > dpdk.org/examples/multi_process/simple_mp/main.c:12
> > > 8
> > >         flags = 0
> > >         ring_size = 64
> > >         pool_size = 1024
> > >         pool_cache = 32
> > >         priv_data_sz = 0
> > >         ret = 9
> > >         lcore_id = 32767
> > >         cl = 0x7ffff7fef718 <_dl_argv>
> > > (gdb) f 1
> > > #1  0x0000000010077d54 in rte_ring_lookup (name=0x102a4e28
> > > "PRI_2_SEC")
> > >     at /home/xueming/mlnx-dpdk.org/lib/librte_ring/rte_ring.c:326
> > > 326                     if (strncmp(name, r->name,
> > > RTE_RING_NAMESIZE) == 0)
> > > (gdb) l
> > > 321
> > > 322             rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
> > > 323
> > > 324             TAILQ_FOREACH(te, ring_list, next) {
> > > 325                     r = (struct rte_ring *) te->data;
> > > 326                     if (strncmp(name, r->name,
> > > RTE_RING_NAMESIZE) == 0)
> > > 327                             break;
> > > 328             }
> > > 329
> > > 330             rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
> > > (gdb) p *te
> > > $1 = {next = {tqe_next = 0x0, tqe_prev = 0x0}, data = 0x0}
> > > (gdb) p *ring_list
> > > $2 = {tqh_first = 0x7ef3eefd5e00, tqh_last = 0x7ef3eec6d100}
> > >
> > > # uname -a
> > > Linux dragon-power-01 4.13.0-rc2 #1 SMP Tue Aug 29 17:27:05 IDT 2017
> > > ppc64le ppc64le ppc64le GNU/Linux
> > >
> > > # cat /etc/issue
> > > Ubuntu 16.04.2 LTS \n \l
> > >
> > > # gcc --version
> > > gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 Copyright (C)
> > > 2015 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions.  There
> > > is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
> > > PARTICULAR PURPOSE.
> > >
> > > # lscpu
> > > Architecture:          ppc64le
> > > Byte Order:            Little Endian
> > > CPU(s):                144
> > > On-line CPU(s) list:   0-87,96-103,112-159
> > > Thread(s) per core:    8
> > > Core(s) per socket:    4
> > > Socket(s):             4
> > > NUMA node(s):          4
> > > Model:                 2.1 (pvr 004b 0201)
> > > Model name:            POWER8E (raw), altivec supported
> > > CPU max MHz:           3690.0000
> > > CPU min MHz:           2061.0000
> > > L1d cache:             64K
> > > L1i cache:             32K
> > > L2 cache:              512K
> > > L3 cache:              8192K
> > > NUMA node0 CPU(s):     0-39
> > > NUMA node1 CPU(s):     40-79
> > > NUMA node16 CPU(s):    80-87,96-103,112-119
> > > NUMA node17 CPU(s):    120-159
> > >
> > > # cat build/.config
> > > # RTE_EXEC_ENV values are the directories in mk/exec-env/
> > > CONFIG_RTE_EXEC_ENV="linuxapp"
> > > # RTE_ARCH values are architecture we compile for. directories in
> > > mk/arch/ CONFIG_RTE_ARCH="ppc_64"
> > > # machine can define specific variables or action for a specific
> > > board # RTE_MACHINE values are architecture we compile for.
> > > directories in mk/machine/ CONFIG_RTE_MACHINE="power8"
> > > # The compiler we use.
> > > # RTE_TOOLCHAIN values are architecture we compile for. directories
> > > in mk/toolchain/ CONFIG_RTE_TOOLCHAIN="gcc"
> > > # Use intrinsics or assembly code for key routines
> > > CONFIG_RTE_FORCE_INTRINSICS=n # Machine forces strict alignment
> > > constraints.
> > > CONFIG_RTE_ARCH_STRICT_ALIGN=n
> > > # Compile to share library
> > > CONFIG_RTE_BUILD_SHARED_LIB=n
> > > # Use newest code breaking previous ABI CONFIG_RTE_NEXT_ABI=y #
> > > Major ABI to overwrite library specific LIBABIVER
> > > CONFIG_RTE_MAJOR_ABI= # Machine's cache line size
> > > CONFIG_RTE_CACHE_LINE_SIZE=128
> > > # Compile Environment Abstraction Layer CONFIG_RTE_LIBRTE_EAL=y
> > > CONFIG_RTE_MAX_LCORE=256
> > > CONFIG_RTE_MAX_NUMA_NODES=32
> > > CONFIG_RTE_MAX_MEMSEG=256
> > > CONFIG_RTE_MAX_MEMZONE=2560
> > > CONFIG_RTE_MAX_TAILQ=32
> > > CONFIG_RTE_LOG_LEVEL=RTE_LOG_INFO
> > > CONFIG_RTE_LOG_DP_LEVEL=RTE_LOG_INFO
> > > CONFIG_RTE_LOG_HISTORY=256
> > > CONFIG_RTE_BACKTRACE=y
> > > CONFIG_RTE_LIBEAL_USE_HPET=n
> > > CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n
> > > CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n
> > > CONFIG_RTE_EAL_IGB_UIO=y
> > > CONFIG_RTE_EAL_VFIO=y
> > > CONFIG_RTE_MALLOC_DEBUG=n
> > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> > > # Recognize/ignore architecture we compile for. AVX/AVX512 CPU flags
> > > for performance/power testing.
> > > # AVX512 is marked as experimental for now, will enable it after
> > > enough #
> > field
> > > test and possible optimization.
> > > CONFIG_RTE_ENABLE_AVX=y
> > > CONFIG_RTE_ENABLE_AVX512=n
> > > # Default driver path (or "" to disable) CONFIG_RTE_EAL_PMD_PATH=""
> > > # Compile Environment Abstraction Layer to support Vmware TSC map
> > > CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
> > > # Compile architecture we compile for. argument parser library
> > > CONFIG_RTE_LIBRTE_KVARGS=y # Compile generic ethernet library
> > > CONFIG_RTE_LIBRTE_ETHER=y CONFIG_RTE_LIBRTE_ETHDEV_DEBUG=n
> > > CONFIG_RTE_MAX_ETHPORTS=32
> > > CONFIG_RTE_MAX_QUEUES_PER_PORT=1024
> > > CONFIG_RTE_LIBRTE_IEEE1588=n
> > > CONFIG_RTE_ETHDEV_QUEUE_STAT_CNTRS=16
> > > CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y
> > > # Turn off Tx preparation stage
> > > # Warning: rte_eth_tx_prepare() can be safely disabled only if using
> > > a #
> > driver
> > > which do not implement any Tx preparation.
> > > CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP=n
> > > # Compile burst-oriented Amazon ENA PMD driver
> > > CONFIG_RTE_LIBRTE_ENA_PMD=y
> CONFIG_RTE_LIBRTE_ENA_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_ENA_DEBUG_TX=n
> > CONFIG_RTE_LIBRTE_ENA_DEBUG_TX_FREE=n
> > > CONFIG_RTE_LIBRTE_ENA_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n
> > > # Compile burst-oriented IGB & EM PMD drivers
> > > CONFIG_RTE_LIBRTE_EM_PMD=y CONFIG_RTE_LIBRTE_IGB_PMD=y
> > > CONFIG_RTE_LIBRTE_E1000_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_E1000_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_E1000_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_E1000_DEBUG_TX_FREE=n
> > > CONFIG_RTE_LIBRTE_E1000_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_E1000_PF_DISABLE_STRIP_CRC=n
> > > # Compile burst-oriented IXGBE PMD driver
> > > CONFIG_RTE_LIBRTE_IXGBE_PMD=n
> > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX_FREE=n
> > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n
> > > CONFIG_RTE_IXGBE_INC_VECTOR=y
> > > CONFIG_RTE_LIBRTE_IXGBE_BYPASS=n
> > > # Compile burst-oriented I40E PMD driver
> > > CONFIG_RTE_LIBRTE_I40E_PMD=y
> CONFIG_RTE_LIBRTE_I40E_DEBUG_RX=n
> > CONFIG_RTE_LIBRTE_I40E_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_I40E_DEBUG_TX_FREE=n
> > > CONFIG_RTE_LIBRTE_I40E_RX_ALLOW_BULK_ALLOC=y
> > > CONFIG_RTE_LIBRTE_I40E_INC_VECTOR=y
> > > CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n
> > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF=64
> > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF=4
> > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM=4
> > > # interval up to 8160 us, aligned to 2 (or default value)
> > > CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL=-1
> > > # Compile burst-oriented FM10K PMD
> > > CONFIG_RTE_LIBRTE_FM10K_PMD=n
> > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX_FREE=n
> > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_FM10K_RX_OLFLAGS_ENABLE=y
> > > CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR=y
> > > # Compile burst-oriented Mellanox ConnectX-3 (MLX4) PMD
> > > CONFIG_RTE_LIBRTE_MLX4_PMD=n CONFIG_RTE_LIBRTE_MLX4_DEBUG=n
> > > CONFIG_RTE_LIBRTE_MLX4_DEBUG_BROKEN_VERBS=n
> > > CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N=4
> > > CONFIG_RTE_LIBRTE_MLX4_MAX_INLINE=0
> > > CONFIG_RTE_LIBRTE_MLX4_TX_MP_CACHE=8
> > > CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS=1
> > > # Compile burst-oriented Mellanox ConnectX-4 & ConnectX-5 (MLX5) PMD
> > > CONFIG_RTE_LIBRTE_MLX5_PMD=y CONFIG_RTE_LIBRTE_MLX5_DEBUG=n
> > > CONFIG_RTE_LIBRTE_MLX5_TX_MP_CACHE=8
> > > # Compile burst-oriented Broadcom PMD driver
> > > CONFIG_RTE_LIBRTE_BNX2X_PMD=n
> CONFIG_RTE_LIBRTE_BNX2X_DEBUG=n
> > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_BNX2X_MF_SUPPORT=n
> > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_PERIODIC=n
> > > # Compile burst-oriented Chelsio Terminator (CXGBE) PMD
> > > CONFIG_RTE_LIBRTE_CXGBE_PMD=y
> > CONFIG_RTE_LIBRTE_CXGBE_DEBUG=n
> > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_REG=n
> > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_MBOX=n
> > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_CXGBE_TPUT=y
> > > # Compile burst-oriented Cisco ENIC PMD driver
> > > CONFIG_RTE_LIBRTE_ENIC_PMD=n CONFIG_RTE_LIBRTE_ENIC_DEBUG=n
> > > CONFIG_RTE_LIBRTE_ENIC_DEBUG_FLOW=n
> > > # Compile burst-oriented Netronome NFP PMD driver
> > > CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_NFP_DEBUG=n #
> > Compile
> > > burst-oriented Broadcom BNXT PMD driver
> > CONFIG_RTE_LIBRTE_BNXT_PMD=y #
> > > Compile burst-oriented Solarflare libefx-based PMD
> > > CONFIG_RTE_LIBRTE_SFC_EFX_PMD=n
> > CONFIG_RTE_LIBRTE_SFC_EFX_DEBUG=n #
> > > Compile software PMD backed by
> > > SZEDATA2 device CONFIG_RTE_LIBRTE_PMD_SZEDATA2=n # Defines
> > firmware
> > > type address space.
> > > # See documentation for supported values.
> > > # Other values raise compile time error.
> > > CONFIG_RTE_LIBRTE_PMD_SZEDATA2_AS=0
> > > # Compile burst-oriented Cavium Thunderx NICVF PMD driver
> > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=y
> > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
> > > # Compile burst-oriented Cavium LiquidIO PMD driver
> > > CONFIG_RTE_LIBRTE_LIO_PMD=y
> > CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_LIO_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_LIO_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX=n
> > > CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS=n
> > > # Compile NXP DPAA2 FSL-MC Bus
> > > CONFIG_RTE_LIBRTE_FSLMC_BUS=n
> > > # Compile Support Libraries for NXP DPAA2
> > > CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=n
> > > CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVA=y
> > > # Compile burst-oriented NXP DPAA2 PMD driver
> > > CONFIG_RTE_LIBRTE_DPAA2_PMD=n
> > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_TX_FREE=n
> > > # Compile burst-oriented VIRTIO PMD driver
> > > CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
> > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DUMP=n
> > > # Compile virtio device emulation inside virtio PMD driver
> > > CONFIG_RTE_VIRTIO_USER=y # Compile burst-oriented VMXNET3 PMD
> > driver
> > > CONFIG_RTE_LIBRTE_VMXNET3_PMD=n
> > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n
> > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_DRIVER=n
> > > # Compile example software rings based PMD
> > > CONFIG_RTE_LIBRTE_PMD_RING=y
> > > CONFIG_RTE_PMD_RING_MAX_RX_RINGS=16
> > > CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16
> > > # Compile software PMD backed by PCAP files
> > > CONFIG_RTE_LIBRTE_PMD_PCAP=n # Compile link bonding PMD library
> > > CONFIG_RTE_LIBRTE_PMD_BOND=y
> > CONFIG_RTE_LIBRTE_BOND_DEBUG_ALB=n
> > > CONFIG_RTE_LIBRTE_BOND_DEBUG_ALB_L1=n
> > > # QLogic 10G/25G/40G/50G/100G PMD
> > > CONFIG_RTE_LIBRTE_QEDE_PMD=y
> > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO=n
> > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX=n
> > > #Provides abs path/name of architecture we compile for. firmware file.
> > > #Empty string denotes driver will use default firmware
> > > CONFIG_RTE_LIBRTE_QEDE_FW=""
> > > # Compile software PMD backed by AF_PACKET sockets (Linux only)
> > > CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y # Compile ARK PMD
> > > CONFIG_RTE_LIBRTE_ARK_PMD=y CONFIG_RTE_LIBRTE_ARK_PAD_TX=y
> > > CONFIG_RTE_LIBRTE_ARK_DEBUG_RX=n
> > CONFIG_RTE_LIBRTE_ARK_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS=n
> > > CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE=n
> > > # Compile WRS accelerated virtual port (AVP) guest PMD driver
> > > CONFIG_RTE_LIBRTE_AVP_PMD=n
> CONFIG_RTE_LIBRTE_AVP_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_AVP_DEBUG_TX=n
> > CONFIG_RTE_LIBRTE_AVP_DEBUG_DRIVER=y
> > > CONFIG_RTE_LIBRTE_AVP_DEBUG_BUFFERS=n
> > > # Compile architecture we compile for. TAP PMD # It is enabled by
> > > default
> > for
> > > Linux only.
> > > CONFIG_RTE_LIBRTE_PMD_TAP=y
> > > # Compile Xen PMD
> > > CONFIG_RTE_LIBRTE_PMD_XENVIRT=n
> > > # Compile null PMD
> > > CONFIG_RTE_LIBRTE_PMD_NULL=y
> > > # Compile fail-safe PMD
> > > CONFIG_RTE_LIBRTE_PMD_FAILSAFE=y
> > > # Do prefetch of packet data within PMD driver receive function
> > > CONFIG_RTE_PMD_PACKET_PREFETCH=y # Compile generic crypto device
> > > library CONFIG_RTE_LIBRTE_CRYPTODEV=y
> > > CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=n
> > > CONFIG_RTE_CRYPTO_MAX_DEVS=64
> > > CONFIG_RTE_CRYPTODEV_NAME_LEN=64
> > > # Compile PMD for ARMv8 Crypto device
> > > CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO=n
> > > CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO_DEBUG=n
> > > # Compile NXP DPAA2 crypto sec driver for CAAM HW
> > > CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=n
> > > CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_DRIVER=n
> > > CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_RX=n
> > > # Compile PMD for QuickAssist based devices
> > > CONFIG_RTE_LIBRTE_PMD_QAT=n
> > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=n
> > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
> > > # Number of sessions to create in architecture we compile for.
> > > session
> > memory
> > > pool # on a single QuickAssist device.
> > > CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
> > > # Compile PMD for AESNI backed device
> > > CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n
> > > CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n
> > > # Compile PMD for Software backed device
> > > CONFIG_RTE_LIBRTE_PMD_OPENSSL=n
> > CONFIG_RTE_LIBRTE_PMD_OPENSSL_DEBUG=n
> > > # Compile PMD for AESNI GCM device
> > > CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n
> > > CONFIG_RTE_LIBRTE_PMD_AESNI_GCM_DEBUG=n
> > > # Compile PMD for SNOW 3G device
> > > CONFIG_RTE_LIBRTE_PMD_SNOW3G=n
> > > CONFIG_RTE_LIBRTE_PMD_SNOW3G_DEBUG=n
> > > # Compile PMD for KASUMI device
> > > CONFIG_RTE_LIBRTE_PMD_KASUMI=n
> > > CONFIG_RTE_LIBRTE_PMD_KASUMI_DEBUG=n
> > > # Compile PMD for ZUC device
> > > CONFIG_RTE_LIBRTE_PMD_ZUC=n
> > > CONFIG_RTE_LIBRTE_PMD_ZUC_DEBUG=n
> > > # Compile PMD for Crypto Scheduler device
> > > CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y
> > > CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER_DEBUG=n
> > > # Compile PMD for NULL Crypto device
> > > CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y
> > > # Compile generic event device library CONFIG_RTE_LIBRTE_EVENTDEV=y
> > > CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n
> > > CONFIG_RTE_EVENT_MAX_DEVS=16
> > > CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64
> > > # Compile PMD for skeleton event device
> > > CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV=y
> > > CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV_DEBUG=n
> > > # Compile PMD for software event device
> > > CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y
> > > CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV_DEBUG=n
> > > # Compile PMD for octeontx sso event device
> > > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y
> > > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF_DEBUG=n
> > > # Compile librte_ring
> > > CONFIG_RTE_LIBRTE_RING=y
> > > # Compile librte_mempool
> > > CONFIG_RTE_LIBRTE_MEMPOOL=y
> > > CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512
> > > CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG=n
> > > # Compile Mempool drivers
> > > CONFIG_RTE_DRIVER_MEMPOOL_RING=y
> > > CONFIG_RTE_DRIVER_MEMPOOL_STACK=y
> > > # Compile librte_mbuf
> > > CONFIG_RTE_LIBRTE_MBUF=y
> > > CONFIG_RTE_LIBRTE_MBUF_DEBUG=n
> > > CONFIG_RTE_MBUF_DEFAULT_MEMPOOL_OPS="ring_mp_mc"
> > > CONFIG_RTE_MBUF_REFCNT_ATOMIC=y
> > > CONFIG_RTE_PKTMBUF_HEADROOM=128
> > > # Compile librte_timer
> > > CONFIG_RTE_LIBRTE_TIMER=y
> > > CONFIG_RTE_LIBRTE_TIMER_DEBUG=n
> > > # Compile librte_cfgfile
> > > CONFIG_RTE_LIBRTE_CFGFILE=y
> > > # Compile librte_cmdline
> > > CONFIG_RTE_LIBRTE_CMDLINE=y
> > > CONFIG_RTE_LIBRTE_CMDLINE_DEBUG=n
> > > # Compile librte_hash
> > > CONFIG_RTE_LIBRTE_HASH=y
> > > CONFIG_RTE_LIBRTE_HASH_DEBUG=n
> > > # Compile librte_efd
> > > CONFIG_RTE_LIBRTE_EFD=y
> > > # Compile librte_jobstats
> > > CONFIG_RTE_LIBRTE_JOBSTATS=y
> > > # Compile architecture we compile for. device metrics library
> > > CONFIG_RTE_LIBRTE_METRICS=y # Compile architecture we compile for.
> > > bitrate statistics library CONFIG_RTE_LIBRTE_BITRATE=y # Compile
> > > architecture we compile for. latency statistics library
> > > CONFIG_RTE_LIBRTE_LATENCY_STATS=y # Compile librte_lpm
> > > CONFIG_RTE_LIBRTE_LPM=y CONFIG_RTE_LIBRTE_LPM_DEBUG=n #
> > Compile
> > > librte_acl CONFIG_RTE_LIBRTE_ACL=y
> CONFIG_RTE_LIBRTE_ACL_DEBUG=n
> > #
> > > Compile librte_power CONFIG_RTE_LIBRTE_POWER=y
> > > CONFIG_RTE_LIBRTE_POWER_DEBUG=n
> > > CONFIG_RTE_MAX_LCORE_FREQS=64
> > > # Compile librte_net
> > > CONFIG_RTE_LIBRTE_NET=y
> > > # Compile librte_ip_frag
> > > CONFIG_RTE_LIBRTE_IP_FRAG=y
> > > CONFIG_RTE_LIBRTE_IP_FRAG_DEBUG=n
> > > CONFIG_RTE_LIBRTE_IP_FRAG_MAX_FRAG=4
> > > CONFIG_RTE_LIBRTE_IP_FRAG_TBL_STAT=n
> > > # Compile GRO library
> > > CONFIG_RTE_LIBRTE_GRO=y
> > > # Compile librte_meter
> > > CONFIG_RTE_LIBRTE_METER=y
> > > # Compile librte_sched
> > > CONFIG_RTE_LIBRTE_SCHED=y
> > > CONFIG_RTE_SCHED_DEBUG=n
> > > CONFIG_RTE_SCHED_RED=n
> > > CONFIG_RTE_SCHED_COLLECT_STATS=n
> > > CONFIG_RTE_SCHED_SUBPORT_TC_OV=n
> > > CONFIG_RTE_SCHED_PORT_N_GRINDERS=8
> > > CONFIG_RTE_SCHED_VECTOR=n
> > > # Compile architecture we compile for. distributor library
> > > CONFIG_RTE_LIBRTE_DISTRIBUTOR=y # Compile architecture we compile
> for.
> > > reorder library CONFIG_RTE_LIBRTE_REORDER=y # Compile librte_port
> > > CONFIG_RTE_LIBRTE_PORT=y CONFIG_RTE_PORT_STATS_COLLECT=n
> > > CONFIG_RTE_PORT_PCAP=n # Compile librte_table
> > > CONFIG_RTE_LIBRTE_TABLE=y CONFIG_RTE_TABLE_STATS_COLLECT=n #
> > Compile
> > > librte_pipeline CONFIG_RTE_LIBRTE_PIPELINE=y
> > > CONFIG_RTE_PIPELINE_STATS_COLLECT=n
> > > # Compile librte_kni
> > > CONFIG_RTE_LIBRTE_KNI=y
> > > CONFIG_RTE_LIBRTE_PMD_KNI=y
> > > CONFIG_RTE_KNI_KMOD=y
> > > CONFIG_RTE_KNI_KMOD_ETHTOOL=n
> > > CONFIG_RTE_KNI_PREEMPT_DEFAULT=y
> > > # Compile architecture we compile for. pdump library
> > > CONFIG_RTE_LIBRTE_PDUMP=y # Compile vhost user library
> > > CONFIG_RTE_LIBRTE_VHOST=y CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > > CONFIG_RTE_LIBRTE_VHOST_DEBUG=n # Compile vhost PMD # To compile,
> > > CONFIG_RTE_LIBRTE_VHOST should be enabled.
> > > CONFIG_RTE_LIBRTE_PMD_VHOST=y
> > > #Compile Xen domain0 support
> > > CONFIG_RTE_LIBRTE_XEN_DOM0=n
> > > # Compile architecture we compile for. test application
> > > CONFIG_RTE_APP_TEST=y CONFIG_RTE_APP_TEST_RESOURCE_TAR=n
> > > # Compile architecture we compile for. PMD test application
> > > CONFIG_RTE_TEST_PMD=y
> > CONFIG_RTE_TEST_PMD_RECORD_CORE_CYCLES=n
> > > CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n
> > > # Compile architecture we compile for. crypto performance
> > > application CONFIG_RTE_APP_CRYPTO_PERF=y # Compile architecture we
> compile for.
> > > eventdev application CONFIG_RTE_APP_EVENTDEV=y
> > > CONFIG_RTE_EXEC_ENV_LINUXAPP=y CONFIG_RTE_ARCH_PPC_64=y
> > > CONFIG_RTE_ARCH_64=y CONFIG_RTE_TOOLCHAIN_GCC=y # Note: Power
> > doesn't
> > > have this support # Note: Initially, all of architecture we
> > compile for.
> > > PMD drivers compilation are turned off on Power # Will turn on them
> > > only
> > after
> > > architecture we compile for. successful testing on Power
> > >
> > > Please refer here for more information on running DPDK multi-process
> > > examples:
> > >
> > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fu
> > > rl
> > > defense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > 3A__dpdk.org_doc_guides_s
> > >
> >
> &data=02%7C01%7Cxuemingl%40mellanox.com%7C9cd11fda7a7f48df7cbd08d
> > 4f9b4
> > >
> >
> 86d8%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C6364080011387
> > 51301&s
> > >
> data=4n76ha0PmSr3wP2c0MAo3PEVYRMKjJVcU9o5xETqhBc%3D&reserved=0
> > > ample-5Fapp-5Fug_multi-5Fprocess.html&d=DwIFAg&c=jf_iaSHvJObTbx-
> > siA1Z
> > >
> >
> Og&r=4TYe4NpqlgVi0ArYpZ76FrWp7IEw8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwu
> > > uryoN7QPg9N-6aTY_jEvDZtMZR0o&s=gupwe5WWQ-
> > ap2jObFko6UJymrcy4jTmF
> > > 2r1dZ2fQN9E&e=
> > >
> > > Best Regards,
> > > Xueming
> >