Re: [dpdk-dev] mutli process C/S model example init failed on xen dom0 with dpdk-16.07 rc2 package

[Olivier MATZ <olivier.matz@6wind.com>]

Hi Huilong,


On 07/12/2016 11:22 AM, Xu, HuilongX wrote:
> Hi all,
>
> I run mutli procee C/S model example failed on xen dom0. Does anyone
> give me some suggest how to debug it?
>
> Thanks a lot
>
> test environment:
>
>        OS&kernel: 3.17.4-301.fc21.x86_64
>
> Gcc version: gcc version 4.9.2 20141101 (Red Hat 4.9.2-1) (GCC)
>
> Package :dpdk.16.07-rc1.tar.gz
>
> Target: x86_64-native-linuxapp-gcc
>
> Compile switch: enable CONFIG_RTE_LIBRTE_XEN_DOM0
>
> Xen version:4.4.1
>
> Test cmdline and result:
>
> /examples/multi_process/client_server_mp/mp_server/mp_server/x86_64-native-linuxapp-gcc/mp_server
> -c f -n 4 --xen-dom0 -- -p 0x3 -n 2
> EAL: Detected 72 lcore(s)
> EAL: Probing VFIO support...
> PMD: bnxt_rte_pmd_init() called for (null)
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: PCI device 0000:01:00.1 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: PCI device 0000:04:00.0 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> Creating mbuf pool 'MProc_pktmbuf_pool' [6144 mbufs] ...
> Port 0 init ... Segmentation fault (core dumped)
>

I reproduced the issue on my platform. In my case, the crash occurs in 
rx_queue_setup():

         /* Free memory prior to re-allocation if needed. */
         if (dev->data->rx_queues[queue_idx] != NULL) {
=>              em_rx_queue_release(dev->data->rx_queues[queue_idx]);
                 dev->data->rx_queues[queue_idx] = NULL;
         }

I don't this we should go in that area for the first rx queue 
initialization. I suspect it could be related to this commit:
http://dpdk.org/browse/dpdk/commit/?id=ea0bddbd14e68f

I think we cannot expect that memory is initialized at 0 when using Xen 
dom0. If I add the following (dirty) patch, I don't see a crash anymore:

--- a/lib/librte_eal/common/eal_common_memzone.c
+++ b/lib/librte_eal/common/eal_common_memzone.c
@@ -258,6 +258,8 @@ memzone_reserve_aligned_thread_unsafe(const char 
*name, size_t len,
         mz->flags = 0;
         mz->memseg_id = elem->ms - 
rte_eal_get_configuration()->mem_config->memseg;

+       memset(mz->addr, 0, mz->len);
+
         return mz;
  }

--- a/lib/librte_eal/common/rte_malloc.c
+++ b/lib/librte_eal/common/rte_malloc.c
@@ -123,7 +123,13 @@ rte_malloc(const char *type, size_t size, unsigned 
align)
  void *
  rte_zmalloc_socket(const char *type, size_t size, unsigned align, int 
socket)
  {
-       return rte_malloc_socket(type, size, align, socket);
+       void *x = rte_malloc_socket(type, size, align, socket);
+
+       if (x == NULL)
+               return NULL;
+
+       memset(x, 0, size);
+       return x;
  }

  /*


Sergio, could you have a look at it?

Regards,
Olivier