Re: [dpdk-users] Unregistered mempool in secondary

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

It would be good to get more detailed logs (--log-level=eal,8 or similar for mempool) to see exactly what fails, and why. However, I’m not sure I understand what is going on there. Since when does testpmd support secondary processes?

Thanks,
Anatoly

From: Shahaf Shuler [mailto:shahafs@mellanox.com]
Sent: Thursday, December 6, 2018 1:46 PM
To: Tom Barbette <barbette@kth.se>; Yongseok Koh <yskoh@mellanox.com>; Raslan Darawsheh <rasland@mellanox.com>; Thomas Monjalon <thomas@monjalon.net>; Burakov, Anatoly <anatoly.burakov@intel.com>; Iremonger, Bernard <bernard.iremonger@intel.com>
Cc: users@dpdk.org
Subject: RE: Unregistered mempool in secondary

Adding some folks which may help.

Raslan – is the secondary process testing pass on our regression?

From: Tom Barbette <barbette@kth.se<mailto:barbette@kth.se>>
Sent: Thursday, December 6, 2018 1:53 PM
To: Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>; Yongseok Koh <yskoh@mellanox.com<mailto:yskoh@mellanox.com>>
Cc: users@dpdk.org<mailto:users@dpdk.org>
Subject: RE: Unregistered mempool in secondary


No, that testpmd problem is with all NICs. I just can't get testpmd to work in primary+secondary mode. Technically the lines below should at least start but it doesn't. So I can't reproduce the problem at hand with testpmd...



If some testpmd expert could jump in here to allow to showcase secondary with testpmd it would be great. Then I may be able to reproduce.



Tom





________________________________
De : Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>
Envoyé : jeudi 6 décembre 2018 10:54
À : Tom Barbette; Yongseok Koh
Cc : users@dpdk.org<mailto:users@dpdk.org>
Objet : RE: Unregistered mempool in secondary

Only w/ Mellanox NICs or in general?

From: Tom Barbette <barbette@kth.se<mailto:barbette@kth.se>>
Sent: Thursday, December 6, 2018 11:38 AM
To: Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>; Yongseok Koh <yskoh@mellanox.com<mailto:yskoh@mellanox.com>>
Cc: users@dpdk.org<mailto:users@dpdk.org>
Subject: RE: Unregistered mempool in secondary


I'm trying to reproduce with testpmd. But when I launch :

sudo ~/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -w "0000:03:00.0" -w "0000:82:00.0" -- -i

sudo ~/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -w "0000:03:00.0" -w "0000:82:00.0" --proc-type=secondary​ -- -i



I get on the secondary:
EAL: Error - exiting with code: 1

Cause: Creation of mbuf pool for socket 0 failed: File exists

I also had that with 18.11.

Tom



________________________________
De : Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>
Envoyé : jeudi 6 décembre 2018 10:21
À : Tom Barbette; Yongseok Koh
Cc : users@dpdk.org<mailto:users@dpdk.org>
Objet : RE: Unregistered mempool in secondary

Do you reproduce this issue w/ testpmd?
Can you provide the instruction on how to?

From: Tom Barbette <barbette@kth.se<mailto:barbette@kth.se>>
Sent: Thursday, December 6, 2018 10:19 AM
To: Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>; Yongseok Koh <yskoh@mellanox.com<mailto:yskoh@mellanox.com>>
Cc: users@dpdk.org<mailto:users@dpdk.org>
Subject: RE: Unregistered mempool in secondary


Hi Shahaf,



Both devices are initialized in the primary with rte_eth_dev_start, and queues also (each with their own packet pool). It's just that the secondary read from the initialized queue.



If you're mentionning something more to do, then I did not understand. Which function would that be? To ask port 0 to register port 1's pools and vice versa?



Thanks !



Tom



________________________________
De : Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>
Envoyé : jeudi 6 décembre 2018 08:18
À : Tom Barbette; Yongseok Koh
Cc : users@dpdk.org<mailto:users@dpdk.org>
Objet : RE: Unregistered mempool in secondary

Hi Tom,

There is a complete isolation between the resources of each port.
Meaning, mempool cannot be registered on one port and be used on the other (w/o explicit registration).

In order your solution to work, the primary process needs to probe both ports, so that the mempool_walk will happen on both and the pool will be registered to both.

From: Tom Barbette <barbette@kth.se<mailto:barbette@kth.se>>
Sent: Thursday, December 6, 2018 1:41 AM
To: Shahaf Shuler <shahafs@mellanox.com<mailto:shahafs@mellanox.com>>; Yongseok Koh <yskoh@mellanox.com<mailto:yskoh@mellanox.com>>
Cc: users@dpdk.org<mailto:users@dpdk.org>
Subject: Unregistered mempool in secondary

Hi mlx5 maintainers,

Since we're using a second ConnectX 5 NIC plugged to our second CPU socket, we see the following message when flowing packets through that port :

net_mlx5: port 1 using address (0x6000712742c0) of unregistered mempool in secondary process, please create mempool before rte_eth_dev_start()

The secondary process has all of the primary process pools registered via rte_mempool_walk (2 as we have 2 numa nodes). But that address in the error message is actually not between any of the pool->mz->addr_64  + pool->mz->size range. So maybe this comes from an automatically created pool of the primary ? If so, how can we force it to be created before we call eth_dev_start?

We think the only change is that the second port is now a dedicated NIC on the second socket, instead of the second port of the same NIC on the first socket. But we are not 100% sure this is the triggering event. The same test worked before, still with DPDK 18.08.

Thanks,
Tom