Re: [dpdk-dev] [PATCH v2] app/testpmd: optimize membuf pool allocation

["Phil Yang (Arm Technology China)" <Phil.Yang@arm.com>]

> -----Original Message-----
> From: Burakov, Anatoly <anatoly.burakov@intel.com>
> Sent: Monday, October 8, 2018 7:36 PM
> To: Ferruh Yigit <ferruh.yigit@intel.com>; dev-bounces@dpdk.org;
> dev@dpdk.org
> Cc: bernard.iremonger@intel.com; Gavin Hu (Arm Technology China)
> <Gavin.Hu@arm.com>; stable@dpdk.org; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>
> Subject: Re: [dpdk-dev] [PATCH v2] app/testpmd: optimize membuf pool
> allocation
>
> On 08-Oct-18 12:33 PM, Ferruh Yigit wrote:
> > On 9/12/2018 2:54 AM, dev-bounces@dpdk.org wrote:
> >> By default, testpmd will create membuf pool for all NUMA nodes and
> >> ignore EAL configuration.
> >>
> >> Count the number of available NUMA according to EAL core mask or core
> >> list configuration. Optimized by only creating membuf pool for those
> >> nodes.
> >>
> >> Fixes: c9cafcc ("app/testpmd: fix mempool creation by socket id")
> >>
> >> Signed-off-by: Phil Yang <phil.yang@arm.com>
> >> Acked-by: Gavin Hu <gavin.hu@arm.com>
> >> ---
> >>   app/test-pmd/testpmd.c | 4 ++--
> >>   1 file changed, 2 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index
> >> ee48db2..a56af2b 100644
> >> --- a/app/test-pmd/testpmd.c
> >> +++ b/app/test-pmd/testpmd.c
> >> @@ -476,6 +476,8 @@ set_default_fwd_lcores_config(void)
> >>
> >>   nb_lc = 0;
> >>   for (i = 0; i < RTE_MAX_LCORE; i++) {
> >> +if (!rte_lcore_is_enabled(i))
> >> +continue;
> >>   sock_num = rte_lcore_to_socket_id(i);
> >>   if (new_socket_id(sock_num)) {
> >>   if (num_sockets >= RTE_MAX_NUMA_NODES) { @@ -
> 485,8 +487,6 @@
> >> set_default_fwd_lcores_config(void)
> >>   }
> >>   socket_ids[num_sockets++] = sock_num;
> >>   }
> >> -if (!rte_lcore_is_enabled(i))
> >> -continue;
> >>   if (i == rte_get_master_lcore())
> >>   continue;
> >>   fwd_lcores_cpuids[nb_lc++] = i;
> >>
> >
> >
> > This is causing testpmd fail for the case all cores from socket 1 and
> > added a virtual device which will try to allocate memory from socket 0.
> >
> >
> >   $ testpmd -l<cores from socket 1> --vdev net_pcap0,iface=lo -- -i
> >   ...
> >   Failed to setup RX queue:No mempool allocation on the socket 0
> >   EAL: Error - exiting with code: 1
> >     Cause: Start ports failed
> >
> >
>
> It's an open question as to why pcap driver tries to allocate on socket
> 0 when everything is on socket 1, but perhaps a better improvement would be to
> take into account not only socket ID's of lcores, but ethdev devices as well?
>
> --
> Thanks,
> Anatoly

Hi Anatoly,

Agree.

Since NUMA-aware is enabled default in testpmd, so it should be configurable for vdev port NUMA setting.

testpmd -l <cores from socket 1> --vdev net_pcap0,iface=lo --socket-mem=64 -- --numa --port-numa-config="(0,1)" --ring-numa-config="(0,1,1),(0,2,1)" -i

...
Configuring Port 0 (socket 0)
Failed to setup RX queue:No mempool allocation on the socket 0
EAL: Error - exiting with code: 1
  Cause: Start ports failed

This should be a defect.

Thanks
Phil Yang
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.