From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 61CF5239 for ; Mon, 15 Oct 2018 12:41:37 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Oct 2018 03:41:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,384,1534834800"; d="scan'208";a="81529831" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.49]) ([10.237.221.49]) by orsmga008.jf.intel.com with ESMTP; 15 Oct 2018 03:41:35 -0700 To: "Phil Yang (Arm Technology China)" , "dev@dpdk.org" Cc: nd , "anatoly.burakov@intel.com" References: <1539336895-22691-1-git-send-email-phil.yang@arm.com> <4b4aeed5-ba6e-8df3-386c-191b05a73586@intel.com> From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= xsFNBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABzSVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+wsGVBBMBAgA/AhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgBYhBNI2U4dCLsKE45mBx/kz60PfE2EfBQJbughWBQkHwjOGAAoJEPkz60Pf E2Eft84QAIbKWqhgqRfoiw/BbXbA1+qm2o4UgkCRQ0yJgt9QsnbpOmPKydHH0ixCliNz1J8e mRXCkMini1bTpnzp7spOjQGLeAFkNFz6BMq8YF2mVWbGEDE9WgnAxZdi0eLY7ZQnHbE6AxKL SXmpe9INb6z3ztseFt7mqje/W/6DWYIMnH3Yz9KzxujFWDcq8UCAvPkxVQXLTMpauhFgYeEx Nub5HbvhxTfUkapLwRQsSd/HbywzqZ3s/bbYMjj5JO3tgMiM9g9HOjv1G2f1dQjHi5YQiTZl 1eIIqQ3pTic6ROaiZqNmQFXPsoOOFfXF8nN2zg8kl/sSdoXWHhama5hbwwtl1vdaygQYlmdK H2ueiFh/UvT3WG3waNv2eZiEbHV8Rk52Xyn2w1G90lV0fYC6Ket1Xjoch7kjwbx793Kz/RfQ rmBY8/S4DTGn3oq3dMdQY+b6+7VMUeLMMh2CXYO9ErkOq+qNTD1IY+cBAkXnaDbQfz0zbste ZGWH74FAZ9nCpDOqbRTrBL42aMGhfOWEyeA1x7+hl6JZfabBWAuf4nnCXuorKHzBXTrf7u7p fXsKQClWRW77PF1VmzrtKNVSytQAmlCWApQIw20AarFipXmVdIjHmJPU611WoyxZPb4JTOxx 5cv9B+nr/RIB+v5dcStyHCCwO1be7nBDdCgd4F6kTQPLzsFNBFfWTL4BEACnNA29e8TarUsB L5n6eLZHXcFvVwNLVlirWOClHXf44o2KnN3ww+eBEmKVfEFo9MSuGDNHS8Zw1NiGMYxLIUgd U6gGrVVs/VrQWL82pbMk6jCj98N+BXIri+6K1z+AImz7ax7iF1kDgRAnFWU0znWWBgM2mM8Y gDjcxfXk4sCKnvf6Gjo08Ey5zmqx7dekAKU2EEp8Q1EJY3jbymLdZWRP4AFFMTS1rGMk0/tt v71NBg1GobCcbNfn9chK/jhqxYhAJqq86RdJQkt3/9x1U1Oq0vXCt4JVVHmkxePtUiuWTTt+ aYlUAsKYZsWvncExvw77x2ArYDmaK0yfjh37wp0lY7DOJHFxoyT8tyWZlLci/VMRG2Ja33xj 0CN4C1yBg+QDeV3QFxQo42iA/ykdXPUR3ezmsND3XKvVLTC4DNb3V/EZQ7jBj64+bEK0VW4G B31VP00ApNQvSoczsIOAKdk97RNbpmPw6q10ILIB+9T1xbnFYzshzGF17oC0/GENIHATx8vZ masOZoDiOZQpeneLgnFE9JfzhLTxv6wNZcc/HLXRQVTkDsQr8ERtkAoHCf1E5+b5Yr7pfnE4 YuhET746o25S53ELUYPIs49qoJsEJL34/oexMfPGyPIlrbufiNyty5jc/1MRwUlhJlJ5IOHy ZUa+6CLR7GdImusFkPJUJwARAQABwsF8BBgBAgAmAhsMFiEE0jZTh0IuwoTjmYHH+TPrQ98T YR8FAlu6CHAFCQXE7zIACgkQ+TPrQ98TYR9nXxAAqNBgkYNyGuWUuy0GwDQCbu3iiMyH1+D7 llafPcK4NYy1Z4AYuVwC9nmLaoj+ozdqS3ncRo57ncRsKEJC46nDJJZYZ5LSJVn63Y3NBF86 lxQAgjj2oyZEwaLKtKbAFsXL43jv1pUGgSvWwYtDwHITXXFQto9rZEuUDRFSx4sg9OR+Q6/6 LY+nQQ3OdHlBkflzYMPcWgDcvcTAO6yasLEUf7UcYoSWTyMYjLB4QuNlXzTswzGVMssJF/vo V8lD1eqqaSUWG3STF6GVLQOr1NLvN5+kUBiEStHFxBpgSCvYY9sNV8FS6N24CAWMBl+10W+D 2h1yiiP5dOdPcBDYKsgqDD91/sP0WdyMJkwdQJtD49f9f+lYloxHnSAxMleOpyscg1pldw+i mPaUY1bmIknLhhkqfMmjywQOXpac5LRMibAAYkcB8v7y3kwELnt8mhqqZy6LUsqcWygNbH/W K3GGt5tRpeIXeJ25x8gg5EBQ0Jnvp/IbBYQfPLtXH0Myq2QuAhk/1q2yEIbVjS+7iowEZNyE 56K63WBJxsJPB2mvmLgn98GqB4G6GufP1ndS0XDti/2K0o8rep9xoY/JDGi0n0L0tk9BHyoP Y7kaEpu7UyY3nVdRLe5H1/MnFG8hdJ97WqnPS0buYZlrbTV0nRFL/NI2VABl18vEEXvNQiO+ vM8= Message-ID: Date: Mon, 15 Oct 2018 11:41:34 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH] app/testpmd: fix vdev socket initialization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2018 10:41:39 -0000 On 10/15/2018 10:51 AM, Phil Yang (Arm Technology China) wrote: >> -----Original Message----- >> From: Ferruh Yigit >> Sent: Saturday, October 13, 2018 1:13 AM >> To: Phil Yang (Arm Technology China) ; dev@dpdk.org >> Cc: nd ; anatoly.burakov@intel.com >> Subject: Re: [PATCH] app/testpmd: fix vdev socket initialization >> >> On 10/12/2018 10:34 AM, phil.yang@arm.com wrote: >>> The cmdline settings of port-numa-config and rxring-numa-config have >>> been flushed by the following init_config. If we don't configure the >>> port-numa-config, the virtual device will allocate the device ports to >>> socket 0. It will cause failure when the socket 0 is unavailable. >>> >>> eg: >>> testpmd -l --vdev net_pcap0,iface=lo >>> --socket-mem=64 -- --numa --port-numa-config="(0,1)" >>> --ring-numa-config="(0,1,1),(0,2,1)" -i >>> >>> ... >>> Configuring Port 0 (socket 0) >>> Failed to setup RX queue:No mempool allocation on the socket 0 >>> EAL: Error - exiting with code: 1 >>> Cause: Start ports failed >>> >>> Fix by allocate the devices port to the first available socket or the >>> socket configured in port-numa-config. >> >> I confirm this fixes the issue, by making vdev to allocate from available socket >> instead of hardcoded socket 0, overall this make sense. >> >> But currently there is no way to request mempool form "socket 0" if only cores >> from "socket 1" provided in "-l", even with "port-numa-config" and "rxring- >> numa-config". >> Both this behavior and the problem this patch fixes caused by patch: >> Commit dbfb8ec7094c ("app/testpmd: optimize mbuf pool allocation") >> >> It is good to have optimized mempool allocation but I think this shouldn't limit >> the tool. If user wants mempools from specific socket, let it have. >> >> What about changing the default behavior to: >> 1- Allocate mempool only from socket that coremask provided (current >> approach) >> 2- Plus, allocate mempool from sockets of attached devices (this is alternative >> solution to this patch, your solution seems better for virtual devices but for >> physical devices allocating from socket it connects can be better) >> 3- Plus, allocate mempool from sockets provided in "port-numa-config" and >> "rxring-numa-config" >> >> What do you think? > > Hi Ferruh, > > Totally agreed with your suggestion. > > As I understand, allocating mempool from sockets of attached devices will enable the cross NUMA scenario for Testpmd. Yes it will. > > Below is my fix for physic port mempool allocate issue. So, is it better to separate it into a new patch on the top of this one or rework this one by adding below fix? I prefer to add a new one because the current patch has already fixed two defects. Anyway, I will follow your comment. +1 to separate it into a new patch, so I will check existing patch. Below looks good only not sure if is should be in `set_default_fwd_ports_config`? Or perhaps `set_default_fwd_lcores_config`? And port-numa-config and rxring-numa-config still not covered. > > 565 static void > 566 set_default_fwd_ports_config(void) > 567 { > 568 › portid_t pt_id; > 569 › int i = 0; > 570 > 571 › RTE_ETH_FOREACH_DEV(pt_id) { > 572 › › fwd_ports_ids[i++] = pt_id; > 573 > + 574 › › /* Update sockets info according to the attached device */ > + 575 › › int socket_id = rte_eth_dev_socket_id(pt_id); > + 576 › › if (socket_id >= 0 && new_socket_id(pt_id)) { > + 577 › › › if (num_sockets >= RTE_MAX_NUMA_NODES) { > + 578 › › › › rte_exit(EXIT_FAILURE, > + 579 › › › › › "Total sockets greater than %u\n", > + 580 › › › › › RTE_MAX_NUMA_NODES); > + 581 › › › } > + 582 › › › socket_ids[num_sockets++] = socket_id; > + 583 › › } > + 584 › } > + 585 > 586 › nb_cfg_ports = nb_ports; > 587 › nb_fwd_ports = nb_ports; > 588 } > > Thanks > Phil Yang > >> >> >>> >>> Fixes: 487f9a5 ("app/testpmd: fix NUMA structures initialization") >>> >>> Signed-off-by: Phil Yang >>> Reviewed-by: Gavin Hu >> >> <...>