From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 499FB5424 for ; Thu, 22 Jan 2015 10:56:55 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP; 22 Jan 2015 01:52:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,448,1418112000"; d="scan'208";a="654774579" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by fmsmga001.fm.intel.com with ESMTP; 22 Jan 2015 01:56:50 -0800 Received: from irsmsx108.ger.corp.intel.com ([169.254.11.64]) by IRSMSX109.ger.corp.intel.com ([169.254.13.11]) with mapi id 14.03.0195.001; Thu, 22 Jan 2015 09:56:49 +0000 From: "Ferriter, Cian" To: "Richardson, Bruce" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] lib/librte_ether: change socket_id passed to rte_memzone_reserve Thread-Index: AQHQLxJxCtZNxstVx0ajqB3SBqs+d5y9xuPQgABMPYCAADmXQIABGZmAgAhk2eCABCI/4A== Date: Thu, 22 Jan 2015 09:56:48 +0000 Message-ID: References: <1421140920-9964-1-git-send-email-cian.ferriter@intel.com> <20150113135549.GA3460@bricha3-MOBL3> <20150114100946.GA10476@bricha3-MOBL3> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] lib/librte_ether: change socket_id passed to rte_memzone_reserve X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2015 09:56:55 -0000 Hey guys, I just wanted to ask is there anything more that can be done with this patc= h or is it in an acceptable state for pushing? Cian -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferriter, Cian Sent: Monday, January 19, 2015 6:39 PM To: Richardson, Bruce Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH] lib/librte_ether: change socket_id passed t= o rte_memzone_reserve I would be happy with the original suggestion. If the ethdev data for a por= t in use is in cache it removes the performance concern associated the curr= ent setup and my fix. The original suggestion also fixes the crash that I w= as seeing because of memory being reserved from a numa node with no "--sock= et-mem" allocated. Cian -----Original Message----- From: Richardson, Bruce Sent: Wednesday, January 14, 2015 10:10 AM To: Ferriter, Cian Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH] lib/librte_ether: change socket_id passed t= o rte_memzone_reserve On Tue, Jan 13, 2015 at 06:05:25PM +0000, Ferriter, Cian wrote: > Comments on alternative solutions: > 1) how would this solution work when there is no NIC present, and "rte_et= h_from_rings" is called? Here, could you have an else where the socket id o= f the master core is passed to the "memzone_reserve"? > 2) how would you advise making this change? I have looked at where "rte_e= th_dev_allocate" is being called and in all but one case, there is a "numa_= id" that could be passed in. This isn't the case for " rte_eth_dev_init" ho= wever, is there an easy solution for this? Would there now need to be an "r= te_eth_dev_data" struct for each socket that there is a NIC attached to, re= serving memory from that socket? >=20 > Cian While I think the issues you highlight can probably be overcome, I'm not so= sure any more how much it matters what numa node this is allocated on. The= ethdev data for any port in use by a port should be in the cache. In that = case, if it doesn't matter, your original suggestion would work fine. /Bruce >=20 > -----Original Message----- > From: Richardson, Bruce > Sent: Tuesday, January 13, 2015 1:56 PM > To: Ferriter, Cian > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH] lib/librte_ether: change socket_id=20 > passed to rte_memzone_reserve >=20 > On Tue, Jan 13, 2015 at 09:23:16AM +0000, Ferriter, Cian wrote: > > Passing a socket id of "rte_socket_id()" can cause problems in non DPDK= applications as there is a dependency on the current logical core we are r= unning on. > > Passing " rte_lcore_to_socket_id(rte_get_master_lcore())" as the socket= id to rte_memzone_reserve resolves these issues as the master lcore doesn'= t change. > >=20 >=20 > The only trouble is that when affinitizing the memory for the NICs to the= socket of the master lcore, it gives us no way to correctly configure an a= pp to use NICs connected to two different sockets on the one system. All me= mory for all NICs will end up on the same socket. Two possible alternative = solutions: > 1) affinitize memory to the socket the NIC is connected to > 2) add a socket parameter to the API calls to allow the user complete=20 > control over their memory allocations >=20 > Obviously the second one breaks backward compatibility (assume we modify = existing API call), but is more powerful. >=20 > Thoughts? >=20 > /Bruce >=20 > > -----Original Message----- > > From: Ferriter, Cian > > Sent: Tuesday, January 13, 2015 9:22 AM > > To: dev@dpdk.org > > Cc: Ferriter, Cian > > Subject: [PATCH] lib/librte_ether: change socket_id passed to=20 > > rte_memzone_reserve > >=20 > > Change the socket id that is passed to rte_memzone_reserve from the soc= ket id of current logical core to the socket id of the master_lcore. > > --- > > lib/librte_ether/rte_ethdev.c | 2 +- > > 1 files changed, 1 insertions(+), 1 deletions(-) mode change > > 100644 =3D> 100755 lib/librte_ether/rte_ethdev.c > >=20 > > diff --git a/lib/librte_ether/rte_ethdev.c=20 > > b/lib/librte_ether/rte_ethdev.c old mode 100644 new mode 100755=20 > > index 95f2ceb..835540d > > --- a/lib/librte_ether/rte_ethdev.c > > +++ b/lib/librte_ether/rte_ethdev.c > > @@ -184,7 +184,7 @@ rte_eth_dev_data_alloc(void) > > if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY){ > > mz =3D rte_memzone_reserve(MZ_RTE_ETH_DEV_DATA, > > RTE_MAX_ETHPORTS * sizeof(*rte_eth_dev_data), > > - rte_socket_id(), flags); > > + rte_lcore_to_socket_id(rte_get_master_lcore()), flags); > > } else > > mz =3D rte_memzone_lookup(MZ_RTE_ETH_DEV_DATA); > > if (mz =3D=3D NULL) > > -- > > 1.7.4.1 > >=20