From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C8EF7A057B; Tue, 14 Apr 2020 13:23:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 20B851C1C4; Tue, 14 Apr 2020 13:23:31 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id EC7B51C1C1 for ; Tue, 14 Apr 2020 13:23:26 +0200 (CEST) IronPort-SDR: tH9FITFootgpuNzt8PGnjpfVnUcDE/aicc3OtBAQbipid1o7dbfIdSRAaQ+flPNvVgoQHuCx4g gvwjzF9RV7Pw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Apr 2020 04:23:25 -0700 IronPort-SDR: x24eXGmut84+Lj0NbatuD9ipRqkIAekDWy750rOI152kOdcZnrKtYcLh+ibz+YuYvJ5COOf/vm uvQLD3htipug== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,382,1580803200"; d="scan'208";a="245390661" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by fmsmga008.fm.intel.com with ESMTP; 14 Apr 2020 04:23:25 -0700 Received: from orsmsx153.amr.corp.intel.com (10.22.226.247) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 14 Apr 2020 04:23:25 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX153.amr.corp.intel.com (10.22.226.247) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 14 Apr 2020 04:23:24 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.44) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 14 Apr 2020 04:23:24 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GpNH7lJ2LJ0kwKcA0Gf7T7osYKw4Mp/Ai6j2ebcN8Gq6DdvPB6Gw1fAIh5rt0DZ5kck6jYA6ZNB/ZURbN+QwlOSp+B1XTUp7ZSWoog3gZJoWhuTdaCnNAzuDcH/T0ItyZx5EmRXV6hwf3wk2fiMrz1txiKiVyOsAL98Ai5EmL5PnM0gmVVX/ixdC2cCMA/SLSVyPJR7GulupOz8/CBImwxxvlY/EV6QbzM/8vvl2O2Otc2DcKypjfBSZlhbRYAhBSNLkk3sCqVCbEvSOMigJNdNQ7QwjzCYazuhyMpUbgxC1yOx9Pb/rUYiDqSb7LDExporxMP7TwZ7cwkARtRUN7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UouHElJr1iU3INuYlldyzx8RR0euql2I4PVJnpU0wco=; b=Tw/GjDF6Ca5XofhUxGh46ZXYAYXK9dj5nxKd37uSan5aXPrSmoPDk//ET54ki/rOMBkXOvWAo0C/kbD/8SGm1F3Iio5NJfjoBr44cXqMbFwMs3jjuqICPpnEznli3tYUl3L6ObxLVG9GkzAgumnXnKihUTw+Avo1o6MZyWWtEQlmRk5tLTqOCKnFHBiQp5FGk39dFvHOaWhzNyvJr431K7B02YBrb6R6vXlxmRaDRlbL5CKOOIykjkzP6D9F8a/13MXb4vB0oQspna9QGI7pvY8k3wfWpQ5s8o6xG9fn20FEYnL+T538QTZdy3aJV9holu0xWK2Ysp5buzD2B+Rg6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UouHElJr1iU3INuYlldyzx8RR0euql2I4PVJnpU0wco=; b=HjIj/P/q6OpvG0S748WTvLq6X2hGX0cf0Ak56o9L3wpHa5PKQ6d/HRr0EjEV35+8EX60czBgTJ9IxFaEosVAy+uiiP5F9+qik2HtZj0QbAS0SIlYyVfwb1Bx5d4u+CXbC3cYbj5+osIfq4+lGPYsiqx3jvmkhp9yM993eOURM/Q= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BYAPR11MB3333.namprd11.prod.outlook.com (2603:10b6:a03:19::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.28; Tue, 14 Apr 2020 11:23:23 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4%6]) with mapi id 15.20.2900.028; Tue, 14 Apr 2020 11:23:23 +0000 From: "Ananyev, Konstantin" To: Christos Ricudis , "Burakov, Anatoly" CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] eal: Place EAL thread stack in a reserved per-lcore memzone Thread-Index: AQHWEXNhl9vSrUz72E6pIC/KGTf/N6h4d7RQ Date: Tue, 14 Apr 2020 11:23:23 +0000 Message-ID: References: <1586768952-10554-1-git-send-email-ricudis@niometrics.com> In-Reply-To: <1586768952-10554-1-git-send-email-ricudis@niometrics.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.172] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2ef1786a-84a4-41da-2973-08d7e0663ea5 x-ms-traffictypediagnostic: BYAPR11MB3333: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:635; x-forefront-prvs: 0373D94D15 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10019020)(376002)(136003)(346002)(396003)(39860400002)(366004)(6506007)(26005)(478600001)(8936002)(81156014)(66946007)(66476007)(64756008)(9686003)(55016002)(66446008)(66556008)(316002)(6636002)(110136005)(5660300002)(76116006)(4326008)(52536014)(2906002)(33656002)(71200400001)(86362001)(186003)(7696005); DIR:OUT; SFP:1102; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: oqSkHgZZ4FwnlOxwPF7aK+RR9B/t9JRbNJxDDziSxNtUb/6zHakfislaTRJlPjIfHoNZV4cO/ZzoNHuPGVGhaVQcAsa8NQBcDjdY6Q+0kdZ2BSM6fi90B0BsXHzxO6VcLnuEc2Pr6Tu909VYRGYGvQSnqqzL/muI5WfSX0ngqtpGYDuzM3KxPGA6jwX/xnn3/67EJe8yNUEP018JTS7pTw4XkkhSl9veMYX5Z1hPBOnxf6ims2V1QV6CCx4Pf8NY4Hvc7QpcDhX+L87Va7WHLK5cWGrjJKPLOt6gZ1FcOBzwU932RMf+N5En5fGrv6CnQvDfGYHzeRvIGLe12eGJ3/oNNhWKZoHif5kxEESy4yMkMm1QY6FsInxGHWb342IOX7eDQZ71Hc/IFL7E+94GMNxcQGYouEWdc0Qd9BcNR186XbsnIR+DZ+HwGzN45AmE x-ms-exchange-antispam-messagedata: cCd40yaQdS0PWSB+W26U+WCfC1U3n0Nz1l+Wgn3SDpPK3nyOZwZhkXYiYO91UFhDz4eFxh8MfoFd1XUZAm0daqSVq35/HkB0xo+CHtpEfn6oQdqwnVlFl4Vd8FhQkMpKL1RxautF8R8oaJ8k5c616Q== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 2ef1786a-84a4-41da-2973-08d7e0663ea5 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Apr 2020 11:23:23.7192 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: L1Y9Z5n6iF4EB1OqnbcJZuuyO3g/1djJeNEVLXgn0TkOxSxH4CROTsspqIDuLLv95OFwiMqwO1TFHWW1/i8Mz6tZ2uP2SArYJXEFzYCGK5E= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3333 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] eal: Place EAL thread stack in a reserved per-lcore memzone X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, >=20 > Reserve a per-lcore 4MB memzone and allocate thread stack of EAL threads = there for better NUMA locality of stack-allocated variables I wonder if there any real performance improvement seen with that change? Any case (existing DPDK app/example) that can demonstrate it?=20 Konstantin >=20 > Signed-off-by: Christos Ricudis > --- > lib/librte_eal/linux/eal.c | 20 ++++++++++++++++++++ > 1 file changed, 20 insertions(+) >=20 > diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c > index 9530ee5..e047107 100644 > --- a/lib/librte_eal/linux/eal.c > +++ b/lib/librte_eal/linux/eal.c > @@ -68,6 +68,8 @@ >=20 > #define KERNEL_IOMMU_GROUPS_PATH "/sys/kernel/iommu_groups" >=20 > +#define THREAD_STACK_SIZE_DEFAULT (4ULL * 1024ULL * 1024ULL) > +#include > /* Allow the application to print its usage message too if set */ > static rte_usage_hook_t rte_application_usage_hook =3D NULL; >=20 > @@ -1224,6 +1226,24 @@ static void rte_eal_init_alert(const char *msg) >=20 > lcore_config[i].state =3D WAIT; >=20 > + pthread_attr_t attr; > + pthread_attr_init(&attr); > + size_t thread_stack_size =3D THREAD_STACK_SIZE_DEFAULT; > + char thread_stack_name[64]; > + snprintf(thread_stack_name, sizeof thread_stack_name, "rte:lcore:%s:%d= :threadstack", rte_eal_process_type() =3D=3D > RTE_PROC_PRIMARY ? "p" : "s", i); > + const struct rte_memzone *mz =3D rte_memzone_lookup(thread_stack_name)= ; > + if (mz =3D=3D NULL) { > + if ((mz =3D rte_memzone_reserve(thread_stack_name, thread_stack_size,= lcore_config[i].socket_id, 0)) =3D=3D NULL) { > + rte_panic("Cannot allocate memzone for thread stack"); > + } > + } > + void *thread_stack =3D mz->addr; > + > + if (pthread_attr_setstack(&attr, thread_stack, thread_stack_size) < 0)= { > + rte_panic("Cannot set thread stack\n"); > + } > + RTE_LOG(DEBUG, EAL, "Thread stack for lcore %d on socket %d set to %p\= n", i, lcore_config[i].socket_id, thread_stack); > + > /* create a thread for each lcore */ > ret =3D pthread_create(&lcore_config[i].thread_id, NULL, > eal_thread_loop, NULL); > -- > 1.8.3.1