From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D809DA04F1 for ; Mon, 9 Dec 2019 14:14:05 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3F55B1B994; Mon, 9 Dec 2019 14:14:01 +0100 (CET) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10087.outbound.protection.outlook.com [40.107.1.87]) by dpdk.org (Postfix) with ESMTP id 8D1231B9B5 for ; Sun, 8 Dec 2019 16:58:31 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dRyH8YDEGC6hwxDCkIRYW+AvJkT9HMOL6k4mQEUM/eZT0JvdC2rQmG7p3QRgNYDHC2Fs1KRRH4jpH1i0SN+NY7EWcJNregwKtvNt/nafnkiNzqMY99vCi+UujA5DKwk8R/ZKd4K0HArde1rLvqghOjxNfCJl8dDWEsPR5IwQusVRxQV4veV7V9m6rhIG+d2CONIXrOaIO7HIlTpXc0+3a9jEmFtCfCcXhs1mp4RYeKktlJPEZptU7yJQu8p/mOAJHPLHBdxJbDbeK+cysj9NDTLJze2O7eR55wzYF3NWfAHdlbYkVxe3Xk0zshgBOCNghZIK5BKPXOMRxpFxPmJopA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Uls1F5jnPES93v/IuKV1LQ+DPjT76fsMDXaeTvDcwjA=; b=RqYtDk2mKOU9jPXbiGvrC+6WPDBRlWwDwDDYTc/XKkeEIWVdM/C++hIVII0L01g9NkglkHUx8Sfb2s+PbAC/CVRMcF88QaTocKCDMh2Q77cUx6zQ/v4YG8ecIVoH8e1l38T1HdgHrCdsi7eaD7X4T/9JV1NM0Xkyn6K1aOgpAoZW+U4io4JTVMJn8EyKLF1TnkD6UZh2Mchyn6+Gpl2mjwh4pnqcXDG6Hzyl6AmS57VAHgxVrBPXxpDymTMDtR8TDStU/91+8LV1+T6px+s39Pp3wo14AXghrP4EdeJszgtAp82Un6UiDdxbIl8QvYyV/Tf0/BBJ+XS/oPPMPMvsSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 193.27.93.20) smtp.rcpttodomain=dpdk.org smtp.mailfrom=verint.com; dmarc=bestguesspass action=none header.from=verint.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=verint.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Uls1F5jnPES93v/IuKV1LQ+DPjT76fsMDXaeTvDcwjA=; b=H6gK2zxPjEQu22arp3yIEleytaTQrSgmzesFO1NOXmqEO++YaI+AdtL3M4BJvp8S4KqiWDQ+pP0WBjiaR0oKkITmVyvIx4IdxTttEyQI037rKTEZGuX8oB+4zql+Vzq6p7y9oKdWVC79EpcjR9ffxx3VpE9w/zUKEiCBplJJ1dQ= Received: from AM5PR0102CA0003.eurprd01.prod.exchangelabs.com (2603:10a6:206::16) by HE1PR01MB3884.eurprd01.prod.exchangelabs.com (2603:10a6:7:a0::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2516.12; Sun, 8 Dec 2019 15:58:30 +0000 Received: from DB5EUR01FT020.eop-EUR01.prod.protection.outlook.com (2a01:111:f400:7e02::201) by AM5PR0102CA0003.outlook.office365.com (2603:10a6:206::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2516.12 via Frontend Transport; Sun, 8 Dec 2019 15:58:29 +0000 Authentication-Results: spf=pass (sender IP is 193.27.93.20) smtp.mailfrom=verint.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=bestguesspass action=none header.from=verint.com; Received-SPF: Pass (protection.outlook.com: domain of verint.com designates 193.27.93.20 as permitted sender) receiver=protection.outlook.com; client-ip=193.27.93.20; helo=mail.verint.com; Received: from mail.verint.com (193.27.93.20) by DB5EUR01FT020.mail.protection.outlook.com (10.152.4.247) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2495.18 via Frontend Transport; Sun, 8 Dec 2019 15:58:28 +0000 Received: from TLVPEXCH1.Verint.Corp.Verintsystems.com (10.61.241.30) by TLVPEXCH1.verint.corp.verintsystems.com (10.61.241.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1847.3; Sun, 8 Dec 2019 17:58:26 +0200 Received: from TLVPEXCH1.Verint.Corp.Verintsystems.com ([10.61.241.30]) by TLVPEXCH1.verint.corp.verintsystems.com ([10.61.241.30]) with mapi id 15.01.1847.003; Sun, 8 Dec 2019 17:58:26 +0200 From: "Ram, Hila" To: "users@dpdk.org" Thread-Topic: Memory allocation issue after upgrading to DPDK18.11 Thread-Index: AdWt4FLrAn0QuidiQzKpqETkKW91aQ== Date: Sun, 8 Dec 2019 15:58:26 +0000 Message-ID: <940315f6101e4662bb87e19ea357c206@verint.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.61.241.83] MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:193.27.93.20; IPV:CAL; CTRY:IL; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(346002)(39860400002)(376002)(396003)(189003)(199004)(86362001)(26826003)(478600001)(5660300002)(36906005)(5640700003)(316002)(2906002)(36756003)(8676002)(336012)(2616005)(108616005)(24736004)(186003)(54896002)(790700001)(70586007)(6916009)(9326002)(70206006)(8936002)(356004)(26005)(1730700003)(102836004)(81156014)(81166006)(71190400001)(76130400001); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR01MB3884; H:mail.verint.com; FPR:; SPF:Pass; LANG:en; PTR:bzq-193.27.93-20.bgp.bezeqint.net; MX:1; A:1; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4d3e9f5b-d6a4-476f-ea1f-08d77bf777f7 X-MS-TrafficTypeDiagnostic: HE1PR01MB3884: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-Forefront-PRVS: 0245702D7B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /HfqvxkPTKU9xR6y4Uk/eFJq5pF0mTHx0V1K5+PeWc2X9MgJCNV4i3aSgSoE7rT4h0BM7hb9AnlttHTD7/j8umzm0OcQD+XaWTeR9VazZ4JlktVnUEZEtDg6vwql2lKQX1Pw1Kdgs2Yr1Y9DLOoRg/cjsMBNWgZETbof7N9F3yB/6KhIY6uenAP3Hr9B8VfcfIyXvUeEpBn7D1EUoAokp5UPN/ty9uy1V7kG28djxFWixSZ8IH6Gd1abv/qWU3pRU4nV6O0tizmns1kw9QwozA8+X3Z12P9tIRm3RvarU64/zYTWeQiyZbcWOEE2qBSfedvapq45Iqru3NKttnFz8zFqVXNLzT6o7btFFbWOLG3ze+C6SQaq0AYppoVaCJnlh/FO25vMihNuZtm7EU6+ajW4GRu8+k1rbBlwUmvbSspGHf1yLOBGMcVSTYs4MSt5 X-OriginatorOrg: verint.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2019 15:58:28.2948 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4d3e9f5b-d6a4-476f-ea1f-08d77bf777f7 X-MS-Exchange-CrossTenant-Id: bb2ed304-4099-49cf-b081-cbb7a3a580ca X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bb2ed304-4099-49cf-b081-cbb7a3a580ca; Ip=[193.27.93.20]; Helo=[mail.verint.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR01MB3884 X-Mailman-Approved-At: Mon, 09 Dec 2019 14:13:59 +0100 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] Memory allocation issue after upgrading to DPDK18.11 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, It's my first time upgrading DPDK version (from DPDK16.07 to DPDK18.11.5). Some setup relevant info: * Two Numas. * Single page size of 1G. * Total of 95 huge pages, 48 are being allocated for socket0 and 47= for socket1. After upgrading to 18.11, I get failure on application startup while trying= to alloc some memory block, which is greater than Greatest_free_size and n= o other pages are left. With the same application and amount of huge pages,= these allocations ended successfully when running with DPDK16.07. So I compared the allocation on 16.07 VS 18.11 and found out that when memo= ry is allocated statically in 16.07 we have no issue (where Free_size alway= s equals to Greatest_free_size), but when allocating memory dynamically we = allocate a new page whenever there is no enough contiguous memory, so I gue= ss we have some holes and as a result no free memory is left. 1. When working dynamically, doesn't the DPDK try to allocate contiguo= usly? If it does, why are we out of memory? In order to overcome it I enabled the memory legacy flag. However, once usi= ng it, I get the following error - EAL: Could not find space for memseg. Pl= ease increase CONFIG_RTE_MAX_MEMSEG_PER_TYPE and/or CONFIG_RTE_MAX_MEM_PER_= TYPE in configuration. With default configuration, each segment list has 32 segments (as being det= ermined by memseg_primary_init), 4 lists per socket. When we get to remap_segment (getting there from eal_legacy_hugepage_init = -> remap_needed_hugepages), and go through each memseg list searching for n= =3D48 free segments. However, on each list we have only 32 segments so we f= ail to remap segments. As a workaround I increased CONFIG_RTE_MAX_MEM_MB_PER_LIST from 32768 to 65= 536 so as a result each segment list holds 64 segments. Modifying CONFIG_RT= E_MAX_MEMSEG_PER_TYPE and/or CONFIG_RTE_MAX_MEM_PER_TYPE as printed in erro= r message had no influence as calculation takes into account the min value = of per_type and per_list, where per list was the smaller one. 2. Why do we check for n segments in a single list instead of in all l= ists? 3. Is it correct handling increasing num of segments per list? What if= we have more huge pages (which result with more than 64 per socket) - we s= hould re-increase it? Thanks, Hila This electronic message may contain proprietary and confidential informatio= n of Verint Systems Inc., its affiliates and/or subsidiaries. The informati= on is intended to be for the use of the individual(s) or entity(ies) named = above. If you are not the intended recipient (or authorized to receive this= e-mail for the intended recipient), you may not use, copy, disclose or dis= tribute to anyone this message or any information contained in this message= . If you have received this electronic message in error, please notify us b= y replying to this e-mail.