From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6937442BAC; Fri, 26 May 2023 17:33:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 020CC40DDA; Fri, 26 May 2023 17:33:38 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 4B86C4068E for ; Fri, 26 May 2023 17:33:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685115216; x=1716651216; h=message-id:date:subject:from:to:cc:references: in-reply-to:content-transfer-encoding:mime-version; bh=ZvM5FsQ/MeL/j1UH4d5HIRgimiQbzDJFLVrWhy7Xjq4=; b=MceJaRpYEsgAUO6q8CgsM4fIJmpaGtxM0YpFslemI5sf20ITj6sPlLFt xyu3wp+Xzp5zxym++Slb+X4tPB3gZSW7+tn4ddHdz2njsHDOz5I3F209j xwueNiIJ5DJRxMs7M6eLf4k5xqCKsIlvzj922/73iEIXObmhKYokL9k7G XfzAPS2gizEh0fB8uNtQGk54AsJo9vJ4PuoLX7zIsRohGCaPISa34UXt7 DMICcbiR9UU2k8SBCY9mVj17BIzMwVFJoou+RZka+tHrHGVr8T6Vb/NYx IWafoetDO0Xlj1427fFVJfU2e2+k3Znmpc9/qBfLtc2RAuamul91N6+fa w==; X-IronPort-AV: E=McAfee;i="6600,9927,10722"; a="356609852" X-IronPort-AV: E=Sophos;i="6.00,194,1681196400"; d="scan'208";a="356609852" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 May 2023 08:32:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10722"; a="879576848" X-IronPort-AV: E=Sophos;i="6.00,194,1681196400"; d="scan'208";a="879576848" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga005.jf.intel.com with ESMTP; 26 May 2023 08:32:41 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Fri, 26 May 2023 08:32:40 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Fri, 26 May 2023 08:32:40 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Fri, 26 May 2023 08:32:40 -0700 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.46) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Fri, 26 May 2023 08:32:39 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h9sNBoU1Cn3fbzoCYmr5tTPBBog35RejJOk/a4hwM3FU2iTG118BDxGDLBBk4Cj/W289qDfm1aJHvH7YPZqc03DoODcp6yn2k3OUW03/RO1d7QF+4KdL/NLQS4rJiC3GWFURCPra5P7pecbRqngfqeClDIE1IAiyp0rgr/OuXPMGh5tnari1s7AKZHfcS1Ocy0w9Lxn6UBIirnwjU9SS30k3Fpl+V/XjuN1tQzhDAFpdgNtQylnNmLVL8rY5kuP0sqt1QGd9zkEsj62VDDPFNWPhbPoDKUCci4qzIrTelSfkj6c7VKCPy0YkacMCK/008Rk/qvZIR8pDdsCiy+ii2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rqFuptcqGMYxjz3W5ulPfjg4h/3AE0Xn0T3oKhvLXFw=; b=NmHl6omrLzGhPiMrFafOTlq+tem4tYdE1ftQIXuMarpkTLGdLfMVPck5tdPsq8sCKDiWyIgnQgEjhZZssgLe98+97i+hQoRFrpmDJgzNL99WdFV6EiV8hZyZuqbNI0ttjlclrTACGGUCc9v/7/XpvbSMPo1WG4CzkI+3pIkfHV3lf55Q0yForIMF1FK0yVhg+R/pamg0X7eAN4hX6zRMaMYYTFb6uK18377Yoq51rYFZr0hEOxYoum+mViYI5rMpVl6t4tsaKGDwTtTsfy97UiKA1EqLuhqTHz1c9W6UdkZECZCfIy90NMKx9GpqI5tsBUa3pUhXeJsiPs6Gl9eYlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by LV2PR11MB6022.namprd11.prod.outlook.com (2603:10b6:408:17c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.28; Fri, 26 May 2023 15:32:38 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::49c0:aa4c:e5b4:e718%5]) with mapi id 15.20.6433.017; Fri, 26 May 2023 15:32:38 +0000 Message-ID: <77e7da3f-6a4c-abf9-bffc-6af2f59e1830@intel.com> Date: Fri, 26 May 2023 16:32:32 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.11.0 Subject: Re: [PATCH v3] eal: fix eal init may failed when too much continuous memsegs under legacy mode Content-Language: en-US From: "Burakov, Anatoly" To: Fengnan Chang , CC: Lin Li References: <20230526034127.18231-1-changfengnan@bytedance.com> <4099387b-9f8a-ea0c-8ab9-4c9de15117c4@intel.com> In-Reply-To: <4099387b-9f8a-ea0c-8ab9-4c9de15117c4@intel.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: DUZPR01CA0184.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b3::7) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|LV2PR11MB6022:EE_ X-MS-Office365-Filtering-Correlation-Id: b881f5f2-d119-49a2-ca89-08db5dfe6f4f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: L31wYALVpub4wL+y1QVI0z3Q5YedHqh0w/TvzUwv+fbmLgH8dvyLAgZShhC931G/8liM0dW1v/iFKG4h53sd+LlVL+RrU9mbhYjtJqubcuUisu6mZPd4ciAEUP9ITKGP+xRTt4fXTy9ATb4C4NhlrPVVDzo9uvByISpdXpeRh733enIXjjLP8URe5wSXCZgAy3KbvaZXykGU1ySTIk1RqBK1cEavGDLLLp91EvuZ448XwDEzIgEVMWg8bWwGV/7W7PWg3nwy7vKwtyirE4V6FLhlDvGvSvhX/urQRGns+D9VzxP47wF7rXCa86gnkotMHcCSeuSLmj6ajMCAAf2RKZfKQ0Bq46817sjhLm4Bxtg50Wa2Z3p9MUNkuVY3xEX3UTz52USXX6iNrkjOGsKxj/ubBe2vha/cICJULuQFf1h9mEYQa36IYxhXsaSxkKGw3IXpKFSk5ES9wKOs6SADUn2TGXKA4sAiRdu8oo1DewCacuTgVwZs699AX/cTx+284B3l8L6THQZFkNdkONphU/uj3IZY889pnM3VlKf58c4Pu4EqX0U/LLvvgUZP5Y7it1Lno2UTrmbwcxczYioN4mT9g6SW0JB1+9hghsjVzNetb3bFEOG82K2U+1qSKm4a5jWdlDj8jQ4GLPqCr5ECdQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(396003)(376002)(39860400002)(346002)(136003)(366004)(451199021)(66476007)(66556008)(4326008)(31696002)(31686004)(66946007)(41300700001)(6666004)(316002)(6486002)(5660300002)(478600001)(8676002)(8936002)(86362001)(82960400001)(26005)(6512007)(6506007)(186003)(53546011)(2906002)(38100700002)(83380400001)(2616005)(36756003)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eGJrT0EvcmZPR3pwcXhnL3pkSDVCN0xkdGFuZEF1eVdaZ1hOZ2FhT2ZiRkR5?= =?utf-8?B?cVRTNVc1MTYydU1rbTdiMXBUSUlJRklFb3YxVWkxdFdZTEozN3lwUHlUZmsy?= =?utf-8?B?aFJNY2VHNGZGRnp4R0htd1dKVUV0dkJrRHhZVlJJb285ZWpVQzBoYWpIeVpn?= =?utf-8?B?ekU0dnBkUlBUTnNMY1ZFTGJyWWJMUlk1cUlhSEVrZ3lzR21mTUlxRGM1KzFK?= =?utf-8?B?ZlJsK3BkcnFNbFdsWEVsWHd2eC85czl4b1FFUEI3M3dCcC9teUNmQnlaYm1r?= =?utf-8?B?VnJubWV2MTJWQmtEUHo3ZEZuZ2E5ZjFndU10OVl4WW1lWGZseDY2bWhGbE5O?= =?utf-8?B?cVhWRkwxa3ZWaS9ZRlhwY2tobDJHVitXblo0MlJORDNyR2ZodWtIRHA3dm5C?= =?utf-8?B?SG1NdkduLzBibjNoaU0rTEdIODZ2VUw4Nmpocm1TZnpkSUdKTU9mQXhwcGN2?= =?utf-8?B?dnVGK0g1M3RncHRRR2tKY0JyMjQvclhxSS8rWDMwL2NhYkJkUmtDbUdvZUx0?= =?utf-8?B?enQ5UjFneXZGTjUxR2RPdGh0R0RrLzhrS2lOam1ZQk93d2p5dFF3VllmTDk1?= =?utf-8?B?QjlHMU5IdjArYUlWbWFscTNISHArS3ViM3h4N1h2SVhKL1dTU0QvMkFlenhS?= =?utf-8?B?SnQ5N2xYK1Rkc3R6MEZwaytnZlR4UzJEcU84cDVKVkpPR2ZHNWtWMk84ZGox?= =?utf-8?B?UHE5aFMxTTlzVU9RSEdreUV0MFlpQ2lsK1FIUVpEbUI4OFFUam1kSS9jVEhF?= =?utf-8?B?d3RMcFNaTHFXTFZINFJyU0twYmg3T2RuTU1LNnV5UVpTOUNxUEt6TjA2NUtM?= =?utf-8?B?K2UxMFhtZ1BMVDFCYVhKZ3UzaTNxamMxTzEwcE9lcVMxd0dicGtIQTFsNTlu?= =?utf-8?B?eDN4MjVwUnAwaEdXUW1ZNEZITVRLdnNlb3o2Q1RJc2tzRTlMZ3paRlpCSkF5?= =?utf-8?B?K3BSU3k0VG94MDJBZVZuSEpteFBnOFptV3NNbVhiUUpxVCs5bDdvdkhWK1ZI?= =?utf-8?B?RW5JZmR2ZzUyRmdVMC9FZmpTcTdrenpMQWl2bXNIdjFOR055MnZ3VjByaE9V?= =?utf-8?B?OUhkNTlOQ0lXNENxbHBIcVp2WlFTMVhKNG9ERE1TR0J2cmhUaktOaTI3VE9r?= =?utf-8?B?bW80UmVwd1ZUVlZMV09IREpUSHVHdERWei92dDlDUjlVMnVGa1czSkNDSldh?= =?utf-8?B?U2tYanRSTCtGRG10Mk5LSDFCZmtBNWw3c1dwZDB0TTlSaUUrY3RxNnpqdFpp?= =?utf-8?B?MVFzSDhqNjdyYVIzK2llNVR6RHpjUFhzWGE0ditOTUlLd0VZNFhwSjA0ZUI0?= =?utf-8?B?RFJvOHErR0tmU2pCeHc0WWJ4SDZXbjRmQ3lDWUlIWEliV2hVbXo4N1c2WFQw?= =?utf-8?B?ZExkOWVUaEJSQktGVk5HTnpUYityYkJYdndVZFZJN21peXlYa3FEVlkvRkVE?= =?utf-8?B?UWtUMkJxQjl3UkNuemFQck1NekJYZDNyVC9WbENheHJWYVc0NzNUZnhDTWsr?= =?utf-8?B?WUdPVGxoWWJiZjcrd1pDQ1NwUnROMm4xeHpQY2g4c0ZTcWRIeXFhQ1RGSm1k?= =?utf-8?B?WVZ3RXM4T0xKOGs0ZkFHQTE5K0Jsdy85V01zTkd6WDdJTGdia2NwWWNlM2lT?= =?utf-8?B?VWVHUEx6ODFzZ1VrTG1DQzZ1enFOaGNjNEgyeWdYNVlSbUpTNjQvQWlZT2g0?= =?utf-8?B?QTNPbG55WExyeWJqbGdrWUJGZXovbXJrK29BN0QzSGZvODZ2ZGxSMWp2Tk9y?= =?utf-8?B?MXAyOEZuandHVlJIMGlYQjZzRXk5T29hZmFNek9HbUY5dlNzQkVrWG5tcXZN?= =?utf-8?B?K01ESDVyNThla2F3Zng1cTdUZDRwaUlaN0RKTWlaYytibUh2akJaeEp0VVEv?= =?utf-8?B?dmhPRnUyWTFGejRDbHpDaFUyeHZCeTAzVGxuOE1aektGdzJJUjMrakNxa1RP?= =?utf-8?B?K2lFM3JaU0NTcnA4aVYzOTgxSDRGdVNZUHhmKzZtbVh3VzBDV3BVRXhlendP?= =?utf-8?B?cklsKzhrREVON3hYT1B0QzBVMU9zOHhkdHVtZ3dmbzIwbHlxOUVXeHV4a2ZV?= =?utf-8?B?aVJqUzlhQ1VOK3ozZHVwcUZaOVRhSytKUGllMmhXU0FPV3J6WEU5dW5HK2w5?= =?utf-8?B?bm9WSEg4UGk5Und4VUZoREpWZHpVeDUwRjVPMU1zeDZXUnpkUDV3eThuVDBY?= =?utf-8?B?dEE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: b881f5f2-d119-49a2-ca89-08db5dfe6f4f X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 May 2023 15:32:37.9880 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RTS+HqKweDlKNBanSalvxqJQ+hSz+P/TYS3hw6o0wKxKzourO10ORdEFGzaZ5yHJSPVOL5Pr4s+I/AKFQZXBjmAkbK0jIVADeiBhKqbkb/k= X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR11MB6022 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/26/2023 3:41 PM, Burakov, Anatoly wrote: > On 5/26/2023 4:41 AM, Fengnan Chang wrote: >> Under legacy mode, if the number of continuous memsegs greater >> than RTE_MAX_MEMSEG_PER_LIST, eal init will failed even though >> another memseg list is empty, because only one memseg list used >> to check in remap_needed_hugepages. >> Fix this by make remap_segment return how many segments mapped, >> remap_segment try to map most contiguous segments it can, if >> exceed it's capbility, remap_needed_hugepages will continue to >> map other left pages. >> >> For example: >> hugepage configure: >> cat >> /sys/devices/system/node/node*/hugepages/hugepages-2048kB/nr_hugepages >> 10241 >> 10239 >> >> startup log: >> EAL: Detected memory type: socket_id:0 hugepage_sz:2097152 >> EAL: Detected memory type: socket_id:1 hugepage_sz:2097152 >> EAL: Creating 4 segment lists: n_segs:8192 socket_id:0 >> hugepage_sz:2097152 >> EAL: Creating 4 segment lists: n_segs:8192 socket_id:1 >> hugepage_sz:2097152 >> EAL: Requesting 13370 pages of size 2MB from socket 0 >> EAL: Requesting 7110 pages of size 2MB from socket 1 >> EAL: Attempting to map 14220M on socket 1 >> EAL: Allocated 14220M on socket 1 >> EAL: Attempting to map 26740M on socket 0 >> EAL: Could not find space for memseg. Please increase 32768 and/or >> 65536 in >> configuration. >> EAL: Couldn't remap hugepage files into memseg lists >> EAL: FATAL: Cannot init memory >> EAL: Cannot init memory >> >> Signed-off-by: Fengnan Chang >> Signed-off-by: Lin Li >> Signed-off-by: Burakov Anatoly > > Hi, > > Thanks for taking my suggested implementation on board! > >> --- >>   lib/eal/linux/eal_memory.c | 55 +++++++++++++++++++++++++------------- >>   1 file changed, 36 insertions(+), 19 deletions(-) >> >> diff --git a/lib/eal/linux/eal_memory.c b/lib/eal/linux/eal_memory.c >> index 60fc8cc6ca..085defdee5 100644 >> --- a/lib/eal/linux/eal_memory.c >> +++ b/lib/eal/linux/eal_memory.c >> @@ -681,6 +681,7 @@ remap_segment(struct hugepage_file *hugepages, int >> seg_start, int seg_end) >>       /* find free space in memseg lists */ >>       for (msl_idx = 0; msl_idx < RTE_MAX_MEMSEG_LISTS; msl_idx++) { >> +        int free_len; >>           bool empty; >>           msl = &mcfg->memsegs[msl_idx]; >>           arr = &msl->memseg_arr; >> @@ -692,24 +693,28 @@ remap_segment(struct hugepage_file *hugepages, >> int seg_start, int seg_end) >>           /* leave space for a hole if array is not empty */ >>           empty = arr->count == 0; >> -        ms_idx = rte_fbarray_find_next_n_free(arr, 0, >> -                seg_len + (empty ? 0 : 1)); >> - >> -        /* memseg list is full? */ >> -        if (ms_idx < 0) >> +        /* find start of the biggest contiguous block and its size */ >> +        ms_idx = rte_fbarray_find_biggest_free(arr, 0); >> +        free_len = rte_fbarray_find_contig_free(arr, ms_idx); >> +        if (free_len < 0) >>               continue; Missed this. When we're splitting up segments, we're looking for contiguous free areas that are at least two memsegs long, so if this memseg is full but contains segments that were split up, there will be a bunch of holes in it, and free_len will be 1 (because every hole will be 1 segment long by definition). So, we should not accept anything that is at least two segments long. -- Thanks, Anatoly