From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 39B4B43FDD for ; Fri, 10 May 2024 17:07:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 24FD7402F1; Fri, 10 May 2024 17:07:47 +0200 (CEST) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by mails.dpdk.org (Postfix) with ESMTP id 8A7A6402C4 for ; Fri, 10 May 2024 17:07:46 +0200 (CEST) Received: by mail-lf1-f45.google.com with SMTP id 2adb3069b0e04-51f99f9e0faso2326602e87.2 for ; Fri, 10 May 2024 08:07:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715353666; x=1715958466; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=HXDf81cK1zqEPIfw0YYM+Ovv8Yuty8NqmwshIC0/ods=; b=NFL+nOVnZdPDbT3Tth5HC10OJKIKq0PYk7tLepkhg+zglMAX5CHwJd4jOJzKvNezEH Sb0hN1x7R8EEiGmEQvNA8Eg26rudAeMHdZLh0ni4HxY5378kfdA40ywDIcBRMDH8j7Gy c5CA2iRw1omk0iAAxX1gDrxA60hhkZavwEYHGTTGHGGRhRfRuyAsw/1iaOkXKZRBOm9b Rxn1tWEMAkgFpUQCBSa9pL/KVez13iR0f3RgPjCOaaVvUXyUJ7l6kEJBd07bWr6tnKmS l78KMMeYPQ3FZo4hpkDfgVHVXPpEsgQxWdkKLQaXJu3ZSf4yLKdFoLVZ1HVEH77WDqH9 H24w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715353666; x=1715958466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HXDf81cK1zqEPIfw0YYM+Ovv8Yuty8NqmwshIC0/ods=; b=l5Ms9EnpsmMWfJIceUvdTNf3jQAQhuLk6FaqSCyZZhXV9Vc2ScaizcXEq70JQ679sa 07vh13fda7N3WJR0SF+Nae05Cid3eycuoIdRayd7lJagM6Gukbkg0jPMdVJPxXGaUUW3 6P3JNXK5qkFs4q0E3c0SP6ZIp1FJ0WxLzGIcJxtZamUVrz7JWz4F3ZnRt7hNDxi34EOl 2zGKWMChXwBAlI2X7nAsL9urlodLf4cjKcimkHUO21/L22HUrUVl247SUV5P8E2WzgfZ tBQtRETvArI0OeOie2ON8jjnuUo7nc9paF3QXMKbyJYCBgPf4F8kxlw59O2ZBroZ2v3Y Yg4g== X-Gm-Message-State: AOJu0YzVFi/T78JCIJjtBaF62dH0vBue1su0FygDg1blq5k4SpAuh59c K1lIPFDZqoGqhLJgNyDQ/gzs5nrSBPgKbIgpmVsxIP+7izyrYmen X-Google-Smtp-Source: AGHT+IGWxltbcqLeDGS0QuqupehNyLpkbDS3NEE+S+bgIrAB5ncu/k/Xex6mDCi0LWPsXQQRxeTCDQ== X-Received: by 2002:a05:6512:2ee:b0:51b:5c9f:992e with SMTP id 2adb3069b0e04-5220ff71d7bmr1947939e87.14.1715353665389; Fri, 10 May 2024 08:07:45 -0700 (PDT) Received: from sovereign (broadband-109-173-110-33.ip.moscow.rt.ru. [109.173.110.33]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-521f38d32fcsm713268e87.167.2024.05.10.08.07.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 May 2024 08:07:44 -0700 (PDT) Date: Fri, 10 May 2024 18:07:43 +0300 From: Dmitry Kozlyuk To: Antonio Di Bacco Cc: users@dpdk.org Subject: Re: Failure while allocating 1GB hugepages Message-ID: <20240510180743.53c37660@sovereign> In-Reply-To: References: X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org 2024-05-10 11:33 (UTC+0200), Antonio Di Bacco: > I have 16 hugepages available per NUMA on a 4 NUMA system: > > [user@node-1 hugepages]$ cat > /sys/devices/system/node/*/hugepages/hugepages-1048576kB/free_hugepages > 16 > 16 > 16 > 16 > > Using the following program with dpdk 21.11, sometimes I can allocate > a few pages but most of the time I cannot. I tried also to remove > rtemap_* under /dev/hugepages. > rte_memzone_reserve_aligned is always supposed to use a new page? > > #include > #include > #include > > #include > #include > > int main(int argc, char **argv) > { > const struct rte_memzone *mz; > int ret; > printf("pid: %d\n", getpid()); > // Initialize EAL > ret = rte_eal_init(argc, argv); > if (ret < 0) { > fprintf(stderr, "Error with EAL initialization\n"); > return -1; > } > > for (int socket = 0; socket < 4; socket++) > { > for (int i = 0; i < 16; i++) > { > // Allocate memory using rte_memzone_reserve_aligned > char name[32]; > sprintf(name, "my_memzone%d-%d", i, socket); > mz = rte_memzone_reserve_aligned(name, 1ULL << 30, socket, > RTE_MEMZONE_IOVA_CONTIG, 1ULL << 30); > > if (mz == NULL) { > printf("errno %s\n", rte_strerror(rte_errno)); > fprintf(stderr, "Memory allocation failed\n"); > rte_eal_cleanup(); > return -1; > } > > printf("Memory allocated with name %s at socket %d physical > address: %p, addr %p addr64 %lx size: %zu\n", name, mz->socket_id, > (mz->iova), mz->addr, mz->addr_64, mz->len); > } > } > > // Clean up EAL > rte_eal_cleanup(); > return 0; > } Hi Antonio, Does it succeed without RTE_MEMZONE_IOVA_CONTIG? If so, does your system/app have ASLR enabled? When memzone size is 1G and hugepage size is 1G, two hugepages are required: one for the requested amount of memory, and one for memory allocator element header, which does not fit into the same page obviously. I suspect that two allocated hugepages get non-continuous IOVA and that's why the function fails. There are no useful logs in EAL to check the suspicion, but you can hack elem_check_phys_contig() in malloc_elem.c.