From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD98F42BD8; Tue, 30 May 2023 03:35:40 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 50C0340F18; Tue, 30 May 2023 03:35:40 +0200 (CEST) Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by mails.dpdk.org (Postfix) with ESMTP id 58E66406BC for ; Tue, 30 May 2023 03:35:39 +0200 (CEST) Received: by mail-pg1-f170.google.com with SMTP id 41be03b00d2f7-52cb78647ecso2384175a12.1 for ; Mon, 29 May 2023 18:35:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20221208.gappssmtp.com; s=20221208; t=1685410538; x=1688002538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=9y6HaPS00mZOjC/ISA9gFHshO8bDH7cdkxA1tgEZ8tw=; b=JwFDF9BhgMb1NOM5iAadhreYdU2fXh/fHA4/iJ9sC/6o2Z1EBm362u/1JnHOtijsOB SB+aGOhr3ddb6Hai2fHP+y544CF7MLBnLDcvCEHbZHL1Ay+WObW62eblghlYxLrokhPa LLlh1VVdDVMTnDnWnLJDlOkh3QlDMec0uT+xqTxOQrarrmzthbnzDiy21BApvxnWuW/+ 4G25XGTwMJuywCaPi1jbaBzV277TQt0fSXrOmilOYV59okM8k+iX2XYbE5Vb3rgORLEB R9p/snQnyrg38ySteYvWLKrrMllP7ZG0xkv/P5W0EkeIYRS7yRcKAVkkn3heQ8S48Kr8 vCnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685410538; x=1688002538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9y6HaPS00mZOjC/ISA9gFHshO8bDH7cdkxA1tgEZ8tw=; b=H7wwv/Q4zt4a/uJTKhn3YZ1RThDSCOxItIxdqz33KK3AUzV4YqhgMS7XE6yEq84wb1 KJW1bVzm6eDHUaXN4fL+vlOyDoj+6PoWYAgikVKLKf2RWUCnXcDOcpkXhkGd1L//dtxi xJMnKzKdaTqdPY1XPwkt15omqas19DU2PF8uaVd+5zIc645L8dk6raTGRZr9rEubNAz1 OUCm2Y950Gs7gSVrMKUfha2QtmT4xTePQ9RqaGHS6HqVnzjpW8jkLFvJkD8N28CND1R+ 1MamI/KSneUnG0ChVCAo60MbmpIJ60EkLZkKjG+oSQv4M6OkX/dofI9u5LI2euT0Z/a5 lTtQ== X-Gm-Message-State: AC+VfDzgj8nupv1KuDfnRR4ntVBpVVq1VBw88CE/raIjSlLWpun/8hfb lwSLGw6Ld9nsUTGUXo5dnLr+gF9QcIHNdVgGIqMcxnQC X-Google-Smtp-Source: ACHHUZ7rQpGlTIxrNdSayIWujbmkGSimMeFZ1sZOSRMDwp/o9EgxCT+paSa4+tNn2biOvbkE8Mm/QA== X-Received: by 2002:a17:903:22c7:b0:1ae:4a0b:5957 with SMTP id y7-20020a17090322c700b001ae4a0b5957mr998586plg.54.1685410538319; Mon, 29 May 2023 18:35:38 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id c9-20020a170902c1c900b001b034faf49csm3980142plc.285.2023.05.29.18.35.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 May 2023 18:35:17 -0700 (PDT) Date: Mon, 29 May 2023 18:35:14 -0700 From: Stephen Hemminger To: Baruch Even Cc: dpdk-dev Subject: Re: Hugepage migration Message-ID: <20230529183514.1febf224@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, 28 May 2023 23:07:40 +0300 Baruch Even wrote: > Hi, > > We found an issue with newer kernels (5.13+) that are found on newer OSes > (Ubuntu22, Rocky9, Ubuntu20 with kernel 5.15) where a 2M page that was > allocated for DPDK was migrated (moved into another physical page) when a > 1G page was allocated. > > From our reading of the kernel commits this started with commit > ae37c7ff79f1f030e28ec76c46ee032f8fd07607 > mm: make alloc_contig_range handle in-use hugetlb pages > > This caused what looked like memory corruptions to us and cases where the > rings were moved from their physical location and communication was no > longer possible. > > I wanted to ask if anyone else hit this issue and what mitigations are > available? > > We are currently looking at using a kernel driver to pin the pages but I > expect that this issue will affect others and that a more general approach > is needed. > > Thanks, > Baruch > Fix might be as simple as asking kernel to lock the mmap(). diff --git a/lib/eal/linux/eal_hugepage_info.c b/lib/eal/linux/eal_hugepage_info.c index 581d9dfc91eb..989c69387233 100644 --- a/lib/eal/linux/eal_hugepage_info.c +++ b/lib/eal/linux/eal_hugepage_info.c @@ -48,7 +48,8 @@ map_shared_memory(const char *filename, const size_t mem_size, int flags) return NULL; } retval = mmap(NULL, mem_size, PROT_READ | PROT_WRITE, - MAP_SHARED, fd, 0); + MAP_SHARED_VALIDATE | MAP_LOCKED, fd, 0); + close(fd); return retval == MAP_FAILED ? NULL : retval; }