From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E32CC42BE1; Tue, 30 May 2023 10:04:46 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7A5B240F18; Tue, 30 May 2023 10:04:46 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id DB849406BC for ; Tue, 30 May 2023 10:04:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685433884; x=1716969884; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=irMpXhGxsDMjcxiVuMqxw9asU+6b9shOfzMosqUOlG8=; b=bgqvCx5HSwTtNYhOnQina6XGW3PBsnKo7Mec4BR2A7CfC9tVrYAuZrHH HaU2rXe7TgJat1UjdU2qIDOwkPpM1ivUqA75BsVmm5nyCpwq0ZTaaQbq3 J7qWfGDZivEyI7mqvzV28h/V6TDx1/WzlSmdcBZbbO7pkKBUnMsV/vBQO sFdFCYWLeEvnxEW2YJELbnVHYdTR8k0p9PsVJC4uP1Srfv0l6POLugECF LHnIx01UAqdfJJBsNneKbUbC14wFzEw/tuujxr/Beyj3+TW+9myg7IP91 1BCRszvw2+BqMPTsJtKnB/uXpb1yzx/7PKvVpn6DQXrLGgwXFIpYN4MWb A==; X-IronPort-AV: E=McAfee;i="6600,9927,10725"; a="383102040" X-IronPort-AV: E=Sophos;i="6.00,203,1681196400"; d="scan'208";a="383102040" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 May 2023 01:04:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10725"; a="953022745" X-IronPort-AV: E=Sophos;i="6.00,203,1681196400"; d="scan'208";a="953022745" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga006.fm.intel.com with ESMTP; 30 May 2023 01:04:42 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Tue, 30 May 2023 01:04:42 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Tue, 30 May 2023 01:04:41 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Tue, 30 May 2023 01:04:41 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.41) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Tue, 30 May 2023 01:04:41 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f9ujgEh5z+FfNLIPyX/7qdiVxXaFGfN7md3HP01qQYgmOO/ezhfxz9TnFWERi7Fhavt9qzDmyMvIjiivaF4ckXDxmOjxzbYOeuFU48T1vrKT9USLncb/444HIKn1wtixXws9CDHsma3tRAW7PnsmaJZn4mNLYcPq6hXBeknEn7Fes2ag07jmnlm6qQ0maHjc0rtbWQ5RfCzl10yzO2OAK/hWDjPfPvFLA3mLs/cUKXq882DveAmSRpR2oZuQxpEu220FmK4RxaMhGEyV8t5k/l6+xi3AiNB+Bg0UVQsdmO+G5Cpo3gsWHa27J655l8+Dtx5UK1fUd6zvJza4EvvVEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=toeVg4xYfIPB7D8ANmpStyJ0rNJShuhgsC9jM054hCA=; b=WnUaGdWnTSZB8qltr87OhFc1MJ+WtPda93vDr3IOJ+KuLaUCHx/DHOwv2VJ4BVyxs6Zugmy/aFrBwRJ+Jed3B2JR4HTyU3eRYrVnIRq5II2QmVF0dtz3rB5JNPpl6/tdLZ7TY4o5eOKRP9QIAj0Dz7akQNDzjYUa8QyZchejFYSyRIif19lwi2BsYR/fx5CX7mQ4Wv/wMquC60nrVFkexn8MWcGt35tJ6RKNWAYAeYkuqrDC6999/koiA3Yx3t50skK0Rzd71PvAK9cXWeHjcVd4vDX2HP30fMFN1OKGQ7pZ849cO52aXUzvIfGsMAABEfY5MYYoI1HuFX+mZ4Ma3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SJ1PR11MB6108.namprd11.prod.outlook.com (2603:10b6:a03:489::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.23; Tue, 30 May 2023 08:04:39 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::b8f3:958:d2c5:2232]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::b8f3:958:d2c5:2232%7]) with mapi id 15.20.6433.022; Tue, 30 May 2023 08:04:39 +0000 Date: Tue, 30 May 2023 09:04:33 +0100 From: Bruce Richardson To: Baruch Even CC: dpdk-dev Subject: Re: Hugepage migration Message-ID: References: Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO4P123CA0329.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18c::10) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SJ1PR11MB6108:EE_ X-MS-Office365-Filtering-Correlation-Id: 77bb0305-9685-4950-2077-08db60e48431 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3uaWOiZXmz836t4HGMrpFXSqE5RaOWwc9T0/WVWehUL2l2KkL7tqbtAuAUkeVssxEXn3VdmBY/qWtC4ofM42NyW8r5Cy8UeI8oDMizKMEAkXw7adtD/kWF5xArRoNhlFcLAgjo83Zn/qsEbosXchA/Urutjyj/mWrZxm4XTtHG0ur/CfUtgQ1bvfR52WdFNIp7xUiSZyJWnrfWeuC+dqshP1cJHVU0PU+wnZtRObEscos7tWosEJk7GGS2k8R2GEwe29V7i6uNJZYDFr5EvKeB1eMwgfSIo4Uxdw7f7RSC1b/FsAn43BtgWv7zaPMIIj0oeVFyfFQHIu1wOe9/lxfVpmwhN/zu0teexVmKnTmQsr8TJzB6XDW+QHIaavZF2+p+B6XqozwrvUZ5/0UEnwzVGfEow+CKOdlq0uSxFb/lieFTQqIhJHS4nwVB2MgvBmCKA60lGCBPQVeMGQpjGrOfOqRRplF5S51g9W4IY4dBYAZHpm4bCSw3HDXHmjVZhy5Uvv6/gd1RGrteOspCIGrn3he27ORNNI0xAdH3Z3daGtDqu0AwVw8Khwy9nI7QT0 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(376002)(396003)(366004)(346002)(136003)(39860400002)(451199021)(3480700007)(82960400001)(478600001)(38100700002)(86362001)(66946007)(66556008)(66476007)(6916009)(4326008)(2906002)(6666004)(6512007)(6486002)(186003)(26005)(316002)(6506007)(41300700001)(44832011)(7116003)(5660300002)(8676002)(8936002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?EkmA1yPluYWuoIVobIpHRagvH+GosSNmsaF/CBTtnZpkuabhxv/ybknc4Bk5?= =?us-ascii?Q?5S8ZxCVM+/Jl4PcGJrpRsFZMm24dC0Qo9wnnoUkVlWPcoOk+csNS6foGwbW2?= =?us-ascii?Q?2wztnYlzmSMDWPtbQ8sg5FbBhuBustUArFZIsAdCklOGNYS4XeQArJu116Z2?= =?us-ascii?Q?LcY82hWV80IMVAb3UVS1xULaLUHqLEOTZoK5BVv35BkH1t116CBME9nKVX2C?= =?us-ascii?Q?/TcTF9L7GfsIupmdPoRD8qWxCTiDKcKE2h6E44Q3xgm4WsgAvIWTUrJGn4EP?= =?us-ascii?Q?2+U7vd4umIRgyDqnosvgJxFjCMSbRShFe0zO9AuJi6I60WlRfaV/kE8NMe/r?= =?us-ascii?Q?LNAgZi9EJ/Kwxdj9RZuKt7Mg+0v8GklcgfiurD+MC2OsHNqrPKQtPeyT44hr?= =?us-ascii?Q?GCcGQy5fSBkx5H28mZqV/Rnj28KpZhfA4OZM3LVA0Pl1VjxUEBmpN6kthjQb?= =?us-ascii?Q?Mpq4x7IFCSg39O/r7WfGBJIBL7U3nvT6Q53ysPjNsN8WU60PsWQq3afsP9qo?= =?us-ascii?Q?Pxx3ggxAoj4Ijvol4fYzYuvhuAVWjyPoqhUdigThqqZKrBtYBnXzeHeDA5Ek?= =?us-ascii?Q?nUYGjPNeDcHS6FPAb1GWvXIBcSmWLCx1s4rx8xXr5oF6HnTIp38Kxws+5aYT?= =?us-ascii?Q?jWD38UkjFfKTd5nSumdHgDNwx/pjxqh7SSHyGLiISP6TVXp/M3Dhuib2zeLA?= =?us-ascii?Q?MpOssZoNHDT9q2vUGfIqvdACtPm+oGNQ8uVoE86j07sTs4TIfovRv3ETf83k?= =?us-ascii?Q?YUtwMiTVvMd5ehic4wRvAp96+rT3w8+/55xWsroZBhaRIUTBv+wzOjV4TR6E?= =?us-ascii?Q?2waZSOLgQgvrN5SQ2w6bhVsQg87j5E/UF8iLS8Di1LI+/vHCmVeBFEfzbsPA?= =?us-ascii?Q?74N9IN+iVWdxuV8sm/RL68oOiqra94XGtS+RRR7nootuBnq+Ve8BXhgLZXa9?= =?us-ascii?Q?EcNV/omy4FDKKdHZWP64eBaRo88fS4u7zR4N2CudhSGZiqSaUnfze2BWrvVR?= =?us-ascii?Q?85dVkSUk+oweacAoNaMChk0U0uJVaclw9qT3TAToh6QCEm9K7pEgQkiEAMjy?= =?us-ascii?Q?oifc5I8mtk29GupZ62BoWnX/PTBAxVlH3C/xQi27dthPCYgwhQfa36HeIxLl?= =?us-ascii?Q?1oKmNq1W82SnhB31NqcG4gvIyd+QnxgfdnD4LcxHhCqh7TqenKuX1g2tFsgb?= =?us-ascii?Q?bRgdSCyl5gpDiUJoajJKQyEISvAB3p3tdkHmi8wzARcS6Kabb5D4OCYSwX73?= =?us-ascii?Q?UR09Rx0BRhUn+k/U4RZhlcq6a0HAJz9CYfasqki5qs+OXdQmtffFV9swHaev?= =?us-ascii?Q?Lw5wb88tL+67ZQbna8iy0HTqAeyuBKr06apqkAR5rPXiM91y339OXPVUf+BL?= =?us-ascii?Q?gsnMbL6Z7Dt68COwnuR5xUZbVL6x3WJz4SsgQcPIy7JR/t/2sN+caHyt73ZA?= =?us-ascii?Q?HK8P02d2UnzEFRI6hEOBt2O4uMHzU/yPlSZ337a0RmMTltU6676Q5ZmgADqZ?= =?us-ascii?Q?1R/GsSb2XbLDC1eKUwPVM9MgoETJUluhoWnHZnNWgp/A23pjq2maY03TU54p?= =?us-ascii?Q?KZB9IVdI/Ap+JW0Ndu96/h38bIQZEhcTAtEUdgqHqh0duiw0aDJWMMvLDYn1?= =?us-ascii?Q?Bw=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 77bb0305-9685-4950-2077-08db60e48431 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2023 08:04:39.1705 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: HWyY3aSzTpOv2gGv79VzvtQUMdGYM3/4qCWgiCGzRqVYUyNZzkUIORWJIr2PKCHQQKCjfScYlHsTz5e6I8jx0TXGW6NKs2OOtQlwW8qaLwA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR11MB6108 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, May 28, 2023 at 11:07:40PM +0300, Baruch Even wrote: > Hi, > We found an issue with newer kernels (5.13+) that are found on newer > OSes (Ubuntu22, Rocky9, Ubuntu20 with kernel 5.15) where a 2M page that > was allocated for DPDK was migrated (moved into another physical page) > when a 1G page was allocated. > From our reading of the kernel commits this started with commit > ae37c7ff79f1f030e28ec76c46ee032f8fd07607 > mm: make alloc_contig_range handle in-use hugetlb pages > This caused what looked like memory corruptions to us and cases where > the rings were moved from their physical location and communication was > no longer possible. > I wanted to ask if anyone else hit this issue and what mitigations are > available? > We are currently looking at using a kernel driver to pin the pages but > I expect that this issue will affect others and that a more general > approach is needed. > Thanks, > Baruch > -- Hi, what kernel driver was being used for the device I/O part? Was it a UIO based driver or "vfio-pci"? When using vfio-pci and configuring IOMMU mappings, the pages mapped should be pinned by the kernel, I would have thought, since the kernel knows they are being used by devices. /Bruce