From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7BC0D429DD; Mon, 24 Apr 2023 18:17:00 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 533C441138; Mon, 24 Apr 2023 18:17:00 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id 088B1410ED for ; Mon, 24 Apr 2023 18:16:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RvXZWyjMxmHnpBO3pNcKmG4u8qTtP22y9LE9m22eszfkSddqbkhA4pj5j58neXE3pyJNc1JHrh43E0yuaK7fylPLtO5biU8Wbp1mdkIvzasjACCXiwnsFcjvMWl5BDoX46XB4V27TNKcZiE3qzmfKPxTUFH0WotTgA/sDipzzjW3PcjYOf9zp8OKcl/rC8j4L6UgtOUYbpRdn1Th7Ik3K6piMIdV3K8fKXfwSoJKSmdIXhezj0kAjlZsJ0l75mZTStW7Y7GPo7GKVSf3b9Hf+NCyIxLbhy0CK+BRm5C4sJHM9Nzq8e5DQEHVVrVTMwF2NxxHTzyNfh609VimwxqKdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tmYBlLlAaHeEeTKxXFMva99LBeWhIsz3O61BxIaT8Xk=; b=CwWYnLZxanzq2gMlng4OUpacxAEzCujkloBOTPtD82dGCXZnUn6T/Oe9flhcOzEo1IcMH+P1GSgyc3MJ5R3r7NLHQZNd20D339L1gTcbXZrvW0ljYz29Ftziikf077H8AfA1jdacwwCQXgn8iaBtoBgTjcpL2ZNa6fBWT5Tthwx8qVkxVp7I/MXd1/zPcKSY1eruMwgzLSEoO5osd2EQgm6/i3GDK4FgS7UWl1hnIIrM5krNkhrgqmCXQJWhLG+yncajG3lsjXDa9EWybB6M3YA5YVQjiiXvY6DGysdOx1Refz9rB1YVjXZd2cD0HKCwSpWklZMmwsI997MP5CDM+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tmYBlLlAaHeEeTKxXFMva99LBeWhIsz3O61BxIaT8Xk=; b=MKNtTd+TZQK38NqSTDlraVCCboAfTxHwkRRc8gfAx3YzI2syJQZbipBB3lP0VyFevHBeRcCglH8kaORI8QQx7yYcHEGVGuQ3dCrPgKR7CHdHeuEGNTnkDYV17CgeHsN/6RTQANhaMfAnvmwKbQeWLepJ1DwGYaAhS2a8oR+6J54= Received: from CH3PR12MB8308.namprd12.prod.outlook.com (2603:10b6:610:131::8) by CY5PR12MB6083.namprd12.prod.outlook.com (2603:10b6:930:29::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.33; Mon, 24 Apr 2023 16:16:57 +0000 Received: from CH3PR12MB8308.namprd12.prod.outlook.com ([fe80::4e4f:3a93:b7bd:489c]) by CH3PR12MB8308.namprd12.prod.outlook.com ([fe80::4e4f:3a93:b7bd:489c%5]) with mapi id 15.20.6319.033; Mon, 24 Apr 2023 16:16:56 +0000 From: "Gupta, Nipun" To: Stephen Hemminger , David Marchand CC: "Burakov, Anatoly" , "dev@dpdk.org" , "thomas@monjalon.net" , "Yigit, Ferruh" , "Agarwal, Nikhil" Subject: RE: [PATCH v2] vfio: do not coalesce DMA mappings Thread-Topic: [PATCH v2] vfio: do not coalesce DMA mappings Thread-Index: AQHZH/wr5WL+1ubhYUiYfLJ4gCqsP667p9IAgAYNGNCAWhtDAIAAFeUAgAFsY4CAHe7QAIAADV+AgAAAlqA= Date: Mon, 24 Apr 2023 16:16:56 +0000 Message-ID: References: <20221230095853.1323616-1-nipun.gupta@amd.com> <20230104051936.2456411-1-nipun.gupta@amd.com> <0d356823-b198-ebd1-7a28-b678ae9edaf9@intel.com> <4e912b50-c17f-cfb4-c8e3-f3db4f96178c@amd.com> <20230424091037.6ef9fabd@hermes.local> In-Reply-To: <20230424091037.6ef9fabd@hermes.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_ActionId=02eedf45-34dc-485b-b36e-bcc97020302e; MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_ContentBits=0; MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_Enabled=true; MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_Method=Privileged; MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_Name=Non-Business-AIP 2.0; MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_SetDate=2023-04-24T16:12:55Z; MSIP_Label_64e4cbe8-b4f6-45dc-bcba-6123dfd2d8bf_SiteId=3dd8961f-e488-4e60-8e11-a82d994e183d; authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR12MB8308:EE_|CY5PR12MB6083:EE_ x-ms-office365-filtering-correlation-id: b1e4123c-e47f-4973-01af-08db44df531f x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Yy90OXk3vyuuKrmegqx2RBOQyhfJVBZ7XFHE4hk8/TnCGeG0PjGVhDcV7dGmAXXmwAkyBu5bG903xTgeNczj7/hm+GFh/YF9MGqXqZO1BRPCSoN4/ZjhzCLRgfY9bWMrmu4iJ/dIYMnns8ldpulaJ60gEizAlvLBz4bjTsKXTFXRHQAcjkgCmiVb43lImf/W6/VSai/rMydclwoSgIOZ8RVptB1AhmSX2lG0tlrS9XkML1nYFNLUwszfIj0YsxmiMfhNVAB01RfV0610ncU0jUCSZa+UmZfxSkKzTuVWAz0FidAh62y2I4YHzWX48KlIA97zY0+uBvYuuWwIFFKQkO1WTgGT6nVR5yz+uyv2NmBuA9NRJv7ig478Q89RQETbwxm7orgZy6mdZ46jaRXP8vp2VPXZHJMQ4XUt7RGuOInA19wsYYpUGCuahDGqxwz4VprvqYTKGo15sxEw7l0D8AvIt/EfihhaTM8USet6HCHnsZFQTwb4VM1uBRaQj7YeSIrgdxL2e1LEPsMVA/eEATn/Vjl5c0Xf7ac+nkWQNzEWCe6BysBUbaa8NYcZdSqw894VuzQrtWYkAQvIzL/hyKPEGJd6oFJsp/tSYYGFmBn2D/CYCOVWxGqAhcyzGQuI x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8308.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(346002)(136003)(39860400002)(366004)(451199021)(2906002)(7696005)(71200400001)(55016003)(9686003)(6506007)(53546011)(186003)(76116006)(66946007)(66556008)(66476007)(66446008)(64756008)(8676002)(8936002)(316002)(41300700001)(4326008)(478600001)(5660300002)(52536014)(54906003)(110136005)(38070700005)(38100700002)(122000001)(86362001)(33656002)(83380400001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?UfEInqMGNi6HvWYlb0KXdWb4e7VtI1jkEBUU47nGVjFDel8zbqmSB/fHK4L+?= =?us-ascii?Q?VfO6jp3gW5b4l8Mm+eZR/1z8IqqfYGdxUjoq4VRVi/qGZEQaDNzVYGcDRYGv?= =?us-ascii?Q?duu4+tD6bJ20BxXzTg3aLy6ZTimDm+wr1v3SOygZRUbkKLg4W0yNI77wI36j?= =?us-ascii?Q?BTjGYSSrUCX2wwuRQXJI64keczMMKb2zTG3fMi43vsCyNaIaBdUPNoxkCTcy?= =?us-ascii?Q?24gGkmByD2HlAhQokAb2xiScDHgnwY/DPTy8ujo55mcWYm6pM/6aPn/8xmfM?= =?us-ascii?Q?wU/Kgmv1UmHUiNXrR2i+S5dxEORCXL0RIEtSG+tFrQ09+MgfHVy8nybKO2oH?= =?us-ascii?Q?k3KcbgoORh5prALqnyHAasyalGb+mPRKuH+YggUYQkewKM2jtwgx3R8pj3pH?= =?us-ascii?Q?oCkAtaRAm00sRQyP5IecjAIbhfxMPBfT1S2YaZICt1YIK8pFs0wS8PUBT8oO?= =?us-ascii?Q?IR1Jvew2wQXCwkrtJOq6aoKYl6PelaPMjMKvbIHhXuO/k7+MMTuTZPv5DCNR?= =?us-ascii?Q?5KXZjjHWnMj5YW71mhXvYqnwTPp5w1xXHA9f4kwoF12pCXIDIOJgGXjZ6IBD?= =?us-ascii?Q?jcegDTPfuyJ8nwA3gkdE1swGuLrjeNCK+N/nA/CvgN8AeD2X9r/6M4lCKs9d?= =?us-ascii?Q?0ZdgP3IYurSrRtAFmqxChYWClfDxiFhDRJBooXtZriik3BeKxIorhoJhHl1o?= =?us-ascii?Q?uikanP9ELgO2fTKYVTgqkofm1CJtYLdO4j07FBMqB9EbdQaPIfgxxa8XSsdZ?= =?us-ascii?Q?ns3QSpMVXcIhjNgvluL5kJpIq6egf6LYDBF77bms8U7/Nw6ue688kR8Odolb?= =?us-ascii?Q?ZiZdiKHXC6/eU9ZqowZ4qv02oNjDjIc5vrq2uNWa5fM8TNSsJX6OkOQOLWlC?= =?us-ascii?Q?SoErdj1hdiAyifzgWYyPW5oEXwQhwQCr175+WyHmig9iWJcx8XJTlaKOlO2N?= =?us-ascii?Q?Ynjnnn0hvYJLAUDdDq+6iiG0m3IjE5FIVq+W8QWV2PXqAAViu0RTgqnpizbz?= =?us-ascii?Q?8CMnSHba+8l6bHt3wylF14LYHwzgucrIjMzf43AhFZz1DStmQf5eouqPv3Tq?= =?us-ascii?Q?0Ar6/zO7oeeGQeRzPRH9LUjMuqGpiV9J7ljEI4nsCE2FpYQYOtCdHEeykGe1?= =?us-ascii?Q?vZIF9FoZSoBiffzhTUOpMWjJDwmzkEA2P0rlIE/PkQGlTiebOdmKeFg51GXa?= =?us-ascii?Q?3Y1240gTyl4K6PFXgHgM4tV21LK+UQv7dpMvEMKcePurTH1gSJM9okJBBuyK?= =?us-ascii?Q?Ekal+95WiFon715R9mAr4B/g9nGZK0jU/d95egWwexWjgdQt2ytgLMTRnTFN?= =?us-ascii?Q?h1RHhSEVIlzxy905pIM9F1JdlprEv94oOnQyTMmmfHx+eG3dh6hjrwptJlaj?= =?us-ascii?Q?XCk5gLs2Nchm++PtGl9sIm3cKz4e2NDzU9QbdgBntB98kpniLbG+woWcjj8w?= =?us-ascii?Q?nKe3AO4zG26G6PZ+idWse4h26WJcJk2JxBF17s7whZZHqzwUo+TzTR6XilRa?= =?us-ascii?Q?9CdfYDYWuZl5lwVA7obP4kwVuC7OqO8D1cCAGGPvgV6mzYUWgO57IbeXqIiZ?= =?us-ascii?Q?oxYkxFofUBZB6MkIlMJ8pgjX6eqDn5XMhRpG9efrT+7aSTnhFKraMFYoVIM/?= =?us-ascii?Q?mHVXwI0uRzI9G6bPNNRZe+I=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8308.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b1e4123c-e47f-4973-01af-08db44df531f X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2023 16:16:56.5378 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Wk5BlPCQTr2Goqp7Umm7LOlC997+JxTBrAE53S4dEEcbDUlCrV6CcW8ww5fHKyIO X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6083 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Stephen Hemminger > Sent: Monday, April 24, 2023 9:41 PM > To: David Marchand > Cc: Burakov, Anatoly ; Gupta, Nipun > ; dev@dpdk.org; thomas@monjalon.net; Yigit, Ferruh > ; Agarwal, Nikhil > Subject: Re: [PATCH v2] vfio: do not coalesce DMA mappings >=20 >=20 > On Mon, 24 Apr 2023 17:22:46 +0200 > David Marchand wrote: >=20 > > > > > > > > > > Thanks, I can reproduce this issue with regular devices too (run test= pmd > > > with no devices, bind a NIC to VFIO, attach it, then quit). You're > > > correct in that since the initial mapping was done with mapping large > > > contiguous zones (such as when mempools are created before attach), a= ny > > > subsequent freeing of memory will cause these errors to happen. > > > > > > I don't think this can be fixed by anything other than not doing the > > > contiguous mapping thing, so provisionally, I think this patch should= be > > > accepted. I'll play around with it some more and get back to you :) > > > > Can we conclude on this topic? > > It is best we merge this kind of change the sooner possible for a relea= se. > > > > Thanks. >=20 > Shouldn't the coalesced mappings be able to have correct datastructure > (accounting) so that on shutdown the unmap's are done for the right size? This issue occurs only on the hotplug case. Other devices which are not hot= plugged and are existing from the start of the application need to have individual = (non- coalesced) mappings. So individual (non-coalesced) mappings are definitely required. IMO we should not maintain separate mapping for each hot-plugged device as it would be unrequired overhead. Regards, Nipun