From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0073.outbound.protection.outlook.com [104.47.36.73]) by dpdk.org (Postfix) with ESMTP id B24F85A44 for ; Tue, 31 Jul 2018 05:48:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PEOmuHRA+z96my/8R7n9br9LQYCFY9AgCqBFQzWqni0=; b=Gt/leP2rbxNACw6T6RoFeNILJI4URII2e+hkwqs6mTcJNRgHWvZi1nMfV6loL7eviODiRh0/frHQ1t/7/3kyR0eEvEU1j0crmtXRPjHNN9FM39B36aQau7N4fpjDpkhBrvFrvjZUPhd76lCeYWqAnJiomC92EZJV6xAyNm2wAeE= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (111.93.218.67) by BL0PR07MB4993.namprd07.prod.outlook.com (2603:10b6:208:49::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.995.19; Tue, 31 Jul 2018 03:47:56 +0000 Date: Tue, 31 Jul 2018 09:17:41 +0530 From: Jerin Jacob To: Anatoly Burakov Cc: dev@dpdk.org, thomas@monjalon.net, t.yoshimura8869@gmail.com Message-ID: <20180731034740.GA3371@jerin> References: <4107730371de4ac30367258cb3419df184d1ec16.1532947452.git.anatoly.burakov@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4107730371de4ac30367258cb3419df184d1ec16.1532947452.git.anatoly.burakov@intel.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: BMXPR01CA0011.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::21) To BL0PR07MB4993.namprd07.prod.outlook.com (2603:10b6:208:49::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2efc0506-5e5c-4f2b-e670-08d5f6986788 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600074)(711020)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(2017052603328)(7153060)(7193020); SRVR:BL0PR07MB4993; X-Microsoft-Exchange-Diagnostics: 1; BL0PR07MB4993; 3:Gd89Xay4udnlFb/0LTvToGgXP1LtA5L+fe7LdVe+3tWQqY5/ODF9do86tsdz1pjZp3KzmQkdaTms+om3NCYwBT/pTMFDF7g0xBjD2Ca4cECFOsirMGuMVPAUMx2yu6OYVbq614Q82Iq7j/VwnBfc63Rw/F4gqBAjD8EH9sjfup8w4CVdJjrXXC3vKprlLnzE2CZD9kZzuCcLFUuyPhwkCRwhrYAe7uMOxZ1xddc5hfw05WMKkeb+DYEL6eguXXql; 25:HZVUyTv2xf3bn9niy7hcb4NtQPHbTB2CeEgwKf+/ShJtNrkJMnASYyduQ4e6TO89oaM/qXpcV2ZG3EbKMTVYjXF5vNc4TTvT1RWSKON3+pc0nGdbn4tfGZhBYP3s5eqLPqtwjEQszQ4ieIhWsDHKQUWR0ISbXwp7rlbLaGcZbcTQFqPmDMummeCzypgaITYGlnsy3TP0RgL6MGQgxC9MWj8NhyCs+ueSuzx963UZlYs3ml0CvdPvP0FLMHLMqVqJxiAbTdh7bwNr1xlVko+Sh1ES7gIK1p+f2UqBxf3T+zaysfgPIGS4LNaFWv8OfuOUF/XiVSLaPZx+TX1OnAA2Gw==; 31:ESLnibJ83pMqUckUFOuzKjtMpArfaTjKNFIaVXgKucZw+jqM/8mJaPrlfts/WYNoe6Z14xGtqbMkTqBXHjk3cfNnR9/9W4KW1v6I1/AaDktagiMBwZFvTI/QlAiM8A3Q36YOk1wp2clYgPnOtg86jhGsftIfYl2g5f4t10hcdxPPUnqaKZhcxFEwk2fRDempTirc5ucY4PhnE0NEWDbXiP4zWo55jVm8nEqRRG8cps4= X-MS-TrafficTypeDiagnostic: BL0PR07MB4993: X-Microsoft-Exchange-Diagnostics: 1; BL0PR07MB4993; 20:y2Yno24CD2CooNg5IUrBUo6sJoXUGBJtHxDinjSVuZdcbtV8QAnvCsjuS9hDmUTtmaYNWDZ2Ebdx+6HZq2YUfhho4VFRnsMWKYCPiEL7LNCMKzqiZw81iFO5kbXagihKsIXAQi8OgCjeU9FMIjsHDOKm8SPU1f5eX6rG40rK8Qv7//rEibiDI+V1eCTS5NUNBangGqQfmlyNqhccsX9AWvhYh6Y+IlH/TujNzRAX+zoZ6dOzlDHBfs3ZQVgvxA/4q62C1GMy5fjiexF/4JXIVJwD0BHRl6vb7h8d2PhqLH2/GWmMOOLcRlbmqNBANcdetgKwT3Imu8YdN0XxVAylxU+JglKmrA90txnrC/IXO+Jpwu2fQkz5jqUgBEqR/E62O82BD/hUhMN3Lde008GtJjOSSe+YsO2/wZPLrDlqRatYFNdchW8srIfekSxpGrkEEvMw0w5tiJuyu8Y88oue6jUW7p/xBtlA2OCcKY+pFiy/CBFTxmWE8tiP6jvDg0Qw4ABSpNueYAadv9/C2VCtUdxBYgPZdgujgoCS2yboXyh9eOxMbyq252TMwbe6HmUcwFlToehcsKm4yWobdgnQTTlPAoH0hFoXQnRQZUSV5xo= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(3231311)(944501410)(52105095)(10201501046)(3002001)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:BL0PR07MB4993; BCL:0; PCL:0; RULEID:; SRVR:BL0PR07MB4993; X-Microsoft-Exchange-Diagnostics: 1; BL0PR07MB4993; 4:PmZ+60Xa3hlikCy0VwJomJl0cjQ/b5c+2UiJc6cGLXhKr10xnNVIKXJV5lAy+dwM4cL6WlZcWk/QCLis/rDPAxHYHirWiINr5u59YZnGS3wJmiAEkIi3XTEbyQbql47IYettd7jPm2fI5864k+Fo4S9GpnrG4i/udtPSb7E9pclMvQb9xcyDZ2h+1eC4468k+g1eUb/vpkxzcCCtPOiHLxNhZ9N7P6avngXijVj/EE+ytb3ga3SWl7mKNLlrW/6MuS9wyshJ8eWmVdMOZtDFpVxqd7tlNUnDmTQRph+PyxmzM43hBALgPPo4iS+l2y16/pN944mSi8RTZPdN9uLPmlr5ORzlM/z5K/fafCHrm6s= X-Forefront-PRVS: 0750463DC9 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(346002)(376002)(396003)(39860400002)(136003)(13464003)(189003)(199004)(6666003)(66066001)(478600001)(486006)(476003)(229853002)(68736007)(33656002)(8676002)(6496006)(6916009)(14444005)(16586007)(23726003)(47776003)(72206003)(58126008)(316002)(16526019)(956004)(186003)(81156014)(81166006)(44832011)(575784001)(305945005)(25786009)(76176011)(50466002)(55016002)(39060400002)(9686003)(33716001)(53936002)(386003)(52116002)(7736002)(446003)(5660300001)(1076002)(3846002)(97736004)(6116002)(5009440100003)(8936002)(6246003)(42882007)(106356001)(33896004)(2906002)(26005)(11346002)(105586002)(4326008)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BL0PR07MB4993; H:jerin; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BL0PR07MB4993; 23:wOXqgK1yL6Gb1Hcal3KSEzmjmoIEmadzLZbcFUkv1?= =?us-ascii?Q?S7zLa9ltwB3DGHvQoHrec5Qzb02b5uY0NBALpzBPZn3wZ5Egt+L9yUmgBvf1?= =?us-ascii?Q?1h/0ebktYB8qL6KH72rq4fq5daK7NTmMm5GdaDXW2mxh06I448i8TnCVCQ6V?= =?us-ascii?Q?ppKmTj3008/NFXUmTUqfGrXqu4WAaxXy16jUXvaBlHszE854eEXho+L8VhbF?= =?us-ascii?Q?r7cQDvugfjDGHfXim9HX32LICxYAMF19TjNRHyEfZ4RifhTqQcpqUWIk99xJ?= =?us-ascii?Q?F0J4QVXZE0DTFPgaeykNorDtR5tr44xaHjY/0bPa/iYXBSsPWEtHpDPVFPSl?= =?us-ascii?Q?B47hGDa/qAaSh7aBGPHVXFHR6NbOC4c8tOMpVhdudLj6qRd+egpWYt33GQpA?= =?us-ascii?Q?M3fD79/SlIKO3pSAvXbOTXN1U/5lctH/QlBtgkd16Mupd0IO4xKMsb1s0ukd?= =?us-ascii?Q?Bso8CfAslMgRBsbCASI/qyrEBjCxGrY1x7y+NCVjSHoVKlGqAOnmGovlst0S?= =?us-ascii?Q?GPXiOg+/lRFhkWTtANEp4n8PuJgsA5ZD/jvj8lcoVQRA7l1wrY3hs2pe5nXu?= =?us-ascii?Q?zwfRSNVYyIZ3Tm0XNDF+AFAI6nkD/XXVpXh495Vgq/9hzK98W5U7IX91Cerr?= =?us-ascii?Q?8Vtq1DQ+wqKyXnJziEzd71u9K6ltg8mK6NJgc8zBc9FvlBoEwnALVa74087T?= =?us-ascii?Q?2wXROrFS8WU8rKZoVeemU+wh9SppNLm5EEgJhUZltaBCYiM4KjFy5mFx4Yvi?= =?us-ascii?Q?To0f4kU2oq/K3NP2D4ahWx2BbTT+0O/FqJbDtHeoOIjo1oy5+nSi7WN2uFbJ?= =?us-ascii?Q?DaOgM8tFtgPJ9aSIbJFyQuwMpZq+6ug8/mPTCKk2gtsL0VdkNUvBFMyko4zw?= =?us-ascii?Q?MBtZexSiACAlVwbqg//b8MSVICHzvzPvLw1PsAAypNdiqD3lkXG391rWEIKQ?= =?us-ascii?Q?Q9uMaUrVkCWGbPpScxhc5RwAgY9ZYhCdZeFSXoRN+f4Z8aw2iib9UB44WJCw?= =?us-ascii?Q?Bwqyk7lMOJ+4N6VPONG/zvb8tJCkpbjEifS+/hFWAyCyuKoBu7fd5iVAFv1i?= =?us-ascii?Q?8D3FJPsI79RIVtAQNVm6l/v9EIiG2bKKpGOVjB7IoADTruK4bnDVxUZ25s+X?= =?us-ascii?Q?ZJUl94uMMFuvceAJAntuAPoyYYSRlqDiaBPZX7Ev80hjY3v9j6PZ6q1Og4us?= =?us-ascii?Q?P4AQIQTKjmPux6vguCxdQzDnBTZ5Z3SqogrK32xQrrSGWc1KEYmv4mWihNGA?= =?us-ascii?Q?oObQFXtn3SCkKcz40YGdH5QemjjLu9fMuTPJyvquRaxZRysvV0GrIRtpxKGr?= =?us-ascii?Q?rNdgEMIOV61R7i+I69OTOOkZwuHGeivB9eY+S6X7psU5cTBVxeCZwZiej4py?= =?us-ascii?Q?e6Hr+09gLI/J+8r5fzTGWEjLNQSTqHM32oD3dMUJEZJFtuvKagb9JrReJQWJ?= =?us-ascii?Q?TusOKY78A=3D=3D?= X-Microsoft-Antispam-Message-Info: NVkYlkMQzhj14iV7RLl/5+zVkOvS9YUv6K5yxVTH01TXy5KWIDvoobXNZiT/nwTt6P9YlqZc90HdYGbhqX0em9S7Y+iaIC950rrDer+6q+U5QOeTAzuQQRUHjhoNeTiaDpBW6lcKksw1ywO32VI6m8xUlyW3XhhmyxD0IjksxZ0L+2OTVdE1JqumodQcZZkCPEcH20TY581JIDG/endJBfeg98a0ScuWazVx3IC5XmmooaPQDwD0kAsNY0OREU+s/fDxrUAU9XB6FtpFtjrC5pM74RQSOy+Ppa3w9XrEhvjE50iU8b1ahhA0Int7Iv9D0WXic7Yyr9VTJdslMZuzOqhpdfAGxcq3PLLer/VkI2w= X-Microsoft-Exchange-Diagnostics: 1; BL0PR07MB4993; 6:DLpZgTd5eVrqqDSoqEfxHRHBbKul0AnBxjApIvlZUEgSYuuvX3FF/iKA1FbpCaBh8+T95HoeWRGYrzy8OhsBJDW52VdkH1gzU2X1KJQWRhHXmM7uyxYecobGlKxMr4IOzp6S2Rc9SCyKLRRI3P0iv82kEir63BqzN3IxhyCZlEXoYKeav09NfBbuMrtlL1NEE1X0Fo1rX+wu+G7XiyVZEub5OEEJKlYx2yyOE0e5J1OjrHG4VVKjfga0oIjLypXwR+spWreTt0HhQYvXY9/E4DHDurqUY8fzCa6thr1c1Il8veatmpQEg4yJZOQhjarLMckoxbMb4R0Z8fLpObYtfxKhtSh/PQ/55tlWqUH1CSGJZFwIOmxgvifm/xNxO4yeVi2lndiZkAW5DJB1Kt6wYXuMhLCKuEmf/XniwOm8Gc0ZzRt5X7RHd8ksZRlJXm2fxzy6yRzq5w2BP/7RVjEIxg==; 5:2BMH1vhDLNYz5F5rMq/9gcNGkku1jt8yKA3ZZ205G/XFccq+EHXrj8UbuXGOpsWdhM7u3huSpsFm5NS1yf9Q2tcyoVs3C+T03wnnyg/YLSWAwg0+gnEcF/UlrbdhDgTXhOBLjO/LJZ5YNi8qK6po881gXof8zmVYIj0W5XMn+N0=; 7:v3JPA3pMPPhEhMBCC2mpv8MajVHGqXUoqkuqD21Wym8LavZh4XNuZdoybDytZjbTE8L+5v4v3i9G1YIHYnqdMa7QQ6DOVPFkUavrlBZ5hkgaK2besqNW/UQxEjfeIK++BmutX74g/hMGwwZInjF228y09KZ2Ng2VpblSggfW7rb+1tjjHN3D/5FGaQ25zpIKjJcKH0kwBP6a4h6qv5chaj/ldOTAidSEE2Xxv3QebYUimnJqGgk6a1Y5EPVvNnac SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Jul 2018 03:47:56.2230 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2efc0506-5e5c-4f2b-e670-08d5f6986788 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR07MB4993 Subject: Re: [dpdk-dev] [PATCH] vfio: revert retry logic for MSI-X BAR mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 03:48:04 -0000 -----Original Message----- > Date: Mon, 30 Jul 2018 11:59:06 +0100 > From: Anatoly Burakov > To: dev@dpdk.org > CC: thomas@monjalon.net, jerin.jacob@caviumnetworks.com, > t.yoshimura8869@gmail.com > Subject: [PATCH] vfio: revert retry logic for MSI-X BAR mapping > X-Mailer: git-send-email 1.7.0.7 > > External Email > > This reverts commit d4774a568ba0a5923229974a002972c83eb04570. > > The patch is incomplete because kernel 4.16+, while being capable > of mapping MSI-X BARs, will also report if such a capability is > available. Without checking this capability, gratuitous errors > are displayed on kernels <4.16 while VFIO is attempting to mmap > MSI-X BAR and fails, which can be confusing to the user. > > Signed-off-by: Anatoly Burakov Acked-by: Jerin Jacob > --- > drivers/bus/pci/linux/pci_vfio.c | 93 ++++++++++++++------------------ > 1 file changed, 41 insertions(+), 52 deletions(-) > > diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c > index bb6ef7b67..686386d6a 100644 > --- a/drivers/bus/pci/linux/pci_vfio.c > +++ b/drivers/bus/pci/linux/pci_vfio.c > @@ -332,59 +332,50 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource *vfio_res, > void *bar_addr; > struct pci_msix_table *msix_table = &vfio_res->msix_table; > struct pci_map *bar = &vfio_res->maps[bar_index]; > - bool again = false; > > if (bar->size == 0) > /* Skip this BAR */ > return 0; > > + if (msix_table->bar_index == bar_index) { > + /* > + * VFIO will not let us map the MSI-X table, > + * but we can map around it. > + */ > + uint32_t table_start = msix_table->offset; > + uint32_t table_end = table_start + msix_table->size; > + table_end = (table_end + ~PAGE_MASK) & PAGE_MASK; > + table_start &= PAGE_MASK; > + > + if (table_start == 0 && table_end >= bar->size) { > + /* Cannot map this BAR */ > + RTE_LOG(DEBUG, EAL, "Skipping BAR%d\n", bar_index); > + bar->size = 0; > + bar->addr = 0; > + return 0; > + } > + > + memreg[0].offset = bar->offset; > + memreg[0].size = table_start; > + memreg[1].offset = bar->offset + table_end; > + memreg[1].size = bar->size - table_end; > + > + RTE_LOG(DEBUG, EAL, > + "Trying to map BAR%d that contains the MSI-X " > + "table. Trying offsets: " > + "0x%04lx:0x%04lx, 0x%04lx:0x%04lx\n", bar_index, > + memreg[0].offset, memreg[0].size, > + memreg[1].offset, memreg[1].size); > + } else { > + memreg[0].offset = bar->offset; > + memreg[0].size = bar->size; > + } > + > /* reserve the address using an inaccessible mapping */ > bar_addr = mmap(bar->addr, bar->size, 0, MAP_PRIVATE | > MAP_ANONYMOUS | additional_flags, -1, 0); > - if (bar_addr == MAP_FAILED) { > - RTE_LOG(ERR, EAL, > - "Failed to create inaccessible mapping for BAR%d\n", > - bar_index); > - return -1; > - } > - > - memreg[0].offset = bar->offset; > - memreg[0].size = bar->size; > - do { > + if (bar_addr != MAP_FAILED) { > void *map_addr = NULL; > - if (again) { > - /* > - * VFIO did not let us map the MSI-X table, > - * but we can map around it. > - */ > - uint32_t table_start = msix_table->offset; > - uint32_t table_end = table_start + msix_table->size; > - table_end = (table_end + ~PAGE_MASK) & PAGE_MASK; > - table_start &= PAGE_MASK; > - > - if (table_start == 0 && table_end >= bar->size) { > - /* Cannot map this BAR */ > - RTE_LOG(DEBUG, EAL, "Skipping BAR%d\n", > - bar_index); > - munmap(bar_addr, bar->size); > - bar->size = 0; > - bar->addr = 0; > - return 0; > - } > - > - memreg[0].offset = bar->offset; > - memreg[0].size = table_start; > - memreg[1].offset = bar->offset + table_end; > - memreg[1].size = bar->size - table_end; > - > - RTE_LOG(DEBUG, EAL, > - "Trying to map BAR%d that contains the MSI-X " > - "table. Trying offsets: " > - "0x%04lx:0x%04lx, 0x%04lx:0x%04lx\n", bar_index, > - memreg[0].offset, memreg[0].size, > - memreg[1].offset, memreg[1].size); > - } > - > if (memreg[0].size) { > /* actual map of first part */ > map_addr = pci_map_resource(bar_addr, vfio_dev_fd, > @@ -393,12 +384,6 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource *vfio_res, > MAP_FIXED); > } > > - if (map_addr == MAP_FAILED && > - msix_table->bar_index == bar_index && !again) { > - again = true; > - continue; > - } > - > /* if there's a second part, try to map it */ > if (map_addr != MAP_FAILED > && memreg[1].offset && memreg[1].size) { > @@ -419,8 +404,12 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource *vfio_res, > bar_index); > return -1; > } > - break; > - } while (again); > + } else { > + RTE_LOG(ERR, EAL, > + "Failed to create inaccessible mapping for BAR%d\n", > + bar_index); > + return -1; > + } > > bar->addr = bar_addr; > return 0; > -- > 2.17.1