From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3B39743CA1; Thu, 14 Mar 2024 11:56:08 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1E5A0406BC; Thu, 14 Mar 2024 11:56:08 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by mails.dpdk.org (Postfix) with ESMTP id C504740297; Thu, 14 Mar 2024 11:56:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710413766; x=1741949766; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=E/a+XUEPBPg45f8x9SiisaFL6PjYvON+8AFL9NI0//4=; b=h4x/DIxwhWzlrsiwjuQ6Dw1ObkGfK8XBRG7UngniC4bvZnMYF49MlXnZ fifqN2LSGfKttvku795kVzBolP0uZ86Sjb4x78vPXWUrmia9Q1AJkZucz EOsQO+5mqnT44tkR1yN8T9D8wAQ6Sl5b9ua2UN0PVpIobCSSFeTVLPU8A 9rg0+AM+AK9wfpqzx+oqa+8SAj4uEhPC6sIF83kvdWr1zmdsFzRCF8WxP WHs94IqvMqTPyObU9mp87/n+ZL+k1kT/TJGJ1LaJiOBQWI1p0F0i9irAg 9gaquJGMGCz1PPk9gwbdJkXyEQAtaDhaMWTh0L0jOYFgq0G6LJdb6ZYmD Q==; X-IronPort-AV: E=McAfee;i="6600,9927,11012"; a="16376571" X-IronPort-AV: E=Sophos;i="6.07,125,1708416000"; d="scan'208";a="16376571" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Mar 2024 03:56:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,125,1708416000"; d="scan'208";a="12179187" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmviesa010.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 14 Mar 2024 03:56:04 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 14 Mar 2024 03:56:03 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Thu, 14 Mar 2024 03:56:03 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.168) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Thu, 14 Mar 2024 03:56:03 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jAoKStOES8Z7v7Qpyhq+9Mizvp1CXSP364mx8ocdpyjCA5tLRmxlSdHjGCj707NHJw9n5OfJFztPskv+tQUW5Vw6Jth8fHx5zUeHnkL3U5Eg3JuEeANUG4mQJX0wbaC1QCCGJwzLCeT6qjpxaTR96/ADOA8INiAN6nMlDgaB+TqkGwJ3M4xOU49XOH+q4ULlPjT5xXF+UJkHCg/yqYdT8E3t5vxszybwPitLGdLJjikKNrCe1n3ur82rAdhTnf15eAosX/4g/3t3Ijpoa5qNDFjKzwBArJ8oeTg3wFxBMyzBuc25/2YnK6xmqbyt/Ns0g6Cla8Dv+oJzn/CYcEj/OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wPsyO8SPkIySRQGBktWkdn1YbLwnOZGrvgpBIkToTsI=; b=dWLqxqgVc3h9sQdwxalq0Sf2N3yUxSw4atJZbe/qW8DmH5HhZ88K0uRYh1Ji+jCwNvu5IGmkMYbiOQ+ea55rOBSx9nZbKB0o4788xZRYwiajWjDSB9/DAaMgUv5/vxS41+v5UxavQCiPI/ffiA5o4/tDBbBav7vV9G/jNrrdXK1AUKNqIHOcVqBvgFiiw7s5FY8UKtUPKqe+5cN4+v70+4BycGEEpEIUi9bZaIk18s1jlGJRSLcuoU3cr47Ez4zSsSu+aXk2kw7qrKjyGSzGUKGHxrNjfgk2oy8yEnbVa2q+UDOMpgslWxlilJxxjroS72cN6dMhlQ/t5iXcrGaYig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by PH7PR11MB5916.namprd11.prod.outlook.com (2603:10b6:510:13d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.18; Thu, 14 Mar 2024 10:55:56 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::ab9d:3c81:51ea:7bb4]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::ab9d:3c81:51ea:7bb4%3]) with mapi id 15.20.7386.015; Thu, 14 Mar 2024 10:55:54 +0000 Message-ID: <75e1581f-ad24-4ced-bd08-2bbc631fd32e@intel.com> Date: Thu, 14 Mar 2024 11:55:48 +0100 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] bus/pci: fix secondary process PCI uio resource map problem Content-Language: en-US To: Chaoyong He , CC: , Zerun Fu , , , Long Wu , "Peng Zhang" References: <20240124104523.2022242-1-chaoyong.he@corigine.com> <20240129092231.3531217-1-chaoyong.he@corigine.com> <20240129092231.3531217-2-chaoyong.he@corigine.com> From: "Burakov, Anatoly" In-Reply-To: <20240129092231.3531217-2-chaoyong.he@corigine.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P265CA0112.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c3::15) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|PH7PR11MB5916:EE_ X-MS-Office365-Filtering-Correlation-Id: 9568519f-9d34-4a8c-939f-08dc44155253 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xk86ob5xsn4hFh/k+5U9ltxy0dvg6drypkUFggVEkZecm6hRzFvOXhQUADctbzG4wcHfgAmrvUjQ/BhgJ2WsH0SvhBFqTsWNs4ldhHNhRi/tVlx3Ryz401JRaKmGaD7X+piSq5Ff8DlW3aLH8yaRQFmXt3006zKQ38b2EtnvKLJBnub1b0W+S6vwTCwuSRB4KMw5aNE/ms1EGozJk7lni5fN8mQxE+/55VliSX95UeU0jBC2fQ6AVffbbDbr+1noFd+pYxFmypn+1ugetihUSp4yzuJS3/mLF3ObTOisNJPOq99DFCGvJcoBCHpYgFGDFMAyMojLyGk5dn+DMqGKXU0+5Tw3szMakZc2AuWfOFIRxY8uUwF9Kv6VG87UpMRvT1RZkPRxdgAq0kDeWBev7UF7rrCatMSNdzmcl0k0YkMR9IsuD6I2P/Sjc637NiVJA2qEuwvnNpVOZt704JNlvURiX00V6tfpdnBjVGYNg/kVQI2BAnUT+x6sQXWPoHGoMjlkgyRiVU03nEKKZ5be8OXJPYmQePxyQF+LZS8Q6ibD2TEWURWIP+5W+WLh6c6++CnKrNz0tndB/Su2TAW4s5iX2D/J7hdE83DcRvkwW1rnrpoxnWgjQnO8HGUY4+5zPuo6qwf4fqwwXk/QxSpeWFKy7bpr8yoVvtK8qZMpSRU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(1800799015)(376005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TXJQc05GOUpPTUtNVDg1YjNaN1NnaTlSUjIzWDlyWTFBRmJTaVlDUUR6eFdU?= =?utf-8?B?TjRxZUtVbkVaVGtkTHRqNWQvRkFkeTJVdGs4cXZjcjkvaCtvNlo1NVJVZXEx?= =?utf-8?B?bnNKaTRsVmM3ZllVdTFiZVVBSjNlQXI2U2xROEtxYVZJRDg3ZmMzOFFHV0ZY?= =?utf-8?B?eUNKNUVvUklWbk8rWDVQUCtLL1hBN2hybGZxWEk5WFlTNjZMUW9qRGZodmox?= =?utf-8?B?N09ZN3NRVXlSVmltZWxqblRocUxqcGFKc1BTRlkxZi9xajhBRlBEZXA4cEFV?= =?utf-8?B?Vnd5amtzcFBJLzc4NWdxTDcwajAySDV6ckt2N0pPZzFNRWxrVFJKajFweEt1?= =?utf-8?B?MDJLaG56M3Y4UnUwTFJ4bjVxdENlOGxTWEtHTENSbTV4czloUlpuQkZCSUhv?= =?utf-8?B?UDQyVUg5cHJnVUZ5T0lPMlJHdzlpaG9jRFViVHplTzdMMHlPN3FPNm9waEgr?= =?utf-8?B?elZaT01JdUpISmNhTDdCSi9VRUJoSllNSnBiMXRkVTlVYlRzUjFhR2ZsTUxz?= =?utf-8?B?MlN6c01lZndkcWJUd1A0NGtISDRuUjFtMFpjTGs2aUdONElPbWs2Vy9od0Y4?= =?utf-8?B?RUxCYjY4aFk4SHJMVEE0bThHVUZFTlB2RnJHMXRCTmlxdnhDcVNkQ0JsbEVE?= =?utf-8?B?WjJwNUFRL2g3emR1Z0JwM2tnd0FNSm83aVhuS0lnQnBOeUFFU0VValZLVWlQ?= =?utf-8?B?SkV4dTZwenVpL0NKL1VWb3JGODFURVY5VGJyYlAxcXBRaiswWVAyYnNxaWdS?= =?utf-8?B?UnZnR2VPWVFUNjhlbGoyNWREOWdWeG96YUlsUm9uSUVYa1R5UjZZSHRqSUNN?= =?utf-8?B?OXF1Yy9XYUNFVzRnRTB2aWRGN1FQbERTMUFoV212U05iMWNFY0g3QXYxdk1p?= =?utf-8?B?bFpheXVBelhJbkhHbzcrOU9hdnhIa0NZc0ZlSmIreXJPZExkK0VUbzdoZkVF?= =?utf-8?B?cit3RXZzenBzdnFiZkJ2b1pFakE1Z3YxZUI2Qjl4M0FIMS9ib0dHMlBFSDFJ?= =?utf-8?B?cHV3N3Y2TUJubTU3dEd2RmtnS1pORWY4UnBWS0Y2QXpNbDdlQ2dIbUlzYWc5?= =?utf-8?B?QVdrQ1I1bDBCbDFHRGJ2aFdjZm1oeXlUc2RFQU5FSHlqZHVsaDN1VWY3dDli?= =?utf-8?B?ZlNnT0dvWlU2Mitnd2xOQkpoeWxQcmxuOFRqaThhcGtsem1nT2RPQVlHOFdB?= =?utf-8?B?YzgrcCtDOVZzVEhGSGllWHZvOW5TRkY3QlNGc25aYzVqZ0Q4L0lBSGNEQ3R1?= =?utf-8?B?U3pCcVM0ckljSTc4RUxJOWM4Rll6S0pyb05KWFZNOGxRNVdWeWowL2ZaQTNZ?= =?utf-8?B?WXloaWlRaFF2aXF0K2J0UkI3UDRIdEhsWnRKWHo1aitPa2lXWmdzV2xndlYv?= =?utf-8?B?a3k5cjhwZ2R1V2ZmSko1dHpIUE9BdkNTNUtaTjZuWVRRdXRBWFljaHVoMFdo?= =?utf-8?B?VEFZWnBDUWZ6dDVSYTZhMnV0cEVuQWE3Mys3M1QyajV3NE1yVC82MHRTN0Uz?= =?utf-8?B?TndGUVFta0NWNk55RDR0bVRMYTBjOU9JRzNwVTgxd21Wais3dUt0bVRiODkr?= =?utf-8?B?eTl0VkxtbDdHeTN0ZEdYb3JKOXVhN2ZickJIT1JWWkd5UUxBblZoNHNUbURx?= =?utf-8?B?WkhUbUY2QzlKcmlQZVFHekNCV054Y3RHR2QyVzI1L3J1Q2dJa1ErYkhCUk5S?= =?utf-8?B?WG5LOFNvQU5qUFg3eTUyVmZMblZ3VTBYVmRVdFA1SVBtODBQdWJTejNxanYw?= =?utf-8?B?aGk5VDc5L2xIakl2SGREcmN0SzhzVG1wVHlOQmdCMWdDMlB3RFRrTEorVEZx?= =?utf-8?B?SEpTSnMvd245WnVzMG1aQk51dmxqTGxsaStva0dMWmpudGFMZFY3Z2s3aGg1?= =?utf-8?B?V3pJZExISS9ET212MVQ3aWNlS2hFczExYVlOTFE1L1c2OTVHYnBESHJLZGcz?= =?utf-8?B?Z01mc05PTVFyMUg2UEViNkQrKzVIVnBvdVFwdEFZU3JmUmNWRGFuWVBybzV3?= =?utf-8?B?UC9PZ1Vlenl5eXRYQkQvb0Z1TVpveWJPdUZnamNlQVBNcjUyaDErdDN6RnJ3?= =?utf-8?B?RkFERUZNQmR0alpvMzMrKy9GNDYrYnl3OWZYTXhHMDdmMjBLTWdtT1RQWmRC?= =?utf-8?B?d1BBai9vZDc2K1N6eGMvcThaRXVtR0NqSjloS3BZSUQ0WFRMUnZtY1JZbDEx?= =?utf-8?B?dnc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 9568519f-9d34-4a8c-939f-08dc44155253 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Mar 2024 10:55:54.7029 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5SanRO7mU1evCP9ac1YiRNpP/EI4Oxl33YTGyE1lUiZjUPyL6VUNv+0Tg7CnpUg6aM04jx53t6UT5mkcNSWn3EiwPWZ0oiMMqTgdpnLQ0Ao= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB5916 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 1/29/2024 10:22 AM, Chaoyong He wrote: > From: Zerun Fu > > For the primary process, the logic loops all BARs and will skip > the map of BAR with an invalid physical address (0), also will > assign 'uio_res->nb_maps' with the real mapped BARs number. But > for the secondary process, instead of loops all BARs, the logic > using the 'uio_res->nb_map' as index. If the device uses continuous > BARs there will be no problem, whereas if it uses discrete BARs, > it will lead to mapping errors. > > Fix this problem by also loops all BARs and skip the map of BAR > with an invalid physical address in secondary process. > > Fixes: 9b957f378abf ("pci: merge uio functions for linux and bsd") > Cc: mukawa@igel.co.jp > Cc: stable@dpdk.org > > Signed-off-by: Zerun Fu > Reviewed-by: Chaoyong He > Reviewed-by: Long Wu > Reviewed-by: Peng Zhang > --- > drivers/bus/pci/pci_common_uio.c | 94 ++++++++++++++++++++------------ > 1 file changed, 60 insertions(+), 34 deletions(-) > > diff --git a/drivers/bus/pci/pci_common_uio.c b/drivers/bus/pci/pci_common_uio.c > index 76c661f054..fcd8a49daf 100644 > --- a/drivers/bus/pci/pci_common_uio.c > +++ b/drivers/bus/pci/pci_common_uio.c > @@ -23,10 +23,57 @@ static struct rte_tailq_elem rte_uio_tailq = { > }; > EAL_REGISTER_TAILQ(rte_uio_tailq) > > +static int > +pci_uio_map_secondary_resource_by_index(struct rte_pci_device *dev, > + int res_idx, struct mapped_pci_resource *uio_res, int map_idx) > +{ > + int fd, i; > + > + if (map_idx >= uio_res->nb_maps) > + return -1; > + > + /* > + * open devname, to mmap it > + */ > + fd = open(uio_res->maps[map_idx].path, O_RDWR); > + if (fd < 0) { > + RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", > + uio_res->maps[map_idx].path, strerror(errno)); > + return -1; > + } > + > + void *mapaddr = pci_map_resource(uio_res->maps[map_idx].addr, > + fd, (off_t)uio_res->maps[map_idx].offset, > + (size_t)uio_res->maps[map_idx].size, 0); > + > + /* fd is not needed in secondary process, close it */ > + close(fd); > + if (mapaddr != uio_res->maps[map_idx].addr) { > + RTE_LOG(ERR, EAL, > + "Cannot mmap device resource file %s to address: %p\n", > + uio_res->maps[map_idx].path, > + uio_res->maps[map_idx].addr); > + if (mapaddr != NULL) { > + /* unmap addrs correctly mapped */ > + for (i = 0; i < map_idx; i++) > + pci_unmap_resource( > + uio_res->maps[i].addr, > + (size_t)uio_res->maps[i].size); > + /* unmap addr wrongly mapped */ > + pci_unmap_resource(mapaddr, > + (size_t)uio_res->maps[map_idx].size); > + } > + return -1; > + } > + dev->mem_resource[res_idx].addr = mapaddr; > + > + return 0; > +} > + > static int > pci_uio_map_secondary(struct rte_pci_device *dev) > { > - int fd, i, j; > + int map_idx = 0, res_idx, ret; > struct mapped_pci_resource *uio_res; > struct mapped_pci_res_list *uio_res_list = > RTE_TAILQ_CAST(rte_uio_tailq.head, mapped_pci_res_list); > @@ -37,41 +84,20 @@ pci_uio_map_secondary(struct rte_pci_device *dev) > if (rte_pci_addr_cmp(&uio_res->pci_addr, &dev->addr)) > continue; > > - for (i = 0; i != uio_res->nb_maps; i++) { > - /* > - * open devname, to mmap it > - */ > - fd = open(uio_res->maps[i].path, O_RDWR); > - if (fd < 0) { > - RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", > - uio_res->maps[i].path, strerror(errno)); > - return -1; > + /* Map all BARs */ > + for (res_idx = 0; res_idx != PCI_MAX_RESOURCE; res_idx++) { > + /* skip empty BAR */ > + if (dev->mem_resource[res_idx].phys_addr == 0) > + continue; > + > + ret = pci_uio_map_secondary_resource_by_index(dev, res_idx, > + uio_res, map_idx); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "Failed to map resources\n"); > + return ret; > } > > - void *mapaddr = pci_map_resource(uio_res->maps[i].addr, > - fd, (off_t)uio_res->maps[i].offset, > - (size_t)uio_res->maps[i].size, 0); > - > - /* fd is not needed in secondary process, close it */ > - close(fd); > - if (mapaddr != uio_res->maps[i].addr) { > - RTE_LOG(ERR, EAL, > - "Cannot mmap device resource file %s to address: %p\n", > - uio_res->maps[i].path, > - uio_res->maps[i].addr); > - if (mapaddr != NULL) { > - /* unmap addrs correctly mapped */ > - for (j = 0; j < i; j++) > - pci_unmap_resource( > - uio_res->maps[j].addr, > - (size_t)uio_res->maps[j].size); > - /* unmap addr wrongly mapped */ > - pci_unmap_resource(mapaddr, > - (size_t)uio_res->maps[i].size); Nitpicking, but I feel like this would've been better done from the caller, not here. This is how it's done in primary process. > - } > - return -1; > - } > - dev->mem_resource[i].addr = mapaddr; > + map_idx++; > } > return 0; > } With fix above, Acked-by: Anatoly Burakov -- Thanks, Anatoly