Re: [dpdk-dev] [PATCH v3] bus/pci: resolve multiple NICs address conflicts

["Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>]

Hi Eric,
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Wangyu (Eric)
> Sent: Tuesday, November 12, 2019 10:23 AM
> To: dev@dpdk.org
> Cc: ferruh.yigit@intel.com; Linuxarm <linuxarm@huawei.com>; humin (Q)
> <humin29@huawei.com>; dengxiaofeng <dengxiaofeng@huawei.com>;
> Liyuan (Larry) <Larry.T@huawei.com>
> Subject: [dpdk-dev] [PATCH v3] bus/pci: resolve multiple NICs address conflicts
Change the headline to "bus/pci: fix mapping issue when page size is greater than bar"?
> NIC address conflicts on 64K pagesize when using multiple NICs,
> 
> as system will mmap 64K pagesize for NIC,
> 
> but dev->mem_resource[i].len is 16K.
I would like the problem is described as follows and followed by analysis and fix.
When integrating NICs that request memory spaces less than a page size, the problem hits as follows:
" Cannot mmap device resource file %s to address: %p "

The root cause is the PCI memory space is mapped at a page size granule, without considering the page size to calculate the next address for future mapping is wrong.

The fix is to calculate the next address for future mappings using the real mapping size.
/Gavin
> 
> Signed-off-by: beard-627 <dengxiaofeng@huawei.com>
> Signed-off-by: Wangyu (Eric) <seven.wangyu@huawei.com>
> Acked-by: Wei Hu <xavier.huwei@huawei.com>
> Acked-by: Min Hu <humin29@huawei.com>
> ---
>  drivers/bus/pci/linux/pci_uio.c  | 2 ++  drivers/bus/pci/linux/pci_vfio.c | 3 +++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c
> index 6dca05a..097dc19 100644
> --- a/drivers/bus/pci/linux/pci_uio.c
> +++ b/drivers/bus/pci/linux/pci_uio.c
> @@ -351,6 +351,8 @@
>  	pci_map_addr = RTE_PTR_ADD(mapaddr,
>  			(size_t)dev->mem_resource[res_idx].len);
> 
> +	pci_map_addr = RTE_PTR_ALIGN(pci_map_addr,
> sysconf(_SC_PAGE_SIZE));
Here "pci_map_addr" is, in effect, exactly the next_addr we talked about in V2, for use of next mapping,  I vote for this fix.
/Gavin
>  	maps[map_idx].phaddr = dev->mem_resource[res_idx].phys_addr;
>  	maps[map_idx].size = dev->mem_resource[res_idx].len;
>  	maps[map_idx].addr = mapaddr;
> diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c
> index b8faa23..64cd84a 100644
> --- a/drivers/bus/pci/linux/pci_vfio.c
> +++ b/drivers/bus/pci/linux/pci_vfio.c
> @@ -750,6 +750,9 @@
>  		bar_addr = pci_map_addr;
>  		pci_map_addr = RTE_PTR_ADD(bar_addr, (size_t) reg->size);
> 
> +		pci_map_addr = RTE_PTR_ALIGN(pci_map_addr,
> +					sysconf(_SC_PAGE_SIZE));
> +
>  		maps[i].addr = bar_addr;
>  		maps[i].offset = reg->offset;
>  		maps[i].size = reg->size;
> --
> 1.8.3.1