[dpdk-dev] 答复: [PATCH v3] bus/pci: resolve multiple NICs address conflicts

["Wangyu (Eric)" <seven.wangyu@huawei.com>]

Thank you for your advice, it's helpful.

I will follow your advice to write a new one.

-----邮件原件-----
发件人: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
发送时间: 2019年11月12日 15:02
收件人: Wangyu (Eric) <seven.wangyu@huawei.com>; dev@dpdk.org
抄送: ferruh.yigit@intel.com; Linuxarm <linuxarm@huawei.com>; humin (Q) <humin29@huawei.com>; dengxiaofeng <dengxiaofeng@huawei.com>; Liyuan (Larry) <Larry.T@huawei.com>; nd <nd@arm.com>
主题: RE: [dpdk-dev] [PATCH v3] bus/pci: resolve multiple NICs address conflicts

Hi Eric,
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Wangyu (Eric)
> Sent: Tuesday, November 12, 2019 10:23 AM
> To: dev@dpdk.org
> Cc: ferruh.yigit@intel.com; Linuxarm <linuxarm@huawei.com>; humin (Q) 
> <humin29@huawei.com>; dengxiaofeng <dengxiaofeng@huawei.com>; Liyuan
> (Larry) <Larry.T@huawei.com>
> Subject: [dpdk-dev] [PATCH v3] bus/pci: resolve multiple NICs address 
> conflicts
Change the headline to "bus/pci: fix mapping issue when page size is greater than bar"?
> NIC address conflicts on 64K pagesize when using multiple NICs,
> 
> as system will mmap 64K pagesize for NIC,
> 
> but dev->mem_resource[i].len is 16K.
I would like the problem is described as follows and followed by analysis and fix.
When integrating NICs that request memory spaces less than a page size, the problem hits as follows:
" Cannot mmap device resource file %s to address: %p "

The root cause is the PCI memory space is mapped at a page size granule, without considering the page size to calculate the next address for future mapping is wrong.

The fix is to calculate the next address for future mappings using the real mapping size.
/Gavin
> 
> Signed-off-by: beard-627 <dengxiaofeng@huawei.com>
> Signed-off-by: Wangyu (Eric) <seven.wangyu@huawei.com>
> Acked-by: Wei Hu <xavier.huwei@huawei.com>
> Acked-by: Min Hu <humin29@huawei.com>
> ---
>  drivers/bus/pci/linux/pci_uio.c  | 2 ++ 
> drivers/bus/pci/linux/pci_vfio.c | 3 +++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/bus/pci/linux/pci_uio.c 
> b/drivers/bus/pci/linux/pci_uio.c index 6dca05a..097dc19 100644
> --- a/drivers/bus/pci/linux/pci_uio.c
> +++ b/drivers/bus/pci/linux/pci_uio.c
> @@ -351,6 +351,8 @@
>  	pci_map_addr = RTE_PTR_ADD(mapaddr,
>  			(size_t)dev->mem_resource[res_idx].len);
> 
> +	pci_map_addr = RTE_PTR_ALIGN(pci_map_addr,
> sysconf(_SC_PAGE_SIZE));
Here "pci_map_addr" is, in effect, exactly the next_addr we talked about in V2, for use of next mapping,  I vote for this fix.
/Gavin
>  	maps[map_idx].phaddr = dev->mem_resource[res_idx].phys_addr;
>  	maps[map_idx].size = dev->mem_resource[res_idx].len;
>  	maps[map_idx].addr = mapaddr;
> diff --git a/drivers/bus/pci/linux/pci_vfio.c
> b/drivers/bus/pci/linux/pci_vfio.c
> index b8faa23..64cd84a 100644
> --- a/drivers/bus/pci/linux/pci_vfio.c
> +++ b/drivers/bus/pci/linux/pci_vfio.c
> @@ -750,6 +750,9 @@
>  		bar_addr = pci_map_addr;
>  		pci_map_addr = RTE_PTR_ADD(bar_addr, (size_t) reg->size);
> 
> +		pci_map_addr = RTE_PTR_ALIGN(pci_map_addr,
> +					sysconf(_SC_PAGE_SIZE));
> +
>  		maps[i].addr = bar_addr;
>  		maps[i].offset = reg->offset;
>  		maps[i].size = reg->size;
> --
> 1.8.3.1