From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 9D9B65F22 for ; Wed, 24 Oct 2018 11:09:21 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Oct 2018 02:09:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,419,1534834800"; d="scan'208";a="268343706" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.99]) ([10.237.220.99]) by orsmga005.jf.intel.com with ESMTP; 24 Oct 2018 02:09:19 -0700 To: "tone.zhang" , dev@dpdk.org Cc: nd@arm.com References: <1540347604-18590-1-git-send-email-tone.zhang@arm.com> From: "Burakov, Anatoly" Message-ID: Date: Wed, 24 Oct 2018 10:09:18 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <1540347604-18590-1-git-send-email-tone.zhang@arm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] pci_vfio: Support 64KB kernel page_size with vfio-pci driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2018 09:09:22 -0000 On 24-Oct-18 3:20 AM, tone.zhang wrote: > With a larger PAGE_SIZE it is possible for the MSI table to very > close to the end of the BAR s.t. when we align the MSI table to > the PAGE_SIZE, the end offset of the MSI table is out the PCI BAR > boundary. > > This patch addresses the issue by comparing both the start and the > end offset of the MSI table with the BAR size. > > The patch fixes the debug log as below: > EAL: Skipping BAR0 > > Signed-off-by: tone.zhang > Reviewed-by: Gavin Hu > Reviewed-by: Honnappa Nagarahalli > Reviewed-by: Steve Capper > --- > drivers/bus/pci/linux/pci_vfio.c | 25 +++++++++++++++++++++---- > 1 file changed, 21 insertions(+), 4 deletions(-) > > diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c > index b1f0683..1373345 100644 > --- a/drivers/bus/pci/linux/pci_vfio.c > +++ b/drivers/bus/pci/linux/pci_vfio.c > @@ -445,9 +445,11 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource *vfio_res, > struct pci_msix_table *msix_table = &vfio_res->msix_table; > struct pci_map *bar = &vfio_res->maps[bar_index]; > > - if (bar->size == 0) > + if (bar->size == 0) { > /* Skip this BAR */ > + RTE_LOG(INFO, EAL, "Skipping this BAR%d\n", bar_index); > return 0; I feel like "this" is unnecessary here - just "Skipping BAR%d" should be enough :) > + } > > if (msix_table->bar_index == bar_index) { > /* > @@ -457,7 +459,12 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource *vfio_res, > uint32_t table_start = msix_table->offset; > uint32_t table_end = table_start + msix_table->size; > table_end = (table_end + ~PAGE_MASK) & PAGE_MASK; > - table_start &= PAGE_MASK; > + table_start = (table_start + ~PAGE_MASK) & PAGE_MASK; IMO these two additions should be replaced by RTE_ALIGN by page size. Makes the purpose of the code much clearer. > + /* after rounding to PAGE_SIZE, it is over the bar->size, > + * fall back to the MSI-X table offset in the bar. > + */ > + if (table_start >= bar->size) > + table_start = msix_table->offset; If i understand things correctly, msix_table->offset value here may be unaligned, so falling back to this value may cause mapping failure, because we later use this value as a size of mapping (which needs to be page aligned). Shouldn't this be aligned using RTE_ALIGN_FLOOR by page size? > > if (table_start == 0 && table_end >= bar->size) { > /* Cannot map this BAR */ > @@ -469,8 +476,18 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct mapped_pci_resource *vfio_res, > > memreg[0].offset = bar->offset; > memreg[0].size = table_start; > - memreg[1].offset = bar->offset + table_end; > - memreg[1].size = bar->size - table_end; > + if (bar->size < table_end) { > + /* > + * after rounding to PAGE_SIZE we don't have any space > + * left after the MSI table, so don't try and map it. > + */ > + memreg[1].offset = 0; > + memreg[1].size = 0; > + } > + else { > + memreg[1].offset = bar->offset + table_end; > + memreg[1].size = bar->size - table_end; > + } > > RTE_LOG(DEBUG, EAL, > "Trying to map BAR%d that contains the MSI-X " > -- Thanks, Anatoly