From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id 5A7C316E for ; Tue, 17 Jul 2018 12:08:22 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jul 2018 03:08:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,365,1526367600"; d="scan'208";a="73011679" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.102]) ([10.237.220.102]) by fmsmga001.fm.intel.com with ESMTP; 17 Jul 2018 03:08:20 -0700 To: Takeshi Yoshimura Cc: dev@dpdk.org References: <20180712030833.4887-1-t.yoshimura8869@gmail.com> <20180717082230.3322-1-t.yoshimura8869@gmail.com> From: "Burakov, Anatoly" Message-ID: <67f6563c-200b-1ec3-899c-83e86fbcf2dd@intel.com> Date: Tue, 17 Jul 2018 11:08:19 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20180717082230.3322-1-t.yoshimura8869@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v4] vfio: fix workaround of BAR0 mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jul 2018 10:08:22 -0000 On 17-Jul-18 9:22 AM, Takeshi Yoshimura wrote: > The workaround of BAR0 mapping gives up and immediately returns an > error if it cannot map around the MSI-X. However, recent version > of VFIO allows MSIX mapping (*). > > I fixed not to return immediately but try mapping. In old Linux, mmap > just fails and returns the same error as the code before my fix . In > recent Linux, mmap succeeds and this patch enables running DPDK in > specific environments (e.g., ppc64le with HGST NVMe) I don't think this applies to BAR0 only - it can be any BAR. Suggested rewording: Currently, VFIO will try to map around MSI-X table in the BARs. When MSI-X table (page-aligned) size is equal to (page-aligned) size of BAR, VFIO will just skip the BAR. Recent kernel versions will allow VFIO to map the entire BAR containing MSI-X tables (*), so instead of trying to map around the MSI-X vector or skipping the BAR entirely if it's not possible, we can now try mapping the entire BAR first. If mapping the entire BAR doesn't succeed, fall back to the old behavior of mapping around MSI-X table or skipping the BAR. > > (*): "vfio-pci: Allow mapping MSIX BAR", > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ > commit/id=a32295c612c57990d17fb0f41e7134394b2f35f6 I think your link is wrong here - at least i can't find the commit when i copypaste the link into my browser. > > Fixes: 90a1633b2347 ("eal/linux: allow to map BARs with MSI-X tables") > > Signed-off-by: Takeshi Yoshimura > --- > + memreg[0].offset = bar->offset; > + memreg[0].size = bar->size; > + do { > void *map_addr = NULL; > + if (again) { > + /* > + * VFIO did not let us map the MSI-X table, > + * but we can map around it. > + */ > + uint32_t table_start = msix_table->offset; > + uint32_t table_end = table_start + msix_table->size; > + table_end = (table_end + ~PAGE_MASK) & PAGE_MASK; > + table_start &= PAGE_MASK; > + > + if (table_start == 0 && table_end >= bar->size) { > + /* Cannot map this BAR */ > + RTE_LOG(DEBUG, EAL, "Skipping BAR%d\n", > + bar_index); > + bar->size = 0; > + bar->addr = 0; > + return 0; You have reserved space for the BAR earlier but do not unmap it on return. Once that is fixed, Reviewed-by: Anatoly Burakov -- Thanks, Anatoly