From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Ravi Kerur <rkerur@gmail.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] IXGBE, IOMMU DMAR DRHD handling fault issue
Date: Sat, 10 Feb 2018 10:11:30 +0000 [thread overview]
Message-ID: <881b4eac-3732-ce0f-c639-7d1ed0df0a61@intel.com> (raw)
In-Reply-To: <CAFb4SLCTpk7eWe6WgybdfEbN9DtW5OTenyxLmtpZ0JxnO55s-w@mail.gmail.com>
On 09-Feb-18 5:41 PM, Ravi Kerur wrote:
>
>
> On Thu, Feb 8, 2018 at 3:20 AM, Burakov, Anatoly
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
>
> On 06-Feb-18 5:55 PM, Ravi Kerur wrote:
>
>
> Hi Anatoly,
>
> I am actually confused with the state of vIOMMU + DPDK. Can you
> please help me clarify?
>
> I tested following DPDK versions
>
> (1) DPDK 17.11, exhibits the issue (IOMMU width as reported by
> RedHat and solution is to prevent using the patch)
> (2) DPDK 17.05.02 (stable release) using 'testpmd' I was able to
> bind a device in VM with VFIO driver and no DMAR error message
> on host
> (3) DPDK 16.11.02 (stable release) using 'testpmd' I was able to
> bind a device in VM with VFIO driver and no DMAR error message
> on host
>
> Clearly issue seen in 17.11 without the patch you mentioned is a
> regression or the issue was masked in earlier DPDK version? I
> did not test traffic with any DPDK version because I wanted to
> first get DMAR errors on host gone.
>
> Our application 'v1' is integrated with DPDK 16.11 and 'v2' is
> integrated with DPDK 17.05.01. In both 'v1' and 'v2' cases I
> don't see IOMMU width error messages on VM, however, DMAR error
> messages are seen host. I am not able to relate what causes DMAR
> error messages on host?
>
>
> Thanks.
>
>
> Hi Ravi,
>
> vIOMMU support is out of our hands, really - we can only make use of
> hardware (or emulation of it) that is available. 39-bit wide address
> *can* work, you just have to be lucky and get PA addresses that
> would fit into 39 bits (so under 512G limit), because we set up IOVA
> addresses to be 1:1 to physical addresses. We could, in principle,
> set up IOVA addresses to go from zero instead of them being a 1:1
> mapping to physical addresses, but that would introduce need to
> translate addresses between IOVA and physical in some cases (e.g. KNI).
>
> I'm not aware of any changes between 16.11 and 17.11 (and indeed
> 18.02) that would make or break support for 39-bit wide PA addresses
> for IOMMU. It is possible that VF/PF drivers do something
> differently which results in DMAR errors showing up sooner rather
> than later, but as far as VFIO support itself is concerned, there
> were no major changes in those releases.
>
>
> Hi Anatoly,
Hi Ravi,
>
> Thank you for your explanation. I would like to ask one more thing as I
> need to get v-iommu+ dpdk working in VM. Can you please tell me what
> determines 'Host Address Width", I know my question has nothing to do
> with dpdk and this is a dpdk list, but if you have any information
> please share it? I googled and found couple of ways to influence 'Host
> Address Width = 46' in guest as well (since dpdk + iommu works fine on
> host and DMAR on host reports address width as 46).
As far as i'm aware, address width supported by IOMMU is taken from
hardware (real or emulated). If e.g. QEMU's vIOMMU emulates an IOMMU
that only supports 39-bit addresses, that's what we'll get.
Not sure what "host address width" means exactly, but if i had to guess,
usually in VM-speak, "host" refers to, well, host - that is, the
physical machine the VM is running on. So, under that assumption, "host
address width" would be telling QEMU maximum width of IOMMU addresses on
the host (i.e. what you have in hardware, on the host - for example, you
might be running on a platform that only supports a 39-bit wide physical
address). Naturally, changing it around wouldn't change much - it would
presumably affect only the way vIOMMU remaps requests from guest to
host, but not the emulated hardware itself. However, i'm not
knowledgeable enough in that area to answer this question definitively.
>
> (1) Qemu has CPU param 'host-phys-bits' boolean, when set to true copies
> it from host
> (2) Qemu has 'phys-bits' integer, when set to '46' should influence guest
>
> Using above options when instantiating a VM doesn't help, Guest VM still
> ends up with 'Host address width = 39'.
>
> (3) There is another Qemu option 'x-aw-bits' which is for VT-d which can
> be set to '39' or '48'. This doesn't help either.
>
> Thanks.
>
> --
> Thanks,
> Anatoly
>
>
--
Thanks,
Anatoly
next prev parent reply other threads:[~2018-02-10 10:11 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-23 17:25 Ravi Kerur
2018-01-24 10:31 ` Burakov, Anatoly
2018-01-24 19:13 ` Ravi Kerur
2018-01-25 10:49 ` Burakov, Anatoly
2018-01-29 22:35 ` Ravi Kerur
2018-01-31 9:59 ` Burakov, Anatoly
2018-01-31 21:51 ` Ravi Kerur
2018-02-01 10:10 ` Burakov, Anatoly
2018-02-01 19:26 ` Ravi Kerur
2018-02-02 10:28 ` Burakov, Anatoly
2018-02-02 20:21 ` Ravi Kerur
2018-02-02 20:51 ` Ravi Kerur
2018-02-05 10:01 ` Burakov, Anatoly
2018-02-06 17:55 ` Ravi Kerur
2018-02-08 11:20 ` Burakov, Anatoly
2018-02-09 17:41 ` Ravi Kerur
2018-02-10 10:11 ` Burakov, Anatoly [this message]
2018-02-10 10:58 ` Burakov, Anatoly
2018-02-10 17:53 ` Ravi Kerur
2018-02-12 10:13 ` Burakov, Anatoly
2018-02-12 22:00 ` Ravi Kerur
2018-02-13 14:31 ` Burakov, Anatoly
2018-02-14 20:00 ` Ravi Kerur
2018-02-15 10:28 ` Burakov, Anatoly
2018-02-15 18:27 ` Ravi Kerur
2018-02-15 20:53 ` Ravi Kerur
2018-02-16 9:41 ` Burakov, Anatoly
2019-01-15 7:07 ` Hu, Xuekun
2019-01-15 11:22 ` Burakov, Anatoly
2019-01-15 13:07 ` Hu, Xuekun
2019-01-21 13:18 ` Hu, Xuekun
2019-01-21 13:39 ` Burakov, Anatoly
2019-01-21 14:44 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=881b4eac-3732-ce0f-c639-7d1ed0df0a61@intel.com \
--to=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=rkerur@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).