From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f52.google.com (mail-lf0-f52.google.com [209.85.215.52]) by dpdk.org (Postfix) with ESMTP id E56B51B7CA for ; Fri, 9 Feb 2018 18:41:34 +0100 (CET) Received: by mail-lf0-f52.google.com with SMTP id a204so12280467lfa.2 for ; Fri, 09 Feb 2018 09:41:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=UHNw0QWsPszmIO+MSoCcT8M2fTRzt/Ig5Mh36qsCA9Q=; b=uh2AQD/HpvwZvQfG7huNuPdvWZ0igAl/jSe9maz3Y8axq9j/AXrk9ATF959uF9T7Ph CSHx3Sfnn77SbLb0/6MAM/6JEAm7jh24NxoRZEpSeyQbYJUfmESAt0SUcoGln8e5jnWP 0RUoPj8SqPr6nhH/EH56W7ZTpK/66aO90aZKMA5dVsy6IQR/Ah4kuODdYwXhUxxSsoev oqYjrEA7pgqlxopkZsdJ1NwdeTBiTvwSWmft7aMOkMNVL2YoesLYOGwlC6JFi0PhfCDo xSmci8xQm3iGBQ/GoMi2VEKlTNhGfzv9cyybMDGzLxwvZI9XEbn+KwHDMI19HJ9cBNtt OHew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=UHNw0QWsPszmIO+MSoCcT8M2fTRzt/Ig5Mh36qsCA9Q=; b=tm6xqSaXvPua/EdLcWeEBgeR2jr4qsblZeDd6Sn0NStQ4q4tXN/kwOKtLZlziAjVPp X+FjQ5cKaZ/LPYT/foIWNL7MCm/MpxFcrC/YyMD9AA0y+AYO4gtCvWqRN77gLFeSeXVr HoMn6tjFh4+D2rkKwFva1aNBaesgClJN99dS/jLswl0FwjVko3Ltxr20zw3cYrKvsS9/ ueIlOgm5v4pB3YM4c+T81tDUtVkH2fWiGSbJxpOj6KmEn+kl0AOdTTH36Clbh3gVRaoZ GW4csuWrMvC2p2MeyJLQJwRZSfW2nfanu6JeSmWmS5tE8ni8sZsSntbQ2yIzHirhYYnw 9kqw== X-Gm-Message-State: APf1xPDY0fV1s9R5IRLCatPk6fBxcplBiHxAwx7Ew+juZDXA9gJop4qC NRxyxrGF3h9iVu6hwa8/5YRn2P3TqQqsiRgG7+k= X-Google-Smtp-Source: AH8x2245xASnVPhWLP+N1INIu0lWwoRcPSlU69SUgErRX6ptwE0y06mEUQgdEfokNVBb4A+f8Y/ckSSJ7GI+AEier3c= X-Received: by 10.46.4.196 with SMTP id a65mr2492461ljf.22.1518198094407; Fri, 09 Feb 2018 09:41:34 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.196.17 with HTTP; Fri, 9 Feb 2018 09:41:33 -0800 (PST) In-Reply-To: References: <8ddb30a3-1253-ff60-20bb-b735fef5a91c@intel.com> <10911b54-57ee-370b-a4f7-f34accf4811e@intel.com> <3464b900-8648-c128-e959-dc60a8883a2d@intel.com> <0120c68f-cf42-5d8b-5600-0514f76209b0@intel.com> <4e5ef551-b7b3-e12d-6254-b882bb952bbb@intel.com> From: Ravi Kerur Date: Fri, 9 Feb 2018 09:41:33 -0800 Message-ID: To: "Burakov, Anatoly" Cc: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] IXGBE, IOMMU DMAR DRHD handling fault issue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Feb 2018 17:41:35 -0000 On Thu, Feb 8, 2018 at 3:20 AM, Burakov, Anatoly wrote: > On 06-Feb-18 5:55 PM, Ravi Kerur wrote: > >> >> Hi Anatoly, >> >> I am actually confused with the state of vIOMMU + DPDK. Can you please >> help me clarify? >> >> I tested following DPDK versions >> >> (1) DPDK 17.11, exhibits the issue (IOMMU width as reported by RedHat and >> solution is to prevent using the patch) >> (2) DPDK 17.05.02 (stable release) using 'testpmd' I was able to bind a >> device in VM with VFIO driver and no DMAR error message on host >> (3) DPDK 16.11.02 (stable release) using 'testpmd' I was able to bind a >> device in VM with VFIO driver and no DMAR error message on host >> >> Clearly issue seen in 17.11 without the patch you mentioned is a >> regression or the issue was masked in earlier DPDK version? I did not test >> traffic with any DPDK version because I wanted to first get DMAR errors on >> host gone. >> >> Our application 'v1' is integrated with DPDK 16.11 and 'v2' is integrated >> with DPDK 17.05.01. In both 'v1' and 'v2' cases I don't see IOMMU width >> error messages on VM, however, DMAR error messages are seen host. I am not >> able to relate what causes DMAR error messages on host? >> >> >> Thanks. >> >> > Hi Ravi, > > vIOMMU support is out of our hands, really - we can only make use of > hardware (or emulation of it) that is available. 39-bit wide address *can* > work, you just have to be lucky and get PA addresses that would fit into 39 > bits (so under 512G limit), because we set up IOVA addresses to be 1:1 to > physical addresses. We could, in principle, set up IOVA addresses to go > from zero instead of them being a 1:1 mapping to physical addresses, but > that would introduce need to translate addresses between IOVA and physical > in some cases (e.g. KNI). > > I'm not aware of any changes between 16.11 and 17.11 (and indeed 18.02) > that would make or break support for 39-bit wide PA addresses for IOMMU. It > is possible that VF/PF drivers do something differently which results in > DMAR errors showing up sooner rather than later, but as far as VFIO support > itself is concerned, there were no major changes in those releases. > > Hi Anatoly, Thank you for your explanation. I would like to ask one more thing as I need to get v-iommu+ dpdk working in VM. Can you please tell me what determines 'Host Address Width", I know my question has nothing to do with dpdk and this is a dpdk list, but if you have any information please share it? I googled and found couple of ways to influence 'Host Address Width = 46' in guest as well (since dpdk + iommu works fine on host and DMAR on host reports address width as 46). (1) Qemu has CPU param 'host-phys-bits' boolean, when set to true copies it from host (2) Qemu has 'phys-bits' integer, when set to '46' should influence guest Using above options when instantiating a VM doesn't help, Guest VM still ends up with 'Host address width = 39'. (3) There is another Qemu option 'x-aw-bits' which is for VT-d which can be set to '39' or '48'. This doesn't help either. Thanks. -- > Thanks, > Anatoly >