From: "Gonzalez Monroy, Sergio" <sergio.gonzalez.monroy@intel.com>
To: "Assaad, Sami (Sami)" <sami.assaad@alcatel-lucent.com>,
"Qiu, Michael" <michael.qiu@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)?
Date: Wed, 20 May 2015 11:56:44 +0100 [thread overview]
Message-ID: <555C686C.3020300@intel.com> (raw)
In-Reply-To: <9478F0FB69DAA249AF0A9BDA1E6ED9521881BDE6@US70TWXCHMBA07.zam.alcatel-lucent.com>
On 19/05/2015 20:23, Assaad, Sami (Sami) wrote:
> Hello Michael,
>
> I've updated the kernel and QEMU. Here are the packages I'm using:
>
> --> CentOS 7 - 3.10.0-229.4.2.el7.x86_64
> - qemu-kvm-1.5.3-86.el7_1.2.x86_64
> - libvirt-1.2.8-16.el7_1.3.x86_64
> - virt-manager-1.1.0-12.el7.noarch
> - virt-what-1.13-5.el7.x86_64
> - libvirt-glib-0.1.7-3.el7.x86_64
>
> I've modified the virtual machine XML file to include the following:
>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <driver name='vfio'/>
> <source>
> <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <driver name='vfio'/>
> <source>
> <address domain='0x0000' bus='0x04' slot='0x10' function='0x1'/>
> </source>
> </hostdev>
>
>
> The syslog error I'm obtaining relating to the iommu is the following:
> #dmesg | grep -e DMAR -e IOMMU
>
> [ 3362.370564] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
>
>
> From the /var/log/messages file, the complete VM log is the following:
>
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): carrier is OFF
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): new Tun device (driver: 'unknown' ifindex: 30)
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): exported as /org/freedesktop/NetworkManager/Devices/29
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (virbr0): bridge port vnet0 was attached
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): enslaved to virbr0
> May 19 15:10:12 ni-nfvhost01 kernel: device vnet0 entered promiscuous mode
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): link connected
> May 19 15:10:12 ni-nfvhost01 kernel: virbr0: port 2(vnet0) entered listening state
> May 19 15:10:12 ni-nfvhost01 kernel: virbr0: port 2(vnet0) entered listening state
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: unmanaged -> unavailable (reason 'connection-assumed') [10 20 41]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: unavailable -> disconnected (reason 'connection-assumed') [20 30 41]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: starting connection 'vnet0'
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 1 of 5 (Device Prepare) scheduled...
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 1 of 5 (Device Prepare) started...
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: disconnected -> prepare (reason 'none') [30 40 0]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 2 of 5 (Device Configure) scheduled...
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 1 of 5 (Device Prepare) complete.
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 2 of 5 (Device Configure) starting...
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: prepare -> config (reason 'none') [40 50 0]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 2 of 5 (Device Configure) successful.
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 3 of 5 (IP Configure Start) scheduled.
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 2 of 5 (Device Configure) complete.
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 3 of 5 (IP Configure Start) started...
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: config -> ip-config (reason 'none') [50 70 0]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: Stage 3 of 5 (IP Configure Start) complete.
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: ip-config -> secondaries (reason 'none') [70 90 0]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: secondaries -> activated (reason 'none') [90 100 0]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): Activation: successful, device activated.
> May 19 15:10:12 ni-nfvhost01 dbus-daemon: dbus[1295]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service'
> May 19 15:10:12 ni-nfvhost01 dbus[1295]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service'
> May 19 15:10:12 ni-nfvhost01 systemd: Starting Network Manager Script Dispatcher Service...
> May 19 15:10:12 ni-nfvhost01 systemd: Starting Virtual Machine qemu-vNIDS-VM1.
> May 19 15:10:12 ni-nfvhost01 systemd-machined: New machine qemu-vNIDS-VM1.
> May 19 15:10:12 ni-nfvhost01 systemd: Started Virtual Machine qemu-vNIDS-VM1.
> May 19 15:10:12 ni-nfvhost01 dbus-daemon: dbus[1295]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
> May 19 15:10:12 ni-nfvhost01 dbus[1295]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
> May 19 15:10:12 ni-nfvhost01 systemd: Started Network Manager Script Dispatcher Service.
> May 19 15:10:12 ni-nfvhost01 nm-dispatcher: Dispatching action 'up' for vnet0
> May 19 15:10:12 ni-nfvhost01 kvm: 1 guest now active
> May 19 15:10:12 ni-nfvhost01 systemd: Unit iscsi.service cannot be reloaded because it is inactive.
> May 19 15:10:12 ni-nfvhost01 kernel: vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
> May 19 15:10:12 ni-nfvhost01 kernel: virbr0: port 2(vnet0) entered disabled state
> May 19 15:10:12 ni-nfvhost01 kernel: device vnet0 left promiscuous mode
> May 19 15:10:12 ni-nfvhost01 kernel: virbr0: port 2(vnet0) entered disabled state
> May 19 15:10:12 ni-nfvhost01 avahi-daemon[1280]: Withdrawing workstation service for vnet0.
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): device state change: activated -> unmanaged (reason 'removed') [100 10 36]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <info> (vnet0): deactivating device (reason 'removed') [36]
> May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: <warn> (virbr0): failed to detach bridge port vnet0
> May 19 15:10:12 ni-nfvhost01 nm-dispatcher: Dispatching action 'down' for vnet0
> May 19 15:10:12 ni-nfvhost01 journal: Unable to read from monitor: Connection reset by peer
> May 19 15:10:12 ni-nfvhost01 journal: internal error: early end of file from monitor: possible problem:
> 2015-05-19T19:10:12.674077Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to set iommu for container: Operation not permitted
> 2015-05-19T19:10:12.674118Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to setup container for group 19
> 2015-05-19T19:10:12.674128Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to get group 19
> 2015-05-19T19:10:12.674141Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: Device initialization failed.
> 2015-05-19T19:10:12.674155Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: Device 'vfio-pci' could not be initialized
>
> May 19 15:10:12 ni-nfvhost01 kvm: 0 guests now active
> May 19 15:10:12 ni-nfvhost01 systemd-machined: Machine qemu-vNIDS-VM1 terminated.
> May 19 15:11:01 ni-nfvhost01 systemd: Created slice user-0.slice.
> May 19 15:11:01 ni-nfvhost01 systemd: Starting Session 329 of user root.
>
>
> Overall Hypothesis: The issue seems to be related to the Ethernet Controller's interfaces which I'm trying to bring into the VM. My Ethernet Controller is : Intel 10G x540-AT2 (rev 01).
> The problem is associated to RMRR.
> Can this issue be attributed to my BIOS? My Bios is the following: ProLiant System BIOS P89 V1.21 11/03/2014.
Hi Sami,
I think the following answer from Lee Roberts could be helpful:
http://dpdk.org/ml/archives/dev/2015-March/015504.html
AFAIK, new Linux kernels (v3.8+) will not attach RMRR-affected devices
to an IOMMU domain.
commit ea2447f700cab264019b52e2b417d689e052dcfd
Author: Tom Mingarelli<thomas.mingarelli@hp.com>
Date: Tue Nov 20 19:43:17 2012 +0000
intel-iommu: Prevent devices with RMRRs from being placed into SI Domain
This patch is to prevent non-USB devices that have RMRRs associated with the
being placed into the SI Domain during init. This fixes the issue where the
for devices being placed in and out of the SI Domain gets lost.
Signed-off-by: Thomas Mingarelli<thomas.mingarelli@hp.com>
Tested-by: Shuah Khan<shuah.khan@hp.com>
Reviewed-by: Donald Dutile<ddutile@redhat.com>
Reviewed-by: Alex Williamson<alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel<joro@8bytes.org>
Sergio
> Thanks in advance.
>
> Best Regards,
> Sami.
>
prev parent reply other threads:[~2015-05-20 10:56 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-14 21:38 Assaad, Sami (Sami)
2015-05-14 23:47 ` Stephen Hemminger
2015-05-15 9:27 ` Bruce Richardson
2015-05-15 9:31 ` Ananyev, Konstantin
2015-05-15 12:59 ` Assaad, Sami (Sami)
2015-05-15 13:56 ` Ananyev, Konstantin
2015-05-15 12:54 ` Assaad, Sami (Sami)
2015-05-15 13:08 ` Bruce Richardson
2015-05-15 19:30 ` Assaad, Sami (Sami)
2015-05-18 10:01 ` Qiu, Michael
2015-05-19 19:23 ` Assaad, Sami (Sami)
2015-05-20 9:14 ` Qiu, Michael
2015-05-20 9:19 ` Qiu, Michael
2015-05-20 10:56 ` Gonzalez Monroy, Sergio [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=555C686C.3020300@intel.com \
--to=sergio.gonzalez.monroy@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=michael.qiu@intel.com \
--cc=sami.assaad@alcatel-lucent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).