* [dpdk-dev] DMAR fault
@ 2013-08-12 20:19 jinho hwang
2013-08-12 20:28 ` Paul Barrette
0 siblings, 1 reply; 5+ messages in thread
From: jinho hwang @ 2013-08-12 20:19 UTC (permalink / raw)
To: dev
[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]
Hi All,
I am using iommu to receive packets both from hypervisor and from VM. KVM
is used for the virtualization. However, after I deliver the kernel options
(iommu and pci realloc), I can not receive packets in hypervisor, but VF
works fine in VM. When I tried to receive packets in hypervisor, dmesg
shows the following:
ixgbe 0000:03:00.1: complete
ixgbe 0000:03:00.1: PCI INT A disabled
igb_uio 0000:03:00.1: PCI INT A -> GSI 38 (level, low) -> IRQ 38
igb_uio 0000:03:00.1: setting latency timer to 64
igb_uio 0000:03:00.1: irq 87 for MSI/MSI-X
uio device registered with irq 57
DRHD: handling fault status reg 2
DMAR:[DMA Read] Request device [03:00.1] fault addr *b9d0f000*
DMAR:[fault reason 02] Present bit in context entry is clear
03:00.1 Ethernet controller: Intel Corporation 82599EB 10 Gigabit Dual Port
Backplane Connection (rev 01)
Subsystem: Intel Corporation Ethernet X520 10GbE Dual Port KX4-KR
Mezz
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 38
Region 0: Memory at *d9400000* (64-bit, prefetchable) [size=4M]
Region 2: I/O ports at ece0 [size=32]
Region 4: Memory at d9bfc000 (64-bit, prefetchable) [size=16K]
Expansion ROM at <ignored> [disabled]
Capabilities: <access denied>
Kernel driver in use: igb_uio
Kernel modules: ixgbe
We can see those addresses are not matched. So the kernel got fault. I am
wondering why this happens?
One suspicion for this is BIOS. I am currently using BIOS version 3.0, but
the latest is 6.3.0. Does this affect the matter?
Any help appreciated!
Jinho
[-- Attachment #2: Type: text/html, Size: 2702 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] DMAR fault
2013-08-12 20:19 [dpdk-dev] DMAR fault jinho hwang
@ 2013-08-12 20:28 ` Paul Barrette
2013-08-12 22:07 ` jinho hwang
0 siblings, 1 reply; 5+ messages in thread
From: Paul Barrette @ 2013-08-12 20:28 UTC (permalink / raw)
To: jinho hwang; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 2140 bytes --]
On 08/12/2013 04:19 PM, jinho hwang wrote:
> Hi All,
>
> I am using iommu to receive packets both from hypervisor and from VM.
> KVM is used for the virtualization. However, after I deliver the
> kernel options (iommu and pci realloc), I can not receive packets in
> hypervisor, but VF works fine in VM. When I tried to receive packets
> in hypervisor, dmesg shows the following:
>
> ixgbe 0000:03:00.1: complete
> ixgbe 0000:03:00.1: PCI INT A disabled
> igb_uio 0000:03:00.1: PCI INT A -> GSI 38 (level, low) -> IRQ 38
> igb_uio 0000:03:00.1: setting latency timer to 64
> igb_uio 0000:03:00.1: irq 87 for MSI/MSI-X
> uio device registered with irq 57
> DRHD: handling fault status reg 2
> DMAR:[DMA Read] Request device [03:00.1] fault addr *b9d0f000*
> DMAR:[fault reason 02] Present bit in context entry is clear
>
> 03:00.1 Ethernet controller: Intel Corporation 82599EB 10 Gigabit Dual
> Port Backplane Connection (rev 01)
> Subsystem: Intel Corporation Ethernet X520 10GbE Dual Port
> KX4-KR Mezz
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort+ >SERR- <PERR- INTx-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 38
> Region 0: Memory at *d9400000* (64-bit, prefetchable) [size=4M]
> Region 2: I/O ports at ece0 [size=32]
> Region 4: Memory at d9bfc000 (64-bit, prefetchable) [size=16K]
> Expansion ROM at <ignored> [disabled]
> Capabilities: <access denied>
> Kernel driver in use: igb_uio
> Kernel modules: ixgbe
>
> We can see those addresses are not matched. So the kernel got fault. I
> am wondering why this happens?
I have seen this happen when VT-d is enabled in the bios. If you are
using dpdk 1.4, add "iommu=pt" to your boot line. Without it, no
packets are received.
Pb
>
> One suspicion for this is BIOS. I am currently using BIOS version 3.0,
> but the latest is 6.3.0. Does this affect the matter?
>
> Any help appreciated!
>
> Jinho
>
[-- Attachment #2: Type: text/html, Size: 5815 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] DMAR fault
2013-08-12 20:28 ` Paul Barrette
@ 2013-08-12 22:07 ` jinho hwang
2013-08-12 22:22 ` Paul Barrette
0 siblings, 1 reply; 5+ messages in thread
From: jinho hwang @ 2013-08-12 22:07 UTC (permalink / raw)
To: Paul Barrette; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 2910 bytes --]
On Mon, Aug 12, 2013 at 4:28 PM, Paul Barrette
<paul.barrette@windriver.com>wrote:
>
> On 08/12/2013 04:19 PM, jinho hwang wrote:
>
> Hi All,
>
> I am using iommu to receive packets both from hypervisor and from VM. KVM
> is used for the virtualization. However, after I deliver the kernel options
> (iommu and pci realloc), I can not receive packets in hypervisor, but VF
> works fine in VM. When I tried to receive packets in hypervisor, dmesg
> shows the following:
>
> ixgbe 0000:03:00.1: complete
> ixgbe 0000:03:00.1: PCI INT A disabled
> igb_uio 0000:03:00.1: PCI INT A -> GSI 38 (level, low) -> IRQ 38
> igb_uio 0000:03:00.1: setting latency timer to 64
> igb_uio 0000:03:00.1: irq 87 for MSI/MSI-X
> uio device registered with irq 57
> DRHD: handling fault status reg 2
> DMAR:[DMA Read] Request device [03:00.1] fault addr *b9d0f000*
>
>
> DMAR:[fault reason 02] Present bit in context entry is clear
>
> 03:00.1 Ethernet controller: Intel Corporation 82599EB 10 Gigabit Dual
> Port Backplane Connection (rev 01)
> Subsystem: Intel Corporation Ethernet X520 10GbE Dual Port KX4-KR
> Mezz
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort+ >SERR- <PERR- INTx-
> Latency: 0, Cache Line Size: 64 bytes
> Interrupt: pin A routed to IRQ 38
> Region 0: Memory at *d9400000* (64-bit, prefetchable) [size=4M]
> Region 2: I/O ports at ece0 [size=32]
> Region 4: Memory at d9bfc000 (64-bit, prefetchable) [size=16K]
> Expansion ROM at <ignored> [disabled]
> Capabilities: <access denied>
> Kernel driver in use: igb_uio
> Kernel modules: ixgbe
>
> We can see those addresses are not matched. So the kernel got fault. I am
> wondering why this happens?
>
> I have seen this happen when VT-d is enabled in the bios. If you are
> using dpdk 1.4, add "iommu=pt" to your boot line. Without it, no packets
> are received.
>
> Pb
>
>
> One suspicion for this is BIOS. I am currently using BIOS version 3.0,
> but the latest is 6.3.0. Does this affect the matter?
>
> Any help appreciated!
>
> Jinho
>
>
>
>
Paul,
thanks. I tried your suggestion, but it works like no iommu command in boot
line. I passed intel_iommu=pt, and receive packets from hypervisor.
However, when I started VM with "-device pci-assign,host=01:00.0", it shows
the following message:
qemu-system-x86_64: -device pci-assign,host=03:10.0: No IOMMU found.
Unable to assign device "(null)"
qemu-system-x86_64: -device pci-assign,host=03:10.0: Device initialization
failed.
qemu-system-x86_64: -device pci-assign,host=03:10.0: Device
'kvm-pci-assign' could not be initialized
The device is detached from kernel, and move to pci-stub. dmesg does not
show any DMAR fault message anymore.
Any idea?
Jinho
[-- Attachment #2: Type: text/html, Size: 5565 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] DMAR fault
2013-08-12 22:07 ` jinho hwang
@ 2013-08-12 22:22 ` Paul Barrette
2013-08-12 22:25 ` jinho hwang
0 siblings, 1 reply; 5+ messages in thread
From: Paul Barrette @ 2013-08-12 22:22 UTC (permalink / raw)
To: jinho hwang; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 3332 bytes --]
On 08/12/2013 06:07 PM, jinho hwang wrote:
>
> On Mon, Aug 12, 2013 at 4:28 PM, Paul Barrette
> <paul.barrette@windriver.com <mailto:paul.barrette@windriver.com>> wrote:
>
>
> On 08/12/2013 04:19 PM, jinho hwang wrote:
>> Hi All,
>>
>> I am using iommu to receive packets both from hypervisor and from
>> VM. KVM is used for the virtualization. However, after I deliver
>> the kernel options (iommu and pci realloc), I can not receive
>> packets in hypervisor, but VF works fine in VM. When I tried to
>> receive packets in hypervisor, dmesg shows the following:
>>
>> ixgbe 0000:03:00.1: complete
>> ixgbe 0000:03:00.1: PCI INT A disabled
>> igb_uio 0000:03:00.1: PCI INT A -> GSI 38 (level, low) -> IRQ 38
>> igb_uio 0000:03:00.1: setting latency timer to 64
>> igb_uio 0000:03:00.1: irq 87 for MSI/MSI-X
>> uio device registered with irq 57
>> DRHD: handling fault status reg 2
>> DMAR:[DMA Read] Request device [03:00.1] fault addr *b9d0f000*
>> DMAR:[fault reason 02] Present bit in context entry is clear
>>
>> 03:00.1 Ethernet controller: Intel Corporation 82599EB 10 Gigabit
>> Dual Port Backplane Connection (rev 01)
>> Subsystem: Intel Corporation Ethernet X520 10GbE Dual
>> Port KX4-KR Mezz
>> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV-
>> VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
>> >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
>> Latency: 0, Cache Line Size: 64 bytes
>> Interrupt: pin A routed to IRQ 38
>> Region 0: Memory at *d9400000* (64-bit, prefetchable)
>> [size=4M]
>> Region 2: I/O ports at ece0 [size=32]
>> Region 4: Memory at d9bfc000 (64-bit, prefetchable)
>> [size=16K]
>> Expansion ROM at <ignored> [disabled]
>> Capabilities: <access denied>
>> Kernel driver in use: igb_uio
>> Kernel modules: ixgbe
>>
>> We can see those addresses are not matched. So the kernel got
>> fault. I am wondering why this happens?
> I have seen this happen when VT-d is enabled in the bios. If you
> are using dpdk 1.4, add "iommu=pt" to your boot line. Without it,
> no packets are received.
>
> Pb
>
>>
>> One suspicion for this is BIOS. I am currently using BIOS version
>> 3.0, but the latest is 6.3.0. Does this affect the matter?
>>
>> Any help appreciated!
>>
>> Jinho
>>
>
>
> Paul,
>
> thanks. I tried your suggestion, but it works like no iommu command in
> boot line. I passed intel_iommu=pt, and receive packets from
> hypervisor. However, when I started VM with "-device
> pci-assign,host=01:00.0", it shows the following message:
>
> qemu-system-x86_64: -device pci-assign,host=03:10.0: No IOMMU found.
> Unable to assign device "(null)"
> qemu-system-x86_64: -device pci-assign,host=03:10.0: Device
> initialization failed.
> qemu-system-x86_64: -device pci-assign,host=03:10.0: Device
> 'kvm-pci-assign' could not be initialized
>
> The device is detached from kernel, and move to pci-stub. dmesg does
> not show any DMAR fault message anymore.
>
> Any idea?
>
> Jinho
Jinho,
you need to specify both
" intel_iommu=on iommu=pt"
Pb
[-- Attachment #2: Type: text/html, Size: 10032 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-dev] DMAR fault
2013-08-12 22:22 ` Paul Barrette
@ 2013-08-12 22:25 ` jinho hwang
0 siblings, 0 replies; 5+ messages in thread
From: jinho hwang @ 2013-08-12 22:25 UTC (permalink / raw)
To: Paul Barrette; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 3420 bytes --]
On Mon, Aug 12, 2013 at 6:22 PM, Paul Barrette
<paul.barrette@windriver.com>wrote:
>
> On 08/12/2013 06:07 PM, jinho hwang wrote:
>
>
> On Mon, Aug 12, 2013 at 4:28 PM, Paul Barrette <
> paul.barrette@windriver.com> wrote:
>
>>
>> On 08/12/2013 04:19 PM, jinho hwang wrote:
>>
>> Hi All,
>>
>> I am using iommu to receive packets both from hypervisor and from VM. KVM
>> is used for the virtualization. However, after I deliver the kernel options
>> (iommu and pci realloc), I can not receive packets in hypervisor, but VF
>> works fine in VM. When I tried to receive packets in hypervisor, dmesg
>> shows the following:
>>
>> ixgbe 0000:03:00.1: complete
>> ixgbe 0000:03:00.1: PCI INT A disabled
>> igb_uio 0000:03:00.1: PCI INT A -> GSI 38 (level, low) -> IRQ 38
>> igb_uio 0000:03:00.1: setting latency timer to 64
>> igb_uio 0000:03:00.1: irq 87 for MSI/MSI-X
>> uio device registered with irq 57
>> DRHD: handling fault status reg 2
>> DMAR:[DMA Read] Request device [03:00.1] fault addr *b9d0f000*
>>
>>
>> DMAR:[fault reason 02] Present bit in context entry is clear
>>
>> 03:00.1 Ethernet controller: Intel Corporation 82599EB 10 Gigabit Dual
>> Port Backplane Connection (rev 01)
>> Subsystem: Intel Corporation Ethernet X520 10GbE Dual Port KX4-KR
>> Mezz
>> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> ParErr- Stepping- SERR- FastB2B- DisINTx+
>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>> <TAbort- <MAbort+ >SERR- <PERR- INTx-
>> Latency: 0, Cache Line Size: 64 bytes
>> Interrupt: pin A routed to IRQ 38
>> Region 0: Memory at *d9400000* (64-bit, prefetchable) [size=4M]
>> Region 2: I/O ports at ece0 [size=32]
>> Region 4: Memory at d9bfc000 (64-bit, prefetchable) [size=16K]
>> Expansion ROM at <ignored> [disabled]
>> Capabilities: <access denied>
>> Kernel driver in use: igb_uio
>> Kernel modules: ixgbe
>>
>> We can see those addresses are not matched. So the kernel got fault. I am
>> wondering why this happens?
>>
>> I have seen this happen when VT-d is enabled in the bios. If you are
>> using dpdk 1.4, add "iommu=pt" to your boot line. Without it, no packets
>> are received.
>>
>> Pb
>>
>>
>> One suspicion for this is BIOS. I am currently using BIOS version 3.0,
>> but the latest is 6.3.0. Does this affect the matter?
>>
>> Any help appreciated!
>>
>> Jinho
>>
>>
>>
>>
> Paul,
>
> thanks. I tried your suggestion, but it works like no iommu command in
> boot line. I passed intel_iommu=pt, and receive packets from hypervisor.
> However, when I started VM with "-device pci-assign,host=01:00.0", it shows
> the following message:
>
> qemu-system-x86_64: -device pci-assign,host=03:10.0: No IOMMU found.
> Unable to assign device "(null)"
> qemu-system-x86_64: -device pci-assign,host=03:10.0: Device
> initialization failed.
> qemu-system-x86_64: -device pci-assign,host=03:10.0: Device
> 'kvm-pci-assign' could not be initialized
>
> The device is detached from kernel, and move to pci-stub. dmesg does not
> show any DMAR fault message anymore.
>
> Any idea?
>
> Jinho
>
>
> Jinho,
> you need to specify both
>
> " intel_iommu=on iommu=pt"
>
> Pb
>
I tried that as well, but it works as if I only add intel_iommu=on.. which
means I do not receive any packets from hypervisor. I also have pci realloc
added. Does this affect?
Jinho
[-- Attachment #2: Type: text/html, Size: 8868 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-08-12 22:25 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-12 20:19 [dpdk-dev] DMAR fault jinho hwang
2013-08-12 20:28 ` Paul Barrette
2013-08-12 22:07 ` jinho hwang
2013-08-12 22:22 ` Paul Barrette
2013-08-12 22:25 ` jinho hwang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).