Re: [dpdk-dev] DPDK doesn't work with iommu=pt

["Zhang, Jerry" <jerry.zhang@intel.com>]

Met the similar issue before.
VT-d enabled? If so you may need to contact HP to upgrade the BIOS or you may disable VT-d and remove iommu=pt intel_iommu=on if you don't need VF function.

>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto
>Sent: Friday, September 26, 2014 5:14 PM
>To: dev@dpdk.org
>Cc: Hayato Momma
>Subject: [dpdk-dev] DPDK doesn't work with iommu=pt
>
>I encountered an issue that DPDK doesn't work with "iommu=pt
>intel_iommu=on"
>on HP ProLiant DL380p Gen8 server. I'm using the following environment;
>
>  HW: ProLiant DL380p Gen8
>  CPU: E5-2697 v2
>  OS: RHEL7
>  kernel: kernel-3.10.0-123 and the latest kernel 3.17-rc6+
>  DPDK: v1.7.1-53-gce5abac
>  NIC: 82599ES
>
>When boot with "iommu=pt intel_iommu=on", I got the below message and no
>packets are handled.
>
>  [  120.809611] dmar: DRHD: handling fault status reg 2
>  [  120.809635] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr
>aa010000
>  DMAR:[fault reason 02] Present bit in context entry is clear
>
>How to reproduce;
>just run testpmd
># ./testpmd -c 0xf -n 4 -- -i
>
>Configuring Port 0 (socket 0)
>PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7ffff54eafc0
>hw_ring=0x7ffff4200000 dma_addr=0xaa000000
>PMD: ixgbe_dev_tx_queue_setup(): Using full-featured tx code path
>PMD: ixgbe_dev_tx_queue_setup():  - txq_flags = 0
>[IXGBE_SIMPLE_FLAGS=f01]
>PMD: ixgbe_dev_tx_queue_setup():  - tx_rs_thresh = 32
>[RTE_PMD_IXGBE_TX_MAX_BURST=32]
>PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7ffff54ea740
>hw_ring=0x7ffff4210000 dma_addr=0xaa010000
>PMD: check_rx_burst_bulk_alloc_preconditions(): Rx Burst Bulk Alloc
>Preconditions: rxq->rx_free_thresh=0, RTE_PMD_IXGBE_RX_MAX_BURST=32
>PMD: ixgbe_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are not
>satisfied, Scattered Rx is requested, or
>RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC is not enabled (port=0, queue=0).
>PMD: check_rx_burst_bulk_alloc_preconditions(): Rx Burst Bulk Alloc
>Preconditions: rxq->rx_free_thresh=0, RTE_PMD_IXGBE_RX_MAX_BURST=32
>
>testpmd> start
>  io packet forwarding - CRC stripping disabled - packets/burst=32
>  nb forwarding cores=1 - nb forwarding ports=2
>  RX queues=1 - RX desc=128 - RX free threshold=0
>  RX threshold registers: pthresh=8 hthresh=8 wthresh=0
>  TX queues=1 - TX desc=512 - TX free threshold=0
>  TX threshold registers: pthresh=32 hthresh=0 wthresh=0
>  TX RS bit threshold=0 - TXQ flags=0x0
>
>
>and ping from another box to this server.
># ping6 -I eth2 ff02::1
>
>I got the below error message and no packet is received.
>I couldn't see any increase RX/TX count in testpmt statistics
>
>testpmd> show port stats 0
>
>  ######################## NIC statistics for port 0
>########################
>  RX-packets: 6          RX-missed: 0          RX-bytes:  732
>  RX-badcrc:  0          RX-badlen: 0          RX-errors: 0
>  RX-nombuf:  0
>  TX-packets: 0          TX-errors: 0          TX-bytes:  0
>
>#################################################################
>###########
>testpmd> show port stats 0
>
>  ######################## NIC statistics for port 0
>########################
>  RX-packets: 6          RX-missed: 0          RX-bytes:  732
>  RX-badcrc:  0          RX-badlen: 0          RX-errors: 0
>  RX-nombuf:  0
>  TX-packets: 0          TX-errors: 0          TX-bytes:  0
>
>#################################################################
>###########
>
>
>The fault addr in error message must be RX DMA descriptor
>
>error message
>  [  120.809635] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr
>aa010000
>
>log in testpmd
>  PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7ffff54ea740
>hw_ring=0x7ffff4210000 dma_addr=0xaa010000
>
>I think the NIC received a packet in fifo and try to put into memory with DMA.
>Before starting DMA, the NIC get the target address from RX descriptors in
>RDBA register.
>But accessing RX descriptors failed in IOMMU unit and reported it to the kernel.
>
>  DMAR:[fault reason 02] Present bit in context entry is clear
>
>The error message looks there is no valid entry in IOMMU.
>
>I think the following issue is very similar, but using Ubuntu14.04 couldn't fix in my
>case.
>http://thread.gmane.org/gmane.comp.networking.dpdk.devel/2281
>
>I tried Ubuntu14.04.1 and got the below error.
>
>  [  199.710191] dmar: DRHD: handling fault status reg 2
>  [  199.710896] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr
>7c24df000
>  [  199.710896] DMAR:[fault reason 06] PTE Read access is not set
>
>Currently I could see this issue on HP ProLiant DL380p Gen8 only.
>Is there any idea?
>Has anyone noticed this issue?
>
>Note: we're thinking to use SR-IOV and DPDK app in the same box.
>The box has 2 NICs, one for SR-IOV and pass through to VM, one (no SR-IOV) for
>DPDK app in host.
>
>thanks,
>Hiroshi