From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id D82395929 for ; Sun, 28 Sep 2014 09:42:27 +0200 (CEST) Received: from azsmga001.ch.intel.com ([10.2.17.19]) by fmsmga103.fm.intel.com with ESMTP; 28 Sep 2014 00:39:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,613,1406617200"; d="scan'208";a="480508774" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by azsmga001.ch.intel.com with ESMTP; 28 Sep 2014 00:48:55 -0700 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.195.1; Sun, 28 Sep 2014 00:48:55 -0700 Received: from cdsmsx101.ccr.corp.intel.com (172.17.3.36) by fmsmsx110.amr.corp.intel.com (10.18.116.10) with Microsoft SMTP Server (TLS) id 14.3.195.1; Sun, 28 Sep 2014 00:48:55 -0700 Received: from cdsmsx102.ccr.corp.intel.com ([169.254.2.14]) by CDSMSX101.ccr.corp.intel.com ([169.254.1.65]) with mapi id 14.03.0195.001; Sun, 28 Sep 2014 15:48:53 +0800 From: "Zhang, Jerry" To: Hiroshi Shimamoto , "dev@dpdk.org" Thread-Topic: DPDK doesn't work with iommu=pt Thread-Index: Ac/ZZvLQu0nCCbsaRVygc3CBHpGSYgBiRPVA Date: Sun, 28 Sep 2014 07:48:53 +0000 Message-ID: References: <7F861DC0615E0C47A872E6F3C5FCDDBD02ADDCDD@BPXM14GP.gisp.nec.co.jp> In-Reply-To: <7F861DC0615E0C47A872E6F3C5FCDDBD02ADDCDD@BPXM14GP.gisp.nec.co.jp> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.17.6.105] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Hayato Momma Subject: Re: [dpdk-dev] DPDK doesn't work with iommu=pt X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Sep 2014 07:42:28 -0000 Met the similar issue before. VT-d enabled? If so you may need to contact HP to upgrade the BIOS or you m= ay disable VT-d and remove iommu=3Dpt intel_iommu=3Don if you don't need VF= function. >-----Original Message----- >From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto >Sent: Friday, September 26, 2014 5:14 PM >To: dev@dpdk.org >Cc: Hayato Momma >Subject: [dpdk-dev] DPDK doesn't work with iommu=3Dpt > >I encountered an issue that DPDK doesn't work with "iommu=3Dpt >intel_iommu=3Don" >on HP ProLiant DL380p Gen8 server. I'm using the following environment; > > HW: ProLiant DL380p Gen8 > CPU: E5-2697 v2 > OS: RHEL7 > kernel: kernel-3.10.0-123 and the latest kernel 3.17-rc6+ > DPDK: v1.7.1-53-gce5abac > NIC: 82599ES > >When boot with "iommu=3Dpt intel_iommu=3Don", I got the below message and = no >packets are handled. > > [ 120.809611] dmar: DRHD: handling fault status reg 2 > [ 120.809635] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr >aa010000 > DMAR:[fault reason 02] Present bit in context entry is clear > >How to reproduce; >just run testpmd ># ./testpmd -c 0xf -n 4 -- -i > >Configuring Port 0 (socket 0) >PMD: ixgbe_dev_tx_queue_setup(): sw_ring=3D0x7ffff54eafc0 >hw_ring=3D0x7ffff4200000 dma_addr=3D0xaa000000 >PMD: ixgbe_dev_tx_queue_setup(): Using full-featured tx code path >PMD: ixgbe_dev_tx_queue_setup(): - txq_flags =3D 0 >[IXGBE_SIMPLE_FLAGS=3Df01] >PMD: ixgbe_dev_tx_queue_setup(): - tx_rs_thresh =3D 32 >[RTE_PMD_IXGBE_TX_MAX_BURST=3D32] >PMD: ixgbe_dev_rx_queue_setup(): sw_ring=3D0x7ffff54ea740 >hw_ring=3D0x7ffff4210000 dma_addr=3D0xaa010000 >PMD: check_rx_burst_bulk_alloc_preconditions(): Rx Burst Bulk Alloc >Preconditions: rxq->rx_free_thresh=3D0, RTE_PMD_IXGBE_RX_MAX_BURST=3D32 >PMD: ixgbe_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are not >satisfied, Scattered Rx is requested, or >RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC is not enabled (port=3D0, queue=3D0). >PMD: check_rx_burst_bulk_alloc_preconditions(): Rx Burst Bulk Alloc >Preconditions: rxq->rx_free_thresh=3D0, RTE_PMD_IXGBE_RX_MAX_BURST=3D32 > >testpmd> start > io packet forwarding - CRC stripping disabled - packets/burst=3D32 > nb forwarding cores=3D1 - nb forwarding ports=3D2 > RX queues=3D1 - RX desc=3D128 - RX free threshold=3D0 > RX threshold registers: pthresh=3D8 hthresh=3D8 wthresh=3D0 > TX queues=3D1 - TX desc=3D512 - TX free threshold=3D0 > TX threshold registers: pthresh=3D32 hthresh=3D0 wthresh=3D0 > TX RS bit threshold=3D0 - TXQ flags=3D0x0 > > >and ping from another box to this server. ># ping6 -I eth2 ff02::1 > >I got the below error message and no packet is received. >I couldn't see any increase RX/TX count in testpmt statistics > >testpmd> show port stats 0 > > ######################## NIC statistics for port 0 >######################## > RX-packets: 6 RX-missed: 0 RX-bytes: 732 > RX-badcrc: 0 RX-badlen: 0 RX-errors: 0 > RX-nombuf: 0 > TX-packets: 0 TX-errors: 0 TX-bytes: 0 > >################################################################# >########### >testpmd> show port stats 0 > > ######################## NIC statistics for port 0 >######################## > RX-packets: 6 RX-missed: 0 RX-bytes: 732 > RX-badcrc: 0 RX-badlen: 0 RX-errors: 0 > RX-nombuf: 0 > TX-packets: 0 TX-errors: 0 TX-bytes: 0 > >################################################################# >########### > > >The fault addr in error message must be RX DMA descriptor > >error message > [ 120.809635] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr >aa010000 > >log in testpmd > PMD: ixgbe_dev_rx_queue_setup(): sw_ring=3D0x7ffff54ea740 >hw_ring=3D0x7ffff4210000 dma_addr=3D0xaa010000 > >I think the NIC received a packet in fifo and try to put into memory with = DMA. >Before starting DMA, the NIC get the target address from RX descriptors in >RDBA register. >But accessing RX descriptors failed in IOMMU unit and reported it to the k= ernel. > > DMAR:[fault reason 02] Present bit in context entry is clear > >The error message looks there is no valid entry in IOMMU. > >I think the following issue is very similar, but using Ubuntu14.04 couldn'= t fix in my >case. >http://thread.gmane.org/gmane.comp.networking.dpdk.devel/2281 > >I tried Ubuntu14.04.1 and got the below error. > > [ 199.710191] dmar: DRHD: handling fault status reg 2 > [ 199.710896] dmar: DMAR:[DMA Read] Request device [21:00.0] fault addr >7c24df000 > [ 199.710896] DMAR:[fault reason 06] PTE Read access is not set > >Currently I could see this issue on HP ProLiant DL380p Gen8 only. >Is there any idea? >Has anyone noticed this issue? > >Note: we're thinking to use SR-IOV and DPDK app in the same box. >The box has 2 NICs, one for SR-IOV and pass through to VM, one (no SR-IOV)= for >DPDK app in host. > >thanks, >Hiroshi