From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A14DBA10DA for ; Wed, 31 Jul 2019 23:20:36 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 84CE21BF0E; Wed, 31 Jul 2019 23:20:35 +0200 (CEST) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by dpdk.org (Postfix) with ESMTP id 9478C1BEE3 for ; Wed, 31 Jul 2019 23:20:34 +0200 (CEST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x6VL7vkv131601; Wed, 31 Jul 2019 17:20:33 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2u3hqajp2b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Jul 2019 17:20:33 -0400 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.27/8.16.0.27) with SMTP id x6VL8O1V133513; Wed, 31 Jul 2019 17:20:33 -0400 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 2u3hqajp1v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Jul 2019 17:20:33 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id x6VLK3gI007415; Wed, 31 Jul 2019 21:20:32 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma02dal.us.ibm.com with ESMTP id 2u0e85vuv9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 31 Jul 2019 21:20:32 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x6VLKUHk55574862 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 Jul 2019 21:20:30 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A7DDCBE05B; Wed, 31 Jul 2019 21:20:30 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 667B9BE04F; Wed, 31 Jul 2019 21:20:29 +0000 (GMT) Received: from davids-mbp.randomparity.local (unknown [9.211.73.252]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Wed, 31 Jul 2019 21:20:29 +0000 (GMT) To: David Marchand Cc: Takeshi T Yoshimura , Maxime Coquelin , dev , "Burakov, Anatoly" References: <20190731033523.2482-1-tyos@jp.ibm.com> <5bb4f727-b07e-5f52-de0e-f5552cb6420a@linux.vnet.ibm.com> From: David Christensen Message-ID: Date: Wed, 31 Jul 2019 14:20:28 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-31_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907310212 Subject: Re: [dpdk-dev] [PATCH] eal: forcing IOVA as PA in ppc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >> Here's the output from my Power 9 system running 19.08-RC2 (I'm not >> using VFIO since I have an MLX5 adapter). >> >> sudo -E LD_LIBRARY_PATH=/home/davec/src/dpdk/build/lib >> /home/davec/src/dpdk/build/app/testpmd --log-level lib.\*:debug -w >> 0000:01:00.0 -w 0000:01:00.1 -l 4,8-79 -n 4 -- --rxq=32 --rxd=1024 >> --txq=32 --txd=4096 --nb-cores=64 -i -a --numa --forward-mode=rxonly >> EAL: Detected lcore 0 as core 0 on socket 0 >> ... >> EAL: Detected lcore 159 as core 2140 on socket 8 >> EAL: Support maximum 1536 logical core(s) by configuration. >> EAL: Detected 160 lcore(s) >> EAL: Detected 2 NUMA nodes >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket >> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or >> directory) >> EAL: VFIO PCI modules not loaded >> EAL: DPAA Bus not present. Skipping. >> EAL: Bus pci wants IOVA as 'DC' > > Ok, I think I understand now. > > Can you try this hunk ? > IOVA mode is now reported as PA. MLX5 PMD fails to load correctly, but that also occurs without the patch and when the EAL command-line option "--iova-mode=pa" is used. This worked previously in RC1 and may be a separate issue. sudo -E LD_LIBRARY_PATH=/home/davec/src/dpdk/build/lib /home/davec/src/dpdk/build/app/testpmd --log-level="lib.*:debug" -w 0000:01:00.0 -w 0000:01:00.1 -l 4,8-79 -n 4 -- --rxq=32 --rxd=1024 --txq=32 --txd=4096 --nb-cores=64 -i -a --numa --forward-mode=rxonly EAL: Detected lcore 0 as core 0 on socket 0 ... EAL: Detected lcore 159 as core 2140 on socket 8 EAL: Support maximum 1536 logical core(s) by configuration. EAL: Detected 160 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory) EAL: VFIO PCI modules not loaded EAL: DPAA Bus not present. Skipping. EAL: PCI driver net_mlx5 for device 0000:01:00.0 wants IOVA as 'PA' EAL: PCI driver net_mlx5 for device 0000:01:00.1 wants IOVA as 'PA' EAL: Bus pci wants IOVA as 'PA' EAL: Bus dpaa_bus wants IOVA as 'DC' EAL: Bus fslmc wants IOVA as 'DC' EAL: Selected IOVA mode 'PA' EAL: No available hugepages reported in hugepages-2048kB EAL: Probing VFIO support... EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) EAL: VFIO modules not loaded, skipping VFIO support... EAL: Ask a virtual area of 0x30000 bytes EAL: Virtual area found at 0x100000000 (size = 0x30000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 4096 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 EAL: Detected memory type: socket_id:8 hugepage_sz:1073741824 EAL: Creating 4 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x100030000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x140000000 (size = 0x800000000) EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x940000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x980000000 (size = 0x800000000) EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x1180000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x11c0000000 (size = 0x800000000) EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x19c0000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 0 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x1a00000000 (size = 0x800000000) EAL: Creating 4 segment lists: n_segs:32 socket_id:8 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x2200000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 8 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x2240000000 (size = 0x800000000) EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x2a40000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 8 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x2a80000000 (size = 0x800000000) EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x3280000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 8 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x32c0000000 (size = 0x800000000) EAL: Ask a virtual area of 0x10000 bytes EAL: Virtual area found at 0x3ac0000000 (size = 0x10000) EAL: Memseg list allocated: 0x100000kB at socket 8 EAL: Ask a virtual area of 0x800000000 bytes EAL: Virtual area found at 0x3b00000000 (size = 0x800000000) EAL: TSC frequency is ~510000 KHz EAL: Master lcore 4 is ready (tid=7fffabb46c90;cpuset=[4]) EAL: lcore 9 is ready (tid=7fffa9c5dc90;cpuset=[9]) ... EAL: lcore 73 is ready (tid=7ffea77edc90;cpuset=[73]) EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 1024MB EAL: PCI device 0000:01:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1019 net_mlx5 EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered net_mlx5: mlx5.c:1683: mlx5_dev_spawn(): MPLS over GRE/UDP tunnel offloading disabled due to old OFED/rdma-core version or firmware configuration EAL: PCI device 0000:01:00.1 on NUMA socket 0 EAL: probe driver: 15b3:1019 net_mlx5 net_mlx5: mlx5.c:1683: mlx5_dev_spawn(): MPLS over GRE/UDP tunnel offloading disabled due to old OFED/rdma-core version or firmware configuration EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) testpmd: /home/davec/src/dpdk/drivers/net/mlx5/mlx5_ethdev.c:520: mlx5_set_txlimit_params: Assertion `config->txq_inline_min >= 0' failed. Aborted Dave