From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 062A2A057B; Mon, 23 Mar 2020 18:41:06 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 477FD1C08E; Mon, 23 Mar 2020 18:41:06 +0100 (CET) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by dpdk.org (Postfix) with ESMTP id 9AFDC1C069 for ; Mon, 23 Mar 2020 18:41:04 +0100 (CET) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02NHY3dP033855; Mon, 23 Mar 2020 13:41:01 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ywf3e4b1x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 23 Mar 2020 13:41:01 -0400 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 02NHbHJ9044028; Mon, 23 Mar 2020 13:41:00 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ywf3e4b1f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 23 Mar 2020 13:41:00 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id 02NHZpAE004127; Mon, 23 Mar 2020 17:40:59 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma03dal.us.ibm.com with ESMTP id 2ywaw96qnh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 23 Mar 2020 17:40:59 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 02NHex5e53281168 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 23 Mar 2020 17:40:59 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EA818112062; Mon, 23 Mar 2020 17:40:58 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E7546112061; Mon, 23 Mar 2020 17:40:57 +0000 (GMT) Received: from ltc.linux.ibm.com (unknown [9.16.170.189]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Mon, 23 Mar 2020 17:40:57 +0000 (GMT) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 23 Mar 2020 10:40:57 -0700 From: dwilder To: Jerin Jacob Cc: Aaron Conole , Michael Santana , Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko , dpdk-dev , "Ruifeng Wang (Arm Technology China)" , David Marchand , David Christensen , wilder@us.ibm.com In-Reply-To: References: <20200220225207.30411-1-dwilder@us.ibm.com> <20200220225207.30411-2-dwilder@us.ibm.com> Message-ID: X-Sender: dwilder@us.ibm.com User-Agent: Roundcube Webmail/1.0.1 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-23_07:2020-03-23, 2020-03-23 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 priorityscore=1501 adultscore=0 mlxscore=0 mlxlogscore=999 impostorscore=0 bulkscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2003230090 Subject: Re: [dpdk-dev] [PATCH v3 1/3] eal/linux: select iova-mode va with no-huge option X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Thanks you for your review Jerin. See my responses are inline. On 2020-03-20 06:24, Jerin Jacob wrote: > On Fri, Feb 21, 2020 at 4:22 AM David Wilder > wrote: >> >> If --no-huge is set and iova-mode has not been specified force VA >> mode. >> If --no-huge and --iova-mode=PA is requested error out as this is >> an impossible configuration. >> >> Signed-off-by: David Wilder >> --- >> lib/librte_eal/linux/eal/eal.c | 14 ++++++++++++++ >> 1 file changed, 14 insertions(+) >> >> diff --git a/lib/librte_eal/linux/eal/eal.c >> b/lib/librte_eal/linux/eal/eal.c >> index 9530ee55f..d3a0a1731 100644 >> --- a/lib/librte_eal/linux/eal/eal.c >> +++ b/lib/librte_eal/linux/eal/eal.c >> @@ -1062,9 +1062,16 @@ rte_eal_init(int argc, char **argv) >> >> /* if no EAL option "--iova-mode=", use bus IOVA scheme >> */ >> if (internal_config.iova_mode == RTE_IOVA_DC) { >> + >> /* autodetect the IOVA mapping mode */ >> enum rte_iova_mode iova_mode = >> rte_bus_get_iommu_class(); >> >> + if (iova_mode == RTE_IOVA_PA && >> !rte_eal_has_hugepages()) { >> + iova_mode = RTE_IOVA_VA; > > What if igb_uio or vfio_nommu has been loaded(i.e no iommu support > enabled from the driver)? This would fail. Yes they would fail. If igb_uio or vfio_nommu (or any driver) cant be forced to VA mode it cant be used with out hugepages. Drivers can be available but not used therefor we print a warning message. > >> + RTE_LOG(WARNING, EAL, "Some buses want 'PA' >> but forcing 'VA' because --no-huge is requested.\n"); >> + RTE_LOG(WARNING, EAL, "Not all buses may be >> able to initialize.\n"); >> + } >> + >> if (iova_mode == RTE_IOVA_DC) { >> RTE_LOG(DEBUG, EAL, "Buses did not request a >> specific IOVA mode.\n"); >> >> @@ -1111,6 +1118,13 @@ rte_eal_init(int argc, char **argv) >> internal_config.iova_mode; >> } >> >> + if (rte_eal_iova_mode() == RTE_IOVA_PA && >> + rte_eal_has_hugepages() == 0) { >> + rte_eal_init_alert("Cannot use IOVA as 'PA' with >> --no-huge"); > > Top of the tree already detecting this case. am I missing anything? > > [master]dell[dpdk.org] $ sudo ./build/app/test/dpdk-test -c 0x3 > --no-huge --iova-mode=pa > EAL: Detected 56 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Static memory layout is selected, amount of reserved memory can > be adjusted with -m or --socket-mem > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: FATAL: Cannot use IOVA as 'PA' since physical addresses are not > available > EAL: Cannot use IOVA as 'PA' since physical addresses are not available > The check you reference is reporting that physical address are not available, for example no permissions to read /proc/self/pagemap. In this case, if --no-huge is set then PA mode is not allowed. There is no guarantee that physical address are persistent with out using hugepages. >> + rte_errno = EINVAL; >> + return -1; >> + } >> + >> if (rte_eal_iova_mode() == RTE_IOVA_PA && !phys_addrs) { >> rte_eal_init_alert("Cannot use IOVA as 'PA' since >> physical addresses are not available"); >> rte_errno = EINVAL; >> -- >> 2.25.0 >>