From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7401DA09FF; Tue, 5 Jan 2021 20:33:27 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2B40B160928; Tue, 5 Jan 2021 20:33:27 +0100 (CET) Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by mails.dpdk.org (Postfix) with ESMTP id C9A58160927 for ; Tue, 5 Jan 2021 20:33:25 +0100 (CET) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 105JVrS9190686; Tue, 5 Jan 2021 14:33:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=+SxTFLBUMT98dI++EUSYlZId5fHpEQjIG+RUiQt7Pa4=; b=Xn+ZcqTKvzgnAFF8m7e1wny/ZvDtQLKkng7HJU0LoNczwX0kyGNXoGhtZgEv1rEO8Tfs bAAuoqtjsb/WVMBLoio3fRbIw/KsO+D8c0oNOpWdHXIe6ne3qZMUuSYc5hT/JraJBfwj 4WIeHkXjlmDkAYC8Qt/P9OVqQAVA0yV7yzXcUinGLdKg7SXXMDx5DSSHaAL/ol7iDUUG ltZyhCMxLEosMgEysPE1ld58nIjCIj2B+gIG86smst9k/QqgFQIEXYSsqhMKXgHKMPsa zpC5iv1q0mx9GuY2zmBoBILpHeBnYf2b/LPBUui3TKeOnTNgc9dRB5jXolAv4Bn/glzO Qw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 35vxbhg1n3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 05 Jan 2021 14:33:24 -0500 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 105JVqTo190617; Tue, 5 Jan 2021 14:33:24 -0500 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 35vxbhg1kw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 05 Jan 2021 14:33:24 -0500 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 105JRtLd002148; Tue, 5 Jan 2021 19:33:22 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 35tgf8wyed-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 05 Jan 2021 19:33:22 +0000 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 105JXMI722938010 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 5 Jan 2021 19:33:22 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F14EB2064; Tue, 5 Jan 2021 19:33:22 +0000 (GMT) Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7812CB205F; Tue, 5 Jan 2021 19:33:21 +0000 (GMT) Received: from Davids-MBP.randomparity.org (unknown [9.163.86.38]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 5 Jan 2021 19:33:21 +0000 (GMT) To: Nithin Dabilpuram , anatoly.burakov@intel.com, david.marchand@redhat.com Cc: jerinj@marvell.com, dev@dpdk.org References: <20201012081106.10610-1-ndabilpuram@marvell.com> <20201217190604.29803-1-ndabilpuram@marvell.com> <20201217190604.29803-4-ndabilpuram@marvell.com> From: David Christensen Message-ID: <935bd057-2ec6-f42c-02a2-9b62784e4950@linux.vnet.ibm.com> Date: Tue, 5 Jan 2021 11:33:20 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-05_05:2021-01-05, 2021-01-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 clxscore=1015 spamscore=0 priorityscore=1501 mlxscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 impostorscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101050109 Subject: Re: [dpdk-dev] [PATCH v6 3/4] test: add test case to validate VFIO DMA map/unmap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hey Nithin, >> +static int >> +test_memory_vfio_dma_map(void) >> +{ >> + uint64_t sz1, sz2, sz = 2 * rte_mem_page_size(); >> + uint64_t unmap1, unmap2; >> + uint8_t *alloc_mem; >> + uint8_t *mem; >> + int ret; >> + >> + /* Allocate twice size of requirement from heap to align later */ >> + alloc_mem = malloc(sz * 2); >> + if (!alloc_mem) { >> + printf("Skipping test as unable to alloc %"PRIx64"B from heap\n", >> + sz * 2); >> + return 1; >> + } >> + >> + /* Force page allocation */ >> + memset(alloc_mem, 0, sz * 2); >> + >> + mem = RTE_PTR_ALIGN(alloc_mem, rte_mem_page_size()); >> + >> + /* map the whole region */ >> + ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD, >> + (uintptr_t)mem, (rte_iova_t)mem, sz); I'm not sure how to resolve this patch for POWER systems. The patch currently fails with the error: EAL: cannot map vaddr for IOMMU, error 22 (Invalid argument) The problem is that the size argument (page size of 64KB * 2) is smaller than the page size set when the DMA window is created (2MB or 1GB depending on system configuration for hugepages), resulting in the EINVAL error. When I tried bumping the sz value up to 2 * 1GB the test also failed because the VA address was well outside the DMA window set when scanning memseg lists. Allocating heap memory dynamically through the EAL works since it's allocated in hugepage size segments and the EAL attempts to keep VA memory addresses contiguous, therefore within the defined DMA window. But the downside is that the memory is DMA mapped behind the scenes in vfio_mem_event_callback(). Not sure how to get around this without duplicating a lot of the heap management code in your test. Maybe others have a suggestion. Dave