From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 97546A04BA; Wed, 7 Oct 2020 19:44:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 03F841B870; Wed, 7 Oct 2020 19:44:47 +0200 (CEST) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by dpdk.org (Postfix) with ESMTP id 977692C16 for ; Wed, 7 Oct 2020 19:44:43 +0200 (CEST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 097HWLf1186464; Wed, 7 Oct 2020 13:44:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=7s0Cew7gAqzYlQ+SF/sQ7mBgcJRR2MW729TyZF5UXAk=; b=AhkAkYSLhZLns/VTDiiercNaAGDrXsGXxJ6wkRLhdTmGhmT1+KL7LOnRxA3kKubUxEPX R+KjLzvZVhqH462oO9bu1Ztq6juUqU3ifkColUZtnVwbjgvgfmzu4oPIenpEbDgjxjC8 D1jsXxWZ4RFkdEazmYQ/KWe7Gfs0WqBz62KknV1EW/xaI7ewIiaXD0xdubHBqWTVNJh0 eoV4+wGvR2dmAI21wbFZtCxGKBtI3z0sEqEr6ydiUt0MqaRyJo7a9gdUCUYrbY7pbE4m sajCFLmycYjNz8Ht2KkGxrk/0fKQ92KJehJo4/oZ10WmZ0aH3M8PEmKaYyu5KKQreDGX PQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 341j4ygcnr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Oct 2020 13:44:41 -0400 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 097HXDJL188507; Wed, 7 Oct 2020 13:44:41 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0b-001b2d01.pphosted.com with ESMTP id 341j4ygcng-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Oct 2020 13:44:41 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 097HgMMf012040; Wed, 7 Oct 2020 17:44:40 GMT Received: from b01cxnp22035.gho.pok.ibm.com (b01cxnp22035.gho.pok.ibm.com [9.57.198.25]) by ppma03wdc.us.ibm.com with ESMTP id 33xgx90w00-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Oct 2020 17:44:40 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 097Hie6Q41288130 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 7 Oct 2020 17:44:40 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4AFF5112064; Wed, 7 Oct 2020 17:44:40 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E5693112061; Wed, 7 Oct 2020 17:44:39 +0000 (GMT) Received: from Davids-MBP.randomparity.org (unknown [9.211.87.34]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 7 Oct 2020 17:44:39 +0000 (GMT) To: "Burakov, Anatoly" , david.marchand@redhat.com Cc: dev@dpdk.org References: <20200630213823.1583764-1-drc@linux.vnet.ibm.com> <20200810210707.745083-1-drc@linux.vnet.ibm.com> <20200810210707.745083-2-drc@linux.vnet.ibm.com> <2c830988-c4db-7bdc-50f3-3fa445a81673@intel.com> From: David Christensen Message-ID: Date: Wed, 7 Oct 2020 10:44:39 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: <2c830988-c4db-7bdc-50f3-3fa445a81673@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-07_10:2020-10-07, 2020-10-07 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 suspectscore=0 impostorscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070108 Subject: Re: [dpdk-dev] [PATCH v3 1/1] vfio: modify spapr iommu support to use static window sizing X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 9/17/20 4:13 AM, Burakov, Anatoly wrote: > On 10-Aug-20 10:07 PM, David Christensen wrote: >> The SPAPR IOMMU requires that a DMA window size be defined before memory >> can be mapped for DMA. Current code dynamically modifies the DMA window >> size in response to every new memory allocation which is potentially >> dangerous because all existing mappings need to be unmapped/remapped in >> order to resize the DMA window, leaving hardware holding IOVA addresses >> that are temporarily unmapped.  The new SPAPR code statically assigns >> the DMA window size on first use, using the largest physical memory >> memory address when IOVA=PA and the highest existing memseg virtual >> address when IOVA=VA. >> >> Signed-off-by: David Christensen >> --- > > > >> +struct spapr_size_walk_param { >> +    uint64_t max_va; >> +    uint64_t page_sz; >> +    int external; >> +}; >> + >> +/* >> + * In order to set the DMA window size required for the SPAPR IOMMU >> + * we need to walk the existing virtual memory allocations as well as >> + * find the hugepage size used. >> + */ >>   static int >> -vfio_spapr_unmap_walk(const struct rte_memseg_list *msl, >> -        const struct rte_memseg *ms, void *arg) >> +vfio_spapr_size_walk(const struct rte_memseg_list *msl, void *arg) >>   { >> -    int *vfio_container_fd = arg; >> +    struct spapr_size_walk_param *param = arg; >> +    uint64_t max = (uint64_t) msl->base_va + (uint64_t) msl->len; >> -    /* skip external memory that isn't a heap */ >> -    if (msl->external && !msl->heap) >> -        return 0; >> +    if (msl->external) { >> +        param->external++; >> +        if (!msl->heap) >> +            return 0; >> +    } > > It would be nice to have some comments in the code explaining what we're > skipping and why. Reviewing this again, my inclination is to skip ALL external memory, which by definition would seem to be outside of IOMMU control, so the code would read: if (msl->external) { param->external++; return 0; } Not sure why existing code such as vfio_spapr_map_walk() distinguishes between heap and non-heap in this situation. Are there instances in x86 where it would matter? > Also, seems that you're using param->external as bool? This is a > non-public API so using stdbool is not an issue here, perhaps replace it > with bool param->has_external? Why do you think the distinction is necessary? Dave