From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 49F49A0C47; Tue, 26 Oct 2021 11:49:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2D909410D5; Tue, 26 Oct 2021 11:49:15 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 3EF90407FF for ; Tue, 26 Oct 2021 11:49:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635241753; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QxIjo+t8shkYv8Vs2zQ+OuZHWXLZKFr+42FuwDMHKPA=; b=DRJpfAaaLmtExzRRtLgbzuDnc1Dk42YMqfsv/1SZ75yosQPaPxGo0WC8RBeal0gvNADMyE r6dnR1d/RaVbYqfmj4jEWriBEe2RVZJ/uhhkWxn92SAkDF5USMTqXZ89h1ZJK6PzOd/OOI iSi8ewIKrPWFILY2OwNrb8wKik6PqvI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-146-H-P2iTFCN1mkFvvMUvkZ9Q-1; Tue, 26 Oct 2021 05:49:12 -0400 X-MC-Unique: H-P2iTFCN1mkFvvMUvkZ9Q-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 604BD10A8E09; Tue, 26 Oct 2021 09:49:11 +0000 (UTC) Received: from [10.39.208.37] (unknown [10.39.208.37]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E136D57CA4; Tue, 26 Oct 2021 09:49:04 +0000 (UTC) Message-ID: Date: Tue, 26 Oct 2021 11:49:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 To: "Ding, Xuan" , "dev@dpdk.org" , "david.marchand@redhat.com" , "Xia, Chenbo" Cc: "Burakov, Anatoly" References: <20211025203353.147346-1-maxime.coquelin@redhat.com> From: Maxime Coquelin In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] vhost: fix async DMA map X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 10/26/21 10:49, Ding, Xuan wrote: > Hi Maxime, > >> -----Original Message----- >> From: Maxime Coquelin >> Sent: Tuesday, October 26, 2021 2:53 PM >> To: Ding, Xuan ; dev@dpdk.org; >> david.marchand@redhat.com; Xia, Chenbo >> Cc: Burakov, Anatoly >> Subject: Re: [PATCH] vhost: fix async DMA map >> >> >> >> On 10/26/21 04:07, Ding, Xuan wrote: >>> Hi Maxime, >>> >>>> -----Original Message----- >>>> From: Maxime Coquelin >>>> Sent: Tuesday, October 26, 2021 4:47 AM >>>> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo >>>> ; Ding, Xuan >>>> Subject: Re: [PATCH] vhost: fix async DMA map >>>> >>>> Hi Xuan, >>>> >>>> On 10/25/21 22:33, Maxime Coquelin wrote: >>>>> This patch fixes possible NULL-pointer dereferencing >>>>> reported by Coverity and also fixes NUMA reallocation >>>>> of the async DMA map. >>>>> >>>>> Fixes: 7c61fa08b716 ("vhost: enable IOMMU for async vhost") >>>>> >>>>> Coverity issue: 373655 >>>>> >>>>> Signed-off-by: Maxime Coquelin >>>>> --- >>>>> lib/vhost/vhost_user.c | 45 +++++++++++++++++++----------------------- >>>>> 1 file changed, 20 insertions(+), 25 deletions(-) >>>>> >>>> >>>> I posted this patch to fix the issue reported by Coverity and also other >>>> issue on NUMA realloc that I found at the same time. But I wonder >>>> whether all this async_map_status is needed. >>> >>> Thanks for your fix! I can help to review and test the patch later. >>> >>> I add the async_map_status in v2 for compatibility. Some DMA device, >>> like DSA, may use kernel idxd driver only. If there is no device bound to >>> DPDK vfio and kernel vfio module is modprobed to ensure >> rte_vfio_is_enabled() is true, >>> we will unavoidably do DMA map/unmap and it will fail. >>> >>> Therefore, the dma_map_status here is used to filter this situation by >> preventing >>> unnecessary DMA unmap. >> >> Ok, then I think we can just remove the async DMA map. >> >>>> >>>> Indeed, if the only place where we DMA map is in >>>> vhost_user_mmap_region(). If it fails, the error is propagated, the mem >>>> table are freed and NACK is replied to the master. IOW, the device will >>>> be in an unusable state. >>> >>> I agree with you, this is the place I consider right to do DMA map >>> because we also do SW mapping here, any suggestions? >> >> No suggestion, I was just explaining that at the only place where >> DMA map were done, mapping errors were properly handled and propagated. > > What about just setting async_copy to false, and allow switching to sync path. > >> >>>> >>>> Removing the async DMA map will simplify a lot the code, do you agree to >>>> remove it or there is something I missed? >>> >>> See above. Indeed, it adds a lot of code. But we can't know the driver for >>> each device in vhost lib, or we can only restrict the user to bind some >> devices >>> to DPDK vfio if async logic needed. >> >> I would think we don't care if DMA unmap fails, we can just do the same >> as what you do for DMA map, i.e. just ignore the error. > > Get your idea, we can do the same as DMA map, and in this way dma_map_status flag can be removed. > >> >> Thanks to this discussion, I have now more concerns on how it works. I >> think we have a problem here in case of DMA device hotplug, that device >> could miss the necessary map entries from Vhost if no VFIO devices were >> attached at VHST_USER_SET_MEM_TABLE time. How would you handle that >> case? > > DMA device is uncore, so I don't see the hotplug issue here. I'm not sure what 'uncore' is, I suppose you mean your device cannot be physically added/removed to the system. I was not clear enough in my question. I meant that for example, the application is started and the Vhost port is created. Then, the DMA device is bound to VFIO, and probed by the DPDK application. Finally, the application register the DMA device as an async channel in Vhost. I think it will not work as the SET_MEM_TABLE will already have happened, so the guest memory won't be mapped in the VFIO container. Do you have the same understanding? > I will have another patch containing compatibility with sync path, and async_map_status flag will be removed. > Hope to get your insights. What do you mean by compatibility with sync path? Thanks for taking care of the async map status removal. Maxime > Thanks, > Xuan > >> >> Regards, >> Maxime >> >>>> >>>> Thanks, >>>> Maxime >>> >