From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 82802A0562; Wed, 14 Apr 2021 12:08:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D456161941; Wed, 14 Apr 2021 12:08:58 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id 35EA616188A for ; Wed, 14 Apr 2021 12:08:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618394936; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=23fY04ypBzSwaAefvEaR0lsdwfleT04fRSarPW2HY2A=; b=QokdFMDV0+kWTrO/VuK7lgzp5qek1IGViqbIeHWdFkl/0YmyDvz2AEFOKFPH9CtN9sNIsb HsjfYV6ynQDXHqEQH2Dvd1eRqMFV3Oy+WIsg5zWFjbXcsDD6IiMjsfWBUnHQN4Q9JLjV7E 7euo2RUxGMlRw5emUxQ9nNFpw3TZF44= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-489-LAm6GHiKMoqkSLfdwiDKWg-1; Wed, 14 Apr 2021 06:08:54 -0400 X-MC-Unique: LAm6GHiKMoqkSLfdwiDKWg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3522A107ACC7; Wed, 14 Apr 2021 10:08:53 +0000 (UTC) Received: from [10.36.110.28] (unknown [10.36.110.28]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B86361F43D; Wed, 14 Apr 2021 10:08:51 +0000 (UTC) To: "Hu, Jiayu" , "dev@dpdk.org" Cc: "Xia, Chenbo" , "Wang, Yinan" , "Pai G, Sunil" , "Jiang, Cheng1" References: <1615985773-406787-1-git-send-email-jiayu.hu@intel.com> <1617368642-131298-1-git-send-email-jiayu.hu@intel.com> <1617368642-131298-4-git-send-email-jiayu.hu@intel.com> <0b974e3cd6534947ba6568d93d9431ab@intel.com> From: Maxime Coquelin Message-ID: Date: Wed, 14 Apr 2021 12:08:50 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <0b974e3cd6534947ba6568d93d9431ab@intel.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2 3/4] vhost: avoid deadlock on async register X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 4/14/21 3:40 AM, Hu, Jiayu wrote: > Hi Maxime, > >> -----Original Message----- >> From: Maxime Coquelin >> Sent: Tuesday, April 13, 2021 7:33 PM >> To: Hu, Jiayu ; dev@dpdk.org >> Cc: Xia, Chenbo ; Wang, Yinan >> ; Pai G, Sunil ; Jiang, Cheng1 >> >> Subject: Re: [PATCH v2 3/4] vhost: avoid deadlock on async register >> >> >> >> On 4/2/21 3:04 PM, Jiayu Hu wrote: >>> Users can register async copy device in vring_state_changed(), >>> when vhost queue is enabled. However, a deadlock occurs inside >>> rte_vhost_async_channel_register(), if >> VHOST_USER_F_PROTOCOL_FEATURES >>> is not supported, as vhost_user_msg_handler() takes vq->access_lock >>> before calling vhost_user_set_vring_kick(). >>> >>> This patch avoids async register deadlock by removing calling >>> vring_state_changed() in vhost_user_set_vring_kick(). It's safe >>> as vhost_user_msg_handler() will call vring_state_changed() anyway. >>> >>> Signed-off-by: Jiayu Hu >>> --- >>> lib/librte_vhost/vhost_user.c | 3 --- >>> 1 file changed, 3 deletions(-) >>> >>> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >>> index 44c0452..8f0eba6 100644 >>> --- a/lib/librte_vhost/vhost_user.c >>> +++ b/lib/librte_vhost/vhost_user.c >>> @@ -1918,9 +1918,6 @@ vhost_user_set_vring_kick(struct virtio_net >> **pdev, struct VhostUserMsg *msg, >>> */ >>> if (!(dev->features & (1ULL << >> VHOST_USER_F_PROTOCOL_FEATURES))) { >>> vq->enabled = true; >>> - if (dev->notify_ops->vring_state_changed) >>> - dev->notify_ops->vring_state_changed( >>> - dev->vid, file.index, 1); >>> } >>> >>> if (vq->ready) { >>> >> >> As replied earlier on v1, I agree the call to vring_state_changed here >> is not needed. But it might not be enough, there are other cases where >> you could have issues. > > vhost_user_notify_queue_state() can be called in three cases: > 1. when vq ready status changes, vhost_user_msg_handler() calls it to notify > backend. But vhost_user_msg_handler() doesn't take lock before calling it. > So in this case, no deadlock occurs in async register. > > 2. if vq->ready is true, vhost_user_set_vring_call() calls it to notify backend > vq is not enabled. Although vhost_user_set_vring_call() is protected by lock, > async register is called only if vq is enabled, so async register will not be called > in this case. > > 3. If vq->ready is true, vhost_user_set_vring_kick() calls it to notify backend > vq is not enabled. Same as #2, async register is called only when vq is enabled. > Even if vhost_user_set_vring_kick() is protected by lock, there is no deadlock in > async register, as it will not be called in this case. > > In summary, I think there is no deadlock issue in async register if we > can remove calling vring_state_change() in vhost_user_set_vring_kick(). But unregister one could be called in theory no? Otherwise it would look unbalanced. At least on disabled notification, the app should make sure the DMA transfers to and from the vring are stopped before it returns from the callabck. Otherwise it could lead to undefined behavior. >> >> Please add stable and Fixes tag. > > Do you suggest to make the patch as a fix for 8639d54563a > ("vhost: introduce async enqueue registration API")? But the > thing is that code removed in this patch is not introduced > by this commit. The commit you need to point to is the one introducing the .vring_state_changed() call. > Thanks, > Jiayu >> >> Thanks, >> Maxime >