From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f65.google.com (mail-pg0-f65.google.com [74.125.83.65]) by dpdk.org (Postfix) with ESMTP id 62C135F3B; Fri, 27 Apr 2018 17:20:36 +0200 (CEST) Received: by mail-pg0-f65.google.com with SMTP id l2-v6so1808452pgc.7; Fri, 27 Apr 2018 08:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=/JYDdHKxQa8cHH+231Qy+HflZDOHXnxAcq4pMkqQ/Us=; b=k60PADzAgXHIi5fRYc35KLGPeO6k0y4v/R6YA6W2C3V3hIfxld+jsNfJBQ8wdDdg3R el87QSAhrKBPYPweXRfEK5dBHvn0ZSY7B49c3CXV3/OlN/E8eQg5HpC1QDqIUYg0VB9f skOARFJCgfSD39ZR4weARw39dQtWZRw3Pr5285RYWa+CoPrvZki/P6Vil9cb3h234OPj P8my2DPZEwU3RVRYugjt37VInF6P+Hbti1/iBaogkI3o7cFLpRLP2Itt/L07bcLaiRc+ yKyTxGFIzYcMdg6LHxl8xU0tnY0Ww6mIIh51dYlb+2oyN8L9PPBTZOFqD3PoIgk2lTYf Uo8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=/JYDdHKxQa8cHH+231Qy+HflZDOHXnxAcq4pMkqQ/Us=; b=bvuM5OcwtO+O7T9cIHCBCJNqhaNhAy42p1lAuCPhxrlc/vg60squ4JYqQ2KKR5RkF/ UQEZLRvf34yk3HIetTFPginSd/RDx5I2VB54hLL/i5FRhUsrzhawQt5V2JIShe033Pke hi40dIsEp1FsEx9pyy6PExEXihD6sW7xWDn/yYQlVwJqrLjrP9InjjTBMNFEfYlXSOz4 e9xWcxtCUuPH/Kk0IKD7eRZ+0T2CBsUwNUjerxvyd2PhlrRufxnJHlWzGRH8BRK1PWjL KaW+aGFqMCrhhzBqjLnLZrKCzeRopo/weSEyPprCjw0RRiLKU/YfZ9XfYJYK9MqC4t+u WUMA== X-Gm-Message-State: ALQs6tBLvo62pr0IMcSTejDvvMvpRDL6HhpWc6qNF618WLcUwydLRwpv vg8sNPe9PPY6gQZHniObmec= X-Google-Smtp-Source: AB8JxZp5jPuryNdu4X+xi/2V+C7FGOPzuvH3ePwajjYWQb3IS6CX4gsJi7Htg3PphAVoOdj7EW5OZw== X-Received: by 2002:a17:902:2006:: with SMTP id n6-v6mr2703815pla.125.1524842435494; Fri, 27 Apr 2018 08:20:35 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([183.240.196.59]) by smtp.gmail.com with ESMTPSA id c18-v6sm2887897pgu.63.2018.04.27.08.20.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Apr 2018 08:20:34 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: maxime.coquelin@redhat.com, jianfeng.tan@intel.com, yliu@fridaylinux.org Cc: dev@dpdk.org, Tonghao Zhang , stable@dpdk.org Date: Fri, 27 Apr 2018 08:19:43 -0700 Message-Id: <1524842385-61707-1-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 Subject: [dpdk-dev] [PATCH 1/3] vhost: fix deadlock due to vhostuser socket and fdset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Apr 2018 15:20:36 -0000 From: Tonghao Zhang When qemu close the unix socket fd of the vhostuser as a server, and then immediately delete the vhostuser port on openvswitch. There will be a deadlock. A thread (fdset event thread): B thread: 1. fdset_event_dispatch rte_vhost_driver_unregister 2. set the fd busy to 1. lock vsocket->conn_mutex 3. vhost_user_read_cb fdset_del waits busy changed to 0. 4. vhost peer closed, remove the conn from vsocket->conn_list: lock vsocket->conn_mutex 5. set the fd busy to 0 Fixes: 65388b43 ("vhost: fix fd leaks for vhost-user server mode") Cc: stable@dpdk.org Cc: Yuanhan Liu Signed-off-by: Tonghao Zhang --- lib/librte_vhost/fd_man.c | 32 ++++++++++++++++++++++++++++++++ lib/librte_vhost/fd_man.h | 1 + lib/librte_vhost/socket.c | 13 ++++++++++++- 3 files changed, 45 insertions(+), 1 deletion(-) diff --git a/lib/librte_vhost/fd_man.c b/lib/librte_vhost/fd_man.c index 8590ee5..b24c27d 100644 --- a/lib/librte_vhost/fd_man.c +++ b/lib/librte_vhost/fd_man.c @@ -174,6 +174,38 @@ return dat; } +/** + * Unregister the fd from the fdset. + * + * If parameters are invalid, return directly -2. + * And check whether fd is busy, if yes, return -1. + * Otherwise, try to delete the fd from fdset and + * return true. + */ +int +fdset_try_del(struct fdset *pfdset, int fd) +{ + int i; + + if (pfdset == NULL || fd == -1) + return -2; + + pthread_mutex_lock(&pfdset->fd_mutex); + i = fdset_find_fd(pfdset, fd); + if (i != -1 && pfdset->fd[i].busy) { + pthread_mutex_unlock(&pfdset->fd_mutex); + return -1; + } + + if (i != -1) { + pfdset->fd[i].fd = -1; + pfdset->fd[i].rcb = pfdset->fd[i].wcb = NULL; + pfdset->fd[i].dat = NULL; + } + + pthread_mutex_unlock(&pfdset->fd_mutex); + return 0; +} /** * This functions runs in infinite blocking loop until there is no fd in diff --git a/lib/librte_vhost/fd_man.h b/lib/librte_vhost/fd_man.h index 76a42fb..3331bcd 100644 --- a/lib/librte_vhost/fd_man.h +++ b/lib/librte_vhost/fd_man.h @@ -44,6 +44,7 @@ int fdset_add(struct fdset *pfdset, int fd, fd_cb rcb, fd_cb wcb, void *dat); void *fdset_del(struct fdset *pfdset, int fd); +int fdset_try_del(struct fdset *pfdset, int fd); void *fdset_event_dispatch(void *arg); diff --git a/lib/librte_vhost/socket.c b/lib/librte_vhost/socket.c index 4a561ad..822db41 100644 --- a/lib/librte_vhost/socket.c +++ b/lib/librte_vhost/socket.c @@ -922,13 +922,24 @@ struct vhost_user_reconnect_list { vhost_user_remove_reconnect(vsocket); } +again: pthread_mutex_lock(&vsocket->conn_mutex); for (conn = TAILQ_FIRST(&vsocket->conn_list); conn != NULL; conn = next) { next = TAILQ_NEXT(conn, next); - fdset_del(&vhost_user.fdset, conn->connfd); + /* + * If r/wcb is executing, release the + * conn_mutex lock, and try again since + * the r/wcb may use the conn_mutex lock. + */ + if (fdset_try_del(&vhost_user.fdset, + conn->connfd) == -1) { + pthread_mutex_unlock(&vsocket->conn_mutex); + goto again; + } + RTE_LOG(INFO, VHOST_CONFIG, "free connfd = %d for device '%s'\n", conn->connfd, path); -- 1.8.3.1